We must bear abundantly the fruit of holiness (see Gal 5:22ff) and evangelization (Jn 15:5). Otherwise, we will be like "a withered, rejected branch, picked up to be thrown in the fire and burnt" (Jn 15:6). We bear fruit abundantly by being attached to and living in the Vine, Jesus Christ (Jn 15:5). We must be abiding in Jesus and He in us, and stay in communion with all the others who abide in Jesus.
In the last few centuries of Christianity, we have the anomaly of Christians who maintain they are abiding in Jesus and yet are cut off from the other branches joined to Jesus, the Vine. Have you ever seen a tree where the branches are connected to the rest of the tree but not to all the branches of the rest of the tree? This is a logical and botanical impossibility. Can you imagine a tree where several of the branches are partially cut off? There are those who are not in full communion with the Church.
Of course, Jesus is speaking analogously, and an analogy can be pressed too far. Nevertheless, Jesus, using another analogy, makes it clear that those baptized into Christ have been baptized into one body (1 Cor 12:13). If we are united to Jesus, the Head of the body, we are to be united to all the other parts of the body. Jesus promised that the world would believe when Christians are one (Jn 17:21). In unity, we will bear the great harvest leading to Jesus' final return.