In genre fiction, the antagonist is usually either known throughout the book or revealed before the end, so that protagonist and antagonist can battle it out during climax.
What I am wondering is, how can a story work out, if the antagonist remains unknown? That is, the protagonist works towards his goal against an opposed force, the source of which is never revealed.
Think of the Lord of the Rings without anyone knowing of Sauron and what he wants. The hobbits and Gandalf would be unable to form a coherent plan. They wouldn't know the goal of their opponent, his weakness, the purpose of his actions, the relationship between him and the Ring. They would flee the Nazgul, seek information and allies, just as they do in Tolkien's novel, but they would have to figure out a solution by trial and error. Maybe they would come to the conclusion that they have to destroy the Ring, and maybe they would manage to do it in the right manner, but all the indication they got of their success would be the letting off of Saurons attacks. Much like the draft stills once you close the window, but you never get to see the source of the wind.
How would one pull that off? How could that be an interesting and satisfying read despite the lacking climactic battle?
And more to the point: What story would we tell? In The Lord of the Rings there is a conflict between two identifiable forces. There are many interpretations, e.g. that Sauron and his forces represent Nazi Germany, or industrialization, or just plain Evil, but despite the final meaning being open, the two forces are unmistakeable and the story tells of their conflict. In a story without antagonist, there would be no conflict, as a conflict needs two or more goals to be pitted against each other. In a story without antagonist, there would be no opposing goal. Even the protagonist does not have a veritable goal except to end the disruption.
The Lord of the Rings without an antagonist would lose both the Lord (the antagonist) and the Ring (the goal of the antagonist) from its title and with it the essence of its story. It would no longer be a story of two opposing forces, but a story of – what?
I guess it would be a story of whatever goes on inside the heroes. It would be a psychological novel. Or it would be a story of a society that has to deal with desaster. It would be a social or political novel. How could it remain fantasy?
Finally, are there examples of novels (or films) where the antagonist remains unknown, so that I could look at a working example?
The reason I ask this question is that I am always dissapointed when the antagonist is revealed in a horror or fantasy novel. When, as a teenager, I watched the old black-and-white Fankenstein movie, there was a strong sense of mystery and suspense, until Boris Karloff appeard. Immediately the wonder of the movie was lost. Special effects have gotten better, but still the same happens to me: as soon as I see the Alien, it appears banal and unterrifying and – as if the movie makers felt the same way – the movie turns from suspense to action.
I have always wondered how you can uphold the mystery. How the supernatural, the terrible, the strange, can remain what they are. And I think the only way is by not revealing them, by having them remain unknown and unknowable (which, I believe, is one reason the judaeo-christian God is such a powerful idea).