And yet somehow, every year, new authors sell blockbusters and earn $millions.
JK Rowling went from nobody to being worth nearly half $billion. So did Dan Brown, so did Stephen King, to name a few, so have many others -- And that is just on their share of the profits, their sales are in the multi-billion range. The entertainment industry with a foundation in writing fiction must be worth $trillions.
Obviously, nobody would buy their work if they did not find it new and interesting. Which is why this:
Add to this the supposition that all plots, themes and genres have been thoroughly explored,
is crap; a false supposition.
As for all the books out there: The world is constantly changing, technology is constantly advancing. This is 2018, and nobody on Earth has seen the Earth of 2019. There are no good stories written in 1970 about teens with cell phones and the Internet, or young superhackers taking down the world economy. Any that come remotely close would fall today on ears too sophisticated to sustain their suspension of disbelief, too many details would be wrong and could not have been anticipated.
The same goes for fantasy worlds. Every person is unique and grows up in a unique way with unique experiences, even twins can argue, and have different milestone experiences with different people. So everyone invents different characters, based on what they want to see, and thus everyone creates a different entertainment.
The point of reading fiction is not to understand the structure of the story, it is to meet new people and have fun.
It doesn't make a difference if the plot can be crammed into a one page outline or is similar to a million other plots. If it is done by new characters, in a new place, and we don't know exactly to the word how it will turn out: It can be entertaining.
Yes, of course I know, watching Sherlock or Elementary or half a dozen other Sherlockian super-detective shows that in the typical episode a puzzle will be introduced, frustration will ensue with wrong leads and dead ends, then it will be solved, and the world reset to do it all again next week. So why do we still watch it? What makes this formulaic story entertaining for fifty episodes in a row?
For that matter, why watch any formulaic entertainment, like competition shows for singing or dancing or cooking, or sports? WHY are they entertaining even though we know very much that one will win, the others will lose?
We watch, read, and listen because of the characters, we humans have the unique ability, given decent writing / acting, to suspend disbelief and feel like fictional characters are real and we are sharing their journey and emotions. Making every new character a different story.
No matter how many books are written, there will always be new characters, like there will always be new humans. In old books, the culture of their time is reflected in the writing, like it or not, and they grow stale and anachronistic. Regardless of the setting; fantasy or reality or scifi, in the past, future, or present, new readers want characters that feel like they feel, and the further the book recedes into the past, the less likely that is.
A story needs a plot like a person needs a skeleton. Absolutely a necessity, and yes, skeletons are all very similar with differences hardly worth mentioning: We can classify them all into roughly two types; male and female. Or go further and subdivide skeletons into more classifications like child and adults, broken or missing parts. But when there is flesh on and in the skeleton, it matters less and what we care about is that flesh, what it looks like, what it thinks and says, and that is what is truly unique about every person: That flesh. And that is what is truly unique about every book: Not the plot skeleton, but (in good writing) the unique characters that are the flesh on those bones, the unique puzzle they must solve.