I'm writing something similar to what you describe (a "War of Worlds" in a SciFi setting with a horrific number of characters, plots, cultures, and locations).
With the risk of being a bit off topic, I am going to list the tools I use and how I use them. I'll also talk about some things I've realized while working on this project with regards to keeping track of everything.
I am, however, not going to list any tool that has explicit handling of several books and a "book superstructure"... The closest I can get is Scrivener that could be described as a file system on steroids (if you were a bit mean about it). And as such can keep track of any form of structure you want.
I need to use several different software tools. Which should be about as surprising as finding more than one type of tool in a carpenter's toolbox...
I have a number of programs that I know how to use and when I need to do something or document something I pick the one that makes the most sense.
And if you stop reading here, you've gotten my best tip: use several tools you know how to use to get/create/keep the information you need.
Don't try to pack it all into a single tool. I've been there. I've tried to build that! And I've decided to go with the carpenter's toolbox analogy and be happy about that.
Timelines in Excel and Aeon timeline
Having a timeline for the overall story helps bring clarity.
I'm using Excel and Aeon Timeline. Actually, both since I feel they give different perspectives on the timeline and allow me to do different things.
If I had to choose, I'd go with Excel only.
Aeon is a bit of spice here. It has a sync function with Scrivener (more about Scrivener below) but when I tried it, it was buggy and now I've realized that's not really the way I want to write this story anyway.
Character lists in Excel
I have a number of different lists of characters in Excel documents.
My main list contains a matrix of all characters and all books, one cell per character-book combination, where I've noted the "size" of the character in the book and if they have POV.
Apart from that, my character lists usually contain a group of characters, maybe a crew on a ship, a rebel group, the advisors to a dictator.
This is a character index that, for most parts, has one or several equivalent documents in Scrivener, and my main benefit is that I get to have a group in a separate Excel document and the characters grouped on size and ordered in alphabetical order in Scrivener.
I guess I could also use metadata and collections in Scrivener to get the characters both by groups and sorted but that just hasn't happened yet.
Wold building and "Snowflakes" in Scrivener
I use Scrivener to record Worldbuilding information and character information (using the Snowflake method, one for each book).
However, it's just like a file system. You need to have a structure for your files and folders.
In my case, this is still a work in progress, and I'm guessing it depends on the story and the author how exactly this structure is designed.
I use Scrivener tags to mark what document belongs in what book, and then a search collection to get only those documents and folders.
Unfortunately, that information isn't that usable in Scrivener, but you can print it out as a pdf and use the pdf if you need to see a single book.
This information could also be well placed in a Wiki, but I have tried several solutions and opted not to use them for two main reasons:
- Backups — I'm paranoid enough to want to have a backup copy of my work (or in fact several) and I'm not keen on having my hard earned information in some cloud solution that happens to have a bad day
- Secrecy – I don't want my information floating around "downtown"... mostly because I don't want it to be out there when I have a readership... a nightmare would be to publish book 1 and then have an enthused fan tell everybody about book 12...
Scrivener's link functionality does, however, give an almost Wiki-like feeling anyway.
Update: I did look into Wikis and found DokuWiki. It apparently can be run from a thumb drive and has, among all *nixes, support for Windows and MacOS, negating all above issues, adding one of technical complexity instead. I haven't tested DokuWiki though.
yEd for relationships, families, etc
yEd is a great tool for visualizing relationships between things, events in a story, family trees, etc.
One of the greatest things, that I really recommend you learn if you are going to use yEd, is it's layout functions.
It has several pretty smart functions for taking a jumble of lines and nodes and lay them out in a way that makes sense.
The three I use most is hierarchical layouts, family tree layouts, and something akin to organic chemistry layout.
Bonus tip: Divide and Conquer
Quite early when I started planning my current project I realized that for all this overall structure and grand plotting, the readers will still want to go to the bookstore to purchase one of my books and have an I-read-a-book-experience from it.
Each book has to be self-contained primarily and rely on earlier books, or foreshadow later books, only if it doesn't break self-containment. (I.e. the reader doesn't get annoyed or confused by it...)
My project contains a number of books and I've spent a very long time doing "Snowflakes" for them, to step 5. Some to step 6.
I have a pretty large number of characters; a handful with POV and their important helpers, their antagonists, and the antagonists' important helpers. And then a bunch of rank and file characters below these characters in importance.
Of these, I've so far "snowflaked" all characters to step 3 and then continued with the POV characters and their main antagonists to step 5.
My original plan was to do snowflakes for all books, one step at a time, and then finally start pumping out first drafts.
However, I've come to realize that after a certain level you need to write a book all the way to the first draft to really get your hands dirty with the plots and characters.
I suggest you do the same thing. Plan the overall world, story and character arcs to a certain level and then allow it to be a chaotic mess and clean it up one book at a time.
Sure, I anticipate I'll shoot myself in the foot with some details that may force me to do changes, but for now, I haven't written any actual scenes, anything that I feel "has my heart in it". I've spent most of my time thinking and making decisions and documenting them on a very high level.
Conclusion
In essence, I suggest that you find several tools and combine them to get what you want, that you accept that you will still feel out of control, do plotting and worldbuilding to a certain level and then write one book at a time and let the rest be what it is.
