How do I organize a massive collection of personal files and writing?

Question

Note: This is a vague question, so I welcome help in improving it.

I have a large amount of writing on my computer, some thousands of RTF files that sometimes start as taking notes in class lectures, sometimes start as unsent letters to friends, sometimes start as pointed reflections on a topic I just realized I have something important to say about it.

But each one of these files might have other random bits of information, statements, ideas, expositions that are not related at all to the main topic of the file.

How can I get started on doing something with all these files? Could be things like "writing exercises" or "file copying and pasting tasks" or even some automation (scripts to process files into some other form), or maybe some open-source application (perhaps console-based or web-based) that is built to aid in this kind of work.

Answer 1

Let me roll up the sleeves and plug OneNote again. I swear, I don't get a commission.

The Answer:
You're looking for some app that'll help you organize bits of info spread across, and mixed up in, thousands of files -- OneNote does that. Of course, it can't know what any given line of text in any given file means to you; it doesn't help you parse those files. What it does give you is a destination for all of those bits of info as you go through and sort everything out (meaning copy/pasting out of the files and into the appropriate spot in a OneNote notebook).

Think of it as an electronic notebook. You can make any given number of notebooks (displayed down the left side; unless you explicitly close one it stays open and is there every time you launch the program), each with any given number of sections (tabbed across the top), each which contain any given number of pages (vertical tabbed down the right side). You can put whatever you want in each page.

So, you might have one notebook called "Letters", where you put all those old unsent letters. Each friend's name could be a section, and each unsent letter to them, or even just ideas of what you want to write them, could go in the section pages.

For class notes, you could have a notebook per class. Each section could be a topic covered, or you could do it chronologically and make each section a week. Whatever works best for you. For those classes where you have only a few notes, you could make an "Other Classes" notebook, and make each section one of those classes.

You can have notebooks for your Random Ideas, Statements (do you mean quotes by that?), and so on.

That's all there is to it. The end result is, once you've gone through those notes once, you can discard those thousands of files and have one spot to go to where all of these things are easy to find, easy to see (not buried where you forget it), and easy to add to in an organized way.

OneNote is a really simple idea, but a tremendously useful one.

List of some more features, that may/may not be useful for this question:
Where it really shines is in all the details. for example:

• Put your cursor anywhere and start typing. That's where your text appears.
• If you copy/paste anything into it, it'll put a little subscript under what you pasted with a link back to the source. If you copied from a web page, it'll link to that page. If you copy pasted from a file on you computer, it'll link with the path to that file
• If you drag and drop a document into it, it gives you three options: a) take that document's text and paste it there, b) make an icon there linked to that document, or c) make an icon but imbed the document into the OneNote behind the scenes so it's always available in that OneNote
• Screen clipping - push a button, OneNote disappears, and you can click-drag to capture whatever is showing on whatever part of your screen. Let up, and OneNote reappears, with that screen clipping inserted wherever your cursor was.
• voice & video recording. Push a button, click record, and start talking. When you click stop it drops that recording in mp3 wherever your cursor was.
• Search, including searching for text within images -- like on a picture of a sign
• various flags, including to-do checkoff flags. I've heard that that can play nice with your outlook tasks too
• Collaboration - You can share a OneNote notebook on a network share. Others can access and use it too, and OneNote keeps it all synchronized between everyone using it. It does a pretty good job of it too. While in this case the main notebook lives on that network share, it'll also keep a local copy, so you can work on it offline. It'll sync as soon as it can.
• Skydrive - same as above, but instead of on a network share, it's in MS's cloud service Skydrive. You get that for free with a live (hotmail) account. Then it's available and sharable anywhere in the world, plus you can open it in a browser in a web version of OneNote (incidentally, you don't have to own OneNote to do that)

And on and on. It just gets out of your way and works the way your mind thinks it will. It's perfect to manage all those notes you've got.

Answer 2

Well, that depends if and what you want to do with it. Having a lot of "great" ideas is easy, until you try to realize them (there is often a discrepancy between what one imagines it to be and what it turns out). And unless you specify what you want to do with it, there is hardly any movement possible.

However, if you want to find out what you have available, I would start by collecting your ideas (and not merely having them somewhere without any structure). There are a lot of ways to do this, from files and folders, to mind/concept maps, outliners (e.g., NeO, OmniOutliner), digital notebooks (Circus Ponies Notebook, OneNote), notes management systems (e.g., DEVONthink), visual notes management systems (Tinderbox, Curio), Wikis (DokuWiki, MediaWiki), etc.

In any case I would recommend using tags to first get an idea in which areas you have ideas, and as you mention that files can have multiple ideas, tags are the way to go (you can tag the quality of ideas, the area like "gift", "character", "plot", "object", etc.). Personally, I love DEVONthink for this, because it augments the File/Folder system, gives you tags and immediately identifies duplicates based on the file content (see this blog entry on my blog). It's a Mac program, but there should be similar solutions for Windows.

After tagging and getting an impression which projects are feasible (creativity is more than one idea and a good collection helps you to generate and collect the necessary ideas) you can decide for a project to realize -- and make something out of your ideas.

For more information I'd point to a book I've written recently -- it's available as PDF 'for free' (Donationware, once I finish with the ePub version). It essentially covers the whole process.

Answer 3

I agree with Daniel Wessel on his recommendations. I would seriously consider DevonThink for your needs. Many people have written about it's ability to find relationships between documents that help in organization. It also has an auto-classify function if you really need something like that.

There are many, many articles describing DevonThink workflows. I would start here with Steven Johnson's often referenced article. And then also search around here. Note: this isn't a paid endorsement. I actually don't like the DevonThink UI that much at all. But unfortunately there really isn't a comparable alternative to its "AI" feature.