FYI, my current file (see description next) has ~14,000 lines (~0.5MB), and my previous non-wiki file had ~55,000 lines (~1.5MB).
I've been using a single file for my professional ProgrammersNotebook since at least 1997. Initially it was a MS Word file (back in my Windows days), but when I moved to Linux I switched over to a simple ASCII file, which I edit in Emacs. The reason I used word was for its outliner - I organized the file by making each day a level 1 entry, and I listed them in reverse chronological order so that I could start at the top when adding the latest entry. The outliner let me structure the file a bit by breaking multi-line activities into separate entities. (Hey - sounds like the GTD principle of making something a project if it requires more than one next action. I'm in trouble - everything has GTD overtones these days...)
I organized the first ASCII file using Emacs' Outline Mode, but organized just like the Word file - reverse chronological, with structure via nesting. Incremental search allowed me to find (but sometimes painfully) items I needed, such as shell script notes, code snippets, and what I had been spending my time on. The problem was that it didn't allow linking and tagging, one of the primary ways I use in structuring information.
So on 2004-07-19 I moved to a second format (still ASCII edited via Emacs), which I described in my post on Photo Blogs, Wikis, and Memories for Life. Briefly, the file has simple entries separated by '----' and a time-stamp at the end. For example:
talked w/PersonOne re: Google-style undergraduate programming
contest. not clear what the topic should be. also talked about future
fun projects. one possibility: ProxIncrVisualization
continued moving information over to planner. ugh- the undated pages
are a pain! wrote PersonTwo re: help, or maybe ordering a dated
The big improvement is linking and tagging via WikiCase (AKA CamelCase or WikiWords). This helps me navigate and find needed information. Of course it opens up another issue, that of consistent tagging. But we'll save that for later. The only other formatting I use in the file is a) I define an entry by placing a WikiWord on the first line by itself, and b) I have some shortcuts for words. The shortcuts are special two- or three-letter words that end with a colon (':') and start a line. My current ones include IN (inbox), MUS (Might Be Useful), IDEA, COOL, and OFF (vacation leave). Finally, URLs are treated specially - I don't mark them up, I just paste them verbatim.
Together these merge (in a very low cost way) some of the good ideas from Wikis, Blogs, and PIM tools, with the simplicity of a text file. (There's a nice discussion of them here.)
Well, not much really. All I have are keystrokes that create a new time-stamped entry and grab a URL's title. In addition I use the usual Emacs features like 'occur', interactive highlighting, and especially hippie-expand. I'd like to do more, but I just haven't had time.
Isn't this just a cheap RDBMS?
At first glance, yes, it's a just a text-based list of free-form records, which could be stored in a Relational Database System. (Actually, I helped build a new kind of database (Proximity) that directly supports representing semi-structured information like this, but that's another story.) My main reasons for not using a database are:
- Easy to set up.
- Customizable editors already available (easy to view, merge, format, search, edit, etc.)
- Easy to backup.
- Easy to write simple external tools to analyze, view, etc.
- Supports schema changes.
Analysis and future
All I'll say here is that I use the file in a few basic ways. (See The Design and Long-Term Use of a Personal Electronic Notebook: A Reflective Analysis - AKA A Personal Electronic Notebook, by Thomas Erickson - for a great analysis of a personal journal tool the author built then used.) Mostly I use it to capture ideas, notes, URLs, and work activity like tasks, coding, and email. I've made myself enter every single URL that I come across that I think might even remotely be useful, because many times I've had to spend a LONG TIME trying to find something I've seen before. (Related: Stuff I've Seen and Keeping Found Things Found.)
To do nicer navigation and browsing I wrote a simple Java program (I used Jetty) to load the file's entries into RAM, show them chronologically, allow search, and turn WikiWords and URLs into links. I've used it a bit, but haven't been motivated to do more.
I think there's a great idea for a Journal Construction Kit that supports the emergence of customized specialization (see Jot for a commercial effort in this area). Here's a question: Is a general tool to support this kind of activity possible? Maybe it would be similar to Jetbrain's Meta Programming System, but for information. Related: Chandler, and these two articles by Martin Fowler.
I'd love to hear from others who have created customized journal tools that support these features. I'm not excited by Emacs programming, I'm just trying to get work done. Any thoughts would be appreciated!