Announcing Edison 2, with Group Experiments and Quantitative Data!

I am absolutely thilled to let you know that Edison 2 is live. It adds two major features: Group experiments and quantitative data. A little about each follows below. I've also added a list view for experiments, which helps to get a sense of the different kinds of activity going on, especially what new group experiments have been created, and which ones you might want to join.

I threw together some documentation on the thinktrylearn.com wiki at EdisonHelp, with screencasts hopefully coming soon. As an experiment, I'm playing with using Edison itself to document how to use it, in the form of an Official Group Experiment. (Note: If you're interested in volunteering, I have plenty of fun ways you can help with Edison, including documentation and screencasts.)

For the development philosophy, I worked hard to adopt the edict of "As simple as possible, but no simpler"[1], part of the lean startup ideas I'm learning about. Here I kept the functionality and designs of the two features to a minimum, including converging on a super simple data model. Like any good experiment, the results will tell me how I did and where to go next.

A huge thanks to my brilliant back end programmer Andy O'Shea, and to and my talented designer and front end coder Zinj Guo. They are both excellent, and made this ambitious release go smoothly.

In the next week or two I'll be announcing a contest with prizes for various categories, such as experiment with the most participants, most data, or experiment with the most surprising results. Stay tuned, and Happy Experimenting!

Group Experiments

There are now basically two experiment types - Individual Experiments and Group Experiments. They are created by the "Create an Experiment" button that's at the top of the right sidebar on most pages, which now pops up a choice of which type you want to create. Individual experiments haven't changed in concept, but they now - like any experiment - have the option of defining measurements that are used for capturing data about your experiment.

Group experiments, on the other hand, are community experiments that multiple people participate in. The way they work is straightforward. You create a group experiment (describing its details and defining its measurements), which people can join by clicking the big "Join" button at the top right of the group experiment's page. Participants who join get an personal instance of the group experiment (called a "peer" experiment) that works like an individual one, but which is linked to the group. In this way you get to make observations and record data on your own experiment page, but you can also interact with the rest of the group by making comments on the group experiment's page. In other words, it's the community page for the group experiment itself.

You can read more in the docs. Again, comments are always welcome.

Quantitative Data

The other big feature is the ability to define what measurements you'd like to take in your experiment. The way you do this is to edit the experiment's details and use the Measurements widget at the bottom to specify each thing you want to track. The data model I decided on is simple:

From DesigningExperiments: Each measurement has four components:

• Name: A few words at most
• Type: Either Number or List
• Units: What units the measurement is in (optional)
• Description: What the measurement is, what it's used for, how and when the measurement is taken, etc. (optional)

This should cover 90% of all measurement tasks, with some work-arounds for the others. (Again, the results of the 2.0 release will tell me a lot.) Once you've defined your measurements you can enter data anywhere you could enter a text observation before. There's also a dedicated Data page for browsing, editing, and deleting.

The Big Bucket Personal Informatics Data Model

Seth Robert's post on Personal Science (especially about "data exhaust" [1]) got me thinking about big data and the implications for the self-tracking work we do. What evidence is there that big data will infiltrate self-experimenting? Under what conditions will self-tracking move from "small data", or "data poor" (a few hundred or a few thousand data points) to "big data" or "data rich" (terminology from The Coming Data Deluge)? Let me share some thoughts and get yours.

First, what does "big data" mean [2]? From Wikipedia:

Big data are datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing.

This identifies an important problem. While it is natural to throw all our personal data into one big database, there are costs associated with doing so. I don't mean those associated with capture (clearly we will solve the technical and cultural challenges), but the costs in sensemaking - turning data into actionable wisdom. Let's put the problem into context and assume the future for personal science looks something like this (help me here):

1. Many of our personal artifacts will be instrumented to know something about us (find many body-oriented ones in Walter's Health Internet of Things), but the sky's the limit. (For some great examples of work data we might capture, see Gary's comment on my post The Quantified Worker.) The idea is that these things will be smart enough to answer questions needed for our experiments, like "How much water did I drink?", "How active was I today?", or "Did I raise my voice this week?"
2. These artifacts will seamlessly transmit data to a central place that each individual owns and has complete control over. Also contributing data are medical professionals and any other person or organization that learns something about us. They will be contractually obligated to share it.
3. This data is augmented by self-tracking tricorders that we may wear, which capture other personal data channels like cognitive states and life events.
4. From all that data the citizen scientist will periodically reflect and analyze via triggers such as periodic reminders, natural events from experiments (e.g., when a question is answered or an experiment ends), or opportunistic situations such as encountering a problem or having a friend asking how we're doing.
5. Finally, the experimenter applies the results by integrating them into new mental models or behaviors, and continues this cycle of thinking up experiments, trying them out, and learning from them.

(Note that these steps are non-linear and are happening in parallel.)

Given this flow, I argue that the hard work is in the final two steps - sensemaking and behavior change. Leaving the latter for now (Ian Ayres on Carrots and Sticks addresses that well), how can we do these effectively when we are collecting a lifetime's worth of data? I don't know, but a few things come to mind including using advanced statistical tools, Visual analytics, and possibly the most important, collaboration. After all, successful researchers know that science works best when collaborating with others. In fact, given this possible future, our relationships with professions may move more in this direction.

What do you think?

References

• [1] My reply: Exhaust usually means waste that's a byproduct of production. However, in our case data is the means of self-improvement. It's like a catalyst for making a change in ourselves. Plus, unlike exhaust, it has value after its use. While factories may capture waste products for other uses, they don't treat the waste as intrinsically useful. That's a big difference.
• [2] Two additional resources you might find helpful are Wired's The End of Theory: The Data Deluge Makes the Scientific Method Obsolete and Nature's Special on Big Data.

"Constraints force creativity"

"Constraints force creativity. Run on limited resources and you'll be forced to reckon with constraints earlier and more intensely. And that's a good thing. Constraints drive innovation."

Getting Real: Fund Yourself

I've also found that constraints simplify. Rather than seeing one as a frustration, I've had luck turning around my perspective and treating it as a relief - one more thing that I don't have to consider. You?

How self-tracking and experimenting can generate wisdom

In How to Track Wisdom, Alex asks about how self-tracking might help us to get wiser, and gets some great responses. Here's my take. I'd love to hear your thoughts.

In Think, Try, Learn I claim you can track almost anything (hopefully debunking a self-experimenting myth), though coming up with specific metrics may require some creativity. My thought is that the relationship between self-experimenting and wisdom is direct, and that experimenting makes a good tool for becoming wiser.

Let's start with a definition:

1. The quality of having experience, knowledge, and good judgment
2. The soundness of an action or decision with regard to the application of such experience, knowledge, and good judgment
3. The body of knowledge and principles that develops within a specified society or period

This provides some clues. First, we gain wisdom through experience, which is exactly what experimenting is about. In that sense, gaining experience is built in because the whole idea is to try things (i.e., to make changes), measure what happens, and then learn from it. Put another way, experimenting plants the seeds for wisdom to sprout.

Second, we gain knowledge because we learn a lot when we are excited about a topic. In the self-experimenting case, our motivation stems from the subject being ourselves. My first step when I design an experiment is to research the issue to understand possible cause-effect relationships, and then choose a starting point of something to try changing. For example, when researching improving my sleep, I learned about the importance of having a regular routine and going to bed as soon as I first feel tired (see Install a new bedtime sleep routine in Edison). Because you're putting the knowledge directly to use in a personally relevant way, it sticks. Just ask someone about an experiment they tried, and you'll see their eyes light up when they share what they learned.

But what specifically should you track? A process that's helped me get insights into my (often irrational) behavior is to model my thinking, put it to the test, and adjust according to what conflicts (or not) I discover. A primary application is in making decisions. I think that every decision has an underlying model or theory that it's based on, which makes for a natural test:

1. Write down your decision when you make it, including a short analysis of what you think's going on in the situation, along with your predicted outcome,
2. Follow the consequences long enough for them to mature, and then
3. Compare the results to your theory when you started, dissect the differences, and learn from it.

I've found that over time this helps me improve my accuracy and understanding of how to world works. It also helps with anxiety, which makes decisions stressful and difficult. By tracking the decision I get some relief through objectivity. Sometimes it even changes my fear into curiosity, because I honestly start to wonder what will actually happen. For fun I'll occasionally assign probabilities to possible outcomes, which usually leads to surprises, such as a less likely one happening, or one happening that I didn't expect at all. This has also helped be more optimistic, since I usually assume the worst, but things don't always work out that way. Finally, this helps to make explicit any hidden biases and assumptions that are at play, such as my noticing how pessimistic I can be.

The tool I use for tracking this is my experimenter's journal (in my case a big text file), where I tag each decision made, then come back to it later and update them with my conclusions. I've been doing this for years, and you can read more in my old post, A key to continuous learning: Keep a decision log.

Another exercise that's been helpful in facilitating wisdom is tracking lessons I've learned. I use them two ways. First, capturing the lesson cements it in my mind and helps to take the sting out of unpleasant ones, and second, tagging it makes it easy to review ones I've made so I don't make them again! Read more at Some thoughts from tracking "lessons learned" for a year.

Finally, I think the experimental mindset can permanently change how we look at the world by switching our thinking from having answers to asking questions. I think every experiment should start with a question, which puts us into a learning mode that makes us open to changing our thinking and behavior. This differs from being stuck in an echo chamber where you get deeper into a mental rut by hearing the same answers repeatedly - a real risk of how we use the internet.

12 Myths about Self-Tracking

[Cross-posted to Quantified Self]

(Let me get a little provocative this time around and share some myths of self-tracking I've been playing with. I'd love to hear your thoughts about these and any other myths you might know about.)

Myth: You have to use technology.
Fact: A good guideline is to use a tool that's appropriate for the job. I know people who get good results using spreadsheets, and paper has some wonder affordances. (Read Malcolm Gladwell's The Social Life of Paper for a fascinating analysis of air-traffic controllers' paper-based system.) Then again, with large sets of data, visualization tools are invaluable.

Myth: Not everything can be measured.
Fact: I suggest that, with a little (or maybe a lot) of creativity, you can come up with something you can measure for any experiment. Check out Alex's post, How To Measure Anything, Even Intangibles. (Bonus: Do you have any that are giving you trouble? Let's play "stump the blogger!")

Myth: You have to be a scientist.
Fact: While it probably helps to have a background in science, and better yet one in statistics, we can still do valuable work with rudimentary skills, given you design a strong experiment that can teach you something.

Myth: You have to start with a goal or theory.
Fact: Sometimes we aren't to the point where we have a working theory, but we are ready to start poking and prodding to see what might emerge. However, I'd argue that, at a minimum, we should always start with a question. What you ask might change, but it'll get you moving. I'm still a believer that simply observing keenly can lead to awareness and eventually change.

Myth: Data-tracking is cold and dispassionate.
Fact: Far from it! Just think back to an experiment of yours where you had an insight or surprise - how exciting was that? Curiosity is an emotion, after all, which drives our love of exploration, adventure, and discovery. Plus, exploring the world with a curious mind is great fun. At the same time, it takes a level of detachment to see results for what they are, especially if they rub up against something that we are attached to.

Myth: Self-experimenting is just for problem-solving.
Fact: While addressing a specific concern is an excellent application of the self-quantification, I think of it more as a life-encompassing mindset. That is, a perspective on how to go about our lives, and one that asserts that our job of learning is never done.

Myth: Self-experimenting is easy.
Fact: I'm sure you've noticed that it takes discipline to consistently track things about your life, and to do the thinking and learning that results from your data. Key is a strong desire to learn something and, ideally, to get an answer.

Myth: Self-experimenting is hard.
Fact: The other side of the coin is that our work comes down to something simple: Thinking of a change to make, trying it while making observations, and then learning via reflection. Start out with something small that's easy to measure, and then work up from there.

Myth: Citizens can't do science.
Fact: This one is about the broader view of the validity of citizens doing science, and was what I was getting at in my post Making citizen scientists - that this work applies especially well to the individual. (For some reasons, see the excellent comments at the bottom of that post.) Related to this are two other myths, the sample size of one is not valid and the results need to be clinical-grade ones.

Myth: It's just for external things.
Fact: Some of the richest territory to mine via experiments is your mind, behaviors, and mental models. Treating your thinking and behavior themselves as data is valuable, and can lead to fresh insights that you can use for improvement.

Myth: You have to use numbers.
Fact: Scandalous, I know, but I've found that in some cases I can test out ideas without having to measure anything. However, I always make entries in my experimenter's journal, which keeps me engaged and gives me something for later analysis.

Myth: It only applies to health.
Fact: Related to my point above about the experimental mindset, I've had a lot of benefit from applying the thinking to all parts of my life. While health is the number one category in Edison, there are lots of other creative applications, including the social and work realms.