In the Edison experiment oil pulling, experimental enthusiast Diane is testing the controversial idea that swishing oil in the mount for ~20 minutes/day lessens the pain of an tooth abscess. As I wrote in Insights, stories, and surprises from Edison, as the TTL leader I'm working to keep my opinions to myself, and focus on encouraging experimentation and moderate if necessary. I've not had to do the latter until passionate comments put the experiment at risk of being "hijacked" as one Edison user put it.
This brought up the question of whether having a control and, doing "real" science is necessary to get valid results, i.e., to have believable evidence that the technique works.
What I'd like to do here is strengthen my understanding of the scientific method by analyzing this experiment's design. Note that in applying Think, Try, Learn we are limited in how rigorously we apply the method. In general we adopt the spirit of science (hence the simplified "Think, Try, Learn" name) but in this case let's play with how to make the experiment as valid as possible given our limitations, such as having a sample of one. (Note: A forthcoming Edison feature is being able to set up group experiments where more than one person can participate in the same experiment, with the system calculating statistically-valid results. This will be extremely cool.)
In all cases, I'm told we are using an "interrupted time-series" design. From A Short Course on Evaluation Basics:
In situations where a control group is not possible, if (1) data on the treatment group can be obtained for several periods both before and after the participants are exposed to the program, and (2) if there is a sudden change in scores, and (3) if there is a continuation of the change, that is considered to be good evidence of the program's impact.
Jumping right in, our question is always:
Will oil pulling Decrease abscess sensitivity?
To get a good run and to give the swishing time to work, how about we test for four weeks? Two weeks doing the swishing followed by two not doing it.
Second, our dependent variable (What we observe) is abscess sensitivity. In researching oral sensitivity I found Measuring Tooth Sensitivity from The American Dental Hygienists' Association, where they recommend stimulating the area then using a Verbal Rating Scale (VRS) to measure results. The scale ranges from 0 (No discomfort/pain, but aware of stimulus) to 3 (Marked discomfort/pain that lasted more than 10 seconds). How we measure this is a good question given that Diane probably does not want to visit her dentist every week for an unpleasant professional measurement session! How about we decide the same area to stimulate each time, and then once a day (or week) press a rubber tipped gum stimulator like this to the area and measure the result using the VRS scale.
Finally, for our independent variable (What to change) will be whether the swishing is done or not. Note: A clever variation suggested is to make the liquid itself the independent variable, say by swishing two weeks with oil followed by two weeks with water. Our controlled variables (What I keep the same) would be the swishing activity itself, the amount and type of oil used, any oral care she's currently doing, etc.
This still doesn't account for the possibility that during the intervening two weeks there was something else going on in the mouth, which is always changing, but I'm shooting for a good start. What do you think?
A variation: Measure toxins absorbed by the oil
The Oil pulling discussion at The Skeptics Society Forum has plenty of other design and variable suggestions, including this one where instead of measuring pain, you would spit the used oil out onto grass and see if it dies from the supposed toxins that the oil absorbs and removes.