Why We Need a Control Group
A common casual discussion topic among my fellow educators is the question of what the essential focus should be for a true universal education system. Having seen an increasing sense of innumeracy and relative ignorance of recent history among my own students, my default position is always that everyone –young and old– could benefit from remediation in both mathematics and general history.
But my good friend, statistician Dr Nicholas Barrowman, once offered that what everyone really needs to understand is a deceptively simple thought: that scientific investigations need a control group. If everyone could wrap their brains around why a control group is needed, then society as a whole would be more rational, less divisive, and better able to navigate life’s boundless waters of chaos and uncertainty.
My first reaction to his position was friendly dismissal. Clearly, Nick was stuck in his own world, his particular professional paradigm which involves the design and critique of medical studies. But years later, I begin to see the true weight and value of his position, especially in the current era of fake news and anti-science.
The importance of a control group to science is, I hope, logically obvious to everyone reading this article. Without a control group, one has nothing to which to compare empirical observations. Absent such relativity, one can reach many unsupported conclusions, and therein construct any number of personally appealing, though empirically unsupported realities. In short: the absence of a control group allows for the ascension of ideology over facts.
Decades ago, I was at a dinner party at which a vocal young woman, passionately opposed to the proliferation of pornography in our society, declared to the group: “Do you know that 100% of convicted sex offenders have looked at porn?” She clearly hoped to demonstrate an inescapable causal link between pornography and sex crime.
As I had not yet learned the virtues of silence, I foolishly responded: “100% of convicted sex offenders have presumably also eaten bread. What’s your point?”
The rare gluten-intolerant sex criminal aside, I could have also offered that 100% of men who are not sex offenders also watch porn. But my point was made. Absent the control group, this young woman’s personal political values had driven her toward one –and only one– interpretation of a given observation, even though her conclusion was not necessarily supported by that observation.
Recently, Amnesty International published its report on women’s experiences of abuse on Twitter, titled, Toxic Twitter: A Toxic Place for Women. The report concluded that “the violence and abuse many women experience on Twitter has a detrimental effect on their right to express themselves equally, freely, and without fear.” The word “equally” suggests that this is in comparison to the experiences of men. But, as Suzie Mulesky pointed out, Amnesty did not assess the Twitter experiences of men, the natural control group. This omission calls into question whether the phrasing of their conclusion is supported by the very data that they presented.
In 2014, a University of Toronto study showed an improvement in ADHD symptoms among subjects receiving homeopathic treatments. Homeopathy is largely considered to be quackery for a host of very good reasons. So for a respected institution to put its name behind a clinical trial showing some benefits to such a controversial approach is, shall we say, newsworthy.
Physician-scientists joyfully criticized the study, and an open letter signed by leading scholars, including two Nobel laureates, chastised the university for having backed it. The Faculty of Pharmacy’s link to the study is now dead. But one of the methodological factors that should have immediately cast some doubt onto its merits was –you guessed it– its lack of a control group.
The study is what we call an “uncontrolled trial”, in which a series of subjects are observed over time to see if an intervention (in this case an homeopathic treatment) caused any change. Uncontrolled trials have a purpose in science, including, for example, the measurement of survival rates after a surgical intervention. But it’s very dangerous to make strong conclusions about therapeutic effectiveness from such trials.
A control group is not always necessary in an experiment, if proper care is taken to limit one’s conclusions. But it behooves us all to understand why a control group is indeed important, and how interpretations driven by ideology can flourish in its absence.
