Scientists Cracked Themselves Up With a Study That Found Parachutes Are No More Effective Than Empty Backpacks
Always read the fine print
/https://tf-cmsv2-smithsonianmag-media.s3.amazonaws.com/filer/01/c0/01c09301-3682-464d-8750-86a3244f0a59/5975442961_0e7772b843_o.jpg)
It is a truth universally acknowledged that if you’re planning to jump out of a plane, it’s a good idea to have a working parachute at your disposal. But a recent study calls this assumption into question, showing that parachutes are no better at protecting people from harm than empty backpacks. A minor detail worth mentioning: all of the participants involved in the research jumped from aircraft that were firmly rooted on the ground.
Published in the Christmas edition of the journal BMJ, the study is a gleeful satire, though you might not know it if you only took a cursory glance at the report. It has all the veneer of a serious investigation: scientific jargon, citations, meticulous dissection of the researchers’ methodology. The joke experiment began with the scientists trying to recruit unsuspecting passengers seated next to them on flights that were taken for work or personal reasons.
“We’d strike up a conversation and say, ‘Would you be willing to be randomized in a study where you had a 50 percent chance of jumping out of this airplane with—versus without—a parachute?’” cardiologist Robert Yeh, an associate professor at Harvard Medical School and attending physician at Beth Israel Deaconess Medical Center, tells NPR’s Richard Harris.
Unsurprisingly, they didn’t get many takers. So the researchers expanded their experiment to include friends, relatives and members of the investigative team, asking them if they “would be willing to be randomized to jump from the aircraft at its current altitude and velocity,” which in this case meant aircraft that weren’t moving, the study authors explain. In total, 23 people agreed to take the very short plunge—either from a biplane at a site in Martha’s Vineyard, Massachusetts, or from a helicopter at a site in Michigan.
The participants were randomly assigned either a parachute or a backpack with nothing in it—a type of experiment known as a “randomized control trial” because it includes a group that receives the “treatment” under investigation, and a control group that does not. None of the parachutes actually opened because the duration of the fall was so short, but no matter: all of the participants emerged equally unscathed.
The study also followed up with the participants 30 days after the jump and found, to no one’s surprise, “no significant difference in the rate of death or major traumatic injury between the treatment and control arms,” leading them to conclude that although “decades of anecdotal experience have suggested that parachute use during jumps from aircraft can save lives, these observations are vulnerable to selection bias and confounding.”
Yes, the study was very silly, but there’s a serious point to it. For one thing, the paper is a reminder of why it is important to read studies carefully before drawing conclusions about their findings.
“It's a little bit of a parable, to say we have to look at the fine print, we have to understand the context in which research is designed and conducted to really properly interpret the results,” Yeh tells Harris.
Selectively including and excluding participants from a study can also lead to absurd results; the new research was effectively useless because it did not involve participants who had any use for a parachute. And, as Saurabh Jha notes in the Health Care Blog, the new study is a humorous demonstration of when randomized control trials are useful—and when they are not.
“The point of the parachute satire is that the obvious doesn’t need empirical evidence,” Jha writes. “It is a joke on non-judgmentalism, or egalitarianism of judgment, on the objectively sincere but willfully naïve null hypothesis where all things remain equally possible until we have data.”
The study authors are not, of course, refuting the importance of randomized control trials, even going so far as to write: “[W]e believe that such trials remain the gold standard for the evaluation of most new treatments.” That being said, they explain, the trial shows that “accurate interpretation requires more than a cursory reading of the abstract.”
It should also be noted that while none of the participants sustained any injuries during the experiment, the same cannot be said of the researchers. As they explain in the footnotes of the study, “All authors suffered substantial abdominal discomfort from laughter.”