There was big news in the field of kidney stone research last week, as anyone who scans the internet would know. “Research Finds Thrilling Cure for Kidney Stones: Roller Coasters,” declared NBC News. “How a Roller Coaster Can Help You Pass a Kidney Stone,” explained Gizmodo. “Little Kidney Stone? Ride a Roller Coaster, Says Study,” reported CNN. “Passing a Kidney Stone Can Be a Real Roller Coaster Ride,” wrote Slate’s own Phil Plait. At last check, there were 149,000 results in Google News.
This was indisputably fun science. Researchers made a 3-D–printed model of a kidney, placed real kidney stones inside, and took the contraption on a roller coaster 20 times to see whether the jostling of the ride helped pass the stones. The conceit is immediately understandable; it’s basically a blue-ribbon science fair project. A hypothesis is offered by professional scientists, evidence is marshaled to support said hypothesis, and the whole thing is written up in a peer-reviewed journal. The solution seems low-cost and fun. More than fun, this is relatable science: Roger Bacon rides the Cyclone.
That it went viral was entirely inevitable. It was also unfortunate.
After watching the news reports on this study balloon, I couldn’t shake the thought that the only good response to a study like this is not to report on it at all. (And yet here we are.) That’s because, beneath all its quirkiness and all that delightful banter about urine, it is not a paper that is ready for prime time. This is a bit of niche research, an exceedingly preliminary finding that is simply one research team saying: Hey, this might be a thing. Could we please look at these nebulous conclusions some more to see if they are real? But inevitably, in the passage from specialized research investigation into wacky-science viral glurge, that cautious incrementalism had become a “thrilling cure.”
This happens even in spite of carefully constructed sentences to couch the conclusions in the tentative language of what “might” be true and what “could” work. But when that kind of language is warranted for even full-blown clinical trials, all the mights and coulds in the world can’t effectively indicate to readers that this research is the babiest of baby steps.
The fact is it’s downright impossible to use the results in this study to reach any kind of conclusion about whether roller coasters in general help people in general pass kidney stones.
The author of the paper was David Wartinger, an osteopathic urologist and professor emeritus at Michigan State’s College of Osteopathic Medicine. He seems fully aware that there is a lot of implicit delight to be found in the tale of a scientist who repeatedly fills a fake kidney up with pee and takes it on a roller coaster in an attempt to help patients. The study even offers the winsome detail that researchers wanted to try the experiment with pig or cow kidneys but decided against it “owing to ambient temperature and the inappropriate display of such material in a family-friendly amusement park.”
I mean, this is a scientist with a sense of humor. We love a scientist with a sense of humor. He’s also a doctor who listens to his patients. We love a doctor who listens to his patients. Here’s how the study came to be: The researchers had heard a number of anecdotes about people who thought that going on a roller coaster had helped them pass a stone. One guy in particular told them that three rides on a specific coaster, Walt Disney World’s Big Thunder Mountain Railroad, had resulted in three passages of three stones. So they scanned the shape and size of his kidney, 3-D–printed a silicone replica, and took it to Disney World to ride that roller coaster. The model was filled with the kidney stones the patient had actually passed. When the researchers sat in the front of the roller coaster; the stones passed four out of 24 times (a pass rate of 17 percent). When they sat in the back of the coaster, they passed 23 out of 36 times (pass rate of 64 percent).
If anything, this was worse than what the guy had experienced himself in real life (remember, he went on the ride three times and passed a stone each time—100 percent pass rate). This may indicate problems with the model—it was 3-D printed out of silicone, not kidney, for example.
For a control, they compared their roller coaster rides with their trips on “the most benign theme park ride we could find” at Disney World, the Railroad. (No stones were passed on the this “ride.”) That’s fair if you’re asking whether a roller coaster is more effective than other amusement park rides at helping you pass a kidney stone.
But what we really need to know is whether riding a roller coaster helps you pass a kidney stone more reliably than just, say, existing in the world. To that end, the study includes one stat: “A renal calculus greater than 6 mm in diameter has about a 1 percent chance of spontaneous passage without intervention.”
That sounds pretty small! The stones they tested passed way more than 1 percent of the time! Of course, the stones they tested were also much smaller than 6 millimeters in diameter (the study gives their size only in cubic millimeters—annoying, certainly; willfully misleading, perhaps—but the biggest stone they tested had a diameter of 4.5 millimeters). And the research from which they pulled their 1 percent chance stat has plenty of other, more helpful numbers that they failed to include: Of patients studied with kidney stones, 83 percent of them spontaneously passed their stones without any medical intervention. For patients who had stones smaller than 2 millimeters, the average time it took for the stones to spontaneously pass was just 8.2 days. Furthermore, “for 95% of stones to pass it took 31 days for those 2 mm, or less, 40 days for those 2 to 4 mm, and 39 days for those 4 to 6 mm.” That study concluded that half of stones bigger than 5 millimeters might require an intervention. (Five millimeters is sort of the turning point for when stones become a problem—which is actually exactly why we want to find reliable ways to help people reliably pass smaller stones.)
All in all, this research shows how one man’s stones in one man’s “kidney” (made out of silicone) reacted to a ride on one roller coaster. But there are a lot of other factors involved in why someone does or doesn’t pass kidney stones. “Size alone doesn’t explain it,” said Ivan Porter II, a nephrologist at the Mayo Clinic. Some people will pass stones naturally while others never will. Movement alone doesn’t explain it, either. “We already have anecdotal evidence that there are certain things that move people around that help,” said Porter. “But in some people it has zero effect.” It might have something to do with the various shapes of people’s kidneys, or the shapes of the stones they develop, or something about their urine (and in this model, they mixed one man’s kidney shape with another man’s urine—the researcher donated to this cause). We already knew the subject of the model was predisposed to passing these exact kidney stones on this exact roller coaster. The experiment was just a way of showing that there was perhaps something to this idea, and getting it on the books in the peer-reviewed literature.
Wartinger explained to me that he actually conducted the study in 2008; he just didn’t get around to publishing it until now. “I put the study in my desk drawer for quite a while,” he said. “It was basically shame that made me write it up.”
It was probably more than shame alone that propelled him, though: If he were going to go on to study this effect in actual humans, he’d need to show off the preliminary reasons for his hunch. “This is the validation model for getting the attention to do the human trial,” he said.
Wartinger is pretty responsible about all of this. When we spoke, he admitted it might sound as if he was overreaching, but he added that, since 2008, he’s taken his fake kidney on roller coasters another couple hundred times and the effect has still stood up. He knew the research wasn’t strong enough to start recommending coaster rides in his practice, though he does recommend them to his friends. “What I would tell my friends is a different standard than what I can recommend for patients,” he said.
If the method does work, that’s very cool, because it shows that the conversations doctors have with their patients are rewarding, as Plait wrote. Plus, it could be a low-cost fix to an expensive problem, one that shows that there are certain ways that humans can retain control over their own health.
But what if it doesn’t?
The reason the roller coaster study has gotten all this attention isn’t because this is clear science backing up a novel new treatment. “To try to think that you could apply this to the general population would be an absolute disaster,” said Porter. The reason it’s gotten so much attention is that people like roller coasters and oddball eureka moments.
And in fact, there’s nothing really wrong with this study. It’s a little thin, and the 6 mm/1 percent stat is on the verge of being deceptive, but none of the problems is significant enough to warrant a debunking. There’s nothing really to debunk. There’s also nothing really to affirm. It took me the better part of 1,500 words to explain exactly what this experiment shows. This is a classic case of “Huh! That might be a thing. Let’s test it a lot more.” Which is exactly what Wartinger was trying to do when he published the study in the first place. He and his researchers had found a weird thing that they wanted to study more rigorously so that eventually they might know how to get people of all types to be able to pass kidney stones of small to medium sizes, before they become a problem that requires medical intervention. Noble goal!
So where did NBC, or anyone else, get the idea that Wartinger had found a “Thrilling Cure”? If you’re looking for culprits, you could begin with the press release blasted out by Michigan State. “GOT KIDNEY STONES?” read the headline, “RIDE A ROLLER COASTER.” Wartinger said he expected the attention, but he sounded a little ashamed of it. “I’ve tried not to look at the online articles,” he said. “Some of them are clearly just meant to fill space and catch people’s attention.
“I wish it was a little bit less quirky and took it a little more seriously,” he added. “It’s difficult to get people to read more than a few paragraphs or get past the byline.”
Much has been said about the bad incentives that prevail in both the scientific literature and the science media. Everyone is encouraged to do some degree of violence to the very science they’re tasked with illuminating. Scientists structure their research in order to get the attention they need to publish papers, get tenure, and continue to do their work. University flacks hype the scientists’ findings in order to win them that attention. Journalists have to simplify and popularize even further in order for people to pay any attention at all. Of course you’re more likely to read “Passing a Kidney Stone Can Be a Real Roller Coaster Ride” than “Validation of a Functional Pyelocalyceal Renal Model for the Evaluation of Renal Calculi Passage While Riding a Roller Coaster.” A more conservative “Scientists Find Some Evidence to Support the Vague Notion That Roller Coasters May Possibly, in Some As Yet Ineffable Way, Help Certain People Pass Certain-Size Kidney Stones” would certainly fall somewhere in the middle. But the internet does not reward cautious explanation of how science plods along in the murky pursuit of truth—it feasts on share-worthy exclamations that presume anything published in the literature is fact.
In reality, science is a very slow process. Researchers need to investigate their ideas incrementally. They need to show their work and make the case for funding so that the next study might be a little more rigorous. As science journalists, we need to be more skeptical of single studies and less quick to tell ourselves that adding a might exonerates us of any charges of overhyping. What we chose to cover matters as much as how we cover it. We should take a cue from the kidneys and let the small stuff pass.
