Should You Trust Predictions About the SCOTUS Ruling on Obamacare?

Answers to your questions about the news.
March 29 2012 4:54 PM

A 5-4 Vote Against? Or 6-3 in Favor?

How reliable are we at predicting the outcomes of cases before the Supreme Court?

Jeffrey Toobin.
CNN legal analyst Jeffrey Toobin called the oral arguments on the Affordable Care Act a "train wreck" for the Obama administration and predicted the Supreme Court will strike down the health care reform law.

Photograph by Amy Sussman/Getty Images.

The Supreme Court heard oral arguments this week on the constitutionality of the 2010 health care reform law. The consensus among the media is that things didn’t go well for President Obama, and some reporters are predicting that the individual mandate will be declared unconstitutional. After making their assessments, court observers usually add the caveat that you can’t reliably predict the outcome of a case based on oral argument. Is that old saw true?

No. For years, the conventional wisdom among legal scholars was that oral arguments had little or no impact on the outcome of a case, and that any predictions based on them would be unreliable. Accordingly, Supreme Court forecasting models used to be based only on contextual information, such as the subject matter being decided (tax law, federalism, civil rights, etc.), the relative wealth of the parties, and whether the lower court decision was conservative or liberal. The best of these complex models correctly predicted about 75 percent of cases and performed extremely poorly on nonideological cases involving tax or maritime law. (As a general rule, legal scholars are able to call around 60 percent of cases correctly, simply on the basis of their expertise and intuition.) Then, in 2004, New York Times Supreme Court reporter Linda Greenhouse noticed that she could at least match the top forecasting models simply by listening to the tenor of oral arguments. Greenhouse argued that the justices tip their hands during questioning, so models that ignored this phase of a case were incomplete.

Also in 2004, future Chief Justice John Roberts, then a Supreme Court attorney, noticed that justices ask more questions of the side that ultimately loses in 86 percent of cases. Though he’d tested his theory on only 28 cases—and no one knew whether its sensational success rate would hold up—there was now ample evidence that oral arguments do, in fact, offer very useful data for predicting the outcome of a Supreme Court case. Drawing on this insight, many political scientists now incorporate language analysis of the arguments into their models. One method is to simply tally up the words spoken to each attorney, consistent with the approach Chief Justice Roberts pioneered. The other method goes slightly deeper, analyzing the emotional quality of the language that the justices use: The more “unpleasant” words a lawyer hears compared with his opponent, the less likely he is to win. (Justice Scalia has a habit of telegraphing his vote by using words like “idiotic” during oral argument.) The most sophisticated models combine these two strategies with the more traditional forecasting factors—ideology, historical voting patterns, etc. These haven’t yet been deployed in enough cases to have a documented success rate, but research on past cases strongly suggest a major improvement in accuracy.


Predicting the outcome of the present health care reform case is difficult, because there are several different questions being debated and more than two attorneys appearing before the court. However, a state-of-the-art model created by professors Ryan Black of Michigan State, Sarah Treul of the University of North Carolina, Timothy Johnson of the University of Minnesota, and Jerry Goldman of Chicago-Kent College of Law suggests that the court will declare the individual mandate unconstitutional by a 5-4 vote. The big question mark, of course, is swing voter Justice Kennedy. He asked Solicitor General Donald Verrilli Jr. two more questions than he asked the challenger’s attorney, Paul Clement, with 14 percent more negative language, suggesting a slight preference for overturning the law. 

Got a question about today’s news? Ask the Explainer.

Explainer thanks Timothy Johnson of the University of Minnesota and Jeffrey Segal of Stony Brook University.

Video Explainer: Why Aren't There More Slam Dunks in Women's Basketball?

This video was produced from an original Explainer by Brian Palmer. Want more questions answered? You can now watch video Explainers at Slate's News Channel  on YouTube.


Frame Game

Hard Knocks

I was hit by a teacher in an East Texas public school. It taught me nothing.

Chief Justice John Roberts Says $1,000 Can’t Buy Influence in Congress. Looks Like He’s Wrong.

After This Merger, One Company Could Control One-Third of the Planet's Beer Sales

Hidden Messages in Corporate Logos

If You’re Outraged by the NFL, Follow This Satirical Blowhard on Twitter

Sports Nut

Giving Up on Goodell

How the NFL lost the trust of its most loyal reporters.

How Can We Investigate Potential Dangers of Fracking Without Being Alarmist?

My Year as an Abortion Doula       

  News & Politics
Sept. 15 2014 8:56 PM The Benghazi Whistleblower Who Might Have Revealed a Massive Scandal on his Poetry Blog
Sept. 15 2014 7:27 PM Could IUDs Be the Next Great Weapon in the Battle Against Poverty?
Dear Prudence
Sept. 16 2014 6:00 AM Can of Worms Prudie offers advice to a letter writer who wants to blackmail a famous ex with tapes of his fetish.
  Double X
The XX Factor
Sept. 15 2014 3:31 PM My Year As an Abortion Doula
  Slate Plus
Tv Club
Sept. 15 2014 11:38 AM The Slate Doctor Who Podcast: Episode 4  A spoiler-filled discussion of "Listen."
Brow Beat
Sept. 15 2014 8:58 PM Lorde Does an Excellent Cover of Kanye West’s “Flashing Lights”
Future Tense
Sept. 16 2014 7:36 AM The Inspiration Drought Why our science fiction needs new dreams.
  Health & Science
Bad Astronomy
Sept. 16 2014 7:30 AM A Galaxy of Tatooines
Sports Nut
Sept. 15 2014 9:05 PM Giving Up on Goodell How the NFL lost the trust of its most loyal reporters.