Should You Trust Predictions About the SCOTUS Ruling on Obamacare?

Answers to your questions about the news.
March 29 2012 4:54 PM

A 5-4 Vote Against? Or 6-3 in Favor?

How reliable are we at predicting the outcomes of cases before the Supreme Court?

Jeffrey Toobin.
CNN legal analyst Jeffrey Toobin called the oral arguments on the Affordable Care Act a "train wreck" for the Obama administration and predicted the Supreme Court will strike down the health care reform law.

Photograph by Amy Sussman/Getty Images.

The Supreme Court heard oral arguments this week on the constitutionality of the 2010 health care reform law. The consensus among the media is that things didn’t go well for President Obama, and some reporters are predicting that the individual mandate will be declared unconstitutional. After making their assessments, court observers usually add the caveat that you can’t reliably predict the outcome of a case based on oral argument. Is that old saw true?

No. For years, the conventional wisdom among legal scholars was that oral arguments had little or no impact on the outcome of a case, and that any predictions based on them would be unreliable. Accordingly, Supreme Court forecasting models used to be based only on contextual information, such as the subject matter being decided (tax law, federalism, civil rights, etc.), the relative wealth of the parties, and whether the lower court decision was conservative or liberal. The best of these complex models correctly predicted about 75 percent of cases and performed extremely poorly on nonideological cases involving tax or maritime law. (As a general rule, legal scholars are able to call around 60 percent of cases correctly, simply on the basis of their expertise and intuition.) Then, in 2004, New York Times Supreme Court reporter Linda Greenhouse noticed that she could at least match the top forecasting models simply by listening to the tenor of oral arguments. Greenhouse argued that the justices tip their hands during questioning, so models that ignored this phase of a case were incomplete.

Also in 2004, future Chief Justice John Roberts, then a Supreme Court attorney, noticed that justices ask more questions of the side that ultimately loses in 86 percent of cases. Though he’d tested his theory on only 28 cases—and no one knew whether its sensational success rate would hold up—there was now ample evidence that oral arguments do, in fact, offer very useful data for predicting the outcome of a Supreme Court case. Drawing on this insight, many political scientists now incorporate language analysis of the arguments into their models. One method is to simply tally up the words spoken to each attorney, consistent with the approach Chief Justice Roberts pioneered. The other method goes slightly deeper, analyzing the emotional quality of the language that the justices use: The more “unpleasant” words a lawyer hears compared with his opponent, the less likely he is to win. (Justice Scalia has a habit of telegraphing his vote by using words like “idiotic” during oral argument.) The most sophisticated models combine these two strategies with the more traditional forecasting factors—ideology, historical voting patterns, etc. These haven’t yet been deployed in enough cases to have a documented success rate, but research on past cases strongly suggest a major improvement in accuracy.

Predicting the outcome of the present health care reform case is difficult, because there are several different questions being debated and more than two attorneys appearing before the court. However, a state-of-the-art model created by professors Ryan Black of Michigan State, Sarah Treul of the University of North Carolina, Timothy Johnson of the University of Minnesota, and Jerry Goldman of Chicago-Kent College of Law suggests that the court will declare the individual mandate unconstitutional by a 5-4 vote. The big question mark, of course, is swing voter Justice Kennedy. He asked Solicitor General Donald Verrilli Jr. two more questions than he asked the challenger’s attorney, Paul Clement, with 14 percent more negative language, suggesting a slight preference for overturning the law. 

Got a question about today’s news? Ask the Explainer.

Explainer thanks Timothy Johnson of the University of Minnesota and Jeffrey Segal of Stony Brook University.

Video Explainer: Why Aren't There More Slam Dunks in Women's Basketball?

This video was produced from an original Explainer by Brian Palmer. Want more questions answered? You can now watch video Explainers at Slate's News Channel  on YouTube.

TODAY IN SLATE

Doublex

Crying Rape

False rape accusations exist, and they are a serious problem.

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

I Bought the Huge iPhone. I’m Already Thinking of Returning It.

The Music Industry Is Ignoring Some of the Best Black Women Singing R&B

How Will You Carry Around Your Huge New iPhone? Apple Pants!

Medical Examiner

The Most Terrifying Thing About Ebola 

The disease threatens humanity by preying on humanity.

Television

The Other Huxtable Effect

Thirty years ago, The Cosby Show gave us one of TV’s great feminists.

Lifetime Didn’t Find the Steubenville Rape Case Dramatic Enough. So They Added a Little Self-Immolation.

No, New York Times, Shonda Rhimes Is Not an “Angry Black Woman” 

Brow Beat
Sept. 19 2014 1:39 PM Shonda Rhimes Is Not an “Angry Black Woman,” New York Times. Neither Are Her Characters.
Behold
Sept. 19 2014 1:11 PM An Up-Close Look at the U.S.–Mexico Border
  News & Politics
Weigel
Sept. 19 2014 9:15 PM Chris Christie, Better Than Ever
  Business
Moneybox
Sept. 19 2014 6:35 PM Pabst Blue Ribbon is Being Sold to the Russians, Was So Over Anyway
  Life
Inside Higher Ed
Sept. 19 2014 1:34 PM Empty Seats, Fewer Donors? College football isn’t attracting the audience it used to.
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Slate Picks
Sept. 19 2014 12:00 PM What Happened at Slate This Week? The Slatest editor tells us to read well-informed skepticism, media criticism, and more.
  Arts
Brow Beat
Sept. 19 2014 4:48 PM You Should Be Listening to Sbtrkt
  Technology
Future Tense
Sept. 19 2014 6:31 PM The One Big Problem With the Enormous New iPhone
  Health & Science
Medical Examiner
Sept. 19 2014 5:09 PM Did America Get Fat by Drinking Diet Soda?   A high-profile study points the finger at artificial sweeteners.
  Sports
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.