Should You Trust Predictions About the SCOTUS Ruling on Obamacare?

Answers to your questions about the news.
March 29 2012 4:54 PM

A 5-4 Vote Against? Or 6-3 in Favor?

How reliable are we at predicting the outcomes of cases before the Supreme Court?

Jeffrey Toobin.
CNN legal analyst Jeffrey Toobin called the oral arguments on the Affordable Care Act a "train wreck" for the Obama administration and predicted the Supreme Court will strike down the health care reform law.

Photograph by Amy Sussman/Getty Images.

The Supreme Court heard oral arguments this week on the constitutionality of the 2010 health care reform law. The consensus among the media is that things didn’t go well for President Obama, and some reporters are predicting that the individual mandate will be declared unconstitutional. After making their assessments, court observers usually add the caveat that you can’t reliably predict the outcome of a case based on oral argument. Is that old saw true?

No. For years, the conventional wisdom among legal scholars was that oral arguments had little or no impact on the outcome of a case, and that any predictions based on them would be unreliable. Accordingly, Supreme Court forecasting models used to be based only on contextual information, such as the subject matter being decided (tax law, federalism, civil rights, etc.), the relative wealth of the parties, and whether the lower court decision was conservative or liberal. The best of these complex models correctly predicted about 75 percent of cases and performed extremely poorly on nonideological cases involving tax or maritime law. (As a general rule, legal scholars are able to call around 60 percent of cases correctly, simply on the basis of their expertise and intuition.) Then, in 2004, New York Times Supreme Court reporter Linda Greenhouse noticed that she could at least match the top forecasting models simply by listening to the tenor of oral arguments. Greenhouse argued that the justices tip their hands during questioning, so models that ignored this phase of a case were incomplete.

Also in 2004, future Chief Justice John Roberts, then a Supreme Court attorney, noticed that justices ask more questions of the side that ultimately loses in 86 percent of cases. Though he’d tested his theory on only 28 cases—and no one knew whether its sensational success rate would hold up—there was now ample evidence that oral arguments do, in fact, offer very useful data for predicting the outcome of a Supreme Court case. Drawing on this insight, many political scientists now incorporate language analysis of the arguments into their models. One method is to simply tally up the words spoken to each attorney, consistent with the approach Chief Justice Roberts pioneered. The other method goes slightly deeper, analyzing the emotional quality of the language that the justices use: The more “unpleasant” words a lawyer hears compared with his opponent, the less likely he is to win. (Justice Scalia has a habit of telegraphing his vote by using words like “idiotic” during oral argument.) The most sophisticated models combine these two strategies with the more traditional forecasting factors—ideology, historical voting patterns, etc. These haven’t yet been deployed in enough cases to have a documented success rate, but research on past cases strongly suggest a major improvement in accuracy.

Predicting the outcome of the present health care reform case is difficult, because there are several different questions being debated and more than two attorneys appearing before the court. However, a state-of-the-art model created by professors Ryan Black of Michigan State, Sarah Treul of the University of North Carolina, Timothy Johnson of the University of Minnesota, and Jerry Goldman of Chicago-Kent College of Law suggests that the court will declare the individual mandate unconstitutional by a 5-4 vote. The big question mark, of course, is swing voter Justice Kennedy. He asked Solicitor General Donald Verrilli Jr. two more questions than he asked the challenger’s attorney, Paul Clement, with 14 percent more negative language, suggesting a slight preference for overturning the law. 

Got a question about today’s news? Ask the Explainer.

Explainer thanks Timothy Johnson of the University of Minnesota and Jeffrey Segal of Stony Brook University.

Video Explainer: Why Aren't There More Slam Dunks in Women's Basketball?

This video was produced from an original Explainer by Brian Palmer. Want more questions answered? You can now watch video Explainers at Slate's News Channel  on YouTube.

TODAY IN SLATE

History

Slate Plus Early Read: The Self-Made Man

The story of America’s most pliable, pernicious, irrepressible myth.

Rehtaeh Parsons Was the Most Famous Victim in Canada. Now, Journalists Can’t Even Say Her Name.

Mitt Romney May Be Weighing a 2016 Run. That Would Be a Big Mistake.

Amazing Photos From Hong Kong’s Umbrella Revolution

Transparent Is the Fall’s Only Great New Show

The XX Factor

Rehtaeh Parsons Was the Most Famous Victim in Canada

Now, journalists can't even say her name.

Doublex

Lena Dunham, the Book

More shtick than honesty in Not That Kind of Girl.

What a Juicy New Book About Diane Sawyer and Katie Couric Fails to Tell Us About the TV News Business

Does Your Child Have Sluggish Cognitive Tempo? Or Is That Just a Disorder Made Up to Scare You?

  News & Politics
Foreigners
Sept. 29 2014 10:00 PM “Everything Must Change in Italy” An interview with Italian Prime Minster Matteo Renzi.
  Business
Moneybox
Sept. 29 2014 7:01 PM We May Never Know If Larry Ellison Flew a Fighter Jet Under the Golden Gate Bridge
  Life
Dear Prudence
Sept. 29 2014 3:10 PM The Lonely Teetotaler Prudie counsels a letter writer who doesn’t drink alcohol—and is constantly harassed by others for it.
  Double X
The XX Factor
Sept. 29 2014 1:52 PM Do Not Fear California’s New Affirmative Consent Law
  Slate Plus
Slate Fare
Sept. 29 2014 8:45 AM Slate Isn’t Too Liberal, but … What readers said about the magazine’s bias and balance.
  Arts
Brow Beat
Sept. 29 2014 9:06 PM Paul Thomas Anderson’s Inherent Vice Looks Like a Comic Masterpiece
  Technology
Future Tense
Sept. 29 2014 11:56 PM Innovation Starvation, the Next Generation Humankind has lots of great ideas for the future. We need people to carry them out.
  Health & Science
Bad Astronomy
Sept. 29 2014 12:01 PM This Is Your MOM’s Mars
  Sports
Sports Nut
Sept. 28 2014 8:30 PM NFL Players Die Young. Or Maybe They Live Long Lives. Why it’s so hard to pin down the effects of football on players’ lives.