Should You Trust Predictions About the SCOTUS Ruling on Obamacare?

Answers to your questions about the news.
March 29 2012 4:54 PM

A 5-4 Vote Against? Or 6-3 in Favor?

How reliable are we at predicting the outcomes of cases before the Supreme Court?

Jeffrey Toobin.
CNN legal analyst Jeffrey Toobin called the oral arguments on the Affordable Care Act a "train wreck" for the Obama administration and predicted the Supreme Court will strike down the health care reform law.

Photograph by Amy Sussman/Getty Images.

The Supreme Court heard oral arguments this week on the constitutionality of the 2010 health care reform law. The consensus among the media is that things didn’t go well for President Obama, and some reporters are predicting that the individual mandate will be declared unconstitutional. After making their assessments, court observers usually add the caveat that you can’t reliably predict the outcome of a case based on oral argument. Is that old saw true?

No. For years, the conventional wisdom among legal scholars was that oral arguments had little or no impact on the outcome of a case, and that any predictions based on them would be unreliable. Accordingly, Supreme Court forecasting models used to be based only on contextual information, such as the subject matter being decided (tax law, federalism, civil rights, etc.), the relative wealth of the parties, and whether the lower court decision was conservative or liberal. The best of these complex models correctly predicted about 75 percent of cases and performed extremely poorly on nonideological cases involving tax or maritime law. (As a general rule, legal scholars are able to call around 60 percent of cases correctly, simply on the basis of their expertise and intuition.) Then, in 2004, New York Times Supreme Court reporter Linda Greenhouse noticed that she could at least match the top forecasting models simply by listening to the tenor of oral arguments. Greenhouse argued that the justices tip their hands during questioning, so models that ignored this phase of a case were incomplete.

Also in 2004, future Chief Justice John Roberts, then a Supreme Court attorney, noticed that justices ask more questions of the side that ultimately loses in 86 percent of cases. Though he’d tested his theory on only 28 cases—and no one knew whether its sensational success rate would hold up—there was now ample evidence that oral arguments do, in fact, offer very useful data for predicting the outcome of a Supreme Court case. Drawing on this insight, many political scientists now incorporate language analysis of the arguments into their models. One method is to simply tally up the words spoken to each attorney, consistent with the approach Chief Justice Roberts pioneered. The other method goes slightly deeper, analyzing the emotional quality of the language that the justices use: The more “unpleasant” words a lawyer hears compared with his opponent, the less likely he is to win. (Justice Scalia has a habit of telegraphing his vote by using words like “idiotic” during oral argument.) The most sophisticated models combine these two strategies with the more traditional forecasting factors—ideology, historical voting patterns, etc. These haven’t yet been deployed in enough cases to have a documented success rate, but research on past cases strongly suggest a major improvement in accuracy.

Advertisement

Predicting the outcome of the present health care reform case is difficult, because there are several different questions being debated and more than two attorneys appearing before the court. However, a state-of-the-art model created by professors Ryan Black of Michigan State, Sarah Treul of the University of North Carolina, Timothy Johnson of the University of Minnesota, and Jerry Goldman of Chicago-Kent College of Law suggests that the court will declare the individual mandate unconstitutional by a 5-4 vote. The big question mark, of course, is swing voter Justice Kennedy. He asked Solicitor General Donald Verrilli Jr. two more questions than he asked the challenger’s attorney, Paul Clement, with 14 percent more negative language, suggesting a slight preference for overturning the law. 

Got a question about today’s news? Ask the Explainer.

Explainer thanks Timothy Johnson of the University of Minnesota and Jeffrey Segal of Stony Brook University.

Video Explainer: Why Aren't There More Slam Dunks in Women's Basketball?

This video was produced from an original Explainer by Brian Palmer. Want more questions answered? You can now watch video Explainers at Slate's News Channel  on YouTube.

TODAY IN SLATE

Politics

The Democrats’ War at Home

How can the president’s party defend itself from the president’s foreign policy blunders?

Congress’ Public Shaming of the Secret Service Was Political Grandstanding at Its Best

Michigan’s Tradition of Football “Toughness” Needs to Go—Starting With Coach Hoke

A Plentiful, Renewable Resource That America Keeps Overlooking

Animal manure.

Windows 8 Was So Bad That Microsoft Will Skip Straight to Windows 10

Politics

Cringing. Ducking. Mumbling.

How GOP candidates react whenever someone brings up reproductive rights or gay marriage.

Building a Better Workplace

You Deserve a Pre-cation

The smartest job perk you’ve never heard of.

Hasbro Is Cracking Down on Scrabble Players Who Turn Its Official Word List Into Popular Apps

Florida State’s New President Is Underqualified and Mistrusted. He Just Might Save the University.

  News & Politics
Politics
Sept. 30 2014 9:33 PM Political Theater With a Purpose Darrell Issa’s public shaming of the head of the Secret Service was congressional grandstanding at its best.
  Business
Moneybox
Sept. 30 2014 7:02 PM At Long Last, eBay Sets PayPal Free
  Life
Gaming
Sept. 30 2014 7:35 PM Who Owns Scrabble’s Word List? Hasbro says the list of playable words belongs to the company. Players beg to differ.
  Double X
The XX Factor
Sept. 30 2014 12:34 PM Parents, Get Your Teenage Daughters the IUD
  Slate Plus
Behind the Scenes
Sept. 30 2014 3:21 PM Meet Jordan Weissmann Five questions with Slate’s senior business and economics correspondent.
  Arts
Brow Beat
Sept. 30 2014 8:54 PM Bette Davis Talks Gender Roles in a Delightful, Animated Interview From 1963
  Technology
Future Tense
Sept. 30 2014 7:00 PM There’s Going to Be a Live-Action Tetris Movie for Some Reason
  Health & Science
Medical Examiner
Sept. 30 2014 11:51 PM Should You Freeze Your Eggs? An egg freezing party is not a great place to find answers to this or other questions.
  Sports
Sports Nut
Sept. 30 2014 5:54 PM Goodbye, Tough Guy It’s time for Michigan to fire its toughness-obsessed coach, Brady Hoke.