How to fix the NFL's instant replay system.

The stadium scene.
Dec. 22 2009 12:25 PM

After Further Review

The NFL's instant replay system doesn't work. Here's how to fix it.

Illustration by Mark Alan Stamaty. Click image to expand.

Every serious football fan knows the central tenet of the NFL's instant replay system: "A decision will be reversed only when the referee has indisputable visual evidence" that the call made on the field was incorrect. While we're all quick to complain when we don't agree with a replay reversal, few observers ever ask the obvious question: Why is the league's replay standard so demanding?

Earlier this month, Duke law professor Joseph Blocher asked that very question at PrawfsBlog, and since then several more bloggers have weighed in. Many start from the common legal wisdom that a fact-finder usually minimizes errors in adjudication by returning the verdict he believes most likely to be right. That's why, in a tort suit or one for breach of contract, the jury will find for the plaintiff if he proves his case by "a preponderance of the evidence"—a fancy way of saying "more likely than not." Sometimes the jury will mistakenly find for the plaintiff when the true facts favor the defendant, and sometimes it will err in the opposite direction. But, on reasonable assumptions, the "more likely than not" standard of proof is the best way to maximize correct verdicts and minimize incorrect ones.


In criminal law, our goal is not to minimize total errors. Believing, with the 18th-century English jurist William Blackstone, that it is "better that ten guilty persons escape than that one innocent suffer," the Anglo-American system deems it especially important to minimize erroneous convictions. That's why a criminal defendant must be acquitted unless the jury is persuaded of his guilt "beyond a reasonable doubt"—about the most demanding standard of proof known to law. It is designed to ensure that there will be very few erroneous convictions. However, it achieves that end at a high cost: lots of erroneous acquittals.

The NFL's "indisputable visual evidence" standard goes still further. Pro football's replay rules would seem to ensure very few erroneous reversals but at the cost of tolerating a great many erroneous initial calls. That tradeoff would make sense if we believe it "better that ten erroneous calls be allowed to stand than that one correct initial call be reversed." But that seems silly. After all, the very point of introducing instant replay was to reduce errors—"to get the play right." If we want fewer errors, our best bet is to allow the referee to reverse an initial call if he concludes that a contrary call is more likely to be right. In the legal world, that's what's known as de novo review.

Notwithstanding the written rule, isn't it possible that referees are applying something close to de novo review? After all, coaches' challenges succeed from  40 percent to 50 percent of the time. While you might think such a high success rate offers little reason to believe refs abide by the indisputable visual evidence standard, consider that the conviction rate in American criminal prosecutions stands just above 90 percent. Still, few if any scholars think that fact alone provides good evidence that juries do not adhere to the "beyond a reasonable doubt" standard of proof; the high conviction rate is consistent with the supposition that, by and large, prosecutors only bring strong cases. Similarly, a high reversal rate for instant replay is consistent with the possibility that coaches use their challenges wisely.

There are reasons to think that's so. First, because coaches have a limited number of challenges and lose a timeout if a challenge fails, they have every incentive not to make challenges that are likely to lose. Moreover, the facts that reversal rates have increased over time—coaches were successful on 29 percent of challenges in 1999—and that coach-initiated challenges succeed at twice the rate of booth-initiated challenges suggest that coaches are learning to issue challenges more judiciously. So while it appears to me—as it does to many fans—that refs don't always adhere to the strict IVE standard, it seems hard to deny that they apply a standard significantly more demanding than de novo review.

Even assuming that's right, many reasons have been proposed to keep the NFL's replay system the way it is now. None seems very convincing.


Medical Examiner

Here’s Where We Stand With Ebola

Even experienced international disaster responders are shocked at how bad it’s gotten.

It’s Legal for Obama to Bomb Syria Because He Says It Is

Divestment Is Fine but Mostly Symbolic. There’s a Better Way for Universities to Fight Climate Change.

I Stand With Emma Watson on Women’s Rights

Even though I know I’m going to get flak for it.

It Is Very Stupid to Compare Hope Solo to Ray Rice

Building a Better Workplace

In Defense of HR

Startups and small businesses shouldn’t skip over a human resources department.

Why Are Lighter-Skinned Latinos and Asians More Likely to Vote Republican?

How Ted Cruz and Scott Brown Misunderstand What It Means to Be an American Citizen

  News & Politics
Over There
Sept. 23 2014 12:16 PM Another Intervention?    Anti-ISIS airstrikes aren’t about keeping Americans safe.  
Sept. 23 2014 12:36 PM Krispy Kreme Stuffed Half a Million Calories into One Box of Doughnuts
The Eye
Sept. 23 2014 11:33 AM High-Concept Stuff Designed to Remind People That They Don’t Need Stuff  
  Double X
The XX Factor
Sept. 23 2014 11:13 AM Why Is This Mother in Prison for Helping Her Daughter Get an Abortion?
  Slate Plus
Slate Plus
Sept. 22 2014 1:52 PM Tell Us What You Think About Slate Plus Help us improve our new membership program.
Brow Beat
Sept. 23 2014 11:48 AM Punky Brewster, the Feminist Punk Icon Who Wasn’t
Future Tense
Sept. 23 2014 10:51 AM Is Apple Picking a Fight With the U.S. Government? Not exactly.
  Health & Science
Bad Astronomy
Sept. 23 2014 11:00 AM Google Exec: Climate Change Deniers Are “Just Literally Lying”
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.