How to fix the NFL's instant replay system.

The stadium scene.
Dec. 22 2009 12:25 PM

After Further Review

The NFL's instant replay system doesn't work. Here's how to fix it.

Illustration by Mark Alan Stamaty. Click image to expand.

Every serious football fan knows the central tenet of the NFL's instant replay system: "A decision will be reversed only when the referee has indisputable visual evidence" that the call made on the field was incorrect. While we're all quick to complain when we don't agree with a replay reversal, few observers ever ask the obvious question: Why is the league's replay standard so demanding?

Earlier this month, Duke law professor Joseph Blocher asked that very question at PrawfsBlog, and since then several more bloggers have weighed in. Many start from the common legal wisdom that a fact-finder usually minimizes errors in adjudication by returning the verdict he believes most likely to be right. That's why, in a tort suit or one for breach of contract, the jury will find for the plaintiff if he proves his case by "a preponderance of the evidence"—a fancy way of saying "more likely than not." Sometimes the jury will mistakenly find for the plaintiff when the true facts favor the defendant, and sometimes it will err in the opposite direction. But, on reasonable assumptions, the "more likely than not" standard of proof is the best way to maximize correct verdicts and minimize incorrect ones.


In criminal law, our goal is not to minimize total errors. Believing, with the 18th-century English jurist William Blackstone, that it is "better that ten guilty persons escape than that one innocent suffer," the Anglo-American system deems it especially important to minimize erroneous convictions. That's why a criminal defendant must be acquitted unless the jury is persuaded of his guilt "beyond a reasonable doubt"—about the most demanding standard of proof known to law. It is designed to ensure that there will be very few erroneous convictions. However, it achieves that end at a high cost: lots of erroneous acquittals.

The NFL's "indisputable visual evidence" standard goes still further. Pro football's replay rules would seem to ensure very few erroneous reversals but at the cost of tolerating a great many erroneous initial calls. That tradeoff would make sense if we believe it "better that ten erroneous calls be allowed to stand than that one correct initial call be reversed." But that seems silly. After all, the very point of introducing instant replay was to reduce errors—"to get the play right." If we want fewer errors, our best bet is to allow the referee to reverse an initial call if he concludes that a contrary call is more likely to be right. In the legal world, that's what's known as de novo review.

Notwithstanding the written rule, isn't it possible that referees are applying something close to de novo review? After all, coaches' challenges succeed from  40 percent to 50 percent of the time. While you might think such a high success rate offers little reason to believe refs abide by the indisputable visual evidence standard, consider that the conviction rate in American criminal prosecutions stands just above 90 percent. Still, few if any scholars think that fact alone provides good evidence that juries do not adhere to the "beyond a reasonable doubt" standard of proof; the high conviction rate is consistent with the supposition that, by and large, prosecutors only bring strong cases. Similarly, a high reversal rate for instant replay is consistent with the possibility that coaches use their challenges wisely.

There are reasons to think that's so. First, because coaches have a limited number of challenges and lose a timeout if a challenge fails, they have every incentive not to make challenges that are likely to lose. Moreover, the facts that reversal rates have increased over time—coaches were successful on 29 percent of challenges in 1999—and that coach-initiated challenges succeed at twice the rate of booth-initiated challenges suggest that coaches are learning to issue challenges more judiciously. So while it appears to me—as it does to many fans—that refs don't always adhere to the strict IVE standard, it seems hard to deny that they apply a standard significantly more demanding than de novo review.

Even assuming that's right, many reasons have been proposed to keep the NFL's replay system the way it is now. None seems very convincing.



Blacks Don’t Have a Corporal Punishment Problem

Americans do. But when blacks exhibit the same behaviors as others, it becomes part of a greater black pathology. 

I Bought the Huge iPhone. I’m Already Thinking of Returning It.

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

Lifetime Didn’t Think the Steubenville Rape Case Was Dramatic Enough

So they added a little self-immolation.

Two Damn Good, Very Different Movies About Soldiers Returning From War

Medical Examiner

The Most Terrifying Thing About Ebola 

The disease threatens humanity by preying on humanity.

Students Aren’t Going to College Football Games as Much Anymore, and Schools Are Getting Worried

The Good Wife Is Cynical, Thrilling, and Grown-Up. It’s Also TV’s Best Drama.

  News & Politics
Sept. 20 2014 11:13 AM -30-
Business Insider
Sept. 20 2014 6:30 AM The Man Making Bill Gates Richer
Sept. 20 2014 7:27 AM How Do Plants Grow Aboard the International Space Station?
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Slate Picks
Sept. 19 2014 12:00 PM What Happened at Slate This Week? The Slatest editor tells us to read well-informed skepticism, media criticism, and more.
Brow Beat
Sept. 19 2014 4:48 PM You Should Be Listening to Sbtrkt
Future Tense
Sept. 19 2014 6:31 PM The One Big Problem With the Enormous New iPhone
  Health & Science
Bad Astronomy
Sept. 20 2014 7:00 AM The Shaggy Sun
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.