Explainer

How Accurate Are Juries?

Your peers are far from infallible.

Prosecutors Hank Goldberg (L) and Marcia Clark (R) stand next to the empty jury box

Prosecutors Hank Goldberg, left, and Marcia Clark stand next to the empty jury box on Sept. 29, 1995, as they listen to Judge Lance Ito warn Clark that she is walking a fine line in her rebuttal closing arguments in the O.J. Simpson murder trial in Los Angeles.

Photo by Eric Draper/AFP/Getty Images

Jurors in the George Zimmerman trial have dominated this week’s news. Juror B37 appeared on CNN with Anderson Cooper, prompting four of the five others to distance themselves from the outspoken juror. The jury sequestration cost Floridians $33,000, which included pedicures, bowling, and admission to the Ripley’s Believe It Or Not! museum. Juries seem like a lot of trouble. How accurate are they?

Ninety percent accurate, at best. Studies suggest that juries reach the correct verdict between 75 and 90 percent of the time. It’s impossible to ascertain whether juries are accurate in individual cases, of course. (If we knew all the facts, we wouldn’t need a jury.) Some legal scholars also insist that the idea of jury error is nonsensical, because guilt or innocence can’t exist independently of a jury verdict. These quasi-metaphysical concerns notwithstanding, the relatively narrow range of estimates that has resulted from decades of research lends some plausibility to the numbers.

The first major study on this question was published in 1966, when University of Chicago law professors Harry Kalven Jr. and Hans Zeisel asked presiding judges to make a note of their own verdicts before the jury ruled. The judge and the jury agreed in approximately 80 percent of criminal cases. A similar study in 2003 found agreement in 77 percent of cases. Although they disagreed with the jury around 20 percent of the time, the judges found the verdict unreasonable in only 9 percent of cases.

Agreement isn’t a guarantee of correctness, of course. The judge and jury could each be wrong more than half of the time, even with a 77 percent agreement rate. To try to reach a better estimate of a jury’s reliability, Northwestern University statistician Bruce Spencer performed a pair of analyses on the existing datasets. First, he assumed that the presiding judge was at least as accurate as the jury. From that admittedly uncertain starting point, Spencer calculated that juries are incorrect in at least 13 percent of cases. (That’s not far from the 9 percent of cases in which judges find a verdict unreasonable.) For his second statistical trick, Spencer generated a mathematical model based on the differential judge and jury conviction rates to determine separately the rates of wrongful convictions and wrongful acquittals. (The model is described on Page 15 of Spencer’s paper.) According to his numbers, juries wrongfully convict in 25 percent of cases.

That doesn’t mean we should abandon the jury system: Spencer estimated that judges wrongfully convicted in 37 percent of cases in his dataset. On the other hand, judges were very unlikely to wrongfully acquit. Spencer’s estimates are extremely controversial, and he admits that his datasets, which are now more than a decade old and drawn from a limited number of jurisdictions, may not be generalizable to the country as a whole.

For an example of how study parameters can affect the results, consider the following statistical quirk. Between 1946 and 1987, criminals tried in federal court were convicted in approximately three-fourths of all cases, regardless of whether they opted for a jury or only a judge. Over the next 15 years, the conviction rate in jury trials jumped 9 percentage points, while the conviction rate in non-jury trials dropped 18 percentage points. Observers suspect that the adoption of federal sentencing guidelines, which took away judges’ discretion in sculpting punishment, caused judges to acquit more defendants.

Technical evidence—which was limited in the Zimmerman trial—often flummoxes juries. Jurors interviewed after a 1984 asbestos liability trial, for example, thought anyone exposed to asbestos for a certain period would develop asbestosis, which was contrary to the medical testimony they had just heard. Several of the jurors refused to believe that smoking could have been partially responsible the plaintiffs’ breathing difficulties, perhaps because the jurors were themselves smokers. Cases involving pharmaceutical liability and financial malfeasance have also been shown to confuse jurors.

Legal scholars emphasize that there’s more to a criminal trial than finding objective facts. Jurors are called upon to assess a defendant’s mental state, and they sometimes override the law to avoid unjust outcomes. They also diffuse political tension, which would otherwise be focused on a single appointed or elected judge.

Got a question about today’s news? Ask the Explainer.

Explainer thanks Neil Vidmar of the Duke University School of Law and Bruce Spencer of Northwestern University.