Jurisprudence

C.S.Oy

Forensic science is badly in need of reform. Here are some suggestions.

How should forensic science be reformed?

Last week, the state of Mississippi terminated its 20-year relationship with medical examiner Dr. Steven Hayne. Hayne has come under fire from fellow medical examiners, criminal justice groups like the Innocence Project, and one of the authors of this article for his impossible workload, sloppy procedures, and questionable court testimony. In the early 1990s, Hayne and his frequent collaborator, now-disgraced forensic odontologist Dr. Michael West, helped secure murder convictions for Kennedy Brewer and Levon Brooks, both later proven innocent through DNA testing. The two were released from prison earlier this year.

Mississippi is hardly alone when it comes to bad forensic science. It now appears that Washington, D.C., may have to retry Angela O’Brien for the 2000 killing of her 2-year-old goddaughter, Brianna Blackmond, after revelations that the prosecution’s star forensic witness, a physicist named Saami Shaibani, lied about his credentials in a Wisconsin murder case. These are only the most recent and dramatic examples of forensics fraud to make the headlines. Over the years, there have been plenty of other hucksters and charlatans happy to take advantage of the ignorance of juries, prosecutors, judges, and defense attorneys in very complicated and difficult-to-understand disciplines.

But the charlatans are only half the story. Courts have also missed plenty of mistakes from well-intentioned, conscientious scientists, too. In fact, these may be even more common—and harder to catch. Studies show that crime lab fiber, paint, and body fluid analyses, for example,  may consistently have error rates of 10 percent or higher. The error rate in fingerprint analysis is possibly between 1 percent  and 4 percent. And bite mark evidence is notoriously unreliable though still widely used. The Chicago Tribune reported in July that L. Thomas Johnson—one of forensic odontology’s pioneers—has been attempting to use statistical models to shore up the reliability of this discredited field. But Johnson’s efforts have been hampered by new DNA testing in a 1984 murder, which concluded that the man convicted of the crime was not the source of saliva found on the victim’s sweater. Johnson testified for the prosecution in that case.

The use of forensic science in criminal trials is critically important. But reforms of the system are also desperately needed. It’s not enough to weed out the incompetent scientists. We need to begin to monitor even the good ones. One major barrier to improving forensic evidence in criminal trials is that in most jurisdictions, the state has a monopoly on experts. Crime lab analysts and medical examiners (and to a lesser extent DNA technicians) typically work for the government and are generally seen as part of the prosecution’s “team,” much like the police and investigators. Yes, science is science, and it would be nice to believe that scientists will always get at the truth no matter whom they report to. But studies have consistently shown that even conscientious scientists can be affected by cognitive bias.

A scientist whose job performance is evaluated by a senior official in the district attorney or state attorney general’s office may feel subtle pressure to return results that produce convictions. In cases in which district attorneys’ offices contract work out to private labs, the labs may feel pressure—even if it’s not explicit (though sometimes it is)—to produce favorable results in order to continue the relationship.

Cognitive bias can be even subtler. For some experts, merely knowing the details of a crime or discussing it with police or prosecutors beforehand can introduce significant bias to a lab technician’s analysis.

A research team led by Seton Hall law professor Michael Risinger published a study in the January 2002 California Law Review identifying five stages of scientific analysis in which bias can affect even the most professional expert’s opinion. The study was careful to note that these biases were unintentional and not the result of outright fraud. But according to the study, cognitive bias can factor into the ways in which a scientist observes the initial data, records that data, and makes calculations and also how he remembers and reinterprets his notes when preparing for trial—a problem that looms larger as time elapses between the lab work and trial testimony.

Most jurors aren’t aware of any of these biases; in fact, most give enormous weight to expert witnesses. Even out-and-out frauds like West and Shaibani can persuade jurors if they’re presented in court as reputable experts, appear likeable, and can testify with conviction. A study of the first 86 DNA exonerations garnered by the Innocence Project estimated that faulty forensic science played a role in more than 50 percent of the wrongful convictions. While it’s obviously not possible to completely eradicate bias and scientific error from the courtroom, a few simple and relatively inexpensive reforms could go a long way toward reducing it. Here are a few more recommendations:

Forensic counsel for the indigent. In many jurisdictions, indigent defendants aren’t given access to their own forensic experts. As a result, the only expert witnesses are often testifying for the prosecution—experts that come prepackaged with the inherent biases noted above. This undermines the whole adversarial basis of our criminal justice system. Indigent defendants should be given vouchers to hire their own experts, who can review the forensic analysis and conclusions of each prosecution expert.

Expert independence. Crime labs, DNA labs, and medical examiners shouldn’t serve under the same bureaucracy as district attorneys and police agencies. If these experts must work for the government, they should report to an independent state agency, if not the courts themselves. There should be a wall of separation between analysis and interpretation. Thus, an independent medical examiner would, for instance, perform and videotape the actual procedure in an autopsy. The prosecution and defense would then each bring in their own experts to interpret the results in court. When the same expert performs both the analysis and interpretation, defense experts are often at a disadvantage, having to rely on the notes and photos of the same expert whose testimony they’re disputing.

Rivalrous redudancy. Whether the state uses its own labs or contracts out to private labs, evidence should periodically and systematically be sent out to yet another competing lab for verification. The state’s labs should be made aware that their work will occasionally be checked but not told when. In addition to helping discover errors that might otherwise go undetected, the introduction of competition to government labs would all but remove any subconscious incentive to appease police and prosecutors and would strengthen the incentive for a more objective analysis.

Statistical analysis. The results from forensic labs should be regularly analyzed for statistical anomalies. Labs producing unusually high match rates should throw up red flags for further examination. For example, in 2004 Houston medical examiner Patricia Moore was found to have diagnosed shaken-baby syndrome in infant autopsies at a rate several times higher than the national average. This led to an investigation—and the reopening of several convictions that had relied on Moore’s testimony.

Mask the evidence. A 2006 U.K. study by researchers at the University of Southampton found that the error rate of fingerprint analysts doubled when they were first told the circumstances of the case they were working on. Crime lab technicians and medical examiners should never be permitted to consult with police or prosecutors before performing their analysis. A dramatic child murder case, for example, may induce a greater subconscious bias to find a match than a burglary case. To the extent that it’s possible, evidence should be stripped of all context before being sent to the lab. Ideally, state or city officials might hire a neutral “evidence shepherd,” whose job would be to deliver crime-scene evidence to the labs and oversee the process of periodically sending evidence to secondary labs for verification.

These proposed reforms would go a long way toward correcting the problems of bias and improper incentives in the forensics system. They’re also relatively inexpensive—particularly when compared with the cost of a wrongful conviction. (In the Brooks and Brewer exonerations noted above, the state of Mississippi paid for both the prosecution and defense in two high-profile murder trials, three decades of unnecessary incarceration, several rounds of appeals, and will likely have to pay each man millions of dollars in compensation.)

The continuing stories of forensics error and wrongful convictions are troubling but not all that surprising. Our criminal justice system is centuries old. It just hasn’t adapted well to the dramatic advances in science and technology over the past 30 years. But as forensic evidence becomes more and more important in securing convictions, the need for monitoring and oversight grows exponentially. Every other scientific field properly requires peer review, statistical analysis, and redundancy to ensure quality and accuracy. It’s past time we applied the same quality-control measures to criminal forensics, particularly given the fundamental nature of what’s at stake.