Future Tense

Wrongful Arrest by Software

A California county is struggling with its new computer-based case management system.

handcuffs.

IPGGutenbergUKLtd/Thinkstock

Managing information is central to the criminal justice system, and so it’s inevitable that mistakes happen. Names get confused, files lost. When these errors occur, the police can mistakenly arrest or detain people with no legal cause. But what happens when software is responsible for a wrongful arrest or detention?

On Aug. 1, 2016, Alameda County, California, replaced its ’70s-era case management system with new software, Tyler Technologies’ Odyssey Case Manager. This wasn’t a radical decision: Most counties around the country use some kind of software to process information about the people in their courts. When a judge issues or recalls an arrest warrant, when a defendant posts bail—all of this is data that the courts and the police rely upon to make decisions about whom to detain, arrest, or release.

But since the software was rolled out in this Northern California county, the public defender’s office has learned of dozens of cases in which people have been wrongfully arrested, detained in jail when they should have been released, or erroneously told to register as sex offenders. For example, in September four police officers showed up at the home of a 24-year-old man in Fremont to arrest him. An arrest warrant had previously been issued for his failure to appear in court on a drug possession charge but it had been dismissed. Yet the warrant mistakenly remained active in the court’s new Odyssey system, so the man was arrested. There have been so many reported errors—on a “semi-daily basis,” according to the East Bay Times—that the Office of the Alameda County Public Defender has filed hundreds of identical motions asking the court to keep accurate records. Similar problems have been reported in some of the 25 other California counties with Odyssey contracts, prompting the creation of a “California Tyler User Group” for court staff. Alameda County itself has decided not to use Odyssey for its family, probate, or civil matters.

No one seems to yet understand the source of the errors behind Odyssey’s case management software. For the moment, many of the mistakes appear to result from a user interface for court employees that is far more complicated than the previous system. The software manufacturer, Tyler Technologies, has had little comment. Yet this 2016 problem reflects concerns by the Supreme Court from more than 20 years ago.

In 1991, a police officer arrested Issac Evans after an identification check during a traffic stop turned up an outstanding arrest warrant. The arrest allowed the officer to search Evans’ car, which turned up a bag of marijuana and a subsequent drug possession charge.

But the outstanding arrest warrant wasn’t valid—it had already been rescinded by the judge who originally issued it for several traffic violations. In such cases, the court clerk was supposed to have called the sheriff’s clerk, who would then remove the active warrant from the sheriff’s computer database. Had the procedure been followed in Evans’ case, it’s quite likely the marijuana would not have been found because no warrant would have justified his arrest.

Because his arrest was based on an invalid warrant, Evans’ Fourth Amendment rights had been violated. Normally, this would mean that the marijuana found as a result of the search would have been suppressed, under the exclusionary rule, which is intended to deter police misconduct. One exception to that rule, however, occurs when the police act in “good faith” on a legal decision that they believe to be correct, even if it later turns out to be wrong. In 1994, the Supreme Court decided in Arizona v. Evans that this exception applied to Evans’ case: The mistake was the fault of the court clerk, not of the arresting officer, who relied in good faith on the invalid warrant.

Issac Evans lost because the Supreme Court was convinced that he fell victim to an isolated error. Justice Sandra Day O’Connor, for example, suggested that the court might reach a different conclusion in a case where “the recordkeeping system itself” contained “no mechanism to ensure its accuracy over time” and “routinely” resulted in false arrests. Likewise, Justice David Souter stated that if a computer database had no way of “keeping the number of resulting false arrests within an acceptable minimum limit,” the exclusionary rule might apply. The software mistakes occurring in Alameda County appear to be more systematic than isolated.

So how do individual criminal defendants identify and challenge the “fruits of computerized error,” as Souter called them in Arizona v. Evans?

The answer is that we don’t have a very good answer. At some point in the future, the Supreme Court may decide to apply the exclusionary rule in a case where systemic software errors violate Fourth Amendment rights. The Alameda County Superior Court will hear the public defender office’s request to intervene in the software errors in January. In the meantime, software problems like those experienced in Alameda County have tangible, real-life consequences. Moreover, not every defendant who has fallen victim to these problems may discover that their issue is the result of a systemic software problem rather than an isolated bookkeeping snafu.

These problems will likely worsen as software increasingly becomes embedded in everything we do. Odyssey clearly has its flaws, but at least court employees can identify a problem like a recalled arrest warrant, even if it’s too late to stop a wrongful arrest. With other types of software, however, errors may be difficult to detect. Algorithms designed to help judges decide bail, or to help the police identify suspicious behavior, may be hard for nonexperts to understand, let alone critique. The private companies that design and sell these products may also be reluctant to share their proprietary information.

Part of the problem is a bad fit. Criminal cases are individual, but in the age of big data, problems and solutions have to be systematic. When there are few incentives to audit databases or check for software errors, mistaken arrests and detentions should be no surprise. Justice Ruth Bader Ginsburg once stated that “electronic databases form the nervous system of contemporary criminal justice operations.” Today software, and increasingly sophisticated software, is part of that nervous system. Yet we fail to ensure the system’s health.

This article is part of Future Tense, a collaboration among Arizona State University, New America, and Slate. Future Tense explores the ways emerging technologies affect society, policy, and culture. To read more, follow us on Twitter and sign up for our weekly newsletter.