In an October 2015 address to the International Association of Chiefs of Police, FBI Director James Comey spoke of an urgent “data shortage” that was making it difficult for him to do his job. Addressing a room of law enforcement executives, Comey said, pointedly, “How can we address a rise in violent crime without good information? And without information every single conversation in this country about policing and reform and justice is uninformed.”
Comey was sounding this alarm to advocate for a new data-gathering standard. I have good news for him: The good crime data he’s looking for is already available in pretty much every city in the United States. The bad news is that nobody is using it.
National crime statistics are currently compiled and promoted under a system called Uniform Crime Reporting. UCR is a tool from an earlier time: It came into existence in 1929 and has been run by the FBI since 1930. The FBI’s official motivation for launching UCR was to compile crime statistics in a consistent way so it would be possible to make comparisons and track patterns. According to a 1999 Bureau of Justice Statistics paper, it was also deployed to prevent newspapers from “manufacturing ‘crime waves’ out of thin air.”
More than 85 years later, the UCR system is still being used to control the narrative about crime. UCR might sound like a valuable resource, and it is better than nothing. Residents of a particular city can parse the data for macro crime trends, like whether the murder rate where they live is rising or falling. But when you compare it with what else is possible, UCR starts to look like an incredibly poor product—one that does more to obscure crime trends than to reveal them. Here are three reasons why.
UCR is imprecise. The UCR system consists of two main components. Part I offenses include violent crimes (criminal homicide, forcible rape, robbery, and aggravated assault) and property crimes (theft, auto theft, burglary, and arson). Part II offenses, which tend to be more minor, range from simple assault to disorderly conduct. While the data on Part I offenses include all incidents, regardless of whether an arrest is made, the Part II component doesn’t.
Because of the way the data is collected and reported, all UCR can say is whether very general categories of crime increased or decreased from year to year. To take one example, UCR’s public output combines all robberies into one category. New Orleans, where I worked for the city’s police department as a crime analyst between 2013 and 2015, saw a 200 percent increase in carjackings during that time period. The publicly available UCR data just recorded it as an increase in robberies, though, with no further clarity. (More later on how I know that carjackings increased by 200 percent.)
Similarly, UCR does not differentiate between the types of weapons used in an assault, so a victim who is shot is counted the same as someone who is stabbed.
Data that lumps together criminal activity in this way are of limited value. Even the FBI agrees, warning that using UCR data to rank cities by crime rate provides “no insight into the many variables that mold the crime in a particular town, city, county, state, region, or other jurisdiction.”
UCR is slow. The FBI accumulates UCR data from local departments and compiles it into a product called Crime in the U.S. This is the official data set on American crime, and it comes out once a year. The problem is, the publication process takes forever: The 2014 edition of Crime in the U.S. was not published until September 2015, meaning that for eight full months, the previous year’s crime statistics were a total mystery to voters and policymakers.
While some individual cities publish monthly or quarterly UCR updates, even those interim reports aren’t published until several weeks after the previous period ended. For example, New Orleans saw a dramatic rise in armed robberies over the last few months of 2015. If you relied solely on the city’s UCR publication system, there would be no way to see that trend until weeks or months after the fact, too late for the police to devote more resources to addressing the problem.
UCR is prone to manipulation. It is up to police departments themselves to aggregate the data they collect before sending them up the chain. There can be powerful incentives for departments to present their numbers in misleading ways. A 2012 New York Times survey of retired NYPD officers, for instance, found manipulation—“downgrading crimes to lesser offenses and discouraging victims from filing complaints to make crime statistics look better”—to be common practice throughout the department. Similarly, the New Orleans Office of Inspector General questioned in 2014 whether robberies and sex crimes were being misclassified and improperly downgraded to make the city’s stats look better.
Now, back to that good news I alluded to earlier. Though it gets far less attention than UCR, there is a better repository of crime data out there—one that addresses all of UCR’s obvious weaknesses. It’s known as calls for service, and it’s precise, fast, and less prone to manipulation.
CFS is a database of every incident a police department responds to in a given year, typically containing both 911 calls and police-initiated events. An individual CFS entry usually includes the incident type, priority, time and date of call, time and date of police response, where the incident occurred, the incident’s disposition (a report was taken, the call was a duplicate, the incident was unfounded, etc.), and much more.
Calls for service data are raw and messy, and that’s precisely why they’re such a rich source of information about crime. The unfiltered nature of CFS gives independent analysts the resources they need to draw their own conclusions rather than having to rely on PR-minded police departments. Most American cities collect CFS data, but to date only New Orleans; Sacramento, California; and Tucson, Arizona provide citizens with anything approaching unfettered access to their CFS databases. That needs to change.
Among the advantages to widespread access to CFS:
CFS is rich in detail. Where UCR lumps all crimes into big overarching categories, CFS is exceedingly granular. Take robberies. Using the FBI’s public-facing UCR system, all you can do is count total robberies in a city in a given year. With CFS you can distinguish between carjackings and other kinds of armed robberies—that’s how I know the former increased by 200 percent in New Orleans between 2013 and 2015—and determine what time and date they’re most likely to occur. The ability to access this level of detail makes it possible to evaluate the effectiveness of a police department’s strategy. You can use CFS to break down crime trends by incident type in your city, evaluate crime patterns in a given neighborhood, and track how your police department is handling domestic violence or sex crimes.
CFS is fast. Calls for service are uploaded with only a short delay to account for changing incident types in the immediate aftermath of a crime. (A death whose cause is initially unknown can later be deemed a homicide). Analysis of CFS in New Orleans enabled very accurate estimates of quarterly crime statistics long before any UCR data was available. CFS was also instrumental in identifying the rise and eventual decline of police response times in the city.
CFS is big. Many cities place some kind of crime data online, but these are typically just a small fraction of their service calls. New York, for example, reported information on about 75,000 UCR crimes in 2015, but the NYPD receives more than 10 million calls for service a year. Similarly, Chicago’s UCR data covers only about 10 percent of the city’s CFS total. Part of that discrepancy is due to the fact that some of the complaints that trigger calls for service turn out to be unfounded or cannot be verified by police. But a lot of the incidents that show up in CFS data simply don’t meet the threshold for inclusion in traditional UCR: crimes like illegal gun possession, drug possession, curfew violation, and some forms of domestic violence all get left out. If you want to see the full scope of crime in a particular place, CFS is your only option.
Right now, CFS is massively underused. While many police departments do publish information about the calls they respond to in some limited fashion—the city of Orlando, Florida, to take one example, regularly updates the 20 or so calls that the Orlando Police Department is actively handling—they don’t create real repositories of data that can be analyzed in an ambitious or revelatory way.
Other departments participate in the National Incident-Based Reporting System—that’s the system that FBI Director James Comey was stumping for in October. About 35 percent of law enforcement agencies nationwide are part of the NIBRS program. Those that do participate provide the FBI with aggregated data on a wider array of crimes than traditional UCR statistics. NIBRS, though, is still slow to produce results, and it’s as prone to potential manipulation as traditional UCR.
If CFS is clearly the best option, what’s preventing more cities from putting their CFS data online? One possible answer is that there’s been no public pressure on them to do so. More precise and comprehensive data inevitably mean more accountability. When given a choice, it makes sense that police departments elect to release less, not more information. It’s up to citizens to demand transparency from their government. When it comes to police data, that means saying UCR is just not good enough. It’s time for everyone, everywhere, to demand CFS.
