Faith-based fudging.

The conventional wisdom debunked.
Aug. 5 2003 12:35 PM

Faith-Based Fudging

How a Bush-promoted Christian prison program fakes success by massaging data.

Illustration by Robert Neubecker

The White House, the Wall Street Journal, and Christian conservatives have been crowing since June over news that President George W. Bush's favorite faith-based initiative is a smashing success.

When he was governor of Texas, Bush invited Charles Colson's Prison Fellowship to start InnerChange Freedom Initiative, a Bible-centered prison-within-a-prison where inmates undergo vigorous evangelizing, prayer sessions, and intensive counseling *. Now comes a study from the University of Pennsylvania's Center for Research on Religion and Urban Civil Society reporting that InnerChange graduates have been rearrested and reimprisoned at dramatically lower rates than a matched control group.

For those who know how hard it is to reduce recidivism, the reported results were impressive. Colson celebrated the report by visiting the White House for a photo op with the president. House Majority Leader Tom DeLay issued a triumphal press release. The Journal smacked critics of faith-based programs for "turning a blind eye to science" by opposing InnerChange. The report heartened officials in the four states that have InnerChange programs and buttressed President Bush's plan to introduce the Christian program in federal prisons.

Advertisement

You don't have to believe in faith-healing to think that an intensive 16-month program, with post-release follow-up, run by deeply caring people might be the occasion for some inmates to turn their lives around. The report seemed to present liberal secularists with an unpleasant choice: Would you rather have people "saved" by Colson, or would you rather have them commit more crimes and go back to prison?

But when you look carefully at the Penn study, it's clear that the program didn't work. The InnerChange participants did somewhat worse than the controls: They were slightly more likely to be rearrested and noticeably more likely (24 percent versus 20 percent) to be reimprisoned. If faith is, as Paul told the Hebrews, the evidence of things not seen, then InnerChange is an opportunity to cultivate faith; we certainly haven't seen any results.

So, how did the Penn study get perverted into evidence that InnerChange worked? Through one of the oldest tricks in the book, one almost guaranteed to make a success of any program: counting the winners and ignoring the losers. The technical term for this in statistics is "selection bias"; program managers know it as "creaming." Harvard public policy professor Anne Piehl, who reviewed the study before it was published, calls this instance of it "cooking the books."

Here's how the study got adulterated.

InnerChange started with 177 volunteer prisoners but only 75 of them "graduated." Graduation involved sticking with the program, not only in prison but after release. No one counted as a graduate, for example, unless he got a job. Naturally, the graduates did better than the control group. Anything that selects out from a group of ex-inmates those who hold jobs is going to look like a miracle cure, because getting a job is among the very best predictors of staying out of trouble. And inmates who stick with a demanding program of self-improvement through 16 months probably have more inner resources, and a stronger determination to turn their lives around, than the average inmate.

The InnerChange cheerleaders simply ignored the other 102 participants who dropped out, were kicked out, or got early parole and didn't finish. Naturally, the non-graduates did worse than the control group. If you select out the winners, you leave mostly losers.

Overall, the 177 entrants did a little bit worse than the controls. That result ought to discourage InnerChange's advocates, but it doesn't because they have just ignored the failure of the failures and focused on the success of the successes.

TODAY IN SLATE

Politics

Talking White

Black people’s disdain for “proper English” and academic achievement is a myth.

Hong Kong’s Protesters Are Ridiculously Polite. That’s What Scares Beijing So Much.

The One Fact About Ebola That Should Calm You: It Spreads Slowly

Operation Backbone

How White Boy Rick, a legendary Detroit cocaine dealer, helped the FBI uncover brazen police corruption.

A Jaw-Dropping Political Ad Aimed at Young Women, Apparently

The XX Factor
Oct. 1 2014 4:05 PM Today in GOP Outreach to Women: You Broads Like Wedding Dresses, Right?
Music

How Even an Old Hipster Can Age Gracefully

On their new albums, Leonard Cohen, Robert Plant, and Loudon Wainwright III show three ways.

How Tattoo Parlors Became the Barber Shops of Hipster Neighborhoods

This Gargantuan Wind Farm in Wyoming Would Be the Hoover Dam of the 21st Century

Moneybox
Oct. 1 2014 8:34 AM This Gargantuan Wind Farm in Wyoming Would Be the Hoover Dam of the 21st Century To undertake a massively ambitious energy project, you don’t need the government anymore.
  News & Politics
Politics
Oct. 1 2014 7:26 PM Talking White Black people’s disdain for “proper English” and academic achievement is a myth.
  Business
Buy a Small Business
Oct. 1 2014 11:48 PM Inking the Deal Why tattoo parlors are a great small-business bet.
  Life
Outward
Oct. 1 2014 6:02 PM Facebook Relaxes Its “Real Name” Policy; Drag Queens Celebrate
  Double X
The XX Factor
Oct. 1 2014 5:11 PM Celebrity Feminist Identification Has Reached Peak Meaninglessness
  Slate Plus
Behind the Scenes
Oct. 1 2014 3:24 PM Revelry (and Business) at Mohonk Photos and highlights from Slate’s annual retreat.
  Arts
Brow Beat
Oct. 1 2014 9:39 PM Tom Cruise Dies Over and Over Again in This Edge of Tomorrow Supercut
  Technology
Future Tense
Oct. 1 2014 6:59 PM EU’s Next Digital Commissioner Thinks Keeping Nude Celeb Photos in the Cloud Is “Stupid”
  Health & Science
Science
Oct. 1 2014 4:03 PM Does the Earth Really Have a “Hum”? Yes, but probably not the one you’re thinking.
  Sports
Sports Nut
Oct. 1 2014 5:19 PM Bunt-a-Palooza! How bad was the Kansas City Royals’ bunt-all-the-time strategy in the American League wild-card game?