Faith-based fudging.

The conventional wisdom debunked.
Aug. 5 2003 12:35 PM

Faith-Based Fudging

How a Bush-promoted Christian prison program fakes success by massaging data.

(Continued from Page 1)

The Penn study doesn't conceal the actual poor outcome: All the facts reported above come straight from that report. But the study goes out of its way to put a happy face on the sad results, leading with the graduates-only figures before getting to the grim facts. Apparently, the Prison Fellowship press office simply wrote a press release off the spin, and the White House worked off the press release. Probably no one was actually lying; they were just believing, and repeating as fact, what they wanted to believe. It's hard to know for sure what those involved were thinking: Study author Byron Johnson canceled a scheduled interview at the last moment. The White House didn't respond to requests for comment.

InnerChange program manager Jerry Wilger says he doesn't know much about research, but he doesn't think it's fair to count the performance of the people who dropped out of his program against him, a fair-sounding objection that misses the point entirely. If InnerChange's 177 entrants were truly matched to the control group but ended up having more recidivism, then either the apparent success with the graduates was due to "creaming" or the program somehow managed to make its dropouts worse than they were to start with. If the program genuinely helped its graduates and didn't harm its dropouts, and if the whole group of entrants was truly matched to the controls, then the group of 177 should have done better than the controls. And they didn't.

So, the feel-good winners-only analysis simply isn't worth the paper it's printed on. Only the full-group analysis (known technically as "intent-to-treat," a holdover term from its origins in medical research) has any real value. And on that analysis, the program has a net effect of zero or a little worse than zero. That makes it a loser.

Advertisement

John DiIulio, an intellectually serious advocate of faith-based programs who was the first director of the Bush administration's faith-based initiatives and the founder of the Penn research center, acknowledges frankly the results weren't what a supporter of such programs would have hoped for. But he points out that a single study almost never provides a convincing yes or no answer on a program concept. "The orthodox believers point to a single positive result and say it proves faith-based programs always work. The orthodox secularists point to a single negative result and say it proves faith-based programs never work. They're both wrong."

The poor result of InnerChange doesn't mean that no faith-based prison program could work, but it does mean that this one hasn't, at least not yet. It joins a long line of what seemed like good ideas for reducing recidivism that didn't pan out when subjected to a rigorous evaluation. Maybe my own pet, literacy training, wouldn't do any better in a real random-assignment trial. But that's why you do evaluations; they tell you things you didn't want to hear. If you're honest, you listen to them.

And if you're smart, you don't listen to the political advocates of "faith-based" this and that when they say they're only asking us to support programs that have been "proven" to work.

Correction, Aug. 6, 2003: The article originally and incorrectly described InnerChange Freedom Initiative as "a fundamentalist prison-within-a-prison." The Prison Fellowship regards itself as being part of the evangelical tradition rather than the fundamentalist tradition. (Return to the corrected sentence.)

TODAY IN SLATE

Medical Examiner

The Most Terrifying Thing About Ebola 

The disease threatens humanity by preying on humanity.

I Bought the Huge iPhone. I’m Already Thinking of Returning It.

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

Students Aren’t Going to College Football Games as Much Anymore

And schools are getting worried.

Global Marches Demand Action on Climate Change

Politics

Blacks Don’t Have a Corporal Punishment Problem

Americans do. But when blacks exhibit the same behaviors as others, it becomes part of a greater black pathology. 

Why a Sketch of Chelsea Manning Is Stirring Up Controversy

How Worried Should Poland, the Baltic States, and Georgia Be About a Russian Invasion?

Moneybox
Sept. 19 2014 1:11 PM Americans' Inexplicable Aversion to the 1990s
  News & Politics
Weigel
Sept. 20 2014 11:13 AM -30-
  Business
Business Insider
Sept. 20 2014 6:30 AM The Man Making Bill Gates Richer
  Life
Quora
Sept. 20 2014 7:27 AM How Do Plants Grow Aboard the International Space Station?
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Tv Club
Sept. 21 2014 1:15 PM The Slate Doctor Who Podcast: Episode 5  A spoiler-filled discussion of "Time Heist."
  Arts
Brow Beat
Sept. 21 2014 2:00 PM Colin Farrell Will Star in True Detective’s Second Season
  Technology
Future Tense
Sept. 19 2014 6:31 PM The One Big Problem With the Enormous New iPhone
  Health & Science
Bad Astronomy
Sept. 21 2014 8:00 AM An Astronaut’s Guided Video Tour of Earth
  Sports
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.