Randomized controlled trials: Do they work for economic development?

What Happens When You Approach Global Poverty as a Science Experiment?

What Happens When You Approach Global Poverty as a Science Experiment?

What happens when industries collide?
March 26 2014 11:53 AM

Random Acts

What happens when you approach global poverty as a science experiment?

Randomistas, proponents of randomized controlled trials, have recently been transforming the way we think about economic development and aid to poor countries.

Photo illustration by Lisa Larson-Walker. Photos by Thinkstock, courtesy UNDP/Wikimedia Commons.

If you want to help people start a business, it’s better to give them a loan than a handout. If you want to prevent the spread of HIV and unwanted pregnancies, teach people about safe sex and make condoms widely available. If you want students to learn, put them in smaller classes with new textbooks.

Joshua Keating Joshua Keating

Joshua Keating is a staff writer at Slate focusing on international affairs. 

These are all reasonable assumptions based on what we know about economics and human nature. But none stand up—or at least, none are as simple as they seem—when subjected to the kind of randomized controlled trials that have recently been transforming the way we think about economic development and aid to poor countries. Is randomization as revolutionary as its proponents claim? Or is it just another illusory fix to the intractable problem of global poverty?

The idea behind randomized controlled trials should be familiar to any high school science student. In order to test the effect of a variable on a given subject, you need a control group where the variable is not present. In a medical trial, this could mean giving a new drug to one group of patients and a placebo to another. In development, it could mean giving loans and business training to the poorest women in one group of villages in a given area, but not another group, and studying what happens over the next few months. 


Nongovernmental organizations and governments have been slow to adopt the idea of testing programs to help the poor in this way. But proponents of randomization—“randomistas,” as they’re sometimes called—argue that many programs meant to help the poor are being implemented without sufficient evidence that they’re helping, or even not hurting.

Economist Michael Kremer, now the Gates Professor of Developing Societies at Harvard University, carried out one of the earliest randomized development experiments to gain widespread attention, in western Kenya in the early 1990s. Schools in the area had a chronic shortage of textbooks; there was a consensus that more were needed to improve educational outcomes. But when 25 schools out of 100 were randomly chosen to receive new textbooks, they showed little change in average test scores—the only students whose performance did seem to improve were those already at the top of their classes.

The problem likely had to do with language: School is taught in English in Kenya, but for most students it’s a third language behind Swahili and local languages. New textbooks might help those who already speak English well, but for the majority of students, they make little difference without new ways of teaching. Another study by Kremer and Edward Miguel found that providing students in Kenyan schools with deworming medicine not only improved health outcomes but also decreased rates of absenteeism by one quarter. 

The gospel of randomized controlled trials, or RCTs, has spread significantly since Kremer’s early studies. “I’m not suggesting that every organization in every situation should be doing this,” Kremer told Slate. “The vast majority of development projects are not subject to any evaluation of this type, but I’d argue the number should at least be greater than it is now.”

According to Abhijit Banerjee, the Ford Foundation International Professor of Economics at MIT, part of the reason NGOs and governments have been slow to adopt RCTs is practical: They’re usually slow and expensive to carry out, particularly in the midst of scarce resources and clear humanitarian need. The idea of conducting “experiments” on the poor also makes people nervous, for obvious reasons. “When you use the word experiment, people have a vision that these are rats in cages and you’re feeding them some strange chemicals and seeing what happens,” Banerjee told Slate. “That perception has changed.”

“Every development program that’s being done is an experiment,” says Chris Blattman, a political scientist at Columbia University and author of a popular blog on international development. “The funny thing about most aid programs is how nontransparent they are.”

If Kremer was the pioneer of this type of research, Banerjee and his MIT colleague Esther Duflo, co-founders of the Abdul Latif Jameel Poverty Action Lab, have been its most effective evangelists. Their co-authored 2012 book Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty helped introduce the idea of RCTs to the wider public; Duflo has delivered a popular TED talk on the subject and has been profiled by The New Yorker.

Some of the examples in Poor Economics seem perfectly tailored for the Malcolm Gladwell–Freakonomics era of counterintuitive pop social science. One RCT carried out in Kenya by Duflo, Kremer, and Stanford University’s Pascaline Dupas found that providing teenage girls with school uniforms reduced rates of teen pregnancy to a greater extent than sex-ed programs. Experts had assumed that a lack of knowledge was the reason girls were having unprotected sex. But often girls with limited opportunities were choosing to get pregnant; giving them uniforms made it easier for them to stay in school instead.