Trial by Camera
Measuring the effectiveness of foreign aid.
The aid industry faces a dilemma. On the one hand, countries are more likely to grow rich if their citizens are provided with some important basics, such as a legal system that works or protection from corrupt officials. Such basics might seem the priority for aid money. On the other hand, it is much easier to measure success in simpler projects, such as building roads and laying pipes.
If development agencies focus on pouring concrete, they may be spending money on infrastructure that will never be used—and perhaps never even be built—because of corruption in the background. But if they focus on the broader stuff—democracy, corruption, human rights—they risk trying to do everything and in fact achieving nothing. William Easterly, the World Bank's most prominent apostate, argues that development agencies love big agendas because their contribution cannot be measured and found wanting. It is a bitterly cynical view, but that doesn't mean he is wrong.
I once gave a talk to a delegation of Danish students in which I advocated careful measurement of results, on the grounds that a lot of development spending is faddy and based on sketchy evidence. The Danes replied that their government concentrated on promoting democracy. "That sounds good," I said. "Does it work?" They didn't know. Nor do I.
But some young economists are changing the terms of this debate. They argue that it may be possible both to focus on broad issues such as corruption, and at the same time to measure results very rigorously with a randomized trial—the sort of trial that would be used to test a new medical drug.
Esther Duflo, a French economics professor at MIT, wondered whether there was anything that could be done about absentee teachers in rural India, which is a large problem for remote schoolhouses with a single teacher. Duflo and her colleague Rema Hanna took a sample of 120 schools in Rajasthan, chose 60 at random, and sent cameras to teachers in the chosen schools. The cameras had tamper-proof date and time stamps, and the teachers were asked to get a pupil to photograph the teacher with the class at the beginning and the end of each school day.
It was a simple idea, and it worked. Teacher absenteeism plummeted, as measured by random audits, and the class test scores improved markedly.
Another young economist, Ben Olken of Harvard, used a similar randomization technique to work out whether corruption in Indonesian road-building projects was best fought top-down, using audits, or bottom-up, soliciting comments from local villagers about whether money was being embezzled. One challenge was to work out how much embezzlement was taking place. Olken enlisted engineers to take samples of the road's structure and to estimate how much it should have cost to build; he compared that estimate with how much spending was claimed in the project's accounts. The missing funds were a rough guide to the amount embezzled.
In contrast to Duflo's results, Olken found that bottom-up monitoring was not effective—it shifted the embezzlement from something the villagers cared about (wages) to something they did not (building materials). The threat of a guaranteed audit—a threat that was later carried out—was much more effective, reducing the estimates of missing funds by a third.
It should not be surprising that it took different approaches in different situations to reduce corruption. Economic development is a process full of special cases. All the more important, then, to discover that big goals can be addressed in little steps—and that it is possible to find out whether the little steps are steps in the right direction.
Tim Harford is a Financial Times columnist. His latest book, The Logic of Life, will be published in paperback on Feb. 10.
Photograph of a digital camera by Ciaran Griffin/Stockbyte.