Moneybox

Is The Reinhart-Rogoff Result Based on a Simple Spreadsheet Error?

Update: Reinhart and Rogoff have responded.

So this is huge. Or, rather, it won’t matter even a tiny little bit but it ought to be a big deal anyway. You’ve probably heard that countries with a high debt:GDP ratio suffer from slow economic growth. The specific number 90 percent has been invoked frequently. That’s all thanks to a study conducted by Carmen Reinhardt and Kenneth Rogoff for their book This Time It’s Different. But the results have been difficult for other researchers to replicate. Now three scholars at the University of Massachusetts have done so in “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff” and they find that the Reinhart/Rogoff result is based on opportunistic exclusion of Commonwealth data in the late-1940s, a debatable premise about how to weight the data, and most of all a sloppy Excel coding error.

Read Mike Konczal for the whole rundown, but I’ll just focus on the spreadsheet part. At one point they set cell L51 equal to AVERAGE(L30:L44) when the correct procedure was AVERAGE(L30:L49). By typing wrong, they accidentally left Denmark, Canada, Belgium, Austria, and Australia out of the average. When you fix the Excel error, a -0.1 percent growth rate turns into 0.2 percent growth.*

This is literally the most influential article cited in public and policy debates about the importance of debt stabilization, so naturally this is going to change everything.

Or, rather, it will change nothing. As I’ve said many times, citations of the Reinhart/Rogoff result in a policy context obviously appealing to a fallacious form of causal inference. There is an overwhelming theoretical argument that slow real growth will lead to a high debt:GDP ratio and thus whether or not you can construct a dataset showing a correlation between the two tells us absolutely nothing about whether high debt loads lead to small growth. The correct causal inference doesn’t rule out causation in the direction Reinhart and Rogoff believe in, but the kind of empirical study they’ve conducted couldn’t possibly establish it. To give an example from another domain, you might genuinely wonder if short kids are more likely to end up malnourished because they’re not good at fighting for food or something. A study where you conclude that short stature and malnourishment are correlated would give us zero information about this hypothesis, since everyone already knows that malnourishment leads to stunted growth. There might be causation in the other direction as well, but a correlation study woudn’t tell you.

The fact that Reinhart/Rogoff was widely cited despite its huge obvious theoretical problems leads me to confidently predict that the existence of equally huge, albeit more subtle, empirical problems won’t change anything either. As of 2007 there was a widespread belief among elites in the United States and Europe that reductions in retirement benefits were desirable, and subsequent events regarding economic crisis and debt have simply been subsumed into that longstanding view.

*Correction, April 16, 2013: This post originally conflated the impact of the Excel error with the two other objections raised by the University of Massachusetts researchers.