Spreadsheets and Strategy: Better Triple Check that Cell ReferencePosted: May 14, 2013
Analysis is often powerful; data has a way of cutting through fancy to fact. Here’s a story to warm the hearts of anyone who has had to debug an Excel spreadsheet, either their own or someone else’s.
Recently the CBC’s The Current headlined a story about the analysis on government debt in proportion to economic health as follows:
Kenneth Rogoff was one of two Harvard economists lauded for their research paper affirming the need for deep government austerity measures what he calls ‘Significant Haircuts’, an argument at the core of the last Republican election platform. But then a graduate student combed through the Harvard data and found errors that debunked their findings.
The Independent wrote:
In 2010, a pair of Harvard economists published a paper that confirmed the warnings of the global austerity movement. “Growth in a Time of Debt”, by Professors Carmen Reinhart and Kenneth Rogoff, concluded that countries with a debt exceeding 90 per cent of their annual GDP experienced slower growth than their thriftier peers.
It was a statistic to which pro-austerity policymakers could cling, and many did. Most notably, it was at the heart of the Republican Party’s “Path to Prosperity” budget, penned by GOP vice-presidential candidate Paul Ryan, which proposed dramatic cuts to public spending in the US.
Now, though, Reinhart and Rogoff’s most famous finding has been debunked by a 28-year-old student. Earlier this month, Thomas Herndon, a graduate in the economics department at Amherst College in Massachusetts, found that they had made fundamental mathematical errors in the study – and all because of a flubbed Excel spreadsheet.
Herndon began studying Reinhart and Rogoff’s paper as part of a course assignment, which demanded he replicate data from a well-known study. He chose “Growth in a Time of Debt”. When he was unable to recreate its results, however, he emailed the two professors, hoping they might share the data on which their findings were based. Finally, Reinhart replied. “She said she didn’t have time to look into my query, but that here was the data, and I should feel free to publish whatever results I found,” Herndon later told New York Magazine.
But when he opened her Excel spreadsheet, Herndon found simple computing mistakes. Reinhart and Rogoff had failed to include five of the 20 countries whose records they had analysed in their cornerstone calculation of average GDP growth in countries with high public debt. They had also omitted figures from several countries which might have contradicted their conclusions – Canada, New Zealand and Australia, for instance, have all experienced healthy growth despite high debt.
When he presented his findings, Herndon’s own teachers were sceptical. Professor Robert Pollin told Reuters: “At first, I didn’t believe him. I thought, ‘OK, he’s a student, he’s got to be wrong. These are eminent economists and he’s a graduate student.’ So we pushed him and pushed him and pushed him, and after about a month I said, ‘Goddamn it, he’s right.’”
Professor Pollin and another professor, Michael Ash, agreed to co-author Herndon’s study, “Does High Public Debt Consistently Stifle Economic Growth? A critique of Reinhart and Rogoff”. As soon as it was published, the paper was taken up by Mike Konczal, an influential economics blogger. Soon it was being hailed by New York Times columnist Paul Krugman, not to mention the next Bank of England governor, Mark Carney. In the global economic debate, it seemed to contradict a central pillar of the pro-austerity argument.
Reinhart and Rogoff were forced to confess to a “coding error” in their calculations, but defended the thrust of their findings. “We do not believe this regrettable slip affects in any significant way the central message of the paper,” they stressed in a statement.
Herndon, who describes austerity measures in the UK as “counterproductive”, doesn’t believe Reinhart and Rogoff skewed their findings in favour of austerity. He found that high debt and slow growth are somewhat correlated, but far less dramatically than the Harvard duo claimed.
“[Our] findings are clearly not consistent with the notion that we consistently observe a sharp fall-off in economic growth when the public debt/GDP ratio exceeds 90 per cent,” Herndon wrote in Business Insider. “It would be absurd to think that governments never have to worry about their level of indebtedness… [but] contrary to R&R, there is no definitive threshold for the public debt/GDP ratio, beyond which countries will invariably suffer a major decline in GDP growth. The implication for policy is that, under particular circumstances, public debt can play a key role in overcoming a recession.”