Just few toughts on the Reinhart-Rogoff case.
What does it mean for economics?
Well, at least a couple of good things.
1) The control process works, except when it doesn't.
Since R-R made their spreadshit available upon the request by a student, Thomas Herndon, not being able to get their results, it took few days to find the mistakes. The easyness with which R-R gave their data to a student proves their professionality and transparency. The fact that such a control was possible, fast and it happened so easily shows that the academic control actually works. This is the good news. What then strikes me is that no one took the time to double check R-R calculations before; after all their work is well published. I wonder why there has been no researcher, editor or referee checking the facts (I know the article didn't appear in a peer reviewed section, but anyway, given its popularity, this is striking... - here and here). This sounds weired since even the best papers are supposed to have been checked, controlled, criticized and revised more than once before they are considered a serious piece fo work or get accepetd for publication. Of course mistakes may happen, this is not the problem. The issue is that the mistake we are talking about was so easy to detect (the one on the excel spreadsheet, which anyway is probably not very relevant for the result - Stevenson-Wolfers).
So, on the good side, the academic control is generally effective. On the bad side, why didn't the mistake come out earlier?
It would be great for the profession if more and more authors and reviews started to make data (can a scientific result be based on proprietary data - nowadays often used - that cannot be made freely available and hence result cannot be replicated?) and methods publicly available [what if the result is obtained with proprietary software?] (Mark Thoma on this). Students and researchers may learn much faster and the control process would be more effective.
(Robert Waldman on R-R)
2) Economics is not, and we should not think about it as an exact science.
The fact that there is no clear-cut evidence of a 90% debt/GDP ratio does not prove the austerity promoters wrong. And in general it wouldn't be enough to prove the opposite side right. The evicende of a negative debt-GDP correlation is nothing new. We can debate on its causes and consequences. An this implies having some model in mind. You need to have a model in mind in order to know what you want to test. Then the test may or may not prove you wrong. But wait, what does it means being wrong in economics? Well, I don't think economics is a discipline where being wrong or right happens once for all, as in exact science.
In physiscs, if I want to test something, I can do an experiment, control the environment and repeat it. And we exactly know at which conditions things happen. Once, we know this, it becomes a law. It always works, maybe with some approximations, which can be refined, but yet it's just a mechanical stuff. Indeed, it makes possible to make previsions and to know exactly what will happens to objects almost exactly.
Economics is not like that. And it's not like that not because we have little understanding of things (which is true), it's like that by its own nature. There is a big difference. Yes, we know we still don't really understand many things, but that's a kind of socratic wisdom: no serious economist would claim to have the magical recipe for the economy. Anyway, even if we had a better understanding of things, I guess that the general answer to many issues would still be "it depends". Of course also in physics the result of an experiment depends on conditions, but we can have a clear understanding of them. In economics, where conditions depend on a mix of physical, social, cultural and psycological factors (many of which not even measurable), we have very little knowledge of them by the very nature of the problem.
So, back to R-R, what's the meaning of a 90% threshold? Nice empirical evidence (it'd've been), but I would be sceptical of interpreting such a result as a general thing anyway [by the way academic economists do rarely generalize so strongly, it's often their audience that likes to go beyond with the interpretation]. Assume that the causation goes from debt to growth, mainly due to credit risk as preceived by the markets (which is more or less the story we have been earing in the media for a while). Which country is risky is inherently a positional thing. So how do you avoid being the risky one? You just need not to be too much defferent from the group, no matter the "absolute" riskiness of the group as a whole. As long as you are not perceived as one of the riskiests you are relatively safe, in the sense that you are not likely to be among the firsts in danger. The critical level maybe 90% today, but it may be 30% or 150% tomorrow: it all depends on relative positioning and people expectations. Maybe. Or maybe not. But my feeling about such a clear message as "do not cross the 90% debt-GDP ratio and you'll be safe" is that it sounds to good to be true, and in fact it isn't. Economics is not an exact science where you can find such precise results. Whenever anyone comes out with such things we should be careful and check them one million times. If the result is sound, then we finally have something that looks like a law. It would be great, but I don't dare wish so much.
UPDATE
A post by brighella on noisefromAmeriKa (in Italian - link) tells more or less the same story, but much better than me. Thanks to the author for being so clear.
Il lavoro di ricerca è in gran parte questo: un sacco di sudore nel
curare dettagli noiosi, il tutto spessissimo per non raggiungere alcun
risultato importante che possa essere dato in pasto alla stampa, ai
blog, agli uomini potenti. La ricerca di qualità è fatta di noiosa lentezza, di pedanti dettagli, e di frequentissima assenza di risultati mirabolanti.
[...]
La teoria economica stessa ci dice, infatti, che dobbiamo ben guardarci
dal dare peso a risultati che indichino che sopra una soglia
predeterminata del X% di indebitamento pubblico il debito abbia un
effetto particolarmente negativo sulla crescita.
[...]
Il vero problema è che il Risultato è stato utilizzato da molti in modo
ingannevole. In particolare, il Risultato è stato abbondantemente
riportato dalla stampa e dai blog come una prova a favore delle misure
di contenimento dei deficit pubblici attualmente in atto in varie parti
del mondo.
[...]
Personalmente credo che la qualità di un giornalista o di un blogger
possa intuirsi anche dal modo in cui ha trattato la vicenda R&R. Se
in passato il blogger in questione vi ha detto che i lavori di R&R
sono una importante prova a favore dell’austerity in Europa, allora
cambiate blogger. Allo stesso tempo, se in questi giorni il blogger vi
ha raccontato che il lavoro di H&A&P conferma che l’austerity in
Europa è un errore allora, esattamente come detto sopra, cambiate
letture. In entrambi i casi, vi trovare di fronte ad un commentatore che
non cerca altro che vendervi un’idea preconfezionata.
