Andrew Gelman at the Monkey Cage blog (which I wish was in my Feedly blog reader but I click on the RSS feed link and all I get is a My Yahoo! link ... I'm still upset that Google got rid of Google Reader):
Adam Marcus reports that an influential paper from 2009 on the economic effects of climate change was recently revised because of errors and omissions in the data. According to the paper’s author, Richard Tol:
Gremlins intervened in the preparation of my paper “The Economic Effects of Climate Change” . . . minus signs were dropped from the two impact estimates . . . [also there were] two overlooked estimates . . .
I’m not sure what is up with the gremlin, but in my own work I’ve introduced major errors into the analyses on occasion. Sometimes it happens when I’m copying numbers from a printed source into a typed document, or even when editing a document. So I could definitely see how minus signs could disappear.
I’m reminded of the recent case of Reinhart and Rogoff who introduced major errors in data processing and analysis into an influential paper on macroeconomic policy. The errors were undiscovered for years.
In both these cases, I think the problem is not so much the errors — mistakes will happen, and it’s understandable that researchers will be less likely to catch their errors if they go in the direction that support their views — but rather that there are many fragile links in the chain that connects data to policy recommendations. This is one reason that many people are starting to recommend that the “paper trail” of the statistical analysis be more transparent, so that researchers (including me!) simply aren’t able to make mistakes such as accidentally removing a minus sign in a computer file or losing a column of data in an Excel spreadsheet. ...
Please read the rest of the post and Tol's comment. There is some great (and simple) stuff on interpreting meta-analytic data.
I live in great fear that I've made a simple data mistake that ends up being published. In our 2000 REE paper, the version that I presented at Colorado State University (thanks to John Loomis for scheduling me when I was thereabouts) was wrong ... although, the errors went in the direction of supporting my view that combining revealed and stated preference data could be beneficial ... which we corrected before submission.
*Here is the first post on this topic.
