Andrew Gelman provides Douglas Campbell the space to describe how difficult it is to publish a contradictory replication in economics ("...it took us seven submissions and four years to publish our results in a minor journal."):
And [Douglas Campbell] concludes with some general comments about replication:
Academic economics dearly needs replication studies to become sexy. There are encouraging signs. Thomas Herndon became famous after catching Reinhart and Rogoff’s excel error. There is a new economics replication wiki, and there will be a panel on replication at the AEA meetings. Some journals, such as the AER, require data to be made available online. (Personally, I believe doing empirical research without posting your data online, at least after publication, should be taboo.) Yet, in a field where building close personal relationships is still easily the best path toward publishing and tenure, more needs to be done. One proposal is for someone to calculate new journal rankings which penalize journals which either do not accept comments on the papers they publish, or rarely publish such comments. It would also be helpful if editors, particularly at leading journals, which have substantial market power, would do more to encourage replication. One proposal is that if the QJE or AER wants to encourage comment papers without hurting their own citation ranking, they could start additional journals focused on replication.
I agree. And this reminded me that American Economic Association is conducting a survey. From the AEA Ad hoc Committee on Journals:
The AEA Executive Committee has appointed an ad hoc committee to review a variety of issues related to journals run by the Association. Some of the items being investigated include the mix of journals, refereeing policies, and data sharing rules. The Committee has crafted a survey that takes no more than 10 minutes to complete and we would greatly appreciate it if you would take the survey ...
I took it and here are my answers to the last couple of questions:
That last comment reads:
The AEA should take leadership in the profession and encourage more communication among researchers in terms of comments, replies and replication. This new journal would be an ideal outlet for this communication.
