Economist's View: Are Working Papers Working?

Mark Thoma provides a long skeptical quote (results from the author's working paper significantly changed as a result of peer review up to publication while impressions about the research did not) and answers:

On the issue of working papers, sensational results -- the ones most likely to be costly if they change later -- are going to leak in their preliminary form, and they are going to be reported. When that happens, I'd rather that the experts in the field be aware of the paper already or have easy access to it so that they can qualify the results as needed, or at least try to. Without the checks and balances of other researchers to help reporters and policymakers with the interpretation of the results, etc., this could lead to even worse policy errors than before. More generally, I'm not convinced that the costs -- the times when economics working papers have caused changes in policy that are later regretted -- exceed the benefits to other researchers of having this information available sooner rather than later (if only, for example, to know what questions other people are working on, new techniques that are being used, and so on -- the results themselves are not the only way this information is helpful).

via economistsview.typepad.com

A few comments:

1. I'm a big (26.4 BMI) fan of working papers. Our department's working paper series has really helped publicize some of the great (ok, good) work that has been done at a mid- (ok, mid to low-) level department. 
2. As an individual producer, my working papers let others know what I am working on (assuming at least someone cares). While I'd prefer to have the published paper cited, I'm always happy to have a citation to a working paper. As a consumer, others' working papers let me know what they are working on. I get ideas for my own research 2-3 years before the paper is published.
3. I know of at least one federal agency that is not allowed to use working papers for policy analysis. The results must be peer reviewed. That seems like good policy, and policy analysis with results from "forthcoming" papers mitigates the publication lag criticism.
4. Authors are free to add the "do not quote" disclaimer to their working paper if they are worried that their results might mess up government policy. 
5. I have several working papers in the department series that will go unpublished. Reader beware! I always remember this caveat as I'm reading working papers. 
6. When shouldn't you post your working paper? Suppose you are in a competitive (in the market structure sense of the term) field where rivals have access to your data (i.e., there is no ownership of an essential resource). For example, much of the research in labor and macro is conducted with data that is publicly and readily available (e.g., I could run a basic wage equation in about an hour starting from scratch). Your working paper could allow your rivals to duplicate your results and scoop your work. Even then, I'd prefer to post my working paper as it is a record of who had the idea first.
7. I would say the scoop is not so easy these days in environmental economics. In the valuation area, the field is not competitive (in the market structure sense of the term). Many researchers have ownership of an essential resource as data collection takes considerable time, effort and money. In the environmental policy area, the data is often publicly available but it doesn't rain from heaven like in labor or macro. Researcher effort generates considerable value added making the scoop less likely.