“What’s to Know About the Credibility of Empirical Economics”

T.D. Stanley in the Journal of Economic Surveys:

Thirty years ago, Edward Leamer (1983) offered his trenchant critique of empirical economics “Let’s take the con out of econometrics.” To bolster the credibility of econometrics, Leamer argued for more rigorous sensitivity analyses (Leamer, 1985), and robustness checking has since become conventional practice.

In recent decades, progress has been made. The reduced cost of data analysis has brought about a flood of empirical economic research. Methods have also advanced. In a symposium that reassesses Leamer’s critique, Angrist and Pischke (2010) argue that the recent acceptance of experimental and quasi-experimental research designs has done much to increase the credibility of empirical economics. Yet, disciplines with long traditions of strong experimental designs have their own crisis of credibility.

In a series of papers published in leading medical and science journals, John Ioannidis questions the validity and credibility of some of the most rigorously designed clinical trials and influential papers in medical research (Ioannidis, 2005, 2011, 2013). In the following “What’s to Know About the Credibility of Empirical Economics,” John Ioannidis and Chris Doucouliagos investigate the scientific integrity of empirical economics from a both a broad interdisciplinary perspective and the experience that dozens of meta-analyses of economics research brings. ...

I'm covering the Ioannidis and Doucouliagos paper in my senior seminar class tomorrow. Here is the abstract of the paper:

The scientific credibility of economics is itself a scientific question that can be addressed with both theoretical speculations and empirical data. In this review, we examine the major parameters that are expected to affect the credibility of empirical economics: sample size, magnitude of pursued effects, number and pre-selection of tested relationships, flexibility and lack of standardization in designs, definitions, outcomes and analyses, financial and other interests and prejudices, and the multiplicity and fragmentation of efforts. We summarize and discuss the empirical evidence on the lack of a robust reproducibility culture in economics and business research, the prevalence of potential publication and other selective reporting biases, and other failures and biases in the market of scientific information. Overall, the credibility of the economics literature is likely to be modest or even low.

Here is my presentation: Scientific credibility of economics.