Environmental Economics

This post name has only one word with more than four letters. Cite me.

Keep it simple stupid:

Research papers containing abstracts that are shorter and consist of more commonly used words accumulate citations more successfully, according to a recent study published in the Journal of Informetrics.

After analyzing more than 200,000 academic papers published between 1999 and 2008, the authors found that abstracts were slightly less likely to be cited than those that were half as long. Keeping it simple also mattered— abstracts that were heavy on familiar words such as “higher,” “increased” and “time” earned a bit more citations than others. Even adding a five-letter word to an abstract reduced citation counts by 0.02%.

I have no idea if I use short words in my abstracts.  I will have to look.

It is time for Oklahoma's oil and gas industry to join the fight in saving the Edmond suburb!

Fight the good fight:

Facing a six-year barrage of increasingly large earthquakes, Oklahoma regulators are effectively ordering the state’s powerful oil-and-gas industry to substantially cut back the underground disposal of industry wastes that have caused the tremors across the state.

On Monday, the state Corporation Commission asked well operators in a Connecticut-size patch of central Oklahoma to reduce by 40 percent the amount of oil and gas wastes they are injecting deep into the earth. The directive covers 411 injection wells in a rough circle that includes Oklahoma City and points northeast.

It follows a February request that imposed a 40 percent cutback on injection wells in a similar-size region of northwest Oklahoma.

The actions significantly increase the effort to rein in the quakes, which the commission has long tried to reduce one well or a handful of wells at a time.

But they are an equally notable challenge to the industry, which will most likely be able to make the cutbacks only by reducing oil and gas production. ...

Although critics contend that earthquakes have caused millions of dollars of damage, Oklahoma’s political leaders have long been reluctant to impose restrictions on an industry that dominates the state’s economy. Until last spring, Gov. Mary Fallin, a Republican, maintained that the cause of the tremors was unclear, and the state Legislature refused to consider legislation addressing the issue.

Ms. Fallin abandoned her position as the number of quakes rapidly increased. But the political leadership was not jolted into action until January, after a series of small earthquakes damaged homes and interrupted power in Edmond, an Oklahoma City suburb home to many in the state’s political and financial elite. ...

via www.nytimes.com

I'm a little skeptical that we can save Edmond. A 40% reduction in oil production would cost almost $200 million in lost revenues per month at last year's prices and production.*

*Fact check (i.e., that number seems huge, please tell me what I'm doing wrong):

• Price per barrel (P) = $39.16
• Oklahoma monthly production (Q) = 12,084,000 barrels 
• Revenue = P*Q = $473,209,440
• 40% of Revenue = $189,283,776

More production numbers are here. 

 

Replication vs Duplication: What's the difference?

Your thoughts on this are appreciated.

There's been a good bit of news lately on the issue of replication in various disciplines.  Psychology may or may not be in the midst of a replication crisis. Experimental economics bats around .600 when is comes to replicability in a very small sample, which is good for baseball, but unclear just how good it is for the state of science in experimental economics.   

Replication is something that has interested me for a while.  A lot of my own research ideas come from trying to figure out and replicate what others have done (copying or standing on the shoulders of giants?).  So for the final project in my PhD Applied Econometrics class this year (1st year PhD class), I decided to have them perform a replication:

Final Project: Replication, review and extension of existing results

1. Identify a published peer-reviewed journal article for which you would like to replicate, review and extend the results.

• The article should be published in a respected economics journal. If you don’t know what qualifies as a respected economics journal, talk to faculty members who are experienced at publishing, or try Googling “journal rankings in economics.”
• The article must have estimable econometric results (don’t choose a theoretical paper).
• The article must have data available for replication purposes. Most journals have a formal data policy (for example, here is the data policy for the American Journal of Agricultural Economics: http://www.oxfordjournals.org/our_journals/ajae/for_authors/general.html). Some journals require authors to make data available to interested readers, others do not. YOU MUST FIND AN ARTICLE FOR WHICH DATA IS AVAILABLE. As an example, the AJAE provides links to data with their on-line content (http://ajae.oxfordjournals.org/content/98/1.toc).
• And most important, the article should interest you. Don’t just pick the first article you find that has data available and you think you can use to satisfy the requirement. READ. A LOT. And then find an article that you’re truly interested in understanding.

2. Write a review of the chosen article in which you:

• Replicate, or at least attempt to replicate, the main result(s) of the paper. To replicate the results, you should write and original program that re-estimates at least one model from the paper. DO NOT USE PROGRAMS PROVIDED BY THE AUTHORS, YOU MUST WRITE AN ORIGINAL PROGRAM TO REPLICATE THE RESULTS OF SOMETHING IN THE PAPER. You should do your best to replicate the main finding by estimating the model. If you can replicate the results, report your results. If you come close, report that. If you can’t replicate the results, explain why (is it because you can’t figure it out, or is the paper wrong, or some combination of those?).
• Critique the paper (comment on the motivation, theory, model, estimation, interpretation, data, any issues you find with the data…). Reference other published papers as appropriate.
• Give a brief description of how you would extend the results of the paper given the chance. What would you do different? What could you do better? What would you do next?

Seems like a reasonably interesting project for a first year PhD students, but the way I have described the project brings a question to my mind:  Is this replication or duplication?  And is there a difference? And yes I know that is two questions.

I understand replication in the lab sciences (including experimental economics).  Simplified: Rerun the experiment under the conditions laid out in the original and see if the results hold up.  This even goes so far as questions in valuation studies which John and I regularly take on.  We can always run another survey and see if the results hold up.  But how does replication work in a truly applied setting, where the experiment (or study conditions) is not replicable.  If we simply take the authors' data set and rerun the same models that the author runs, is that replication, or duplication?  Sure we can learn something about the assumptions made in collecting data, cleaning data, deriving an estimable model from theory, model specification, statistical assumptions, etc, but do we really learn anything about the replicability of the results by simply redoing exactly what the authors did with the same data? 

Is that replication or is that duplication?

And does it matter?

Thanks in advance for your thoughts.

 

"More than half of top-tier [lab experiment] economics papers are replicable"

Retraction Watch:

Approximately six out of 10 economics studies published in the field’s most reputable journals — American Economic Review and the Quarterly Journal of Economics — are replicable, according to a study published today in Science.

The authors repeated the results of 18 papers published between 2011 and 2014 and found 11 — approximately 61% — lived up to their claims. But the study found the replicated effect to be on average only 66% of that reported in the earlier studies, which suggests that authors of the original papers may have exaggerated the trends they reported.

Colin Camerer, a behavioral economist at the California Institute of Technology in Pasadena, who co-authored the study, “Evaluating replicability of laboratory experiments in economics,” told us:

Four clearly failed to replicate, three were near misses (large effects but not highly significant by conventional p-value) and 11 replicated rather well.

As he and his co-authors note in the paper:

…replication in this sample of experiments is generally successful, though there is room for improvement.

Interestingly, the authors asked peer traders (those working as experimental economists) to predict the rates of replication before the replication experiments were actually conducted. They also surveyed traders on their beliefs of the probability of replication.

On average, the market prediction of the replication rate was 75.2%, and the survey belief was 71.1%, both of which turned out to be higher than the actual replication rate of 61.1%.

Here is the link to the paper (the above link didn't work for me): http://science.sciencemag.org/content/early/2016/03/02/science.aaf0918.abstract. It's gated so I don't know which studies replicated and which ones did not. 

Here is what they said about the paper at CalTech:

The authors suggest that there are some methodological research practices in laboratory experimental economics that contribute to the good replication success. "It seems that the culture established in experimental economics—incentivizing subjects, publication of the experimental procedure and instructions, no deception—ensures reliable results. This is very encouraging given that it is a very young discipline," says Michael Kirchler, another coauthor and collaborator from the University of Innsbruck.

"As a journal editor myself, we are always curious whether experimental results will replicate across populations and cultures, and these results from multiple countries are really reassuring," says coauthor Teck-Hua Ho from the National University of Singapore.

Coauthor Magnus Johannesson from the Stockholm School of Economics adds, "It is extremely important to investigate to what extent we can trust published scientific findings and to implement institutions that promote scientific reproducibility."

"For the past half century, Caltech has been a leader in the development of social science experimental methods. It is no surprise that Caltech scholars are part of a group that use replication studies to demonstrate the validity of these methods," says Jean-Laurent Rosenthal, the Rea A. and Lela G. Axline Professor of Business Economics and chair of the Division of the Humanities and Social Sciences at Caltech.

You ... you will never be a member of Bushwood!

My take on P-values: Tell me why I am wrong.

How big is your P?  I'm sure Donald Trump could answer that, but I'm talking about the big news today that statisticians think that the general science/social science/pseudo-science community have been using P-values wrong and it has affected the state of scientific research and publication in general.  I'm not a statistician, but I play one in my economics classes.  I used to think I knew what a P-value was, but after reading a number of convoluted and some not-so-convoluted attempts at defining it I'm no longer sure I know what I thought I knew.  So here's my understanding of a P-value in overly simplistic form (typical for Env-Econ).  Let me know where I've gone wrong.

Suppose I have a hunch that male students at Ohio State are taller than female students on average.  But, I can't measure every student at Ohio State, so instead, I am going to use the heights of the male students and the female students in my class of 20 students to see if I can learn anything about all students at Ohio State.  First I am going to make an ass out of you and me and assume that the 20 students in my class are drawn completely at random with respect to their height from the general population of students at Ohio State.  That is, there's no correlation between the students' heights and the likelihood they will end up in my class.  Everyone in the full population of students has an equal probability of ending up in my class.  

Next, I am going to play dumb and assume that male and female students at Ohio State are the same height on average.  Under that assumption (called the null hypothesis), I would expect that the male and female students in my class are going to be roughly the same height, on average.  But because I am only drawing a small subsample from the population, I am not going to expect the null hypothesis to hold exactly with my subsample of 20 students.  Or put another way, if we were to repeatedly redraw samples of 20 students from the full population of students, we would expect to get different average heights for each subsample, but we would expect the average of the averages to be the same as the population average if we take a large enough number of subsamples.  This is called sampling variation, and it is this sampling variation that creates the need for a P-value.

Now to test the null hypothesis that male and female students are the same height on average, I calculate a t-statistic for differences in means using the data I have from my sample of 20 students.  I then calculate the corresponding P-value under the assumption that the null hypothesis is true. 

So suppose I get a p-value of 0.04.  What does that mean?  The p=0.04 means that if I were to redraw 100 new random samples of 20 students from the full population of Ohio State students, I would expect that only 4 of those new samples would generate differences in means equal to or smaller than the difference between male and female heights I observe in my class*. 

So a p-value just gives us information on the likelihood that the data we observe comes from a data generating process consistent with the null hypothesis. 

Alright, now tell me what I screwed up.

*Yes, John, I'm ignoring the one-tail/two-tail issues.

We're #4! And #7! And They're #22!

Most read articles from Applied Economics Policy and Perspectives in February, 2016: 

1. Jonas Kathage, Menale Kassie, Bekele Shiferaw, and Matin Qaim "Big Constraints or Small Returns? Explaining Nonadoption of Hybrid Maize in Tanzania" Appl. Econ. Perspect. Pol. (2016) 38 (1): 113-131 first published online April 7, 2015 doi:10.1093/aepp/ppv009
2. Harry de Gorter and David R. Just "The Social Costs and Benefits of Biofuels: The Intersection of Environmental, Energy and Agricultural Policy" Appl. Econ. Perspect. Pol. (Spring 2010) 32 (1): 4-32 doi:10.1093/aepp/ppp010
3. Peggy J. Liu, Jessica Wisdom, Christina A. Roberto, Linda J. Liu,and Peter A. Ubel "Using Behavioral Economics to Design More Effective Food Policies to Address Obesity" Appl. Econ. Perspect. Pol. (2014) 36 (1): 6-24 first published online October 2, 2013 doi:10.1093/aepp/ppt027
4. Timothy C. Haab, Matthew G. Interis, Daniel R. Petrolia, and John C. Whitehead "From Hopeless to Curious? Thoughts on Hausman's “Dubious to Hopeless” Critique of Contingent Valuation" Appl. Econ. Perspect. Pol. (2013) 35 (4): 593-612 first published online October 22, 2013 doi:10.1093/aepp/ppt029
5. Richard G. Frank "Long-term Care Financing in the United States: Sources and Institutions" Appl. Econ. Perspect. Pol. (Summer 2012) 34 (2): 333-345 doi:10.1093/aepp/pps016
6. Pierre Chandon "How Package Design and Packaged-based Marketing Claims Lead to Overeating" Appl. Econ. Perspect. Pol. (2013) 35 (1): 7-31 first published online July 10, 2012 doi:10.1093/aepp/pps028
7. Timothy C. Haab, Matthew G. Interis, Daniel R. Petrolia, and John C. Whitehead "Interesting Questions Worthy of Further Study: Our Reply to Desvousges, Mathews, and Train's (2015) Comment on Our Thoughts (2013) on Hausman's (2012) Update of Diamond and Hausman's (1994) Critique of Contingent Valuation" Appl. Econ. Perspect. Pol. (2016) 38 (1): 183-189 doi:10.1093/aepp/ppw003

Emphasis added for self-promotional purposes. 

Oddly, the reaction to #4 which prompted #7 only came in at #22.

William Desvousges, Kristy Mathews, and Kenneth Train "From Curious to Pragmatically Curious: Comment on “From Hopeless to Curious? Thoughts on Hausman's ‘Dubious to Hopeless’ Critique of Contingent Valuation”" Appl. Econ. Perspect. Pol. (2016) 38 (1): 174-182 doi:10.1093/aepp/ppv023

 

 

Results from the NABE Economic Policy Survey

The National Association for Business Economics is having their policy conference in DC this week. You may have heard about results from the policy survey about the Fed and interest rates on NPR this morning. The survey also included questions of interest to environmental and resource types. Here is the summary of those results:

Should the federal government choose to increase revenues resulting from the CBO’s projected long-term deficits, 63% of panelists indicate that individual and/or corporate tax bases should be broadened. Enacting a broad-based energy or carbon tax or increasing Social Security and Medicare contributions have relatively less support (46% and 36% of panelists, respectively). A small percentage of panelists support a national sales or value-added tax (21%), an increase in individual income tax rates (17%), or an increase in the corporate income tax rate (13%). ...

President Obama recently proposed a $10 per barrel fee on imported crude oil to pay for annual transportation infra-structure investments. Over half (55%) of the panelists believe that this fee will be paid for primarily by consumers through higher gasoline prices. About 38% of respondents think the tax burden will be shared by both oil producers and consumers. Just 4% believe the tax burden will fall primarily on oil producers.

The survey offered a list of five possible results from the enactment of such an oil tax. (Respondents selected all the proposed effects they thought would occur.) A reduction in oil consumption was the most frequently cited outcome, with almost two-thirds (63%) of panelists believing that consumers will curb their oil consumption if the tax is approved. Just over half (54%) believe that the tax will encourage a shift to alternative forms of energy. Forty-four percent of panelists think that the proposal will “hamper economic growth” through reduced consumer spending, and 40% believe that the tax will “benefit domestic oil producers over offshore producers.” Only 18% of panelists think it will “impose a hardship on energy companies.”

My responses are here and here.

More on the American Statistical Associations's 6 principles for the use of p-values

Following Tim's post, from the Q&A with Ron Wasserstein (ASA’s executive director):

Retraction Watch: You note in a press release accompanying the ASA statement that you’re hoping research moves into a “post p<0.05” era – what do you mean by that? And if we don’t use p values, what do we use instead?

Ron Wasserstein: In the post p<0.05 era, scientific argumentation is not based on whether a p-value is small enough or not.  Attention is paid to effect sizes and confidence intervals.  Evidence is thought of as being continuous rather than some sort of dichotomy.  (As a start to that thinking, if p-values are reported, we would see their numeric value rather than an inequality (p=.0168 rather than p<0.05)). All of the assumptions made that contribute information to inference should be examined, including the choices made regarding which data is analyzed and how.  In the post p<0.05 era, sound statistical analysis will still be important, but no single numerical value, and certainly not the p-value, will substitute for thoughtful statistical and scientific reasoning.

In business stats I teach that the p-value is a continuous measure (i.e., p=0.0168), not discrete (p<0.05), and the quickest way to get an idea about statistical significance from computer output. I say look for those p-values below 0.10 and then consider the effect sizes. But ignorance of effect size is not such a problem in applied disciplines. For example, policy-relevant environmental and resource economics is focused on estimation of benefits and costs and other policy impacts are often the research goal. 

Also, we should be doing more one-tailed tests when theory suggests it. For example, the contingent valuation and choice experiment price (i.e., tax, bid) and scope tests should be one-sided (I think this is what we did in SEJ 1998 for the scope test but not the price test, ugh). Many scope tests classified as "fail" by Desvousges, Mathews and Train (2012) would be reclassified as "mixed" or "pass" if one-tailed tests were the norm. 

Cho and Abe (Is two-tailed testing for directional research hypotheses tests legitimate?) begin like this:

Standard textbooks on statistics clearly state that non-directional research hypotheses should be tested using two-tailed testing while one-tailed testing is appropriate for testing directional research hypotheses (e.g., Churchill and Iacobucci, 2002 and Pfaffenberger and Patterson, 1987). However, during the actual conduct of statistical testing, this advice is not often heeded. According to our observation of 492 recent empirical articles that have used structural equation modeling (SEM), regression analysis, and analysis of variance (ANOVA) in five selected marketing research-related journals, the Journal of Marketing, Journal of Marketing Research, Marketing Science, Journal of Consumer Research, and Advances in Consumer Research (2001–2005), there were 2703 (N = 2703) research hypotheses in total. Overall, 90.9% (n = 2458) of them are expressed in directional form, but only 9.1% (n = 245) of them are described in non-directional form.

How did research evolve to this point? Is it just laziness? Or, did someone prominent criticize a paper for having weak statistics because the author used theory to inform inference?

I don't have any problem if researchers report that a one-tailed test is statistically significant at the 90% confidence level (i.e., a t-statistic of 1.37 is enough to justify discussion of an interesting result and economics is a social science). I think I'll put this into practice and this and see what happens (CNREP is just around the corner).

All that said, I've been guilty as a referee of scolding authors for discussing coefficients where p>0.10.

 

The American Statistical Association sets out six principles for proper use of P-values (And you won't believe #2 or #4)*

Via Retraction Watch:

After reading too many papers that either are not reproducible or contain statistical errors (or both), the American Statistical Association (ASA) has been roused to action. Today the group released six principles for the use and interpretation of p values. P-values are used to search for differences between groups or treatments, to evaluate relationships between variables of interest, and for many other purposes.  But the ASA says they are widely misused. Here are the six principles from the ASA statement: 

1. P-values can indicate how incompatible the data are with a specified statistical model.
2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
4. Proper inference requires full reporting and transparency.
5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

...

Ron Wasserstein [ASA Executive Director]: If the statement succeeds in its purpose, we will know it because journals will stop using statistical significance to determine whether to accept an article. Instead, journals will be accepting papers based on clear and detailed description of the study design, execution, and analysis, having conclusions that are based on valid statistical interpretations and scientific arguments, and reported transparently and thoroughly enough to be rigorously scrutinized by others.  I think this is what journal editors want to do, and some already do, but others are captivated by the seeming simplicity of statistical significance. [emphasis added]

*I've been clicking on too many Facebook ads.

The BP/Deepwater Horizon federal CVM study estimated damages of $17 billion

Go to the Deepwater Horizon Response & Restoration website and click on:

• 5. Preassessment/Assessment
  • 5.15 Other lines of investigation
    • 5.15.3 Other materials

Obvious, right?

You'll see a bunch of memos from the "Total Value Team" that can be pieced together to form a coherent report. Here is the bottom line: