Environmental Economics

A benefit-cost analysis of a red drum stock enhancement program in South Carolina

R. J. Rhodes, J. C. Whitehead, T. I. J. Smith and M. R. Denson 

https://doi.org/10.1017/bca.2018.1

Published online: 11 May 2018

Abstract: Recreational saltwater anglers from the mid-Atlantic through the Gulf of Mexico commonly target red drum. Due to concerns about overharvesting within South Carolina coupled with regional management actions, South Carolina explored the technical feasibility of stocking hatchery-produced juvenile red drum as a technique to augment the abundance of South Carolina stock. In order to assess a continued program, in 2005 a mail survey was used to collect data for estimating the economic benefits with the contingent valuation method. The theoretical validity of willingness to pay was assessed by comparison to the value of a change in red drum fishing trips that would result from the program. Benefits were compared to estimated, explicit stocking costs. We illustrate how a certainty recode approach can be used in sensitivity analysis. The net present values (NPVs) for the stocking program are positive suggesting that the program would have been economically efficient relative to no program.

I like this paper for a number of reasons:

• It was first presented over 10 years ago at the Southern meetings. It sat idle for 10 years and JBCA was the first journal where it was submitted. The lesson is that ... don't let your data sit idle for 10 years. 
• The paper is a benefit-cost analysis, actually used by the agency (see Stocking South Carolina's Favorite Saltwater Fish). I've been working on benefits forever but rarely get to actually compare them to costs. In addition, I got to work with real-life agency fisheries economists and biologists who had real-world constraints on the survey questions they were able to ask. I maybe learned more from this project than any other. 
• I've taught this case study for a number of years in my benefit-cost analysis course. I haven't taught the course in years (department chair duties got int the way) but am scheduled to teach it this fall. I'll be using it again this fall to help explain sensitivity analysis, discounting, horizon values, etc. 
• We did revealed preference (travel cost method) and stated preference (contingent valuation method) analyses to estimate benefits. The revealed preference estimates were less informative than the stated preference estimates so the latter were used in the policy analysis. This illustrates my contention that stated preference data can be useful for policy analysis but (at least for recreation) you are better off if you also consider revealed preference estimates (as a validity check -- and see the next bullet point).
• The journal referees were not much impressed with our assertion that an old case study was worthy of publication. So, I went back to my all-time favorite paper for some theory and found convergent validity between single-site recreation demand and contingent valuation. The lesson here may be that if a researcher is looking for a better understanding of behavior and preferences from stated preference data then things can be discovered and learned. On the other hand, if a researcher is looked to break stated preference methods then that can be arranged too.
• Hypothetical bias is an issue that will likely never go away in contingent valuation (or choice experiments if folks doing that sort of stated preference study were targeted by industry-funded critics) regardless of assertions about consequentiality. In this paper we used a certainty follow-up and the analysis in the preceding bullet point to best match the willingness to pay estimates with a revealed preference estimate. From this, we used alternative certainty corrections for sensitivity analysis. As of today, this is how I think benefit-cost analysis with stated preference methods should be done. 

New publication: Estimating the Value of Medal Success in the Olympic Games

From the inbox:

Your article Estimating the Value of Medal Success in the Olympic Games has now been published in Journal of Sports Economics Volume 19 Issue 3, April 2018 and can be viewed at doi.org/10.1177/1527002515626221

Here is the abstract:

We estimate Canadians’ willingness to pay (WTP) for medals won by Team Canada in the 2010 Winter Olympic Games using data from contingent valuation method (CVM) surveys of nationally representative samples conducted before and after the Games. The results permit an assessment of Own the Podium, a government program designed to increase Canada’s medal count. International prestige and national pride are important determinants of WTP. The results are sensitive to cost and scope, respondents’ beliefs about the effectiveness of the program, as measured by expected medal count. WTP estimates suggest that Own the Podium generated benefits above its cost to a degree unique in the growing literature of sport CVM studies.

Notes:

• This paper uses data from what could be the last phone survey ever.
• Errata: Ugh, the sample size indicator (n) is lower case in Table 6 and upper case (N) in Table 7.
• Yes, we tested for endogeneity of the medals variable (Table 7). It's not a clean comparison since we sum the gold and other medals variables for the test, but still. Using the attitudinal (Table 2) variables as instruments correction for endogeneity does not affect the value of medals in the pre-Olympic data (but the predicted (i.e., endogeneity corrected) medals coefficient is statistically significant at less (more than?) than p=.10 [t=1.51]). In the post-Olympic data the endogeneity corrected model leads to a WTP for an additional medal of $58 versus $24 in the biased model. The confidence intervals overlap so I don't feel bad about presenting the more conservative, and more realistic, estimates.

If smart people include PhD economists at elite universities

Environmental valuation surveys don't provide useful information.

— John C. Whitehead (@johnwhitehead81) March 6, 2018

New publication: "Convergent validity of stated preference methods to estimate willingness-to-pay for seafood traceability"

Whitehead, John, O. Morgan, and William Huth. "Convergent validity of stated preference methods to estimate willingness-to-pay for seafood traceability: The case of Gulf of Mexico oysters." Economics Bulletin 38, no. 1 (2018): 326-335.

Abstract: In this study we compare willingness to pay for a seafood traceability system from contingent behavior demand and contingent valuation referendum vote models using data from a survey of Gulf of Mexico oyster consumers following the BP oil spill in 2010. We estimate a random effects model of oyster demand using contingent behavior data and find that a traceability program increases demand and consumer surplus. We estimate a referendum model for the seafood traceability program using contingent valuation data. We find that welfare estimates from the contingent behavior and contingent valuation methods are convergent valid under certain conditions.

URL: http://www.accessecon.com/Pubs/EB/2018/Volume38/EB-18-V38-I1-P31.pdf

For me, the most interesting part of the paper is our robustness checks on the contingent behavior and contingent valuation data. Here is what we say about the CVM data:

The traceability program increases consumer surplus per meal by $0.95 with a 95% confidence interval of $0.37 to $1.53. The consumer surplus estimates are robust to alternative econometric models such as random effects Poisson and random and fixed effects ordinary least squares models (these results are available upon request). ...

The conditional mean willingness-to-pay per meal ... constrains willingness-to-pay to be positive (Hanemann 1989). The 95% confidence interval is constructed using a bootstrapping procedure (Krinsky and Robb 1986). The willingness-to-pay per meal is $2.69 with a 95% confidence interval of 1.23 to 4.15. ...

We find that differences in the consumer surplus and mean willingness-to-pay estimates are not statistically significant since the 95% confidence intervals overlap. However, this obscures large differences in the point estimates. Mean willingness-to-pay is 183% higher than the consumer surplus estimate from the demand model.

When referendum data exhibits “fat tails” ... welfare measures will be less robust to alternative models relative to textbook data. The conditional mean welfare measure is not robust to alternatives such as the log-linear logit model (median WTP = $0.27 [-$0.03, $0.56] with the 95% confidence interval in brackets) and the Turnbull (Haab and McConnell 1997) nonparametric estimate (WTP = $1.47 [$1.20, $1.74]) which are not sensitive to the tail of the distribution. However, these two estimates also lead to the conclusion of convergent validity with the consumer surplus estimate.

On the other hand, the conditional mean willingness-to-pay estimate is robust to comparison with the Kriström (1990) nonparametric estimate (WTP = $2.65 [$2.23, $3.06]) (Boman, Bostedt, and Kriström 1999). But the confidence interval of the Kriström estimate does not overlap the confidence interval for the consumer surplus estimate. This lack of convergent validity is due to the narrow Kriström confidence interval which is partially an artifact of the smoothing of the data at the upper two bid amounts.

This sensitivity analysis was prompted by referee comments and my experience with two recent contingent valuation studies that, in my opinion, relied too heavily on the Turnbull estimator, especially with "smoothed" due to non-monotonicity (see Whitehead 2017a, 2017b). Smoothing data is equivalent recoding the dependent variable to fit the theory. Typically, recoding the dependent variable to fit the theory is a strong no-no. In the willingness-to-pay context it is OK in order to provide a lower bound willingness-to-pay estimate. But this estimate should not be used for hypothesis testing in the absence of other estimators. Here is the last line of Whitehead (2017b): 

With what Haab and McConnell (2002) call “difficult data,” the entire range of nonparametric and parametric WTP estimates should be examined for validity testing and benefit-cost analysis.

References

Whitehead, John C., “Who Knows What Willingness to Pay Lurks in the Hearts of Men? A Rejoinder to Egan, Corrigan, and Dwyer,” Econ Journal Watch 14(3), 346-361, September 2017a. https://econjwatch.org/articles/who-knows-what-willingness-to-pay-lurks-in-the-hearts-of-men-a-rejoinder-to-egan-corrigan-and-dwyer?ref=articles

Whitehead, John C. "A replication of willingness-to-pay estimates in An adding up test on contingent valuations of river and lake quality’(Land Economics, 2015). Economics Discussion Papers, No 2017-55." Kiel Institute for the World Economy. http://www. economics-ejournal. org/economics/discussionpapers/2017-55 Page 2, no. 2 (2017b). http://www.economics-ejournal.org/economics/discussionpapers/2017-55/file

"Exxon pushes back at #exxonknew study"

From TheHill's Overnight Energy email (March 1):

Exxon pushed back in a report released Thursday against claims from a study last year on its public statements on climate change.

The August study from two Harvard University -- one of whom, Naomi Oreskes, is an outspoken Exxon critic -- used content analysis on decades of Exxon advertisements and statements to conclude that it misled the public about climate change.

But Exxon retained Kimberley Neuendorf, a Cleveland State University content analysis expert, to rebut the conclusions.

"I have concluded that [the authors'] content analysis does not support the study's conclusions because of a variety of fundamental errors in their analysis. [The authors' content analysis lacks reliability, validity, objectivity, generalizability, and replicability," she said.

"These basic standards of scientific inquiry are vital for a proper content analysis, but they are not satisfied by the [authors'] study."

Neuendorf wrote the book that laid out the methods that the Harvard researchers cited.

In her report for Exxon, Neuendorf argues that the study suffered problems with sampling, coding, consensus measurement, lack of disclosure and other issues.

Read the study here.

This sounds too familiar.

Let's say you have coded your data and estimated your first model and, gulp, there are no results*; but then it turns out this is a @SurveyMonkey problem ...

Suppose you have a 3 x 3 experimental design in a stated preference survey with cost $100, $250, and $500 and benefit is 25%, 50% and 75% improvement (in quality or quantity) variables. You ask more than one choice question. SurveyMonkey spits out the data with a number code which you assume is ordinal. For example:

Cost	Code
$100	1
$250	2
$500	3

But actually you find that the code is not in order. For example:  

Cost	Code
$100	3
$250	2
$500	1

Similar things happen to the benefit variable. There is no pattern to the number code assignment across choice question. So, when coding the variable in your statistical software you must compare the the qualitative data (don't forget your labels John Whitehead!) to the quantitative data. 

SurveyMonkey recently "upgraded" their product and this quirk seems to be an unintended result. 

*For example, no scope effect! BP-funded economists are going to hop on that train big-time.

The cost of offshore drilling safety regulations seem low relative to the costs

In case you missed it over the holiday break:

The Trump administration is poised to roll back offshore drilling safety regulations that were put in place after the 2010 Deepwater Horizon oil rig disaster in the Gulf of Mexico that killed 11 people and caused the worst oil spill in American history.

A proposal by the Interior Department’s Bureau of Safety and Environmental Enforcement, which was established after the spill and regulates offshore oil and gas drilling, calls for reversing the Obama-era regulations as part of President Trump’s efforts to ease restrictions on fossil fuel companies and generate more domestic energy production.

Doing so, the agency asserted, will reduce “unnecessary burdens” on the energy industry and save the industry $228 million over 10 years. ...

Industry groups like the American Petroleum Institute have long opposed the safety rules, calling them “flawed and costly.” The fossil fuel interest group warned in 2015 that the regulations would reduce capital investment in the Gulf by $4 billion a year and threaten 50,000 industry jobs. ...

The Obama-era rules, written in 2016, tightened controls on blowout preventers, devices that are intended to stop explosions in undersea oil and gas wells, and called for rig operators to have third parties certify that the safety devices worked under extreme conditions. In the Deepwater Horizon spill, a supposedly fail-safe blowout preventer failed after a section of drill pipe buckled.

As always with industry estimates you need to put them in context (ignoring the need to recognize that industry cost estimates are usually exaggerated to avoid regulations). The U.S. Energy Information Administration says that Gulf of Mexico oil production is 1.6 million barrels per day (2016). If the price per barrel is $65, then that leads to revenues of about $37 billion annually. The cost of the Obama-era rules is $22.8 million per year, about 0.06% of revenues or $456 for each lost job. The regulatory costs seem low to me, especially considering that the BP Deepwater Horizon blowout lead to 11 deaths and damages of $17.2 billion. 

Note, that in contrast to API's warning that 50,000 industry jobs were at risk in 2015, the EIA said that "Oil production in federal Gulf of Mexico expected to continue increasing."

What am I missing?

A short comment on a discrete choice experiment in the AER*

A cool new paper is the first (?) to use a discrete choice experiment in the AER:

Mas, Alexandre, and Amanda Pallais. 2017. "Valuing Alternative Work Arrangements." American Economic Review, 107(12): 3722-59.

Abstract: We employ a discrete choice experiment in the employment process for a national call center to estimate the willingness to pay distribution for alternative work arrangements relative to traditional office positions. Most workers are not willing to pay for scheduling flexibility, though a tail of workers with high valuations allows for sizable compensating differentials. The average worker is willing to give up 20 percent of wages to avoid a schedule set by an employer on short notice, and 8 percent for the option to work from home. We also document that many job-seekers are inattentive, and we account for this in estimation.

I might be in too deep (ya think?) but there are some questionable statements made on the quality of stated preference data. Before I go on, let me emphasize that this is a cool paper. Stated preference research can push the frontier in many fields besides environmental economics. The problem is that environmental economic stated preference researchers must run the 110 high hurdles while this paper gets entered into the 100 dash (to use a track analogy). 

On page 3724:

In this paper we report estimates of worker valuations over alternative work arrangements from a field experiment with national scope. The experiment elicits preferences on work arrangements by building a simple discrete choice experiment into the application process for a national call center. In this way we employ a method that can flexibly back out a willingness to pay (WTP) distribution from close to real market transactions.⁹

I don't consider something a "field experiment" unless there is a real monetary transaction.

Here is footnote 9: 

Discrete choice experiments are an extension of the contingent valuation literature whereby rather than directly asking people for valuations over an attribute (the stated preference method), people are given the choice of two or more scenarios and are asked to choose their preferred option. These scenarios usually vary the attributes and the prices and WTP can be estimated using random utility models (McFadden 1973; Manski 1977). Choice experiments have been shown to have better properties relative to stated preference valuation methods (Hanley, Wright, and Adamowicz 1998). A question is whether these experiments, which are usually survey-based, correspond to actual market behavior. This is something we can overcome by embedding the choice in a real market setting. Diamond and Hausman (1994), who critique stated preference valuation methods, hypothesize that the problem with the approach is not methodological but due to “an absence of preferences” over the attributes they are being asked to value. This is far less of a concern here since we are asking people to make choices over realistic work arrangements.

Is this the sort of lazy assertion that flies in labor economics papers at the AER? It wouldn't fly at JEEM, JAERE, Land Econ, Env and Res Econ, Res and Energy Econ, MRE, and etc (basically any environmental and resource economics journal). 

So:

• "Choice experiments have been shown to have better properties relative to stated preference methods" Huh? See "CVM surveys suck so we're using choice experiments, Q.E.D. (#174)".
• A "discrete choice experiment" is one type of "stated preference method" and choice experiments have lots of problems just like contingent valuation. See "What is Contingent Valuation?".
• Market experience does not eliminate hypothetical bias. See Morgan et al. (2016) for the case of oyster demand. The same problem can be found in the context of trip taking decisions and voting. 

Here is one of the hypothetical situations:

This question might be inconsequential (i.e., your choice will not affect anything whatsoever). If so, there is little reason for respondents to take it very seriously. See Groothuis et al. (2017).

The authors collect more data in online survey. Here is one scenario (parenthetical added):

Imagine that you are applying for a new job in your [current line of work, same line of work as your last job], and you have been offered two positions. Both positions are the same as your [current/last] job in all ways, and to each other, other than the work schedule and how much they pay. Please read the descriptions of the positions below. 

Position 1) This position is 40 hours per week. The work schedule is Monday–Friday 9 am–5 pm. This position pays the same as your [current/last] job.

Position 2) This position is 40 hours per week. The work schedule in this position varies from week to week. You will be given your work schedule one week in advance by your employer. The hours can be morning through evening, weekdays and weekends, but not nights. This position pays “X” (e.g., "15% more than") your [current/last] job.

Which position would you choose?

Another hypothetical question for those having a "flexible job" is:

Suppose your primary employer gives you the option of working a fixed work schedule, Monday-Friday during the daytime. Under this arrangement you would continue to work your usual number of hours but once your schedule is set you may not change the times and days of work. In exchange for having this fixed rather than flexible schedule you would get [2/5/10/20/35]% higher pay. Would you agree to this arrangement if given the choice?

Labor markets are characterized by negotiation. It is difficult to imagine that every survey respondent did not see these questions as the opening round of a negotiation. In both questions, for example, the respondent might be saying I choose position 2 but will only take the job if I can negotiate higher pay. Market researchers face the same issues in a market setting. If you are asked to purchase a new and improved product at a higher price you might answer no, thinking that the product might be offered anything and your signal will result in a lower price. 

These are issues that stated preference folks get hassled about over and over again. In contrast, Ivy leaguers write a dismissive footnote and get an AER**. Ok, the whine is over. 

 

---

*I've been enjoying my semester long sabbatical from blogging (and teaching and dressing like a grown-up) ... until this. 

**And they compare their "field experiment" surveys to the online survey results to deal with a referee who thinks that Diamond's adding up test is a paper killer. Here is footnote 43:

Diamond (1996) recommends testing for internal consistency in contingent valuation surveys. We go further in Section IV by comparing WTP estimates in the market setting to estimates from a nationally representative survey.

Do you think BP is going to let you get past the adding up test by asking some more hypothetical questions? I think not!

What is contingent valuation?

Hey fisheries/econ people, is contingent valuation really on the way out? What's replacing it?

— Michael C. Russell (@ProfRussellCIE) October 20, 2017

I'd be interested how @johnwhitehead81 and @tim_env_econ would answer this question.

— Ngaio Hotte (@nhotte) October 21, 2017

I went on a rant (the kids call it a "tweet storm") and since I don't know how to thread my tweets (and I don't want to lose them when Congress shuts down twitter) I thought I'd post them here (with spell check turned on). Tim was also in on it. This happened after I presented a paper using "discrete choice experiment" (DCE) data in Tim's department (that was fun!) so the thing was on our minds.

Here we go:

• John: Yes, CVM is out. And “discrete choice experiments” are in. And, they are the same thing (under certain conditions)!
• John: They are the same thing when the chioce is a for/against referendum with varying cost and benefit of a movement away from the status quo.
• Tim: CV is a DCE with variation in one attribute (typically) and the price. DCE is more flexible but subject to potentially more biases.
• John: I’m not sure why folks are describing CVM as something other than DCE, other than to try to avoid BP funded criticism.
• Tim: To understand the criticism, follow the $.
• John: Given all the CVM nonsense, you’re kinda stupid to call your study CVM and not DCE (call me stupid)
• Tim: For fun, let’s write a paper called “Scope Effects in DCE” and just use one attribute and price and see if anyone notices.
• John: First, write the CVM paper and send to Land Econ (or Env Res Econ) and get rejected because of Hausmann, Diamond and McFadden 1/n
• Tim: Then global replace “scope” with “attribute” and “CVM” with “DCE” and get R&R at Env Res Econ (or Land Econ) 2/n
• John: I honestly believe that’s how dumb it’s become 3/n
• John: I’m working on an R&R where a referee says you can’t (i.e., it’s not allowed) to ask a follow up CVM question anymore 4/n
• John: We explicitly test for changes/biases with the second question 5/n
• John: Research is about trying new things and discovery, not following the rules of the authority figures 6/n
• John: The NRDA context is very different than RESEARCH, the consultants and those who read them uncritically don’t care to differentiate 7/7

Here is the brief off-the-top-of-my-head history of thought:

• Mitchell and Carson (Using Surveys to Value Public Goods, RFF Press, 1989) define CVM as a hypothetical yes/no response to a randomly assigned cost variable. Variations in attributes of the policy/program (aka "scope") is one internal validity test.
• Carson and Louviere (Env and Res Econ, 2011) define referendum CVM with a scope test as a special case of DCE (as in the John/Tim tweets above). Payment card and open ended elicitation questions are CVM only. 
• Carson and Czajkowski (chapter 9 in Hess and Daly 2014) make the point in Carson and Louviere a bit more clear
• Zawojska and Czajkowski (J Env Econ Policy 2017) take it one step too far and say that referendum CVM with a scope test is DCE and not CVM.

Who Knows What Willingness to Pay Lurks in the Hearts of Men?*

From the inbox:

The new issue of Econ Journal Watch is online at http://econjwatch.org.

In this issue:

Who Knows What Willingness to Pay Lurks in the Hearts of Men? John Whitehead argues that certain authors claim to know, based on their survey data, more than they do about people’s willingness to pay for a wetlands project. The article continues an exchange from the Journal of Environmental Economics and Management.

Here is the abstract:

The present article is the fourth piece in a sequence of articles, the three previous having appeared in Journal of Environmental Economics and Management. In JEEM, Kevin Egan, Jay Corrigan, and Daryl Dwyer published an article with findings from their survey of Ohio residents, collecting contingent valuation data for a wetlands project. They elicit willingness to pay (WTP) with annual and one-time payment schedules in dichotomous choice contingent valuation method questions and compare these to consumer surplus estimates derived from the travel cost method. Using a conservative nonparametric estimator, they conclude that WTP from the annual payment schedule is a “better match” with consumer surplus estimates. Combined with two other analyses, Egan, Corrigan, and Dwyer conclude that “these three arguments make a strong case for matching the duration of a [contingent valuation] survey’s payments with the duration of the proposed benefits, which will most often mean presenting survey respondents with ongoing annual payments” (2015, 134). Dissatisfied with their strong claims, I showed that the contingent valuation method data is of relatively low quality and, with an alternative nonparametric estimator, showed that WTP from the one-time payment schedule is a better match with the consumer surplus estimates. Given the low-quality data, the range of WTP estimates is too wide to draw any conclusions about the most appropriate payment schedule (Whitehead 2017a). In a reply, the authors defend the theoretical validity of their data by appealing to the statistical significance of the entire bid schedule. They conduct three sensitivity analyses, and write that they “overwhelmingly support our original conclusion in favor of using annual payments in” contingent valuation surveys (Egan, Corrigan, and Dwyer 2017). Here, I use parametric approaches to measure WTP and conduct a more complete sensitivity analysis. Egan, Corrigan, and Dwyer’s conclusion that the annual payment results “better match” the consumer surplus estimates is not supported by my analysis. This paper reinforces my initial contention that their contingent valuation data is not useful for the payment schedule issue.

Here is the article: Whitehead, John C., Who Knows What Willingness to Pay Lurks in the Hearts of Men? A Rejoinder to Egan, Corrigan, and Dwyer, Econ Journal Watch 14(3):346–361, September 2017. https://econjwatch.org/file_download/1036/WhiteheadSept2017.pdf?mimetype=pdf

*The Shadow knows ... a paper title suggested by EJW.