Environmental Economics

Reply to "Reply to Whitehead" by Desvousges, Mathews and Train: (4) My treatment of the weighted WTP is biased in favor of the DMT (2015) result/conclusion

DMT (2020) draw attention to my treatment of the weighted WTP estimates. The regression model for the second scenario has a negative sign for the constant and a positive sign for the slope. When I "mechanically" calculate WTP for the second scenario it is a positive number which adds weight to the sum of the WTP parts. This is in contrast to the unweighted data for which WTP is negative. Inclusion of the data from this scenario biases the adding-up tests in favor of the conclusion that the WTP data does not pass the adding-up test. 

The motivation for my consideration of the weighted data was DMT's (2015) claim that they found similar results with the weighted data. My analysis uncovered validity problems with two of the five scenarios which, when included in a adding-up test, led to a failure to reject adding-up. At this point in the conversation it will be instructive to visually examine the weighted data to see if it even passes the "laugh" test. In my opinion, it doesn't. 

Below are the weighted votes and theTurnbull for the whole scenario (note that the weights are scaled to equal to sub-sample sizes). The dots and dotted lines represent the raw data. Instead of a downward slope, these data are "roller-coaster" shaped (two scary hills with a smooth ride home). The linear probability model (with weighted data) has a constant equal to 0.54 (t=9.73) and a slope equal to -0.00017 (t=-0.69). This suggests to me that the whole scenario data, once weighted, lacks validity. While lacking validity, the solid line Turnbull illustrates how a researcher can obtain a WTP estimate with data that does not conform to rational choice theory. The Turnbull smooths the data over the invalid stretches of the bid curve (the "non-monoticities" using the CVM jargon) and the WTP estimate is the area of the rectangles. In this case WTP = $191 which is very close to the unweighted Turnbull estimate. But, a researcher should consider this estimate questionable since the underlying data does not conform to theory. As a reminder, the WTP for the whole scenario is key to the adding up test as it is compared to the sum of the parts. The WTP estimate from linear logit model is $239 with the Delta Method [-252, 731] and Krinsky-Robb [-8938, 9615] confidence intervals. Given the statistical uncertainty of the WTP estimate, it is impossible to conduct any sort of hypothesis test with these data. 

Below are the weighted votes and the (pooled) Turnbull for the second scenario. The dots and dotted lines represent the raw data. Instead of a downward slope, these data are "Nike swoosh" shaped. The linear probability model (with weighted data) has a constant equal to 0.13 (t=2.46) and a slope equal to 0.00107 (t=4.19). This suggests to me that the second scenario data, once weighted, lacks validity. Again, the Turnbull estimator masks the weakness of the underlying data. In this case, the Turnbull is essentially a single rectangle. With pooling the probability of a vote in favor is equal to 28.06% for the lower bid amounts. With pooling the probability is 27.56% for the higher bids. The Turnbull WTP estimate is $112 which appears to be a reasonable number, hiding the problems with the underlying data. 

DMT reestimated the full data model with the cost coefficients constrained to be equal. In a utility difference model the cost coefficient is the estimate for the marginal utility of income. There is no reason for marginal utility of income to vary across treatments unless the clean-up scenarios and income are substitutes or complements. This theoretical understanding does not explain why the weighted models for the whole and second scenarios are not internally valid (i.e., the cost coefficient is not negative and statistically different from zero). The model that DMT refer to passes a statistical test, i.e., the model that constains the cost coefficient to be equal is not worse statistically than an unconstrained model, but it should be considered inappropriate due to the lack of validity in the weighted whole and second scenario data sets. Use of the model with a constrained cost coefficient amounts to hiding a poor result. The reason that the weighted model with the full data set takes the correct sign is because the scenarios with correct signs outweigh the scenarios with incorrect or statistically insignificant signs. The reader should attach little import to DMT's (2015) claim that their result is robust to the use of sample weights. 

Reply to "Reply to Whitehead" by Desvousges, Mathews and Train: (2) What is the effect of negative WTP?

When dichotomous choice CVM data is of low quality, the measure of central tendancy is sensitive to assumptions. As I showed in a paper presented earlier this year (Landry and Whitehed 2020), with the highest quality data it makes no difference the WTP estimator that is used. The Turnbull, Kristrom, linear logit (under both zero WTP assumptions), and linear probability models all produce the same estimate. 

As data quality falls, however, the choice of WTP estimate can matter a great deal. In this situation, so as to avoid sponsor and other biases, it is important for the CVM researcher to present the full range of WTP estimates and avoid the impression that results have been cherry picked. This range of WTP provides a more complete depiction of analyst uncertainty and allows for sensitivity and other analyses.

I have grown accustomed to intense suspicion whenever I see hypothesis tests conducted with only the Turnbull WTP estimate. First, it is a lower bound WTP estimate and potential differences are minimized. Second, its standard errrors are smaller (relative to the mean) than parametric WTP estimates. This second observation is due to the way that the standard errors are calculated and to the fact that the data are smoothed when there are non-monotonicities. As Haab and McConnell (1997, p 253) explained (emphasis added): "We demonstrate that the Turnbull model ... provides a straightforward alternative to parametric models, so long as one simply wants to estimate mean willingness to pay." When hypothesis tests are being conducted, a range of WTP estimates should be used to determine if the results are robust to estimation method. 

So, is it reasonable to include the linear-in-bid parametric model in this collection of WTP estimates? Hanemann (1984, 1989) showed that in a linear utility model, U = a(Q) + bY where Q is a good and Y is income, the mean (and median) willingness to pay is WTP = -a*/b, where a* is the change in utility from changes in Q and b is the marginal utility of income. One benefit of this estimate is that it is insensitive to fat tails. However, this estimate allows for negative WTP values unless the probability of a yes response to a dichotomous choice question is 100% when the bid amount is zero. Negative WTP values can enter into the analysis in two ways. First, the WTP estimate itself can be negative. This will occur when the probability of a yes response at the lowest bid amount is less than 50% (it is this possibility that, I think, motivated Haab and McConnell). The second possibility is that the empirical distribution of WTP can include negative values. This is of little consequence to the analysis unless the confidence interval includes zero. Both circumstances arise with the DMT (2015) data. 

DMT (2020) dismiss outright the possibility of negative WTP. Their dismissal is consistent with Haab and McConnell's argument that since public goods are freely disposable, negative WTP is only an empirical artifact of a distributional assumption. But, with government policy free disposal is not always possible. In the case of a clean up of natural resource damages, the clean up could be considered a wasteful intrusion into a private business decision. Bohara, Kerkvliet and Berrens (2001) discuss how and why negative WTP values might arise, along with empirical examples. Considering this, I would not be surprised if some of the respondents to CVM scenarios demanded compensation for environmental cleanup.  

There have been a number of suggestions about how to handle negative willingness to pay. Many of these involve obtaining more data with follow-up questions (Landry and Whitehead). Unfortunately, the DMT (2015) survey data does not have any of this supplemental information. In that case, in my opinion, an assumption that negative WTP is a possibility can not be ruled out. Inclusion of the linear model allowing for negative WTP, as long as it is presented along with other estimates, should not be dismissed outright. 

DMT (2020) state: "This means that adding-up passed in his calculations on linear models not because of the data but because of his implausible additional assumption that many people have a negative WTP for the environmental programs." It is not true that the linear model finds that "many" people have negative willingness to pay in each of the scenarios. According to the Krinsky-Robb WTP simulation, the percentage of negative WTP values for the whole, first, second, third, and fourth scenarios in the DMT (2020) data are 2%, 0.01%, 77%, 25% and 0.83%. The WTP from the second scenario is negative (situation 1 above). The WTP from the third scenario has a Delta method confidence interval that includes zero (situation 2 above). 

If the negative mean WTP from the second scenario is set equal to zero then the difference in WTP for the whole and the sum of its parts is statistically significant at the p=0.088 level with the Delta Method confidence intervals. The Krinsky-Robb confidence interval is [68, 788] which includes the sum of the WTP for the parts with WTP from the second scenario set equal to zero ($467) indicating that the adding-up test is supported. It is still my contention that the adding-up passed in the (untruncated) linear model not because of the data.

My conclusion is that the negative WTP values do not have an important effect on the adding-up tests. Dismissing these tests because negative WTP values are implausible ignores the literature and the empirical evidence. 

References

Bohara, Alok K., Joe Kerkvliet, and Robert P. Berrens. "Addressing negative willingness to pay in dichotomous choice contingent valuation." Environmental and Resource Economics 20, no. 3 (2001): 173-195.

Landry, Craig, and John Whitehead, "Estimating Willingness to Pay with Referendum Follow-up Multiple-Bounded Payment Cards," paper presented at the 2020 W-4133, Athens, GA, February. 

Haab, Timothy C., and Kenneth E. McConnell. "Referendum models and negative willingness to pay: alternative solutions." Journal of Environmental Economics and Management 32, no. 2 (1997): 251-270.

Hanemann, W. Michael. "Welfare evaluations in contingent valuation experiments with discrete responses." American journal of agricultural economics 66, no. 3 (1984): 332-341.

Hanemann, W. Michael. "Welfare evaluations in contingent valuation experiments with discrete response data: reply." American journal of agricultural economics 71, no. 4 (1989): 1057-1061.

 

Reply to "Reply to Whitehead" by Desvousges, Mathews and Train: (1) Is the log-linear model meaningless?

When dichotomous choice CVM data has a negative WTP problem, one of the standard corrections is to estimate a log-linear model and present the median WTP. With many estimated log-linear models the mean WTP is undefined. This is because the log-linear model flattens the estimated survival curve and, in contrast to a linear model, the probability of a no response does not approach zero at any reasonable bid amount. The median WTP estimate from the log-linear model tends to be a useful supplement to the welfare measures available from the linear model. 

Desvousges, Mathews and Train (2020) rightly argue that (1) the sum of medians is not equal to the median of the sums and (2) the mean WTP estimates for each of their five scenarios are basically infinity (or in the millions of dollars when the data are estimated with a log-linear probit). Given this empirical fact, there seems to be no median estimate available for the sum of the four individual WTP amounts (that could be compared to the median of the whole). However, DMT (2020) are able to estimate the "median of the sum of WTPs through simulation" to be $4904. They then explain that since $4904 is 24 times that of the median of the whole scenario, $201, is "clearly a violation of adding up". Given that the confidence interval from the Delta Method, [-47, 449], does not include $4904 we could reject the notion that the willingness to pay estimates pass this version of the adding-up test. 

Previously however, I've argued that these standard errors are likely the wrong ones to use since the cost parameter is measured without much precision. In this case the Krinsky-Robb confidence intervals are more appropriate. The Krinsky-Robb confidence interval for the median WTP estimate for the whole scenario is [54, 9558]. Since the median of the sum of the WTP estimates from the four adding-up scenarios, estimated by DMT (2020) to be $4904, lies within the 95% Krinsky-Robb confidence interval then we fail to reject the adding-up hypothesis at the 95% confidence level. In contrast, the 95% Krinsky-Robb confidence interval estimated with the whole scenario data from Chapman et al. (2009), which I've argued is higher quality data, is relatively tight around the median of $167: [134, 217]. 

The only other adding-up test that can be conducted with median WTP estimates is to compare the median for the whole with the sum of the medians for the four parts. In this test I found that the median WTP estimates passed the adding up test using the confidence intervals from the Delta Method (Whitehead 2020). It still seems to me that this test is a useful supplement when one is inclined to consider the robustness of the adding-up test conducted with only the Turnbull estimator (as in DMT 2015). Otherwise, we're treating mean WTP estimates in the millions as if they are meaningful. 

To answer the question in the title of this post, the log-linear model is not meaningless. In fact, it is a better model statistically than the linear model for the first and third WTP scenarios in DMT (2015). Information gleaned from the log-linear model provides insights into the quality of the DMT (2015) data. 

While I argue with DMT over the minutiae of these tests and different estimators, the reader shouldn't lose site of how silly the debate over DMT (2015) has become. The bottom line is that the DMT (2015) data is of low quality data and do not rise to the threshold that is needed to support an adding-up test, which requires estimates of willingness to pay as ratios of coefficients.

Reply to "Reply to Whitehead" by Desvousges, Mathews and Train: (6) Are the DMT (2015) data problems typical in other CVM studies?

In Whitehead (2020) I describe the problems in the DMT (2015) data. It is full of non-monotonicities, flat portions of bid curves and fat tails. A non-monotonicity is when the percentage of respondents in favor of a policy increases when the cost increases. In other words, for a pair of cost amounts it appears that respondents are irrational when responding to the survey. This problem could be due to a number of things besides irrationality. First, respondents may not be paying close attention to the cost amounts. Second, the sample sizes may be simply too small to detect a difference in the correct direction. Whatever the cause, non-monotonicities increase the standard errors of the slope coefficient in a parametric model.

Flat portions of the bid curve exist when the bid curve may be downward sloping but the slope is not statistically different from zero. This could be caused by small differences in cost amounts and/or it is due to sample sizes that are too small to detect a statistically significant difference. For example, there may be little perceived difference between a cost amount of $5 and $10 compared to $5 and $50. And, even if the percentage of responses in favor of a policy is economically different between two cost amounts, this difference may not be statistically different due to small sample sizes. 

Fat tails may exist when the percentage of respondents who are in favor of a policy is high at the highest cost amount. However, this is only a necessary condition. A sufficient condition for a fat tail is when the percentage of respondents who are in favor of a policy is high at two or more of the highest cost amounts. In this case, the fat tail will cause a parametric model to predict a very high cost amount that drives the probability that respondents are in favor of a policy to (near) zero. A fat tail will bias a willingness to pay estimate upwards because much of the WTP estimate is derived from the portion of the bid curve when the cost amount is higher than the cost amount in the survey. 

DMT (2020) state that these problems also occur in Chapman et al. (2009) and a number of my own CVM data sets. They are correct. But, DMT (2020) are confusing the existence of the problem, in the case of non-monotonicity and flat portions, with the magnitude of the problem. And, they are assuming that if the necessary condition for fat tails exists then the sufficient condition also exists. Many, if not most, CVM data sets will exhibit non-monotonicities and flat portions of the bid curve. But, these issues are not necessarily an empirical problem. The extent of the three problems in DMT (2015) is severe -- so severe that it makes their attempt to conduct an adding up test (or any test) near impossible.

To prove this to myself I estimated the logit model, WTP value and 95% Krinsky-Robb confidence intervals for 20 data sets. Five of the data sets are from DMT (2015), 2 are from Chapman et al. (2009) and 13 are from some of my papers published between 1992 and 2009 (DMT (2020) mention 15 data sets but two of the studies use the same data as in another paper). The average sample size for these 20 data sets is 336 and the average number of cost amounts is 5.45. The average sample size per cost amount is 64, which is typically sufficient to avoid data quality problems (a good rule of thumb is that the number of data points for each cost amount should be n > 40 in the most poorly funded study).

These averages obscure differences across study authors. The average sample size for the DMT (2015) data sets is 196. With 6 cost amounts the average sample size per cost amount is 33. The Chapman et al. (2009) study is the best funded and the two sample sizes are 1093 and 544. With 6 cost amounts the sample sizes per cost amount are 182 and 91. The Whitehead studies have an average sample size of 317 and with an average of 5 cost amounts, the sample size per cost amount is 65 (the variance of these means are large). Already, differences across these three groups of studies emerge. 

There are a number of dimensions over which to compare the logit models in these studies. My preferred measure is the ratio of the upper limit of the 95% Krinsky-Robb confidence interval for WTP to the median WTP estimate. This ratio will be larger the more extensive is the three empirical problems mentioned above. As this problem worsens, hypothesis testing with the WTP estimates (again, a function of the the ratio of coefficients) becomes less feasible. It is very difficult to find differences in WTP estimates when the confidence intervals are very wide. To suggest that this measure has some validity, the correlation between the ratio and the p-value on the slope coefficient is r = 0.96. 

The results of this analysis are shown below. The ratio of the upper limit of the confidence interval to the median is sorted from lowest to highest. The DMT (2015) values are displayed as orange squares, the Chapman et al. (2009) values are displayed as green diamonds and the Whitehead results are displayed as blue circles and one blue triangle. The blue triangle is literally "off the chart" so I have divided the ratio by 2. This observation, one of three data sets from Whitehead and Cherry (2007), does not have a statistically significant slope coefficient. 

Considering the DMT data, observation 19, with a ratio of 4.82 (i.e., the upper limit of the K-R confidence interval is about 5 times greater than the median WTP estimate), is the worst data set. Observation 8, the best DMT data set, has their largest sample size of n=293. The Chapman et al. (2009) data sets are two of the three best in terms of quality. The Whitehead data sets range from good to bad in terms of quality. Overall, four of the five DMT data sets are in the lower quality half of the sample (defined by DMT*). 

Of course, data quality should also be assessed by the purpose of the study. About half of the Whitehead studies received external funding. The primary purpose of these studies was to develop a benefit estimate. The other studies were funded internally with a primary purpose of testing low stakes hypotheses. In hindsight, these internally funded studies were poorly designed with sample sizes per bid amount too small and/or poorly chosen bid amounts. With the mail surveys the number of bid amounts was chosen with optimistic response rates in mind. With the A-P sounds study a bid amount lower than $100 should have been included. Many of the bid amounts are too close together to obtain much useful information. 

In contrast, considering the history of the CVM debate and the study's funding source (Maas and Svorenčík 2017), the likely primary purpose of the DMT (2015) study is to discredit the contingent valuation method in the context of natural resource damage assessment. In that context, the study is very high stakes and, therefore, its problems should receive considerable attention. The DMT (2015) study suffers from some of the same problems that my older data suffers from. The primary problem with the DMT (2015) study is that the sample sizes are too low. It is not clear why the authors chose to pursue valuation of 5 samples instead of 3 to conduct their adding up test (DMT (2012) describe a 3 sample adding up test with the Chapman et al. (2009) study). Three samples may have generated confidence intervals tight enough to conduct a credible test. 

In the title of this post I ask "Are the DMT data problems typical in other CVM studies?" This subtitle should really be 'Are the DMT data problems typical of Whitehead's CVM data problems in a different era? The survey mode for my older studies was either mail of telephone. Both survey modes were common back in the old days but they have lost favor relative to internet surveys. The reasons are numerous but one is that internet surveys are much more cost-effective and the uncertainty about a response rate is non-existent. Another reason is that internet survey programming is much more effective (with visual aids, piping, ease of randomization, etc). Many of the problems with my old data was due to small sample sizes. This was a result of either poor study design (in hindsight, many CVM studies with small samples should have reduced their bid amounts) or unexpectedly low mail response rates.

It is not clear why DMT (2020) chose to compare their data problems to those that I experienced 15-30 years ago. Unless, in a fit of pique at my comment on their paper, they decided it would be a good idea to accuse me of hypocrisy. I've convinced myself that my data compares favorable to the DMT (2015) data. Especially considering the goals of the research. My goals were more modest than testing whether the CVM method passes an adding-up test for which a WTP estimate (the ratio of two regression coefficients) is required (as opposed to considering the sign, sign and significance of a regression coefficient).

*****

*Note that there are more Whitehead data sets than are called out by DMT. I haven't had time to include all of these into this analysis. But, my guess is that the resulting circles would be no worse than those displayed in the picture below. 

Reference

Maas, Harro, and Andrej Svorenčík. "“Fraught with controversy”: organizing expertise against contingent valuation." History of Political Economy 49, no. 2 (2017): 315-345.

 

The Information Value of Pigouvian Pricing

I recently read an article in the journal Economics and Philosophy, written by Lisa Herzog, which has nothing whatsoever to do with environmental economics but nonetheless I think has interesting implications for it and for Pigouvian pricing in particular.  

In case you are unfamiliar with it, the journal Economics and Philosophy is a scholarly journal publishing articles on topics related to (wait for it) economics and philosophy.  And, often, the philosophy of economics.  This is an area that I have become interested in recently, influenced no doubt by my years as an undergraduate philosophy major.  

In the article, Herzog addresses and basically dismantles Hayek's model of the price mechanism in markets as being an efficient way of processing and distilling information necessary for buyers and sellers.  Hayek's idea is basically that the world is a complicated place, and markets have lots of complicated information about costs, benefits, demands, etc.  But, once markets determine a price for something, that price contains all of the information that buyers and sellers need to make their optimal decision. Herzog quotes Hayek: markets process "dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess." 

Herzog's argument is that Hayek is ignoring important pieces of knowledge that buyers and sellers need to make morally informed choices and that are missing from the price signal.  Without this information, or "epistemic infrastructures, market participants do not act as morally responsible agents: when acting in such markets, it often seems fair to say that they do not know what they are doing."  The motivating example is buying clothes made in sweatshops overseas.  The price mechanism contains lots of information about production and transportation costs, etc., but it is missing important moral information, like the conditions of the workers and their outside opportunities.  Without this information buyers cannot make morally complete (and therefore optimal) decisions.

What does this have to do with environmental economics?  Of course, when there are negative externalities (like pollution), the unregulated market price will not reflect those externalities and thus will not provide buyers and sellers the necessary information to make optimal choices.  This is not new; this is Pigou, and even the most die-hard Hayekian would admit that market failures like externalities need to be incorporated somehow into the price signal to achieve efficiency. (Aside: maybe it's not actually Pigou, according to this working paper by my colleague Spencer Banzhaf.) 

But I think there is a deeper and less obvious point than that.  Supposing that there is a Pigouvian price on pollution that accounts for its negative externalities.  Does this price still contain the necessary "epistemic infrastructure" for all buyers and sellers to make morally responsible decisions?  If the Pigouvian price is "right" then it accounts for the spillover costs, but it's not obvious that these costs are all that is morally necessary to make informed choices.   Just like with sweatshops, there potentially are moral considerations above and beyond any cost considerations (even spillover costs) that need to be incorporated.  It it not clear that a Pigouvian price would incorporate any of this. 

Anyways, some Pigouvian price on pollution is still no doubt better than no Pigouvian price on pollution, but there is potentially a strong argument to be made here based on Herzog's article that any Pigouvian price can't get us enough information to make morally optimal choices.