Desvousges, Mathews and Train (2015) [DMT] find that their contingent valuation method (CVM) survey does not pass the adding up test. In a previous post (Whitehead, Sept. 20, 2016) I examined the data used by Desvousges, Mathews and Train (2015) and find that "it is ... not clear if [the data] passes the most basic validity test in contingent valuation over 89% of the range of bids."
There are other things wrong with the paper (see also cognitive burden, cost). DMT says this about what we said about the theory:
Haab et al. (2013, 10) state that the adding-up test imposes additional structure on preferences beyond that imposed by the scope test and that the additional structure is unnecessary. For the scope test, they say, “A simple theoretical model of WTP, a difference in expenditure functions with changes in quality or quantity, can be used to show that WTP is nondecreasing in quality or quantity (Whitehead, Haab, and Huang 1998).” The same theoretical model, with differences in expenditure functions (as described above), is all that is needed to show the adding-up condition. The assumptions that Whitehead, Haab, and Huang (1998) use to show nonnegative scope effects for the scope test are sufficient to show the adding-up condition for the adding-up test. No additional assumptions or structure is required.
The basic scope test theory defines WTP(1) and WTP(1+2) and shows that WTP(1+2) > WTP(1). The difference is WTP(2). Estimating WTP(2) is where the additional structure on preferences must be imposed during the survey. If one tries to elicit WTP(1) and WTP(2) independently, as in DMT (2015), it is likely that WTP(1) + WTP(2) > WTP(1+2) due to substitution and income effects, which is the result in DMT (2015).
In order to accurately elicit WTP(2) one would need to describe the provision of WTP(1) and how its provision has reduced the income/budget of the survey respondent. WTP(2) is actually WTP(2|1, B-A) where 2|1 means 2 given the provision of 1, B is the budget for environmental goods and A is the amount of money taken from the respondent to pay for provision of 1 [1]. This counterfactual imposes additional cognitive burden and I'm pessimistic that a CVM survey (or a discrete choice experiment) can demonstrate adding up (it hasn't been done in lab experiments with real money and market goods).
DMT (2015) don't describe B-A to survey respondents but conduct a simulation to show that income effects are small. The simulation isn't convincing because respondents may have a budget for environmental goods, Y > B where Y is income, and the payment for good 1 (i.e., A) takes a larger percentage from their budget than it doesn't from their income. The amount A might use up their environmental goods budget leading to WTP(2|1, B-A) < WTP(2, B). I can't find any evidence that DMT (2015) attempt to describe 2|1 to respondents to deal with potential substitution effects.
Finally, if the theory in Whitehead, Haab and Huang (1998), which isn't original with us, is sufficient for an adding up test then Diamond (1996) would not have been necessary.
Note:
[1] This description of WTP(2|1, B-A) is in Whitehead (2016).
References
Desvousges, William, Kristy Mathews, and Kenneth Train. “An Adding Up Test on Contingent Valuations of River and Lake Quality.” Land Economics 91(2015): 556-571. http://le.uwpress.org/content/91/3/556.refs
Diamond, Peter. "Testing the internal consistency of contingent valuation surveys." Journal of Environmental Economics and Management 30, no. 3 (1996): 337-347.
Whitehead, John C. "Plausible responsiveness to scope in contingent valuation." Ecological Economics 128 (2016); 17-22. http://www.sciencedirect.com/science/article/pii/S0921800916302890
Whitehead, John C., "A comment on '“An Adding Up Test on Contingent Valuations of River and Lake Quality,'” The Environmental Economics Blog, September 20, 2016. http://www.env-econ.net/2016/09/a-comment-on-an-adding-up-test-on-contingent-valuations-of-river-and-lake-quality.html
Whitehead, John C., Timothy C. Haab, and Ju-Chin Huang. "Part-whole bias in contingent valuation: Will scope effects be detected with inexpensive survey methods?." Southern Economic Journal (1998): 160-168.
