In the case of mercury rules:
The Environmental Protection Agency’s top clean-air official aggressively pushed back Wednesday against claims that upcoming pollution regulations will cause so many power plant closures that the country could face power outages. ...
[Gina McCarthy, assistant administrator at EPA’s Office of Air and Radiation] criticized recent studies that claim EPA regulations will cause reliability issues, arguing they are based on incorrect assumptions about the agency’s rules.
The studies, McCarthy said, are inaccurate because they assume the regulations are more expensive than they are, include plants that will be shut down regardless of the EPA rules because they are old or inefficient and don’t take into account other tools for ensuring reliability.
“These types of worst-case assumptions, when not clearly described as more stringent than EPA’s rules, can generate more confusion than insight,” she said.
“[M]any of these studies do not make it clear that they are looking at an extreme case that does not reflect EPA’s actual rules and, as a result, overstates potential impacts.”
McCarthy specifically criticized a study released this week by the North American Electric Reliability Corporation (NERC), an industry group charged with developing reliability standards.
The study assumes “that every uncontrolled coal unit will install the most expensive controls available to meet the mercury and air toxics standards requirements,” McCarthy said. “I think we all know that this isn’t what will happen.”
The NERC study said EPA’s regulations “may significantly affect bulk power system reliability depending on the scope and timing of the rule implementation and the mechanisms in place to preserve reliability.”
Worst-case scenarios are just that. If everything that can go wrong does go wrong then something very bad will happen. However, there is little chance that all that stuff will go wrong.
There are several ways to conduct sensitivity analysis when doing economics. One is to assess the base case, best case and worst case. Objective analysts should present all three. Unfortunately, interest group motivated analyses tend to present either the best case or worst case, depending on their goals, analyses. Beware those studies.
Another, more thorough type of sensitivity analysis is Monte Carlo simulation. Monte Carlo simulation takes random draws over all uncertain values in the best-worst case scenarios. A distribution of values can be developed and, not surprisingly, the best and worst case scenarios are found to be not likely.
Here is the simple example I use in my benefit-cost analysis class (Download Monte_Carlo_simulation). There are three estimates of benefits and costs:
| Benefits | WTP | POP | TB |
| Best Case | 6.5 | 250000 | $ 1,625,000 |
| Base Case | 4.5 | 175000 | $ 787,500 |
| Worst Case | 2.5 | 100000 | $ 250,000 |
| Costs | Q | AC | TC |
| Best Case | 150 | 1290 | $ 193,500 |
| Base Case | 225 | 1990 | $ 447,750 |
| Worst Case | 350 | 2690 | $ 941,500 |
| Net Benefits | TB | TC | NB |
| Best Case | $ 1,625,000 | $ 193,500 | $ 1,431,500 |
| Base Case | $ 787,500 | $ 447,750 | $ 339,750 |
| Worst Case | $ 250,000 | $ 941,500 | $ (691,500) |
where WTP is willingness to pay, POP is the affected population, TB is total benefits (TB = WTP*POP), Q is the quantity purchased, AC is average cost and TC is total costs (TC = Q*AC). The best case scenario net benefits is $1.432 million and the worst case scenario is -$691.5 thousand.
Conducting a Monte Carlo simulation with 100 draws from uniform distributions for POP, Q and AC and a normal distribution for WTP (mean = 4.5, sd = 2) leads to a distribution of net benefits that looks like this:
The mean net benefit is $308,845 with a standard deviation of $444,396. Trimming the lower and upper five values gives a 90% confidence interval of [-$338,354, $1,191,952] indicating there is little chance that the best case or worst case scenarios will play out.
Consumers of biased studies put out by interest groups should demand sensitivity analysis.
