From the outset, some critics leveled three arguments against these programs. Those arguments are:

• If these energy efficiency measures are really so beneficial, then consumers would adopt the measures on their own.
• The methods used to evaluate these programs are flawed.
• There are insufficient evaluation data to demonstrate that energy efficiency programs are cost-effective.

Perhaps the most-cited example of these arguments is Paul Joskow and Donald Marron's 1992 Energy Journal article "What Does a Negawatt Really Cost?"

Immediately following the appearance of that article, and in the quarter-century since, energy efficiency program supporters have responded to those arguments. For example, in the same journal in 1994, Amory Lovins authored "Apples, Oranges, and Horned Toads: Is the Joskow & Marron Critique of Electric Efficiency Costs Valid?" In another article in that journal in 1996 titled "The Total Cost and Measured Performance of Utility-Sponsored Energy Efficiency Programs," Joseph Eto et al. examined 20 resource-oriented utility programs and confirmed the cost effectiveness of those programs and their viability as a utility resource option.

Rebutting the Arguments

Those responses and many years of subsequent field testing and program evaluation have persuaded energy regulators. Total annual utility spending on energy efficiency programs has increased seven-fold since 1996. However, given that the old criticisms have resurfaced in recent years, this brief article offers some updated responses.

Consumer choices? / With respect to the argument that consumers would adopt energy-saving measures on their own if the measures were truly efficient, the obvious response is, "Then why haven't they?" There is plenty of cost-effective energy efficiency improvement available to be captured by energy efficiency programs year after year.

Considerable research has identified the market failures and obstacles to customer implementation of energy efficiency measures, e.g., lack of information, lack of easy access in the local market, lack of capital, etc. For examples of this literature, see the 2015 U.S. Department of Energy Report to Congress Barriers to Industrial Energy Efficiency and the Lawrence Berkeley National Laboratory report Market Barriers to Energy Efficiency by William Golove and Joseph Eto.

Flawed research? / With respect to the criticisms of evaluation methods, some of those concerns had some validity in the early days of energy program planning (e.g., over-reliance on ex-ante engineering projections, failure to account for "free-riders" who would have adopted the efficiency measures without the programs). But practitioners have subsequently recognized those concerns and taken steps to address them. For example, there are 75 peer-reviewed papers on the International Energy Program Evaluation Conference (IEPEC) website (www.iepec.org) about free-ridership. Evaluators now routinely take free-ridership into consideration when evaluating programs.

Similarly, evaluators have contributed feedback to improve ex-ante engineering expectations in program planning, which has led to less divergent estimates between the ex-ante and ex-post estimates. Most importantly, at this point professional practice within the evaluation industry would never simply claim ex-ante engineering estimates as the reported energy savings from a program without any ex-post analysis and verification.

More recently, critics of energy efficiency programs have argued that in order to produce sufficient evidence of the programs' benefits, a true randomized experiment (where subjects are randomly assigned to treatment and control conditions) must be conducted. Practitioners in the field know that because of practical constraints and regulatory concerns about customer access, it is seldom possible to randomly assign customers to receive a program. Instead, the program evaluation profession uses a variety of technically sophisticated quasi-experimental methods to try to answer important policy questions using the best empirical evidence. Such methods are widely used in many other professions, from advertising to education to mental health. While we support the greater use of randomized experimental design in the evaluation of energy efficiency programs, to suggest that anything short of a true randomized experiment is not methodologically sufficient is a poor criticism and ignores the limitations of such a methodology. Our view is echoed by the State and Local Energy Efficiency Action Network, which provides guidance and recommendations on methodologies that can be used for estimating energy savings resulting from energy efficiency programs.

Insufficient data? / A particularly objectionable criticism of energy efficiency programs that has rarely but occasionally been raised is that evaluations that show utility energy efficiency programs are cost-beneficial must somehow be biased. This claim clashes with the seriousness of program evaluation professionals. IEPEC has been providing training and conducting conferences on program evaluation for over 30 years, with hundreds of peer-reviewed professional papers being published and cited in regulatory proceedings as well as the academic literature. The IEPEC website provides free access to all its published papers since 1997.

Utility regulators also take their jobs very seriously and offer another buttress against the claim that evaluations of energy efficiency programs produce biased results. States universally require utility energy efficiency programs to pass cost-effectiveness tests (with the exception of low-income programs, which are justified by equity considerations). Nearly all states require that evaluations be conducted by independent contractors rather than by utility staff. Proposed utility programs and utility energy efficiency program results are typically examined in contested case proceedings where all interested parties are free to challenge those results. (One of the authors of this paper was the director of evaluation at a utility regulatory commission for 10 years and would argue that, dollar for dollar, no other area of utility expenditures receives as much scrutiny as energy efficiency programs.)

As for the question of whether "government mandates" are necessary to achieve utility energy efficiency programs, there is a fundamental market failure at play here. Despite extensive evidence that energy efficiency programs are much less expensive than building, fueling, and operating power plants and delivering that energy through extensive transmission and distribution networks, utilities inherently would rather sell more energy than less. Absent requirements and regulatory mechanisms such as decoupling and performance incentives, utilities simply would not provide—and historically have not provided—serious energy efficiency programs for their customers. Utilities are regulated monopolies that operate under all sorts of government mandates in exchange for their franchise. So, if energy efficiency programs are in the public interest (e.g., they reduce total costs to customers for the utility system), it is entirely appropriate for government to require and/or incentivize utilities to provide these programs.

Finally, as to the fundamental question of whether these utility energy efficiency programs are a good value, the evidence is overwhelming. There are literally thousands of individual reports documenting the effects of these programs (many of which are cited in the IEPEC archives noted above). This extensive analysis is itself testimony to the fact that utility regulatory commissions require extensive scrutiny of energy efficiency programs. How many independent evaluation reports have been required and published for other utility expenditures, from bucket-trucks, to transformers, to billing systems, to company management structures, etc.?

For a good overview of utility energy efficiency results, comprehensive summaries are available from the American Council for an Energy Efficient Economy (ACEEE) as well as from the Lawrence Berkeley National Laboratory (LBNL). In a 2018 ACEEE analysis, Maggie Molina and Grace Relf examined results for the 49 largest electric utilities in the United States and found an average cost of saved electricity of 3.1¢ per kilowatt-hour. A previous ACEEE report by Molina summarized the results from 10 different states across four years of programs and found an overall utility cost of 2.8¢/kWh saved for electric programs and 35¢ per therm (100,000 Btu) saved for natural gas programs. An earlier 2009 ACEEE study by Katherine Friedrich et al. across 10 states for electricity and six states for natural gas found average costs of 2.5¢/kWh and 37¢/therm. A 2018 LBNL study by Ian Hoffman et al. examined the cost performance of 8,790 electricity efficiency programs between 2009 and 2015 for 116 investor-owned utilities and other program administrators in 41 states and found an average cost of 2.5¢/kWh saved. A 2014 LBNL study by Megan Billingsley examined over 100 programs across 31 states over a three-year period and found an average total utility cost of 2.1¢/kWh and 38¢/therm.

All of these average costs are well below the utility system avoided cost for delivered electricity and natural gas, demonstrating clearly that energy efficiency is indeed a cost-effective utility resource. This is true even in an era of very low well-head natural gas prices. (And the risks associated with that era being temporary is a whole other subject.) Furthermore, this assessment of energy efficiency program value does not attempt to incorporate the value of any additional benefits commonly associated with energy efficiency programs, such as improved customer health and safety, business productivity and operation and maintenance savings, or reduced environmental emissions.

Conclusion

The best available evidence robustly demonstrates that utility energy efficiency programs have been a cost-effective utility resource. The public policies that require and encourage these programs are well-justified, both conceptually as a response to market failures and monopoly power, as well as empirically given the demonstrated cost-effective results. Regulators should of course continue to exercise good oversight, but there is no basis for abandoning the policy and regulatory framework that facilitates these programs.

Readings

• "A National Survey of State Policies and Practices for the Evaluation of Ratepayer-Funded Energy Efficiency Programs," by Martin Kushler, Seth Nowak, and Patti Witte. ACEEE Research Report U122, 2012.
• "Apples, Oranges, and Horned Toads: Is the Joskow & Marron Critique of Electric Efficiency Costs Valid?" by Amory B. Lovins. The Electricity Journal 7(4): 29–49 (May 1994).
• Barriers to Industrial Energy Efficiency: Report to Congress. U.S. Department of Energy, June 2015.
• "Does Efficiency Still Deliver the Biggest Bang for Our Buck? A Review of Cost of Saved Energy for US Electric Utilities," by Maggie Molina and Grace Relf. Proceedings, ACEEE Summer Study, 2018.
• "Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs: Issues and Recommendations," by Annika Todd, Elizabeth Stuart, Steven R. Schiller, and Charles A. Goldman. Lawrence Berkeley National Laboratory, 2012.
• "Experimentation and the Evaluation of Energy Efficiency Programs," by Edward Vine, Michael Sullivan, Loren Lutzenhiser, et al. Energy Efficiency 7: 627–640 (2014).
• Market Barriers to Energy Efficiency: A Critical Reappraisal of the Rationale for Public Policies to Promote Energy Efficiency, by William H. Golove and Joseph H. Eto. Energy & Environment Division, Lawrence Berkeley National Laboratory, March 1996.
• "Saving Energy Cost-Effectively: A National Review of the Cost of Energy Saved through Utility-Sector Energy Efficiency Programs," by Katherine Friedrich, Maggie Eldridge, Dan York, et al. ACEEE Research Report U092, 2009.
• "The 2018 State Energy Efficiency Scorecard," by Weston Berg, Seth Nowak, Grace Relf, et al. ACEEE Research Report U1808, 2018.
• "The Best Value for America's Energy Dollar: A National Review of the Cost of Utility Energy Efficiency Programs," by Maggie Molina. ACEEE Research Report U1402, 2014.
• The Cost of Saving Electricity through Energy Efficiency Programs Funded by Utility Customers: 2009–2015, by Ian M. Hoffman, Charles A. Goldman, Sean Murphy, et al. Lawrence Berkeley National Laboratory, 2018.
• The Program Administrator Cost of Energy Saved for Utility Customer-Funded Energy Efficiency Programs, by Megan A. Billingsley, Ian M. Hoffman, Elizabeth Stuart, et al. Lawrence Berkeley National Laboratory, 2014.
• "The Total Cost and Measured Performance of Utility-Sponsored Energy Efficiency Programs," by Joseph Eto, Edward Vine, Leslie Shown, et al. The Energy Journal, 17(1): 31–51 (1996).
• "What Does a Negawatt Really Cost? Evidence from Utility Conservation Programs," by Paul Joskow and Donald B. Marron. The Energy Journal 13(4): 41–74 (1992).