Title:
A Credible Approach to Benefit-Cost Evaluation for Federal Energy Technology Programs
A Credible Approach to Benefit-Cost Evaluation for Federal Energy Technology Programs
Author(s)
Jordan, Gretchen B.
Ruegg, Rosalie
Ruegg, Rosalie
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Lawmakers want to know the returns on their investment. The difficulties in measuring benefits and costs, and attributing some portion of those to specific research and technology development (R&D) programs, has meant that many such studies are not seen as credible by those lawmakers. This paper describes a methodology that improves upon an already credible approach developed for a 2001 National Research Council study: "Energy Research at DOE: Was It worth It?" Three benefit-cost studies using this modified approach will be completed by the U.S. Department of Energy's Energy Efficiency and Renewable Energy in 2009. The key evaluation questions to be answered in these studies are: * To what extent have programs produced "actual" economic benefits (energy-savings, renewable market growth, and other positive economic effects) relative to the next best alternative? * To what extent have programs promoted environmental benefits and enhanced energy security by providing alternative energy sources and energy efficiency, and protecting existing sources? * Would today's commercialized technologies have happened at the same time, with the same scope and scale, and with the same extent of deployment without DOE involvement? * To what extent do benefits attributable to DOE involvement exceed DOE R&D expenditures? The modified methodology will implement the retrospective benefits estimation recommendations of the May 2002 EERE Strategic Technical Review and the DOE Benefits Workshop held in 2003. Economic performance metrics that are calculated are Net benefits, Benefit-cost ratio, and Internal Rate of Return. All of these are social and public returns that include private returns but these are not calculated separately. The types of benefits and costs included in calculation of metrics are Investment costs, Energy costs, Other operating costs, Maintenance, repair, and replacement costs, and Rents or royalty payments. Benefits and costs for selected technology "winners" are calculated compared against the next best alternative. Differences are expressed as year-by-year dollar cash flows, adjusted to constant dollars, and amounts are "discounted" for the time value of money (apart from inflation) using OMB-specified real discount rate. Additionally, a "Cluster approach" is used that compares benefits of larger elements of a program to investment costs of the entire program. The cluster approach describes the larger program and government's role, as well as the specific technology elements. It generates a minimum estimate of return for the program without performing detailed analysis of everything in it, thereby increasing feasibility of the analysis. Environmental and Security benefits are also assessed. Environmental benefits quantified will focus on reduction in air pollution. Security benefits will focus on cumulative fuel savings (oil and natural gas), expressed in physical terms. Important benefits that cannot be measured quantitatively within the scope of a study are discussed qualitatively. Knowledge Benefits were added to the NRC framework by experts attending the 2003 DOE Benefits Conference. This EERE benefit-cost methodology calls for a more comprehensive and quantitative assessment of knowledge benefits of the technologies selected for study than have previous benefit-cost studies. Studies identify and document linkages between more than three decades of DOE R&D and, in the 2009 studies, commercial renewable power generation. Studies use interviews with scientists, engineers, and administrators in government, as well as manufacturers and their customers. They also review documents and databases for evidence of linkages, and apply the tools of patent and publication citation analysis, and publication co-author analysis to shed further light on the paths through which outputs of DOE-funded R&D have been disseminated to producers and users. In contrast to the 2001 NRC study, the modified approach requires a case-by-case assessment of additionality - the difference that DOE made in the outcome. Rather than use a rule of thumb, the EERE modified NRC approach will examine an array of ways additionality can occur. These include acceleration in the development and introduction into the market of a technology, change in the scale, performance characteristics, function, and cost of a technology (e.g., its emissions profile or its durability), acceleration in the rate at which a technology gains share of the existing domestic market after it is initially introduced, increase in access of U.S. firms to foreign markets, expansion in the total size of domestic and foreign markets, and reduction in the total investment costs (combined public and private) of achieving the desired outcome as compared with achieving the same with industry going it alone. To have more comparability across EERE benefit-cost studies, this approach will use generic logic models to describe the changes over the time period in technologies and markets using a common language. Activities, external influences, and progress over time in R&D will use the stages of the Stage Gate process: Preliminary investigation, detailed investigation, development, validation, and commercial launch. Activities, external influences, and progress over time in the "readiness" of the market and technology adoption will use four market domains. In addition to the usual domain of the end user that looks at adoption of a technology, there are three market infrastructure domains: Information, Policy/Government, and Business. Examples of activities in these infrastructure domains are the Wind Resources database, EERE efforts on new appliance standards, and partnerships with industry on solar manufacturing R&D.
Sponsor
U.S. Department of Energy
National Oceanic and Atmospheric Administration
Canadian Academy of Health Sciences
National Oceanic and Atmospheric Administration
Canadian Academy of Health Sciences
Date Issued
2009-10-02
Extent
Resource Type
Text
Resource Subtype
Proceedings