Is US efficiency score really that bad? Questioning ACEEE’s International EE Scorecard

Zondits enjoys the reports put out by ACEEE. In our opinion they do very good work with the best intentions, and we fully support their efforts. But friends should challenge friends at times; the recent release of ACEEE’s International EE Scorecard constitutes one of these times.

Developing the International EE Scorecard was and will continue to be a worthwhile effort on their part. Naming and shaming has done wonders with the US state-based scorecard. The same tactic underlies many EE behavioral initiatives, such as Local Law 84 in New York City, which publishes benchmarking scores for all large buildings. There is bipartisan agreement that the US can do better when it comes to energy efficiency, and nothing pushes politicians to act like telling them we’re worse than China, India, and South Korea, and just one step better than Russia! That said, after more than 30 years of highly successful and entrenched efficiency programs, is the US energy efficiency approach really worse than so many of these countries’ as the International EE Scorecard suggests? In fact, for the past 10 to 20 years China has been receiving considerable help in efficiency program planning and delivery from US organizations, including assistance from the Lawrence Berkeley National Laboratory, and also ACEEE.

The Zondits staff has read the ACEEE report cover to cover, and in thinking it over, we are left with a few nagging questions:

When so many attributes are combined into single numbers, do those numbers lose their meaning?
When the report clearly and repeatedly notes all of the shortcomings of their own methodology for so many of the measures, which we all know will never get reported with the resulting numbers, do the numbers become misleading?
And when the analysis by design excludes a host of city and regional building energy compliance laws, is the final score an accurate reflection of real performance?

Developing a multi-national comparison of energy efficiency initiatives is no small task. The challenges of data and information are numerous. However, a report of this stature surely deserves some questioning, and we invite additional comments and response. Here are just a few of our questions:

Is there enough accurate data to even compare?

We all know that EE data is imperfect, and we accept that we have to live with the data we have. But our question here is driven mostly by the selection of countries to compare. The US and Europe have a long history of tracking energy data, and as good as those numbers are we all know that sometimes these can be sketchy. If we then add in China, India, and Russia—not to disparage their efforts—can the data really be compared? We of course trust the IEA’s and especially the ACEEE’s ability to account for that, but the other countries’ sources and rationale should raise eyebrows, especially when the US is ranked behind China and a baby step ahead of Russia.

How can city, state or regional efforts be included?

The report is biased toward laws and goals set at a national or federal level. This favors the top-down approaches employed in Europe and China and, more importantly, handicaps the US model that devolves much of the responsibility to regional, state and local levels. The report appears not to account for initiatives like the laws in New York City and other large American cities outlawing fuel oil or mandating energy audits and benchmarking, yet they impact a huge percentage of the GHG emissions for the largest population centers in the US. Likewise, we must assume that regional initiatives like the RGGI are also not accounted for. Of course, the US is not alone in having regional or municipal ordinances of this sort, but the omissions likely skew the US score downward. We were pleased to see that state level efficiency programs have been captured through the “utility spending” metric; these should be regarded as significantly as comprehensive national efforts since these are the major basis of US efficiency best practices, as has been reported in other ACEEE documents. Future iterations of the International EE Scorecard should find a way to account for city and regional efforts as well.

Why no metric for total energy intensity?

GDP is used to normalize some variables, but full GDP-factored energy intensity is only addressed for the rate of change. At the same time, the industrial and agricultural segments get full measure for energy intensity. Together this results in a perverse bias; as noted in the report, India and China consume 3 times and 2.5 times, respectively, as the US on a total energy intensity basis, yet they are deemed more energy efficient. In search of an explanation for that, we would note that China’s high score in change has at least as much to do with prior excess and bad practices as current reform, and the scores of both India and China are seemingly given a boost by factors such as oxen-driven agriculture. Here again, we think the selection of metrics skews the US relative position downward for reasons not related to efficiency.

As a final rhetorical question, by the ACEEE report’s own admission, due to differences in the standard of living the residential and commercial building metrics for energy intensity do not equate to efficiency – so then why are they included in a ranking for energy efficiency? We understand fully that measuring and comparing energy use in building stock is no easy task, but if simply removing all heating and cooling in buildings can yield a higher score, then the metric as a measure of efficiency is flawed.

Of course, none of our questions means that the ACEEE report should not be written. It just needs to improve and continue to evolve, and clear biases need to be eliminated. Such biases make us believe that the real objective of ACEEE’s report is the shaming of the US and an associated prompt for the US federal government (clearly the core focus of ACEEE) to develop an efficiency-focused energy policy. While Zondits fully supports the efforts to achieve a national consensus toward greater energy efficiency in the US, facts and sensible results should not be compromised in an effort to achieve these objectives. We look forward to future years’ reports – and to seeing improvement in the US and the world over.