2011 KPIA Response – 4.0 Analysis and Assessment
Response of the National Round Table on the Environment and the Economy to its Obligations under the Kyoto Protocol Implementation Act - July 2011
| Previous | Table of Contents | Next |
Summary of key KPIA improvements in response to NRTEE recommendations
The government has taken steps to improve its emissions forecasts substantially since the first KPIA Plan in 2007. We summarize key improvements in the Plans that have responded to NRTEE recommendations as follows:
In its 2007 Response, the NRTEE recommended that the government report on measures using an integrated approach to account for interaction effects between programs that could result in an overstatement of emissions reductions. Also in its 2007 Response, the NRTEE recommended that future KPIA Plans include greater transparency and clarity related to key assumptions and methods. Beginning with the 2008 and subsequent Plans, the government Plan has responded to these recommendations by using an integrated modelling framework and including greater detail on its modelling assumptions.
In its 2008 Response, the NRTEE recommended that only impacts of announced and reasonably expected provincial actions be included in KPIA modelling. In its 2009 Response, the NRTEE recommended that the Plan estimate emissions reductions in terms of projected changes in Environment Canada’s GHG Inventory. The NRTEE also recommended that future KPIA Plans reflect both emissions forecasts and actual emissions data documented in the Environment Canada GHG Inventory. The government responded to these recommendations in its 2010 Plan. The integrated modelling in the 2010 Plan used a consistent definition for an emission reduction, namely the difference between a forecast including federal policies and programs, and a forecast excluding these measures. This approach also served to more transparently account for the effect of provincial policies. Actual emissions were shown for 2008 and compared to the integrated modelling forecasts, providing context for actual emissions in the Kyoto period.
In its 2010 Response, the NRTEE recommended that emissions reductions for each individual measure be consistently estimated as the difference in emissions between a policy case and a reference case. The 2011 Plan now includes assumptions for a reference case for each measure. The NRTEE made this recommendation as an approach to accounting for additionality in the measure-by-measure estimates. While assumptions for the reference case are still not fully consistent throughout all estimates in the Plan, estimates for some programs, like the ecoENERGY for Retrofit Initiative, have for the first time accounted for additionality.
The 2010 NRTEE Response also recommended that the government could further improve transparency by making publicly available the details of both a reference case and policy scenario in the context of Canada’s longer-term emissions targets. The NRTEE notes that in January 2011, Environment Canada published overviews of reference case and policy scenarios to 2020 on its climatechange.gc.ca website.
The NRTEE’s 2011 analysis and assessment examines key improvements in the 2011 KPIA Plan as well as remaining issues in the forecasts.
Improvements in the 2011 Plan
The 2011 Plan improves on the previous Plan and explicitly addresses some of the NRTEE’s 2010 recommendations. Overall, the 2011 Plan is more transparent than the 2010 Plan, and the changes made since 2010 improve the reliability of some estimates and provide greater clarity. We outline these improvements here, highlighting a general improvement in transparency across all the measure-by-measure estimates as well as specific improvements in two individual measure-by-measure estimates.
The methodology presented in the Plan to estimate measure-by-measure emissions reductions is more transparent.
Language throughout the measure-by-measure forecasts clearly defines the reference case assumptions — that is, what the analysis assumes would have occurred in the absence of any Additional detail in the parameters used to estimate measure-by-measure reductions further improves transparency. For example, in assessing the impact of Renewable Fuels Regulations, the 2011 Plan includes more detail with regard to emissions factors used to calculate impacts, and the Plan acknowledges that some level of imported biofuels from the U.S. would be required to meet the regulation during the first three years to allow for growth in Canadian capacity. The 2010 NRTEE Response raised the issue of accounting for trade effects in calculating emissions reductions. Similarly, the 2011 Plan provides more detail on the capacity factors, expected production, and expected emission reductions under the ecoEnergy for Renewable Power program.
The 2011 Plan improves the reliability of estimated emissions reductions from the ecoENERGY Retrofit Initiative using the results of ex-post (after the fact) program effectiveness evaluation to better account for additionality and consumer behaviour, as recommended by the NRTEE in previous KPIA Responses.
For the ecoENERGY Retrofit Initiative, the Methodological Annex in the 2011 Plan notes that “a 2010 evaluation that covered the elements of this program concluded that net-to-gross ratios for their impacts ranges from 0.26 to 0.84.” [iv] The net-to-gross ratio measures the degree to which previous estimates predicted actual emissions reductions. This evaluation [v] assessed the impacts of free-ridership, rebound, and additionality on the effectiveness of the retrofit program, as the NRTEE recommended in each previous KPIA Response. It indicated that these factors led to fewer emissions reductions under the program than previously expected. As a result, estimated emissions reductions from this program were revised down, and the NRTEE considers the updated forecast to be a reliable estimate of the program impacts.
The updates to the estimates for the ecoENERGY Retrofit Initiative demonstrate the value of on-going evaluation of program effectiveness. The government could use these evaluations both to refine emissions reduction estimates for existing programs and to inform estimates for future initiatives. For example, the NRTEE’s assessments of the ecoENERGY Retrofit program in previous KPIA Responses referenced program effectiveness analysis from the similar EnerGuide for Houses program [vi] as evidence that the estimates for the ecoENERGY Retrofit Initiative should account for free-ridership and rebound effects. Similarly, the 2011 Plan documents the use of an ex-post program evaluation for assessing forecasts under the ecoAUTO Rebate Program. Closing the loop between program evaluation and previous forecasts of expected emissions reductions in this fashion leads to improvements of forecasting processes over time.
The 2011 Plan provides an alternative methodology for estimating emissions reductions from an individual measure in the assessment of the Passenger Automobile and Light Truck GHG Regulations.
In previous KPIA Responses, the NRTEE suggested that the integrated modelling framework could be used to further explore the incremental effects of individual policies and programs. The NRTEE proposed that integrated modelling could assess how much emissions would increase if each individual measure were removed from the full suite of programs and policies in order to assess its incremental contribution to emissions reductions. The 2011 KPIA Plan explores this approach in assessing the Passenger Automobile and Light Truck GHG Regulations as an alternative assessment in the Methodological Annex of the Plan.
Environment Canada published the emissions projections under its reference case and policy scenarios to 2020.
In its 2010 Response, the NRTEE recommended that the government “make publicly available the details and underlying assumptions for both a consistent reference case and a policy scenario.” While not part of the KPIA process, Environment Canada responded to this recommendation by making these two scenarios available on its www.climatechange.gc.ca website in January 2011. While the scenarios do not include sector- and regional-level detail, they do increase transparency in the government’s assessment of its climate change policies. These scenarios also respond to NRTEE recommendations by providing longer-term estimates of Canadian emissions and emissions reductions under current federal policies.
Remaining Issues in the 2011 Plan
A key issue identified in previous NRTEE Responses remains in the 2011 Plan. While some of the measure-by-measure estimates have been improved in 2011, the NRTEE notes that questions remain regarding the degree to which estimates account for additionality (estimates for some measures do while others do not). Partially as a result of these issues, substantial inconsistency emerges in the Plan between measure-by-measure estimates and the integrated modelling; while some inconsistency should be expected, the measure-by-measure estimates aggregated together result in approximately double the emissions reductions estimated using the integrated modelling.
While the 2011 Plan better defines reference case assumptions for each measure-by-measure estimate, for some measures — including the ecoENERGY for Renewable Power, ecoENERGY for Buildings and Houses, ecoENERGY for Industry, and Pulp and Paper Green Transformation programs — the Plan requires further evidence to support these assumptions.
In its 2010 KPIA Response, the NRTEE stated:
In order to ensure consistency across all forecasts in the KPIA Plan, including the integrated modelling and measure-by-measure assessments from various government departments, the NRTEE recommends: a) that emissions reductions projections for each program and measure in the Plan be consistently estimated as the difference in expected GHG inventory emissions between a reference case without the measure in place, and a policy case with the measure in place; and-, b) that consistent assumptions be used to define the reference case across all estimates.
As noted above, the 2011 Plan has partially met this recommendation. It now explicitly and consistently estimates emissions reductions as the difference in estimated emissions under a reference case and policy case, meeting the first part of the NRTEE recommendation. However, assumptions underlying the reference cases used in some of the measure-by-measure estimates must be further supported with evidence in the Plan. In the ecoENERGY for Renewable Power program, for example, the Plan explicitly states, “The program considers that all projects funded are incremental and would not have been done without program support.” [vii] As the NRTEE noted in its 2010 Response, this assumption is likely not reliable. Five projects, accounting for 370 megawatts (MW), received financing through the program despite being commissioned prior to the announcement of the program. [viii] Further, provincial and municipal policies and programs also provide support to renewable power projects. As a result, some additionality concerns remain in this estimate, making it a likely overestimate of emissions reductions from this measure. Appendix A provides more detailed assessment of estimates for individual policies and programs.
Definitions for Additionality and Rebound Effect
Additionality issues refer to challenges in differentiating actions by firms and households to reduce emissions that are induced by a policy from actions that they would have taken even if that policy had not been implemented. Some firms and households choose to reduce their emissions based on existing preferences and market signals, while others would take action only if and when the policy is in place. For example, some households would install new windows even if the government did not provide incentives for making homes more efficient. Estimated emissions reductions from such a government incentive should include only the incremental, or additional, households that choose to make the renovation as a result of the government program to avoid double counting.
The related problem of free-ridership arises when stated reductions include the actions of industry and households that are rewarded but not influenced by the policies. Free-ridership occurs when subsidies are paid to all firms or individuals that take an emissions-reducing action, regardless of whether they did so because of the subsidy. Those who would have taken the action regardless are termed free riders, and their behaviour has already been accounted for in the reference case. Not correcting for this issue implies that induced emissions reductions will be over-estimated by the proportion of free-riders.
The Rebound Effect describes the increased use of a more efficient product. For example, more energy efficient cars are cheaper to drive and so people may drive them more.
Similar to previous NRTEE KPIA responses, significant differences exist between the total emissions reductions predicted on a measure-by-measure basis and the emissions reductions predicted through integrated modelling. The NRTEE supports the forecasts from the integrated modelling in the Plan as reliable, as they account for policy interaction, additionality, and rebound effects. However, when summed together, we would expect estimates from the individual policies and programs to lead to comparable estimated emissions reductions with those reported from the integrated modelling approach. Part of this discrepancy results from the additionality and free-ridership issues in some of the measure-by-measure estimates.
Figure 2 illustrates the difference between the two sets of emissions reduction projections based on a comparison of the two approaches to estimation. Curve A in the figure is calculated by adding up the reductions attributed to each individual measure in the measure-by-measure estimates contained in the 2011 Plan. Collectively, these estimates suggest approximately 18 megatonnes (Mt) of emission reductions for 2012 are attributed to the full set of programs and policies included in the 2011 Plan. Curve B represents the estimated overall emission reductions from all programs and policies in the Plan as calculated by the integrated modelling. As the figure illustrates, the 2011 Plan forecasts that when implemented together, all the programs and policies in the Plan will result in about 9 Mt of emissions reductions in 2012. By the end of the Kyoto period, the cumulative emissions reductions are forecast to be 54 Mt on an individual measure-by-measure basis but only 27 Mt on an integrated modelling basis.
Figure 2: Comparison of Total Emissions Reductions Attributed to Individual Measures in the 2011 KPIA Plan and Emissions Reductions of these Measures Projected Using Integrated Modelling, 2008–2012
(Data from or derived from 2011 KPIA Plan; both estimates include all policies and programs reported in the 2011 Plan)
[D]
As set out in previous Responses, the NRTEE acknowledges that some of this discrepancy between the integrated modelling and the program-by-program analysis is unavoidable due to policy interaction effects. In particular, as noted by Environment Canada, the integrated modelling simultaneously accounts for changes in the emissions intensity of energy supply as well as changes in energy consumed, while the measure-by-measure analysis isolates these effects, not taking both into account. For example, the ecoENERGY for Renewable Power program will likely reduce the emissions intensity of electricity supply. As a result of this change, the effect of other programs that reduced energy consumption — for example, of the energy efficiency regulations — would be dampened. While the integrated modelling would account for this effect, the measure-by-measure assessments do not, thereby likely overestimating the expected emissions reductions estimated for some individual measures.
Further, some differences between the forecasts could result from inconsistent assumptions with respect to international trade. The measure-by-measure estimate for the ecoENERGY for Renewable Power program assumes that all new generation displaces existing generation, while the integrated modelling accounts for electricity trade, so new generation could also displace imported power or additional generation could be exported. Given the uncertainty with this issue, either assumption could be legitimate, but the assumption will affect the estimates.
Overall, the discrepancy between the integrated modelling and the measure-by-measure estimates is too large to result only from policy interaction effects. The NRTEE analysis of the estimates for individual policies and programs in the Plan — as presented in Appendix A — suggests that a substantial share of this discrepancy is likely also due to inconsistent consideration of free-ridership, other additionality issues, and rebound effects, which lead to likely overestimates of emissions reductionsm for some individual measures. [6]
____________________
[6]It is important to recognize that individual programs and policies can have multiple policy objectives beyond emissions reductions, including providing public information, incenting technology adoption, or acting as a catalyst for other policy goals. In the context of the KPIA, however, we are assessing only one projected outcome of policies — forecasted emissions reductions.
| Previous | Table of Contents | Next |
© 2013