Internal Audit Report Demand Side Management - BC Hydro · Audit and Risk Management ......

April 19, 2002

To:

G. MagelG.D. HenriquesA.W. LeonardK.J. McDonaldR.E. MarchantB. Van RuyvenC.S. ThomasG.N. SherlockB.G. EltonL.I. BellM. CostelloAudit and Risk Management Committee

AU0215MP&A

Audit Services

Internal Audit Report

Demand Side Management

Prepared by:C.D. Eng

G. ParmarR. Spellman (GDS Associates)

Internal Audit Report Page 1

Executive Summary

Power Smartq Since its inception as BC Hydro’s demand side management function in the late 1980’s, Power

Smart has evolved from a peak of activity in 1992, to a move away from incentives in 1995 inanticipation of retail access and competition, to a low level of activity in 1998 with theimplementation of a retail strategy, to 2001 when supply tightened and the cost of gas andmarket prices arose creating market opportunities for BC Hydro and it’s customers.

q In 2001, recognizing Power Smart as its number one resource for new energy, BC Hydromoved to begin implementing residential, commercial and industrial initiatives and to develop aten year plan. The plan was completed in November, 2001 and requires an investment of $600million to achieve an ongoing annual level of savings of 3,500 GWhs.

q At the close of fiscal 2002, several programs were introduced and implemented includingSUCH (for Schools, Universities, Colleges and Hospitals), a TLP (traffic light program), CCQ(for the Courtenay Comox and Quesnel communities), CFL (compact fluorescent light program)and an energy savings rewards program (e-points). Power Smart management also set anenergy savings target of 120 GWhs and is striving to surpass that goal for F2002.

q For fiscal 2003, plans and targets for each customer sector (i.e. Industrial, Government,Institutional, Property Management & Retail, High Tech & Manufacturing, and Residential) havebeen set to achieve a target run rate of 360 GWhs for the year. The focus will be on theexpansion of residential initiatives throughout Vancouver Island, business initiatives to thelargest customers, and on evaluating the results of the Comox and Quesnel activities.

Auditq Recognizing the magnitude of the investment, an audit of the ten year plan and the

implementation activity and processes should provide the Board of Directors with overallassurance that the plan incorporates the necessary supporting business processes anddiscipline to achieve key business plan objectives.


Executive SummaryAudit Findingsq While aggressive, the plan’s goal to invest $600 million over ten years to achieve an annual

rate of savings of 3,500 GWh is reasonable as it is within the range of savings achieved byother demand side management programs in North America.

q The plan provides an appropriate level of strategic direction for BC Hydro’s DSM Programs.During fiscal 2002, Power Smart has made significant progress with plan development andimplementation of several DSM programs (i.e. LED, SUCH, CFL).

q Power Smart is established in most areas of the financial management plan (i.e. accountingprocesses & procedures, multi-year funding for programs, and program financing) and therisk management plan. Additionally a defined incentive payment process has been developedwith appropriate controls, although no payments have currently been made under this newlydeveloped incentive contract.

q Continuous improvement is needed in several planning, research, monitoring andimplementation areas. Valuation of DSM savings (benefit/cost analyses), tracking andmeasuring GWh acquired, business model, scheduling, resourcing and reporting all requirefurther attention.

q Evaluation plans and measurement & verification plans have only been addressed in acursory fashion at this point and require significant development. The input, accuracy, andsupport of the energy savings tracking system are also other areas under development andrequire further attention.

Management’s Responseq Management have reviewed the audit report and agree that it is a fair reflection of the plan

and implementation activities to date.q Management concur with the finding and recommendations of the audit and have provided a

response for each recommendation with a commitment to act within a set timeframe.


Executive Summary

AUDIT AREA

Degree ofCompletion /Development Ranking

RiskLevel for

Plan COMMENTS

DSM Planning

Financial Management Plan

Valuation of DSM Partially High Demand savings are not reported at the program level. Total Resource Cost(TRC) calculation for overall Power Smart program undervalues benefits by $545million (benefits truncated after 2012). Long term market effects on the benefits ofthe DSM program are not included in calculation of the TRC Test. Avoidedcapacity cost for generation in Long Run Marginal Cost (LRMC) appears to beconservative.

Accounting for DSM programs

Processes and Procedures Fully Medium Inclusion of F2002 energy savings and costs as part of DSM plan is required.

Accounting Treatment Fully N/C High Treatment of Plan costs as deferred capital will be reviewed by external auditors

Tracking & Measuring Energy and Demand Savings

Partially High Setting energy goals by plan and by program improved for F2003. Need to trackand report demand savings. Develop appropriate key performance indicators.

Multi-year funding for proposed programs

Fully Low No significant issues.

Cost of Capital / Program Financing Fully Low No significant issues.

Implementation Plan

Business Model Substantially High Business cases exist, but some program logic model elements are missing, andlong term market effects of programs are not included in TRC ratio calculations.

Schedule Substantially Medium Detailed program schedules are limited. Integrated scheduling tool needed.

Resources Partially High Resource analysis by expertise is needed.

Risk Management Plan Fully High Plan prepared with assistance from "Big 5" audit firm.

Communication Plan N/C N/C Medium Review was completed for CFL, LED, SUCH (see appendices).

Customer Generation Plan N/C N/C Medium Not reviewed as part of audit scope.

DSM Implementation

Incentive Process

Payment

Fully

UnderDevelopment

N/C

High

High

Incentive payment process appears adequate. No incentive payments have beenmade under the new incentive contract for the Power Smart program.

Energy Tracking Process

System (Converge) Substantially High System and process documentation is being developed.

Input Partially High Energy savings tracking needs segregation of duties & support documents.

Reporting Process Substantially Medium Need to streamline and develop more effective reports to meet needs of users.

Measurement & Verification Process Under development N/C High M&V terms of reference exists. No overall or individual program M&V plan exists.

Evaluation Under development N/C High No evaluation plan exists as process is under development.

Program Reviews: For detailed reviews see appendix.

LED (Light Emitting Diode) Substantially High Well designed, financial analysis and budget assumptions clearly documented.Program savings may be overstated.

CFL (Compact Fluorescent Light) Substantially High Well planned, implementation is on schedule. Evaluation plan is well developed.

SUCH (Schools, Universities, Colleges, Hospitals)

Substantially High Well developed implementation plan. Program plans do not integrate programcomponents in a strategic fashion. Goals may be aggressive.

Legend: Satisfactory, no improvement neededSome improvemen t neededAddressed only in cursory fashion, significant improvement needed

N/C No audit comment (no testing completed/not in scope)


Contents

Executive Summary

1. Background

2. Audit Objective and Scope

3. Observations and Recommendations

3a. 10 Year Plan Overview3b. Key Processes3c. Program Reviews

i. Light Emitting Diode (LED) Programii. Compact Fluorescent Light (CFL) Programiii. Schools, Universities, Colleges, Hospital (SUCH) Program

Appendices


1. Background

1989- 1995q Power Smart initiated energy efficiency programs in the late 1980’s as the cost of new

generation was high and the awareness of DSM was low. Investment into Power Smartincluded education, information and awareness programs as well as initiatives towardsstandards, regulations and legislation. Incentive programs such as the refrigerator recoveryprogram were started as was retail promotion and partnering.

1995-1998q BC Hydro moved from incentives to cost recovery and promotion as legislation, regulations and

standards were in place and the cost of new supply fell. It was believed that the market hadtransformed and this was reflected in BC Hydro’s reduced commitment of capital funding to thePower Smart business unit.

1998-2000q In anticipation of deregulation and competition, BC Hydro focused on a retail strategy for the

middle market. A minimal investment in small, select legacy programs was maintained. Resultsare baseline savings of 2,500 GWh each year.

2000-2012q The focus of Demand Side Management (DSM) is a priority for BC Hydro. Selecting DSM in the

stack for new resource acquisition is the number one priority for the corporation, followed bycustomer based generation (co-generation and self-generation), green generation, acquisitionfrom Independent Power Producers (IPP’s) and Combined Cycle Gas Turbine (CCGT).

q The focus is on the top 2,400 industrial and commercial customers with the first 3 years of theplan to focus more on resource acquisition and the latter years shifting to market transformation.

q The plan responds to Senior management’s view of DSM as an important and cost-effectivesource of energy supply. The plan requires three year funding of $335 million to achieve anadditional 1,300 GWh and funding of $600 million over ten years to reach a total of 3500 GWh.


Objectiveq To assess if Power Smart Demand Side Management (DSM) plan incorporates the necessary

supporting business processes and discipline to achieve key business plan objectives.

Scopeq Review risks, processes, procedures, and controls related to, but not restricted to:

q DSM ten-year plan – Review plan assumptions, goals, methodology, principles,supporting analyses, and cost/benefit calculations.

q DSM Implementation – Review factors that lead to a successful implementation:

• Program goals and objectives• Resources and Timing– Staffing, Funding, Scheduling, Milestones• Processes – Customer Selection process, Contract Management process,

Incentive Payment process, Measurement and Verification process.• System – supporting energy tracking system• Reporting – performance and management reporting• Cost Management – accounting and tracking of DSM costs• Risk Management and Quality Assurance Standards• Governance Structure - accountability, roles and responsibilities

q Research industry practices that will assist the DSM implementation.

2. Audit Objective and Scope


3a. DSM Plan – Financial Mgmt: Measures & Processes for Valuation

Valuing Benefits of Programsq Energy and demand savings can postpone the capacity provided by a new power plant at the

system level (system benefits), the need for a substation upgrade in a specific region (localarea benefits), or reduce energy purchase requirements or make available existing generationfor sales to export markets.

q Demand and energy savings are undervalued in the overall Total Resource Cost ratio in theTen Year Plan:

u Demand (KW) savings for the overall Power Smart Ten Year plan are projected to be 457MW by 2012. Actual demand savings from Power Smart programs are not currentlytracked or reported for all programs.

u The current TRC benefit/cost ratio reported for the overall Power Smart Ten Year Planperiod truncates program savings after 2012, thus understating the program benefits by$545 million. Business cases underpinning requests for release of funds prior to programlaunch, such as the Courtney/Comox/Quesnel (CCQ), have extended the period forbenefit valuation to include the expected duration of the period over which the benefits willconservatively extend regardless of the perceived BC Hydro standard discount period often years.

u Given projected capacity needs at the system level and regional level, current calculationsof benefits of demand savings may be conservative based on current estimates of longrun marginal cost (LRMC). Only $5/MWh of the $49/MWh of the LRMC estimate isdemand related. While this $5/MWH value is sufficient for the short-term, this valueappears to be very conservative for the entire Ten-Year Period.

u Demand savings will have a much higher value at Vancouver Island and other areasfacing T&D constraints.


Valuing Benefits of Programsq Energy savings are undervalued in Total Resource Cost ratios because sustainable “market

effects” of DSM programs are not included in current projections of TRC calculations.q Undervaluing demand savings could impact the selection of DSM resources. It is important to

note that demand savings have value at the overall system level, and additional benefits in localareas facing T&D constraints.

q Power Smart staff is examining the targeting of DSM programs to specific geographic areas.

The Value of Capacityq According to the BC Hydro Resource Management Department, the value of avoiding

generation capacity with DSM ranges from a low of $10 to $15 per KW / year to as much as$70 / KW year, depending on the timing and location of the DSM savings. The ResourceManagement Department reports that the value of new generation capacity included inprojections of BC Hydro’s LRMC ranges from $15 to $50 per kW/year. Using these values, theestimated value of capacity saved in year ten of the plan ranges from approximately 4 - 13 % ofthe total estimated value of the plan in that year.

TYPE OF GENERATION CAPACITY GENERATION CAPACITYVALUE

Value based on short term market information $10 - $15 / KW / YR

Additional turbine, existing dam $25 / KW / YR

Capacity value in LRMC of $49 / MWh $15 - $50 / KW / YR

BC Hydro Industrial Rate $53 / KW /YR

Vancouver Island $65 / KW / YR

Combined Cycle Combustion Turbine $70 / KW / YR

Highest Market Price Pacific Northwest – Last 24months

Higher than all of the above

Power Smart BenefitsCumulative PV of Plan Benefits - $1.3 billion over 20

years

10%

90%

Demand Benefits($5)

Energy Benefits($44)




Recommendations Management Action Plans

q Transmission and Distribution lines of businessshould identify local areas in BC where DSM couldpromote substantial load relief for the T&D system.Power Smart staff could then assess DSM as aresource option and proceed accordingly. For eachPower Smart program, consider a component thattargets capacity constrained areas. The T&Dbusiness lines need to establish an “early warning”system to identify such local area opportunities forthe Power Smart Team.

q Management will establish a crossfunctional team includingrepresentation from Transmission,Distribution and Generation LOB’s,and others, to identify opportunitiesfor Power Smart to provide loadrelief in capacity constrained areas.To initiate this effort, work iscurrently underway evaluating suchan opportunity in the Fort Nelsonarea.

q Review and document the short and long-termvalue of generation capacity savings to BC Hydroat the system level, as well as Transmission andDistribution savings in constrained areas. In DSMbenefit/cost models that calculate the TRC test,explicitly show the demand savings in the model,and value these demand savings where they havereal economic value to the Company.

q Management will, where applicablehenceforth explicitly identify andvalue demand savings inbenefit/cost models that calculatethe TRC test.

q Eliminate the practice of truncating savings in TRCcalculations for the overall Power Smart program(master roll-up Excel spreadsheet).

q Management will in future valuebenefits over the full expectedeconomic life of its investmentprovided that it continues to bematerial in amount.



q Track and report demand savings of Power Smart programs. q Management will track andreport demand savings wherethey are material and where it ispractical to do so.

q The Power Smart Team should provide more detailedinformation on short-term and long-term DSM impacts toResource Management. Where not already provided to theResource Management Department, DSM impact informationneeds to be detailed by light load, heavy load and super peakload periods.

q Management concurs and willundertake to acquire andprovide this information for thoseprograms where it is practical todo so.

q BC Hydro should consider acquiring a “state of the art”spreadsheet model to calculate TRC benefit/cost ratiosaccording to DSM industry practice to explicitly value demandsavings, to explicitly value sustainable market effects, and toexplicitly allow penetration modeling of (1) baseline energyefficiency, (2) impacts due to program participants, and (3)impacts due to sustainable markets effects. The currentspreadsheet model used by BC Hydro undervalues the benefitsof DSM programs. The Company should closely examine theNSTAR benefit/cost model, the Northeast Energy EfficiencyPartnerships (NEEP) model and the Northwest EnergyEfficiency Alliance (NEEA) model.

q In the second quarter of fiscal2003, Management willbenchmark its methodologiesand model used for programdevelopment and fundingapprovals against “state of theart” models used elsewhere anddetermine if the incrementalcosts associated with the use ofsuch models are warranted.

q The Audit Services Division will provide further information (viamemo) to the Power Smart Team on specific benefit/costanalysis techniques that can be improved.

q Management will consider thesewhen available.



Program Screeningq The energy efficiency programs in the Ten-Year Plan cover the Residential, Commercial and

Industrial markets. Summary program plans are included in the Ten Year Plan and detailedbusiness cases exist for: Light Emitting Diode (LED) Traffic Light, Schools, Universities,Colleges, Hospitals (SUCH) and Compact Fluorescent Lights (CFL) programs.

q In order to go forward, the portfolio of programs must cost less than 2 cents per kWh saved forthe utility test and 5 cents per kWh saved for the TRC test.

Program Ranking and Prioritizationq No written documentation exists at BC Hydro describing the process used in Fiscal 2002 for

screening, ranking and prioritizing energy efficiency program options and to identify whichenergy efficiency options or programs were rejected.

q Additionally, for programs like SUCH and Power Smart Industrial, the program implementationhas not yet addressed the targeting of incentives to geographic areas in need of load relief toalleviate generation, transmission or distribution constraints.


q On a going forward basis, a systematicprocess for screening, ranking andprioritizing energy efficiency optionsshould be developed and documented.

q Management will, on a going forward basisconsider a well documented systematicprocess which will rank energy efficiencyoptions in keeping with overall corporateobjectives.



Treatment of Forecast Power Smart Savingsq There is a general lack of common understanding across BC Hydro of the treatment of forecast

Power Smart savings included in the BC Hydro load forecast by the Distribution LOB, in therevenue forecast by Corporate Finance, and in the resource planning by the Generation LOB.

q Corporate practice is to allow for initiated DSM GWh savings (for programs that have started)into the load forecast and other identified program savings (for programs not started) into theresource stack. Over time as further initiatives are commenced, the balance in the forecastshould increase with the offset coming from the resource stack.

u The Distribution LOB subtracts highly probable savings from the load forecast (e.g. 54%of GWh savings in F2003). Less likely savings are placed into the resource stack and arenot subtracted from the load forecast (e.g. 46% of GWh savings in F2003).

u The revenue forecast has 100% of DSM GWh savings counted for F2003.u The resource plan separates available resources including DSM savings into levels of

probability.q In 2007, the BC Hydro load forecast assumes that 83% of plan DSM savings (1,597 of

1,935 GWh) are not yet highly probable.


q The BC Hydro Power Smart loadforecasting, revenue forecasting andresource staff need to have commonunderstanding of DSM in the forecastversus projected DSM savings not in theforecast.

q Management will create a cross-functionalDSM resource team during the first quarter ofFiscal 2003 which will address this issue.



Treatment of Freeridershipq A freerider is a participant in a Power Smart program who receives financial assistance for

energy efficiency measures, but would have implemented the energy measures in the absenceof the program. Programs with large freeridership are not likely to be cost effective.Freeridership energy savings are not explicitly counted in the total resource test benefit costratio, although program costs for freeriders are counted.

q To accurately know and assess the future impact of Power Smart programs on future BC Hydroresource needs, the assumptions about the naturally occurring conservation in the load forecastneed to be as consistent as possible with program freeridership estimates.

q Programs need to be designed to optimize freeridership so that BC Hydro does not pay forsavings that would have occurred anyway.

q Based on interviews with program staff, there is not a common understanding of freeridershipissues, and there is lack of up-to-date baseline information on current levels of energy efficiencyin markets served by Power Smart programs.


q Programs need to be designed tominimize the impact of freeriders onenergy savings assumptions. Marketresearch needs to be conducted inadvance of program planning andimplementation to assess baseline levelsof penetration of energy efficiencyequipment or practices in each market tobe addressed by Power Smartprograms.

q Management is satisfied that currentprograms have been designed to optimize theimpact of freeriders with respect to energysavings assumptions. On completion of theConservation Potential Review (CPR) in3rd quarter Fiscal 2003 management will be ina position to re-assess baseline levels ofpenetration of energy efficiency equipmentand practices in each target market.




q Program Managers should be providedfurther education and information onfreeridership issues and should also beprovided the results of the marketresearch.

q Management will immediately undertake areview of freeridership issues with allprogram managers.



3a. DSM Plan – Financial Mgmt: Accounting for DSM Programs

Accounting Proceduresq The Ten-Year Plan and business plans for specific programs describe the accounting

management procedures to be used to track and report Power Smart program costs.q The overall Power Smart Program is using standard BC Hydro cost accounting standards,

thus no further development is needed for accounting procedures for Power Smart programcosts.

Accounting Treatmentq The accounting treatment, as determined by Corporate Finance and Power Smart

Management, to treat all Power Smart expenditures as deferred capital, was investigated,documented and agreed to in consultation with the Legal and Regulatory Affairs Departments.Internal Audit conducted no audit review of the accounting treatment. This issue will beexamined in detail by the BC Hydro external auditors as part of their year end audit work.

Matching of costs and benefitsq The DSM plan commences in Fiscal 2003 and runs for 10 years. As energy savings programs

were launched and systems developed during Fiscal 2002, many costs were incurred inFiscal 2002 relating to the 10 year plan. It was agreed with Power Smart management thatFiscal 2002 Power Smart Program costs and savings should both be included in the trackingof cumulative costs and benefits of the Ten-Year Plan and not just costs or the energysavings.


3a. DSM Plan – Financial Mgmt: Multi-Year Funding and Cost of Capital

Multi-Year Fundingu The 10 year plan requires funding of $600 million over 10 years. The funding is front loaded

with multi-year funding of $335 million for the first 3 years of the plan.u Business cases that meet the funding approval criteria require proper approvals for each

program initiative. Annually, the supporting business case for each program will be re-evaluatedfor program viability prior to approval for release of funds.

u The Corporate Finance Evaluation group will be conducting reviews before the implementationof each program where funding is required.

u Post Implementation reviews will also be conducted.u No significant issues were found.

Cost of Capitalu As DSM shares the same risk profile with BC Hydro, the required rate of return on equity would

be the same as the BC Hydro rate (15.24% for Fiscal year 2002).u The applicable cost of borrowing reflects BC Hydro’s Canadian borrowing rate (7.53% is the

base case cost of capital for evaluating the DSM program).u The above cost of capital rate has been applied to individual DSM business cases that were

reviewed. This cost of capital rate differs from the corporate calculated rate of 9% that isrecommended for use by the Lines of Business for their business case financial models.However, the 7.53% rate was agreed to for DSM program evaluation with Treasury and theController’s Division.


3a. DSM Plan – Financial Mgmt: Tracking & Measuring GWh’s Acquired

Target Settingq An energy resource acquisition goal of 120 GWh was established as a performance goal for

Power Smart management at the beginning of Fiscal 2002. Plans for the DSM initiative weredeveloped part way through the Fiscal 2002 year so the 120 GWh target was incorporated intothe 10 year business plan. The 120 GWh target was detailed only to the market sector leveland not down to a program level.

q Throughout Fiscal 2002 the regular weekly and monthly reporting and tracking of the GWhtarget has been under development and is not yet fully complete.

q It appears that program savings targets have been adjusted for freeridership. Actual reportedGWh savings have not been explicitly adjusted for freeridership – this is the job of the evaluationgroup.

Reasonableness of Program Targetsq One of Power Smart’s goals is to be in the top quartile of DSM providers in North America.

Power Smart has identified two key performance indicators (% of annual sales saved, % ofannual revenue invested) for tracking progress to meet this goal. Power Smart also plans toparticipate in DSM benchmarking studies (such as the Canadian Energy Efficiency Alliance) tocollect the data necessary to progress to achieve this goal.

q The Ten-Year Plan has a target of 3,500 GWh savings (annual run rate) – this represents areduction of 6% in Fiscal 2012 of BC Hydro’s annual GWh sales. The demand savings run ratetarget is 457 MW by 2012 – this represents a reduction of 4% in Fiscal 2012 of BC Hydro’s peakload (peak load before Power Smart demand reductions).

q The target energy savings percentage reduction of 6% is well within the range of savingsachieved by the top quartile of DSM utilities in the U.S. according to research conducted byAudit Services. However, the demand savings target of 4% is not within the top quartile.



Reasonableness of BC Hydro DSM TargetsVersus Other DSM Leaders

EnergySavings (1)(% of Sales)

DemandSavings (2)(% of Peak

Load)BC Hydro – Planned Savings

• Ten Year Plan (2012) 6% 4%

Top 50 U.S. Utilities – Actual Savings

• Lowest % Savings 2% 13%

• Weighted Average of all 50 Utilities 5% 16%

• Highest % Savings 11% 53%

Note: This table shows that BC Hydro’s goal of 6% for Demand Side Managementenergy savings is reasonable and achievable. The BC Hydro demand savingstarget in Fiscal 2012, however, is below the range achieved by the Top 50 US DSMUtilities, and far below the weighted average demand savings.

(1) Energy savings percentage is the percent of 2012 annual KWh sales saved.(2) Demand savings percentage is the percent of 2011/2012 system peak load

saved.



Reasonableness of BC Hydro DSM Targets(BC Hydro Goals Versus Savings Achieved By Largest US Utilities Doing DSM)

Energy Savings (1)(% of Sales)

Demand Savings (2)(% of Peak Load)

BC Hydro – Planned Savings

• Ten Year Plan (2012) 6% 4%

Largest U.S. DSM Utilities – Actual Savings

Number of U.S. DSM Utilities 45 43

• Lowest % Savings 0.1% 0.1%

• Weighted Average of all 50 Utilities 2% 4.4%

• Highest % Savings 9.1% 17%

Note: This table compares BC Hydro’s Ten-Year Plan goals for DSM savings to actualenergy and demand savings already achieved by the largest 45 US Utilities havingDSM Programs.

(1) Energy savings percentage is the percent of 2012 annual KWh sales saved.

(2) Demand savings percentage is the percent of 2011/2012 system peak load saved.



Reasonableness of BC Hydro DSM Targets(BC Hydro Goals Versus Savings Achieved By Top Quartile of US Utilities Doing DSM)

Energy Savings(1)

(% of Sales)

Demand Savings (2)(% of Peak Load)

BC Hydro – Planned Savings

• Ten Year Plan (2012) 6% 4%

Top Quartile of U.S. DSM Utilities – Actual Savings

Number of U.S. DSM Utilities in Top Quartile 71 79

• Lowest % Savings 1.6% 7.4%

• Weighted Average of DSM Utilities – Top Quartile 4.4% 12.4%

• Highest % Savings 11.2% 52.6%

Note: This table compares BC Hydro’s Ten-Year Plan goals for DSM savings to actualenergy and demand savings already achieved by the top quartile of US Utilities havingDSM Programs.

(1) Energy savings percentage for the Ten-Year Plan is the percent of 2012 annual KWhsales saved.

(2) Demand savings percentage for the Ten-Year Plan is the percent of 2011/2012system peak load saved.




q Identify benchmarking studies (that include bothCanadian and U.S. utilities) which will provide the mostvalue to Power Smart management to help itcontinuously improve program design, implementationand evaluation practices, and the overall levelized costper kWh saved. The benchmarking studies should alsohelp BC Hydro to achieve its DSM leadership role inNorth America.

q Management will undertake toidentify and participate inbenchmarking studies in the3rd quarter of Fiscal 2003.

q Need to develop appropriate Key Performance Indicators(KPI’s) to track progress towards the goal to be in the topquartile of DSM providers in North America (indicatorssuch as % of peak load saved with DSM, % of energysales saved with DSM, overall levelized cost per kWhsaved, etc.).

q Management will develop theappropriate KPI’s as part of thecontinuing development ofmethodologies and tools fortracking and evaluation ofenergy savings.

q F2003 targets have been set at a sector and programlevel so this level of transparency has been created forthe future. Detailed annual targets at the program levelshould also be developed for the next three years of theplan.

q Management will undertake toset annual targets at theprogram level for the nextthree years of the plan.

q Actual GWh savings for Fiscal 2002 will need to beadjusted as appropriate for freeridership and otherpertinent issues by the evaluation group to providerelevant and accurate information.

q Management agrees that thiswill be done as part of theevaluation process.


3a. DSM Plan - Implementation Plan : Business Model for Power Smart


q Conduct a gap analysis of marketresearch, then update the overallresearch plan for Fiscal 2003 asnecessary (currently, up-to-date energyefficiency baseline and marketassessment information are notavailable for all Power Smart marketsegments).

q Upon completion of the aforementionedCPR study, management will assess theneed for the recommended gap analysis.Management is satisfied that currentinformation by market sector on its top2400 commercial and industrial customers isadequate to justify moving forward with theplan as presently constituted.

q Consider including logic models in allnew business cases and develop anoverall evaluation plan at all programlevels.

q In the 2nd quarter of Fiscal 2003,management will undertake to assess itsprogram development methodology andbusiness case model in light of other modelscurrently available. The evaluationmethodology currently under developmentwill be extended to encompass the overallplan.

q Implement third party evaluationoversight.

q Management concurs and will, as in the past,implement third party evaluation oversightprior to BC Hydro’s next revenuerequirements hearing.


3a. DSM Plan - Implementation Plan:Resource Requirements & Organization

Staffingq Currently BC Hydro has a hiring freeze and Power Smart does not plan to add full time

regular staff until later in Fiscal 2003.q Several Power Smart department managers, however, have indicated an immediate need

for approximately 20 additional FTE’s in the key areas of measurement and verification,evaluation and key accounts.

Fundingq Power Smart Fiscal 2002 spending of approximately $35 million is budgeted at $94 million

in Fiscal 2003, almost a three-fold increase from Fiscal 2002 levels.q The charts below indicate that funding will increase significantly for the first three years to

meet the energy savings targets, then will decrease significantly after Fiscal 2006 as energysaving targets reduce.

Plan Funding by Year

0

50

100

150

F03 F04 F05 F06 F07 F08 F09 F10 F11 F12

Plan Fiscal Year

Pla

n Fu

ndin

g $M

Plan $

Energy Saving targets by Year

0100

200

300

400

500

600

F02 F03 F04 F05 F06 F07 F08 F09 F10 F11 F12

Plan Fiscal Year

En

erg

y S

avin

gs

(GW

h)

Plan GWh


q The Fiscal 2002 incremental GWh savings target is doubling from 120 to 240 GWh inFiscal 2003.

q Several new energy efficiency initiatives will be launched in Fiscal 2003 and the need fordetailed market assessment research, Measurement and Verification (M&V) and programevaluation will increase substantially.

q Although there are immediate staffing requirements in key areas, it is difficult to ascertain theoverall staffing adequacy without further research to determine staffing levels and required skillsets at other comparable utilities with a DSM program.

q Power Smart management continues to use alternate ways to meet short-term staffingrequirements to address the funding and energy saving targets in the three year plan withouthaving excess staff after three years when funding is scaled back.

q The aggressive staged implementation rollout and plan spending in Fiscal 2003 to Fiscal 2005intensifies the risk of program failure because there will not be sufficient time to develop, trackprogress and learn lessons from preliminary small-scale efforts. This spending trajectorymakes it all the more urgent to get all pieces of the plan (e.g., tracking, reporting, research,M&V, and evaluation) in place immediately to ensure progress in being in the top quartile ofDSM providers in North America.


q Power Smart management should re-assess staffing needs by comparingstaffing levels and skill sets to levels atother comparable DSM utilities, and thenadd (or re-allocate) the staff or outsidecontractor help as indicated by theneeds analysis.

q Management concurs and will undertake astaffing needs assessment by end of2nd quarter of Fiscal 2003.



q Although there is a terms of reference for an M&V plan, no overall M&V plan exists for thePower Smart Program, nor is there a written M&V plan for each program.

q Converge is an in-house developed sales management system that subsequently was modifiedto track energy savings in GWh. It tracks several, but not all Power Smart Initiatives (forexample, the E-Points and residential programs are tracked separately).

q Currently there is limited documentation of data requirements for Converge. Converge does notspecifically identify a field for tracking freeridership.

q No plans are in place yet for an independent group, such as DSM Quality Assurance orMeasurement and Verification, to verify reported energy savings to source documentation likethe energy savings report.


q A M&V plan needs to be completed for theoverall Power Smart effort and for eachPower Smart Program.

q M&V plans for the overall Power SmartPlan and for each program will bedeveloped by the end of the 2nd quarter ofFiscal 2003.

q Immediate attention needs to be given todeveloping and documenting an M&V planfor E-Points, and to reporting E-pointssavings on regular program trackingreports.

q Management agrees and will finalize theplan within the first quarter of Fiscal 2003and will continue reporting the savings ona monthly basis.



Governanceq The Ten-Year Plan and specific business plans provide information about the governance

procedures for management, planning, implementation and evaluation of Power Smartprograms. Organization charts exist that describe the responsibilities of key members of thePower Smart management team. The Power Smart management team meets on a weeklybasis to manage, direct and assess the Power Smart programs. No audit issues were noted inthis area.


q The DSM Quality Assurance functionshould perform regular tests to compareenergy and demand savings estimates for asample of projects in Converge to sourceEnergy Saving Report documentation. Thepurpose of such tests is to validate the on-going savings estimates reported inConverge.

q Management concurs and will incorporateregular tests as recommended to validateongoing savings.



3a. DSM Plan - Implementation Plan: Implementation Schedule & Tools

Monitoring Cycle Times of Key Processesq A critical element of a successful DSM effort is to monitor the cycle time of key business

processes (e.g., the time to issue an RFP and evaluate proposals that are received; the time todeliver services to a commercial or industrial customer after an order has been placed; the timeto design and launch a new program), and to take steps to reduce the cycle time of keyprocesses when appropriate. A detailed review of three implemented programs indicated, thatwhile program procedures had been described in business plans and other documents, therewas no clear evidence that continuous improvement for cycle times of key business processesis to be undertaken.

Scheduling / Control Systemq As the DSM program launches completely for Fiscal 2003, it will increase from a budget in

Fiscal 2002 of $34 million and an energy target of 120 GWh run rate to a budget of $94 millionand an incremental energy target of 240 GWh run rate. Managing all of the various programsand projects with their various milestones and critical deliverables is a challenge that iscurrently not sustainable with the current project management tools being utilized. It is criticalto have an integrated scheduling and project management system that captures key dates forM&V of incentive calculation and key roles and responsibilities.


q Implement an integrated scheduling tool. q Management remains unconvinced that anintegrated scheduling tool would be asubstantial improvement to its currentapproach of assigning program managersand staff to specific programs.


3a. DSM Plan - Implementation Plan: Implementation Schedule & Tools


q Plans for continuous improvement ofcycle times of such processes should beexplicitly addressed either in businesscases or other planning documents forall programs.

q Management does not concur thatcontinuous improvement of general businessprocesses, particularly service delivery andsupply chain management, should beexplicitly addressed within program businesscases; these are best addressed as part ofthe overall corporate focus on businessprocess improvement.


3a. DSM Plan – Risk Management Plan / Customer Generation Component

Risk Management

q The Ten-Year Plan includes a detailed risk assessment for the overall plan. Also, businessplans for specific programs include detailed risk assessments.

q The Risk Management Plan was prepared with assistance from a ‘Big 5’ auditing firm andcovers all related major risks.

q The Audit Team reviewed this risk assessment information, discussed cost recovery and otherrisks with the Power Smart Team, and determined that the risk assessment methodology andfindings to be satisfactory.

Customer Generation Component

q The Audit did not review the Customer Generation Component of the Power SmartTen-Year Plan as part of the 10 year DSM business plan due to audit timing and the largescope of the audit.


3b. Key Processes - Incentives Process

q The use of a competitive bidding or “call” process with the largest 2500 commercial andindustrial customers to solicit competitive proposals for energy efficiency projects at specificfacilities is a standard, well known, successful, and commonly used procedure. Suchcompetitive DSM bidding is used in many jurisdictions including the New York State EnergyResearch and Development Authority (NYSERDA), the State of Wisconsin Focus on EnergyPublic Benefits Program, and by many other DSM utilities in the United States.

q A major component of the DSM initiative is the use of incentive funding in the first three years ofthe 10 year plan. A projected $250 million out of the budgeted $335 million (75%) is targeted forincentives in the first three years.

q The incentives process is not standardized for all DSM programs as a variety of programsoffering different forms of incentives have been developed with cash payments as the mostcommon form of incentive funding that will be utilized.

q Although the incentive process is under development, programs currently reviewed indicate thatauthorization levels for payments exist, controls exist within the approval process, and thatqualifications for participant eligibility (i.e., Power Smart Partners) have been established.

q Recent revisions to the incentive contracts also reflect stronger control over the payment of thefunds. The early incentive contracts that were utilized in F2002 for the Jumpstart and IndustrialPrograms had the following payment provisions in the contract:

u 25% payment at signing of contract, 50% payment at ordering of equipment and25% payment when the project is commissioned and in service.

q The newly revised incentive contracts have the following provisions:u 25% payment on project initiation, 50% payment at Project Completion and Inspection,

25% payment 1 year after project completion when measurement and verification iscompleted.

u No incentive funds have been paid out under the new incentives process.


3b. Key Processes - Incentives Process

q Findings from review of the early incentives process for the University of British Columbia (UBC)under the Industrial Program include:

u No formal scheduling process exists to track and record the timing of future incentivepayments, clawbacks, and measurement and verification to ensure key provisions of eachcontract are not missed.

q Project selection process documentation requires further transparency to ensure that selectioncriteria, ranking of projects, and justification of projects selected is adequate.

q The credit review process currently does not address the viability of customers who may receivecash incentives. To date credit reviews have been minimal.


q Modify the credit checking function togain assurance of the ongoing viability ofcustomers who may receive cashincentive and perform appropriate creditreviews.

q Management is satisfied that current creditreview processes coupled with its intimateknowledge of its top customers hasadequately mitigated its financial exposurebut will undertake review of credit quality aspart of its ongoing continuous improvement.

q Power Smart should maintain a tablethat clearly explains incentivemechanisms available for eachPower Smart Program.

q Management concurs and will undertake todo so in the 2nd quarter of Fiscal 2003.


3b. Key Processes - Tracking Process

Tracking Processq Sales personnel are initially responsible for entering estimated energy savings to Converge.

Previously, attention to timely data input was given low priority. Senior sales personnel are nowemphasizing the importance of recording energy savings on an accurate and timely basis.

q Converge uses various system controls to ensure data integrity. They include automatic systemassignment of opportunity identification numbers, controlled customer name creation, dataexception reports, and computer user access controls. However no system edit exists to detectenergy savings (GWh) input errors.

q Procedural controls are also used to ensure completeness of energy savings. Time deadlinesregarding the input of key sales data have been recently communicated to sales staff andmanagement’s weekly review of energy savings opportunities with sales forces remains the keyprocess control to ensure energy savings are accurate.

q The source of reported estimated energy savings is the energy savings report. This reportidentifies the GWh savings and is meant to be reviewed by BC Hydro’s Measurement andVerification (M&V) group for accuracy and completeness before the data is entered toConverge.

q During Fiscal 2002, project energy savings were entered to Converge by Sales (Key Accounts)and then, later by Delivery staff. In absence of energy savings entered by Delivery, Convergeused the sales’ entered project energy savings for reporting energy savings to seniormanagement. Given sales performance contracts are based on achieving targeted estimatedenergy savings and sales’ input energy savings have been reported, a conflict of interest exists.There is ongoing discussion as to who and how a project’s energy savings will be entered toConverge on a go-forward basis. Other groups such as M&V and Evaluation may enter energysavings in the future. Superseding rules in Converge regarding the one reported energysavings for a project entered by more than one group will need to be implemented and tested.



q Six of twenty audit items tested indicate that previously reported energy savings to seniormanagement were overestimated, due to either incorrect input or lack of supportingdocumentation. One test identified excess energy savings recorded for two instead of oneoption given while two other tests reported energy savings that were overestimated andunsupported (but later corrected during our audit). Support documentation for the remainingthree tests was not filed in central files (but supporting documentation for one of these tests waslater provided and authorized by Power Smart management).

q During the time of our audit testing, management has been re-verifying and changing thereported energy savings for the year.

q Converge computer systems are located in a physically secured room at one location.Management indicated that tape backup is performed for backup and recovery purposes.


q For reported energy savings to seniormanagement, an independent group shouldbe reviewing estimated energy savings andentering their authorized version ofestimated energy savings on a timely basisto Converge. Staff whose compensationmay be tied to reported energy savingsshould be restricted from entering reportedenergy savings.

q All managerial staff within Power Smarthave their incentive compensation tied toachievement of the energy savingstargets – except the evaluation groupwho, as noted earlier, will report toQuality Assurance. No staff withinQuality Assurance will have theircompensation tied to reported energysavings.



q Continue to review and correct significantenergy savings in GWhs that are notsupported by Energy Saving Reports or filedmanagement approval but are energysavings that are reported to seniormanagement given their relatedopportunities are at the required reportinglevel in Converge (i.e., report energy savingsfor opportunities at the step 8 level).

q All data will be continuously reviewed andcleansed to ensure no materialmisstatement of reported energy savings.

q Implement a system edit check on thereasonableness of input entered into theenergy savings field and audit trail featurefor changes made to this field.

q The cost of automated edits versus thepotential benefits will be investigated and arisk effected decision will be made by theend of the second quarter of Fiscal 2003.

q File support documentation for energysavings reported (GWh).

q Management agrees and will take stepsimmediately to ensure compliance.



3b. Key Processes - Reporting Process

q A comprehensive reporting package (compiled by the Group Controller) comprised of monthlyand weekly reports is prepared for the Senior Vice President, Public Affairs and Power Smartand the Vice President, Power Smart. Individual components of this reporting package aredistributed to the Power Smart management team. Due to the nature and relevance of somereports, they do not all meet the management teams’ specific operational needs.

q A substantial amount of information is available in the reporting package and communicationamongst the management team appears to be very good with meetings scheduled on a regularbasis to identify issues. However, concerns were raised within the management team with thelack of support provided in analyzing the report results and providing them guidance withinformation in the reporting package on an on-going basis.

q The reporting package, although under development, needs to be better aligned to meet theneeds of its users.

q General observations of the reporting include:u Energy savings are currently reported on a “gross basis” and will be adjusted during

evaluation.u Presently levelized cost indicators (cost per kWh saved) are not prepared by program and

not reviewed overall on a regular basis.u Market transformation indicators are absent from reporting. These should be considered

for future reporting when the evaluation group does their evaluation.


3b. Key Processes - Reporting Process


q Need to streamline and develop moreeffective reports that meet operationaland senior managements needs.

q Management concurs and will immediatelyundertake to streamline existing reports andto better identify its operational reportingrequirements.

q Review and adopt reporting formats ofreports used by top five DSMorganizations in North America.

q Management agrees that, as part of itscommitment to continuous improvement, itwill review and consider adoption of reportingformats used by selected top DSMorganizations.

q Report Enhancements / Content

♦ Report levelized ¢ / KWh saved byprogram.

♦ Report actual vs. planned savings perprogram.

♦ Provide support to operationalmanagement to better understandreporting package.

♦ Add market transformation indicators.

q Management agrees and will be developinga quarterly report including the forecastlevelized cost per kWh saved.

q Actual vs. Planned savings per program arealready included in the monthly Power SmartProgram Analysis report as requested by theSenior Vice President, Public AffairsPower Smart and the Chair andChief Executive Officer.

q Upon development by the evaluation group,market transformation indicators will beincorporated in the appropriate reports.


3c. Program Review - Light Emitting Diode (LED) Program

Background

q BC Hydro has the opportunity to influence the BC market to adopt energy efficient traffic lightsthrough a two year Power Smart “Traffic Lights Program” (TLP). There are 3,637 “known”traffic light controlled intersections in BC, using an estimated 50 GWh of electricity annually.Light Emitting Diode (LED) traffic lights consume approximately 85-90% less electricity thanincandescent bulbs. Annual electricity savings of approximately 41 GWh are projected to beachieved by the completion of this program in March 2004.

q A number of BC municipalities are testing LEDs, and have confirmed that they cannot alwaysjustify full LED conversion, or else do not have the necessary capital required forLED purchases. To overcome this, BC Hydro will provide 100% of the up-front capital costs toBC municipalities and Ministry of Transportation to supply red, green and “don’t walk”pedestrian LED lamps to replace incandescent bulbs and will recover 50% of these costs overa five year period.

Summary of Findings

q The TLP program has been designed in a thoughtful manner. Financial analysis and budgetassumptions are clearly documented, but may be overstating the actual GWh savingsachievable from effective implementation (due mainly to a lack of baseline penetrationinformation on the number of traffic intersections already using, or currently planning to install,LED technologies outside of the program). Demand savings (kW) exist but have not beenestimated, tracked, or reported.

q Implementation and evaluation plans and program goals have been clearly communicated, butseem heavily focused on near-term (immediate) outcomes. Development of a logic model forthe TLP and more attention to the identification and tracking of intermediate and longer termmarket transformation indicators could help to improve the cost-effectiveness of this programand ensure the formation of a sustainable LED traffic light infrastructure within BC Hydro’sservice territory.


3c. Program Review - Compact Fluorescent Light (CFL) Bulb Program

Backgroundq BC Hydro has the opportunity to influence the BC market to replace incandescent light bulbs

with compact fluorescent light (CFL) bulbs. The target over three years will be the installationof over 2 million CFL bulbs. This program sets out to transform the residential lighting marketby showing customers the benefits of CFL’s by working with manufacturers and retailers toimplement this strategy.

q Mass public information initiatives will serve to educate, inform and encourage behavioralchange in the general public of BC including residential and business customers. A stagedimplementation of this program includes Courtenay, Comox and Quesnel (CCQ - Stage 1) andVancouver Island (Stage 2). The program is projected to save 155 GWh/year by three years,and 490 GWh by ten years.

Summary of Findingsq The CCQ pilot for the CFL Program has been very well planned, implementation appears to be

on schedule, and the evaluation plan is well developed. Baseline evaluation research is nearlycomplete and should offer a good foundation for assessing results of the pilot.


3c. Program Review - Schools, Universities, Colleges and Hospitals

Background

q There are 506 customers with more than 2,670 buildings that consume 1,277 GWh annuallyfrom the Schools, Universities, Colleges and Hospitals (SUCH) sector. BC Hydro has anopportunity to reduce this consumption by 10%. Although there are significant energy savingsfor these customers they face two key barriers that hinder the realization of the fullPower Smart Potential:

u lack of human resources to focus on and implement energy efficiency measures

u lack of financial resources to fund energy efficiency projectsq BC Hydro has the opportunity to influence and remove these barriers to help these customers

improve their electrical efficiencies, reduce operating costs and free up resources for othercritical demands. In return, BC Hydro realizes energy savings which, in addition to providing anet financial benefit, contribute toward environmental sustainability and social mandate.Energy efficiency initiatives in this program, (the corporate target is to deliver 960 GWh andon-site generation is targeted to deliver 400 GWh), combine together to create a total of1,360 GWh energy savings per the Business Case (November 1, 2001).

Summary of Findings

q The SUCH Program has well-defined goals, documented budgets, developed andcommunicated implementation plans.

q Areas of possible improvement are:

u Integrating the plan components in a strategic fashion.

u Obtaining more accurate and current market baseline and characterization research.

u Conduct “process” evaluations and benefit/cost analysis subsequent to planimplementation.

Internal Audit Report Demand Side Management - BC Hydro · Audit and Risk Management ......

Documents

Transcript of Internal Audit Report Demand Side Management - BC Hydro · Audit and Risk Management ......