Evaluating ESPCProject Results

Alice G. Dasek (DOE), Robert Slattery (ORNL), Peter Larsen (LBNL), and Torsten Kieper (The Cadmus Group)Better Buildings ESPC Webinar SeriesSession 5 – Evaluating ESPC Project ResultsSeptember 13, 2018

Overview

Overview of the ESPC Toolkit

Guidelines for ESPC Measurement & Verification

ESPC Project Benchmarking

Tools for Economic Impact Analysis

2

Accelerator Profile

Timeframe2014-2016

Partners25 partners (18 states, six cities, one school district)

PurposeExpand access to Energy Savings Performance Contracting (ESPC) as a promising option for financing energy efficiency retrofits in the public sector

OutcomeMore than $2 billion invested in MUSH ESPC contracts3

4

Area 1: Streamlining the ESPC Process• Partners reviewed existing model ESPC documents

Area 2: Empowering the Market• Partners participated in ePB feedback & training

Area 3: Resolving Individual ESPC Barriers• Support successful, permanent, innovative, and

replicable resolution of individual partner barriers

Accelerator Activity Areas

The ESPC Toolkit

Considering ESPC

Implementing ESPC

Establishing ESPC

Expanding ESPC

Evaluating ESPC5

https://betterbuildingssolutioncenter.energy.gov/energy-savings-performance-contracting-espc-toolkit

Guidelines for M&V -- Overview of Economic Impact Analysis Tools

Benchmark Sheets State & Local Governments K-12 Schools Colleges & Universities Public Housing

6

Evaluating ESPC Project Results

Guidelines for ESPC Measurement & Verification

8U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

Webinar: Evaluating ESPC ResultsFEMP M&V Guidelines v4.0Bob Slattery - Oak Ridge National Laboratory (supporting DOE FEMP)Sept 13, 2018

9U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

FEMP M&V Guideline v4.0

https://betterbuildingssolutioncenter.energy.gov/evaluating-espc

10U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

M&V Throughout the Federal Contract Process•Acquisition Planning Phase 1

•ESCO Selection 2

•Preliminary Assessment 2

•Notice of Intent to Award 2

•Request for Proposal 3

•Investment-Grade Audit 3

•Proposal 3

•Task Order Award 3

•Final Design and Construction 4

•Project Acceptance 4

•Post-Acceptance Performance Period 5

• Outline of M&V Plan established

• Baselines established per M&V Plan

Government Witnessing

Government Witnessing

Government Witnessing

• Full M&V Plan part of Proposal

• Post Installation Report – M&V per Plan

• Annual M&V Report per M&V Plan

11U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

DOE FEMP M&V 4.0 Guidance - Structure

• Overview of M&V Implementation– 6 process steps (ex: develop M&V plan,

define baseline, etc.)

• Risk and Responsibility in M&V• Detailed M&V Methods (A, B, C, D)• Selecting an M&V Approach

– Key considerations, cost/rigor, uncertainty

• Guidance For Specific ECMs – 19 ECM examples

• Appendix– Federal Requirements– M&V Plan Outline– Post-Installation Report Outline– Annual Report Outline

12U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

FEMP v4.0:1. Condenses the document, moving many auxiliary sections to stand-alone documents

available on the FEMP website, federal requirements moved to appendix – i.e. Guidance no longer specific to federal projects.

2. Strengthens Option A by defaulting to annual measurement of a key parameter (draws on IPMVP concept of “Operational Verification” – ensure performance to intended function)

3. Refocuses Option C to stress short-term applicability (IPMVP acknowledges challenge of facility changes, silent on alternate options in performance period)

4. Clarifies Option D approach in performance period - revert to other M&V options (More aligned with IPMVP)

5. Adds M&V approaches for 19 ECM examples (IPMVP has 12 examples in 2012 vol. 1)6. Modifies M&V annual report template for added transparency in documenting

customer impacts to savings7. Integrates customer “witnessing” requirements throughout M&V process

M&V v4.0: Changes from v3.0 & IPMVP alignment

FEMP M&V Guideline 4.0 and IPMVP continue to be in alignment. Subtle nuances exist

13U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

•Interest Rates•Construction costs•M&V confidence•Energy-related cost savings•Delays•Major changes in facility

Financial

•Operating Hours•Loads•Weather•User Participation

Operational

•Equipment performance•Operations •Preventative Maintenance•Equipment repair & replacement

Performance

FEMP Risk and Responsibility and Performance (RRPM) Matrix

Addresses 14 risk topics and assigns responsibility for each between contractor and customer

14U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

DOE FEMP RRPM Example

Template language Agreed Upon Approach

15U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

• RRPM ensures that important risks are addressed and responsibilities assigned

• Dialog fosters mutual understanding of the deal• RRPM is a key item for review in Preliminary

Assessment (Walk Through Audit)• Use the RRPM to guide final proposal review ―

details in M&V plan and other parts should not conflict with RRPM

• Don’t take a responsibility that your organization can’t handle well!

Lessons Learned on RRPM

16U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

M&V 4.0: Annual Reporting TemplateFEMP M&V Guidelines v4.0 provides a template for M&V reporting. • Structured to convey all critical project

performance metrics within executive summary– Project background/overview– ECM descriptions– ECM proposed and verified savings– Customer impacts to savings– Performance / O&M issues summary

• Body of report goes into detail by each ECM– M&V approach– M&V activities conducted– Calculations– Summary of O&M and R&R (items to be addressed)

17U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

M&V Reporting: Transparency of Customer Impacts to savings

1.5 Performance and O&M Issues Note effect of operating deficiencies or enhancements on generation of savings. Note effect of maintenance deficiencies on generation of savings. Detail any deficiencies that need to be addressed by ESCO or government in Table E-4.

18U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

Standardized M&V Reporting & Annual Monitoring of Program Performance

Annual Review of M&V Reports• Full accounting of program ”reported” savings from all

active projects in performance period phase• Of 172 M&V reports examined for FY16, projects

reported achieving • 99% of estimated cost savings • 108% of guaranteed cost savings

151 projects above guarantee 7 projects at guarantee

14 projects below guarantee

19U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

Additional tools for ensuring long term integrity of M&V objectives

M&V Witnessing:Guide to Government Witnessing and Review of M&V Activities

M&V Report Review:Reviewing Post-Installation and Annual Reports For Federal ESPC Projects

O&M Verification Guide:How to Determine and Verify O&M Savings in ESPCs

Energy Escalation Rate Tool:Standard for setting energy escalation rates for federal ESPC

M&V Plan Review:Reviewing Measurement and Verification Plans For Federal ESPC Projects

https://www.energy.gov/eere/femp/resources-implementing-federal-energy-savings-performance-contracts

20U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

Bob SlatteryOak Ridge National Laboratory

slatteryrs@ornl.gov

21U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

ESPC Project Benchmarking

ENERGY TECHNOLOGIES AREA ENERGY ANALYSIS AND ENVIRONMENTAL IMPACTS DIVISION

Benchmarking the Performance of Projects Installed by the U.S. ESCO Industry

Peter LarsenResearch Scientist/Deputy Group Leader

September 13, 2018

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

Benchmarking Timeline 3

Before 2013

• 2000s: DOE-WIP funds LBNL to collect ESCO project information—and write reports about interesting project-level trends, statistics, etc.; Partnership with NAESCO

• 2011: DOE-WIP supports creation of project “benchmarking sheets”• 2012: DOE-FEMP and WIP support developing online, standardized ESCO project

development and tracking system (eProject Builder)

2013-14

• 2014: eProject Builder (ePB) Version 1.0 released• 2014: As part of ESPC Accelerator – LBNL and our partners begin collecting

state/local feedback on design, functionality, and performance of eProject Builder

2015 and beyond

• 2015-17: ePB made more flexible for state/local market based on feedback• 2018: NAESCO transitions to ePB to collect project-level data for accreditation• 2018: ePB hits milestone of 500+ projects and 1,000 users trained• 2018: ePB named finalist for R&D 100 Award (“Oscars of Innovation”)• 2019: Planned updates to benchmarking sheets and online statistics

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

Benchmarking Sheets (circa 2011) 3

• Created using LBNL/NAESCO database of ESCO projects (predecessor to ePB)

• Range and median of observed values for key project metrics including:

Project installation costAnnual saved energy (kBtu)Annual saved electricity (kWh)Annual % savings relative to baselineSimple payback time

• Six different sheets: Federal Govt, State/Local Govt, K-12 Schools, University/College, Healthcare, Public Housing

• Projects completed from 1990 through 2008

• Part of the U.S. DOE ESPC Toolkit

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

Plan for Benchmarking Project Performance

PastData source:LBNL/NAESCO database of projects (1990-2008)Distribution method: PDF files meant to be printed; ESPC Toolkit

Present/Near-termData source:LBNL/NAESCO database of projects (1990-2018)Distribution method: PDF files and online access via ePB and ESPC Toolkit

Future/Long-termData source:eProject BuilderDistribution method:Online access via ePB and hyperlinks provided via ESPC Toolkit (?)

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

eProject Builder (ePB) Overview

Secure, online energy project data tracking & reporting system Funded by U.S. DOE: FEMP and WIP Free tool developed/maintained by LBNL Enables ESCOs and their customers to securely:

Preserve, track and report project information (including M&V) in perpetuity

Generate range of reports on portfolio of projects Develop project scenarios using standardized calculations Benchmark proposed projects against historical information Enter and track ESPC, UESC, ESA, and other contract types Be audit-ready

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

Overview (continued)

Who is using it? Federal: FEMP ESPC, UESC, ENABLE; Army Corps MATOC State ESPC programs: State Energy Offices/Dept of General

Services ESCOs: free/flexible data archive, easy access for proposals NAESCO Applicants: upload projects for accreditation applications

Why ePB is being offered beyond the federal government? The long game: grease the skids for market/savings growth Standardization and data preservation Enables reliable reporting to stakeholders, ability to tout project

success stories; M&V data holds ESCO and agency staff accountable

Overcome skepticism, smooth transaction costs, and enable market growth

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

Overview (continued)

Inputs and database content Collects data at project- and ECM-level Available data includes: Key project dates Financing and guarantee terms; escalation rates Implementation price (i.e., project installation cost) Baseline usage; estimated and guaranteed savings M&V savings over time

Outputs – report and analysis features Project financial and savings schedules Other project-related documents Portfolio-level analysis: data in form of summary tables and graphs M&V savings results – each year and cumulative

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

ePB Status

Participation

Contains 552 projects representing implementation costs of $4.9B and cumulative guaranteed savings of $10.6B

397 federal projects (IDIQ, ENABLE, Army Corps, UESC, GSA Area-wide)

155 state/local/educational/commercial/industrial/other

> 1,000 individuals have trained on ePB: agency, ESCO, and facilitator staff

Partners, Requirements, and Endorsements

Required for DOE-FEMP IDIQ Version 3 contracts

Used for DOE Qualified List of ESCOs accreditation process

Required or being used by increasing number of state ESPC programs

Endorsed by Energy Services Coalition; ESC is a key training/outreach partner

Endorsed by NAESCO – ePB required as platform for annual accreditation process

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

ePB Benchmarking Page (online)

Benchmarking statistics for different combinations of metrics, market segments, and years

Users query aggregated data from the LBNL/NAESCO database of ~ 6,000 ESCO projects (not projects in ePB)

E NE R G Y T E C HNO L O GIE S AR E A E NE R G Y ANAL Y S I S AND E NV I R O NME NT AL I M P ACT S D I V I S I O N

Near-term Upgrades to Benchmarking Sheets/ePB 3

•New sheets will incorporate data from projects completed over the past decade (2009-2018).

•PDFs provided via ESPC Toolkit and online access via eProject Builder

•Individual sheet for each of six markets: Federal government, State/Local Government, K-12 Schools, University/College, Healthcare, Public Housing

•Range and median of the following metrics:

Project cost/sq. ft. (without financing)Annual saved energy (kBtu) and electricity (kWh) per sq. ft.Annual % savings relative to baselineSimple payback timeAnnual saved energy (kBtu) per dollar invested—NEW Annual saved electricity (kWh) per dollar invested—NEW

ENERGY TECHNOLOGIES AREA ENERGY ANALYSIS AND ENVIRONMENTAL IMPACTS DIVISION

Peter Larsen, PhDEmail: PHLarsen@lbl.govPhone: (510) 486-5015

https://eprojectbuilder.lbl.gov/

https://emp.lbl.gov/projects/energy-saving-performance

33U.S. DEPARTMENT OF ENERGY OFFICE OF ENERGY EFFICIENCY & RENEWABLE ENERGY

Analyzing the Economic Impacts of ESPC Projects

Economic Impact Analysis: Analytical Approaches

Torsten Kieper, Cadmus

Why Economic Impacts?

• Enriched perspective of program benefits• Enhanced cost-effectiveness reviews• Positive messaging to stakeholders, regulators, investors, and customers

• Improved resource planning• Demand-Side Management (DSM) within context of multiple resource options

35

Model Options

• RIMS II• Static regional input-output multiplier matrix

• IMPLAN• Static regional economic impact model

• REMI PI+• Dynamic regional economic impact and forecasting model

36

9/19/2018

Regional Input-Output Modeling System II (RIMS II) Multipliers

Benefits

• Ease of use• Cost (low)• Transparency

Limitations

• Accuracy• Granularity/Flexibility• Static economic conditions• Geography (single region)• No time dimension

9/19/2018 37

IMPLAN

Benefits

• Ease of use• Cost (medium)• Geography (can support multi-

region IO)• Granularity/Flexibility• Transparency

Limitations

• Cost (medium)• Static economic conditions• No time dimension

9/19/2018 38

REMI PI+

Benefits• Dynamic economic conditions• Geography (multi-region)• More variable levers (econometrics and

general equilibrium models)• Granularity/Flexibility

• REMI has many fewer sectors than IMPLAN (for U.S.). IMPLAN has ~540, REMI has ~140 at most

• Time dimension (regional economic forecast baseline)

• Net impacts – Automatically compares gross impacts to baseline “business as usual” scenario

• E3 is new model designed for assessing environmental policies. Optional Emissions Module with energy consumption forecast and CO2 emissions

Limitations

• Cost ($48,000 CAD for 1 year PI+ subscription for Ontario region)

• Complexity of underlying model equations

9/19/2018 39

https://www.focusonenergy.com/sites/default/files/Marketing_Resources/FocusOnEnergy_EconomicImpact_Final_0.pdf

https://www.focusonenergy.com/sites/default/files/WI%20FOE%202015%20to%202016%20Econ%20Impact%20Report-%20Summary-%20Final.pdfhttps://www.focusonenergy.com/sites/default/files/WI%20FOE%202015%20to%202016%20Econ%20Impact%20Report-%20Final.pdf

Wisconsin Focus on Energy TV Ad

• https://www.youtube.com/watch?v=hq1VBRaI5tg

Torsten KieperCadmus

Torsten.Kieper@cadmusgroup.com

44

Questions?

For additional information, contact

Alice Dasekalice.dasek@ee.doe.gov

Thank You!