Residential Behavior ProgramsRTF Subcommittee

Ryan FirestoneSeptember 17, 2015

2

Presentation Outline

• RTF history• 7th Power Plan• Where do savings come from?• Modeling principals for the RTF• Measure categorization (Standard vs. Custom

Protocol)• Next steps for staff and subcommittee

3

RTF History• Standard Protocol approved in March, 2010

– Experimental design (treatment and control/comparison group)– Billing analysis– Normalization for weather and other factors, difference-in-difference

measure of savings– Actually more like custom guidance than a prescriptive methodology

– “The RTF didn’t take up the issue of whether behavior-based programs meet the definition of conservation.” meeting minutes from that meeting

• Status set to “Under review for compliance w/ RTF Guidelines” after the Guidelines were developed.

4

Power Plan• Residential behavior is being modeled as a conservation resource for the first time in the 7th

Plan (draft).– Focus on long term savings

• Savings that aren’t expected to persist weren’t counted

– Curtailment ≠ Conservation• Savings from reductions in utility (e.g., thermostat adjustment) aren’t considered conservation under the Power

Act.

– Not double counting• Potential from Plan-modeled equipment/appliances not re-counted in behavior analysis

– Council staff estimated actions that would satisfy Plan requirements, and saving potential• Reduce water heater setpoint• Reduce lighting HOU• Reduce HVAC usage

– Up to 70% of homes (need control group) – 172 kWh/home– 48 aMW potential

• Analysis files (see “Residential Behavior” section)• 7th Plan Conservation Resource Advisory Committee Presentations

– December 17, 2014– January 16, 2015

5

Uniform Methods Project (UMP)• DOE protocols for determining savings from energy efficiency measures and

programs• Chapter 17: Residential Behavior Programs• Applicable to residential behavior programs with large (1,000’s-10,000’s) number of

participants– Each with individual billing data (e.g., by house)

• Experimental Design: – Randomized Control Trial – subjects randomly assigned to group that gets or does not get

messaging– Randomized Encouragement Design – all subjects can opt in, subjects randomly assigned

to group that gets or does not get encouragement to participate.• Analysis:

– Difference (kWhcontrol – kWhtreatment)

– Difference-in-Difference ( (kWhpre - kWhpost)treatment – (kWhpre-kWhpost)control

– Simple average, panel regression w/ or w/out fixed-effects– Avoid double counting of trackable program savings – analyze participation data– Avoid double counting of untrackable (upstream) program savings (e.g., lighting) – use

surveys

6

Where Do Savings Come From?

Purchase Behavior Usage Behavior

Traditional difference-in-difference billing analyses measure total savings relative to control group

Savings can be from changes in equipment and/or changes in the use of existing equipment

Even more categorization is necessary to determine “claimable” savings and align them with other claimed savings

Avoid double counting with non-behavior program

claimed savings

Conservation CurtailmentUpstream program

participation

Non-program purchase

Non-program removal

Trackable program

participation

Avoid double counting with non-behavior program

claimed savings

Avoid double counting with “momentum”

savings

Need to consider

persistence

Not considered conservation

under standing interpretation of the Power Act.

Not to scale

7

Modeling Principles for an RTF Behavior Measure

• Don’t count savings from curtailment (e.g., thermostat adjustment)

• Avoid double counting– Deduct program-incented equipment-based savings from

equipment programs or from behavior savings– Deduct non-incented equipment-based savings from

momentum savings or from behavior savings

• Don’t apply risk mitigation and capacity credits to things we don’t expect to persist– Near-term energy savings still have value

8

What Kind of Measure is This?• Standard Protocol? (Protocol text plus calculator)

– Requires prescriptive best practice method for estimating savings• No best practice yet for disaggregating savings: Evaluations to date have not had much success

– Method may be dependent on:• Targeted behaviors• Control vs. comparison group• Granularity of billing data• Normalization requirements• Program size

• Custom Protocol? [Staff/CAT recommendation] (Protocol text)– Weigh-in on areas we have something to say, but leave details to programs/evaluators

• Experimental design: sample size, control and treatment group selection• Normalization for weather, rate changes, etc.• What to actions count/not count• Estimating total savings• Estimating disaggregated savings• How to handle persistence

• Subcommittee thoughts? • Would it be worthwhile for the RTF to develop this measure as Standard

Protocol? As Custom Protocol?

9

Measure Cost• For Standard Protocol and Custom Guidance, RTF does

not estimate costs or cost effectiveness

• However, Guidelines say “costs and benefits should be estimated and documented as described in these Guidelines, as appropriate. The RTF may review the research plan for a Program Impact Evaluation, including costs and benefits estimation methods”

• Regional costs include not only program costs, but also customer costs to acquire new equipment

10

Next Steps

• Staff/CAT plan to present this material to RTF at October 20, 2015 meeting.

• Recommendations from Subcommittee?

• Requested actions of staff and subcommittee before October meeting?

11

Additional Material

Slides from January 16, 2015 CRAC Presentation

12

13

14

15

16

17

Additional Material

Slide on Curtailment from March 2, 2010 RTF Meeting

Slide 18

B O N N E V I L L E P O W E R A D M I N I S T R A T I O N

The Power Act and RBBP Historically, the Council has excluded measures and practices that reduced the level of

service or utility (in economic terms) provided to consumers by the current measure or practice. – 839a(3). "Conservation" means any reduction in electric power consumption as a result of

increases in the efficiency of energy use, production, or distribution. [Northwest Power Act, §3(3), 94 Stat. 2698.]

– Savings from lowering thermostats for space heating have not been considered conservation under the Act

– Savings from lowering the thermostat on water heaters from 140 F or 130 F to 120 F have been.

Rationale for behavioral change programs as “utility neutral”, or non-sacrificial?– Persistent savings are unlikely to be sacrifices– Programs are asking customers to reduce behavior when it does not change utility (e.g.,

turning off lights or thermostat down when not in the room)– Recent interest in sustainability has created utility for reducing energy consumption,

particularly enabled through technology or information