Post on 08-May-2022
REPORT
Default Time-of-Use Pricing Pilot Interim Evaluation
Submitted to Southern California Edison
April 1, 2019
Principal authors:
Stephen George, Ph.D., Senior Vice President
Eric Bell, Ph.D., Principal
Aimee Savage, Consultant II
Tyler Lehman, Project Analyst II
CALMAC ID: SCE0434.01
Default Time-of-Use Pricing Pilot Interim Evaluation i
Contents
1 Executive Summary ............................................................................. 1
1.1 Pilot Design & Evaluation .......................................................................... 1
1.2 Overall Findings ......................................................................................... 3
1.2.1 Load Impacts ..................................................................................... 3
1.2.2 Structural Bill Impacts ........................................................................ 5
1.2.3 Customer Attrition .............................................................................. 5
2 Introduction .......................................................................................... 7
2.1 Experimental Design .................................................................................. 8
2.2 Pilot Treatments ....................................................................................... 11
3 Methodology ....................................................................................... 25
3.1 Load Impacts ............................................................................................ 25
3.2 Structural Bill Impacts ............................................................................. 29
3.3 Opt-Out Analysis ...................................................................................... 30
4 Load Impacts ...................................................................................... 31
4.1 Summary of Pilot Rates ........................................................................... 32
4.2 Rate 4 ......................................................................................................... 32
4.3 Rate 5 ......................................................................................................... 37
4.4 Post-enrollment Treatments .................................................................... 41
4.4.1 Enhanced E&O ................................................................................ 41
4.4.2 Level Payment Plan ......................................................................... 44
4.5 Comparison Across Rates ....................................................................... 44
5 Structural Bill Impacts ....................................................................... 47
5.1 Rate 4 ......................................................................................................... 47
5.2 Rate 5 ......................................................................................................... 51
Default Time-of-Use Pricing Pilot Interim Evaluation ii
5.3 Comparison Across Rates ....................................................................... 55
6 Customer Attrition .............................................................................. 57
6.1 Pre-enrollment Opt-outs .......................................................................... 57
6.1.1 Pairwise Comparisons ..................................................................... 57
6.2 Post-enrollment Opt-Outs ........................................................................ 62
7 Key Findings ....................................................................................... 67
7.1 Load Impacts ............................................................................................ 67
7.2 Structural Bill Impacts ............................................................................. 68
7.3 Customer Attrition .................................................................................... 68
7.4 A Note About Comparing Default and Opt-in Results ........................... 69
Default Time-of-Use Pricing Pilot Interim Evaluation iii
Glossary of Acronyms
CARE California Alternate Rates for Energy
CCA Community Choice Aggregator
CPUC California Public Utilities Commission
DiD Difference-in-differences
E&O Education and outreach
FERA Family Electric Rate Assistance
IOU Investor owned utility
ITT Intention-to-treat
LPP Level payment plan
ME&O Marketing, education and outreach
OAT Otherwise applicable tariff
RED Randomized encouragement design
TOU Time of use
WG Working Group
Default Time-of-Use Pricing Pilot Interim Evaluation 1
1 Executive Summary
This document constitutes the interim evaluation report for Southern California Edison’s,
residential default time-of-use (TOU) pricing pilot. This pilot was implemented in response to
California Public Utilities Commission (CPUC) Decision 15-07-001. A key objective of the pilot is
to develop insights that will help guide SCE’s approach to implementation of default TOU pricing
for the majority of residential electricity customers and the CPUC’s policy decisions regarding
default pricing.
Findings from the first summer—June through September 2018—are documented in this report.
This report also contains detailed background information on the pilot, describes the pilot design
and the evaluation methodology used for analysis, discusses SCE’s pilot implementation and
treatments, and presents load impacts, bill impacts, and opt-out findings covering the 2018
summer period.
The pilot tested two different TOU rate options. Approximately 200,000 households were
assigned to each of the TOU rates, and an additional 200,000 were retained in the study on the
standard tiered rate to act as a control group for those who were placed on the new tariffs. After
receiving multiple notifications regarding the fact that their rate will change if they did not take
action by a certain date, customers had the option of opting out prior to the rate change and
staying either on their otherwise applicable tariff or choosing an alternative rate plan other than
the one they were to be defaulted on. If a customer took no action, they were placed on the
default rate associated with their assigned group. The initial default notifications are described in
detail in Section 2.2. These notifications included a rate analysis comparing each customer’s bill
based on the new TOU rate with their bill under the otherwise applicable tariff using historical
customer data along with additional education and outreach (E&O) material.
1.1 Pilot Design & Evaluation Evaluation of the default pilot focused on a number of important research objectives, including:
SCE’s operational readiness to default large numbers of customers onto TOU rates
over a short time. Relevant metrics include call volume, billing exception processing,
database capabilities, tracking systems, rate change and bill processing, system
enhancements, and bill protection processing.
The impact of different marketing, education and outreach (ME&O) strategies on
awareness of rate options, opt-out rates, engagement with the TOU rate and customer
perceptions while on a TOU rate. Specific ME&O options examined included variation in
the type of structural bill information provided in conjunction with the default notifications,
two messaging strategies, and different format and content for welcome package
materials.
The average peak and off-peak change in energy usage by customers enrolled on
each default rate (referred to as rates 4 and 5 to reflect differences in the start time for
the peak period, 4 PM versus 5 PM).
SECTION 1 EXECUTIVE SUMMARY
Default Time-of-Use Pricing Pilot Interim Evaluation 2
The bill impacts for customers enrolled onto each rate.
The opt-out rate for customers defaulted onto each rate under each notification
treatment.
The impact of options such as level payment plans (LPP) on customer retention on
each rate as well as on load and bill impacts and customer perceptions while on their
default TOU rate.
An assessment of operational readiness is not included in this report. Survey-related metrics
such as awareness, customer satisfaction, and others have been obtained through two surveys
and are reported elsewhere.
The pilot was structured as a randomized encouragement design (RED) experiment. With a
RED, different randomly selected samples of customers are offered different experimental
treatments (in this case, a TOU rate or different content or messaging in the recruitment
materials) and another random group of customers is not offered anything (e.g., the control
group). Some who are offered the treatment take it and some do not. Because each sample is a
statistical clone of the other due to the random selection (especially in this case where sample
sizes are quite large), comparing the behavior of the encouraged group with that of the control
group allows for an unbiased assessment of the impact of the treatment. This analysis requires
a two-step process in order to isolate the impact of the encouragement (e.g., the offer of a
treatment) from the treatment itself, as explained more fully in Section 3.
Load and bill impacts were estimated for four different climate regions in SCE’s service territory
(hot, moderate, cool, and Climate Zone 10). For the moderate and cool climate regions,
estimates were also made for two customer segments, CARE/FERA customers and non-
CARE/FERA customers. CARE/FERA customers in the hot climate region and Climate Zone 10
were not allowed to be enrolled on TOU tariffs using default recruitment. As such, comparisons
across the two hot and two more moderate regions not only reflect differences in climate but
also differences in the mix of customers. Also, differences in load impacts across customer
segments at the service territory level reflect not just differences across segments, but also
differences in the mix of customers across climate regions for each segment. These differences
must be kept in mind when making comparisons across segments and climate regions.
The difference in bills on the TOU rates compared with bills under the otherwise applicable tariff
(OAT) are comprised of two components – differences due simply to the rates, holding behavior
constant, and differences due to changes in behavior as a result of the difference in price
signals. The first type of difference is known as a structural bill impact, and can be computed
based on usage data prior to customers enrolling on the new rate. Because bill impacts can
vary rather significantly across seasons, and since this report is based only on usage for the
summer season, it does not present behavioral impacts or total bill impacts. Those will be
presented after customers have been on the new tariffs for a full year. This report presents
information on structural bill impacts for summer, winter and an entire year based on
pretreatment data.
In addition to load and bill impacts, another important metric is customer opt-out rates.
Comparisons of pre-enrollment opt-out rates across rate options are indicators of the relative
SECTION 1 EXECUTIVE SUMMARY
Default Time-of-Use Pricing Pilot Interim Evaluation 3
preferences of customers for each rate option and comparisons across notification content and
messaging within a rate option can provide insights about the relative effectiveness of these
notification alternatives. Comparisons across customer segments and climate regions reflect the
influence of these factors on customer acceptance. In this report, pairwise comparisons of opt-
out rates are presented by rate, type of rate comparison, and type of messaging presented in
the pre-enrollment informational communications. Finally, post-enrollment opt-out rates are
presented by rate, CARE/FERA status, climate region, and post-enrollment treatment.
1.2 Overall Findings The first summer of SCE’s default TOU pilot has produced a large amount of information that
will help guide SCE’s approach to implementation of default TOU pricing. However, it must be
kept in mind that these load impact findings are based on only the summer months. Load
impacts are going to differ significantly during winter months and the actions of TOU pilot
participants may be quite different over the course of a full year.
As described above, differences in load and bill impacts and opt-out rates across customer
segments at the service territory level reflect not just differences across segments, but also
differences in the mix of customers across climate regions. CARE/FERA customers in the hot
climate region and Climate Zone 10 were not allowed to be enrolled on TOU tariffs using default
recruitment. Comparisons between CARE/FERA and non-CARE/FERA customers are valid for
the moderate and cool climate regions and comparisons across all four climate regions are valid
for non-CARE/FERA customers. However, comparisons across segments at the service territory
level reflect both differences in behavior across segments as well as differences in the
participation of segments across climate regions.
If comparisons are made between SCE’s default rates and the prior opt-in pilot, it is
important to note that the months included in the evaluation, peak period hours, prices,
and inclusion of CARE/FERA customers all changed between the opt-in and default
pilots. Therefore, the differences observed between the pilots are not solely a difference in
customer response to opt-in versus default enrollment strategies. With these cautions in mind,
the remainder of this section provides a high level summary of key findings.
1.2.1 Load Impacts
Table 1-1 presents the average weekday peak period load reduction for each pilot rate. Key
findings for load impacts are summarized in following the table.
SECTION 1 EXECUTIVE SUMMARY
Default Time-of-Use Pricing Pilot Interim Evaluation 4
Table 1-1: Peak Period Load Reductions on Average Weekday
Utility Metric Rate 4 Rate 5
SCE
Peak Period Hours
4-9 PM 5-8 PM
% Impact 1.50% 2.00%
Absolute Impact (kW)
0.02 kW 0.03 kW
On average, default customers on both Rates 4 and 5 produced small but statistically
significant, peak-period load reductions. Peak period load reductions averaged roughly
1.5% for Rate 4 and 2.0% for Rate 5.
Load reductions for the common hours shared by the two rates (5 to 8 PM) were greater
for Rate 5 than for Rate 4, likely because of the higher peak period price per kWh. It’s
also possible the shorter peak period of Rate 5 allowed for greater flexibility in customer
response to the price signal. The difference was statistically significant for the territory as
a whole, the moderate climate region, and Climate Zone 10.
Statistically significant but small reductions in daily electricity use were found for both
rates and in all climate regions. It appears that the average customer in SCE’s service
territory was more likely to reduce overall usage during the peak period rather than shift
usage to off-peak hours.
The pattern of load reductions across climate regions in absolute terms was consistent
between the two rates but was slightly different in percentage terms. Absolute peak
period load reductions were largest in Climate Zone 10 and the hot climate regions, but
these segments did not include CARE/FERA customers. Absolute impacts were smallest
in the cool climate region, which included CARE/FERA and non-CARE/FERA
customers.
In the moderate and cool climate regions, non-CARE/FERA customers typically had
statistically significantly greater peak period impacts compared to CARE/FERA
customers. One exception was households in the moderate climate region on Rate 4,
where the difference was not statistically significant. This finding is consistent with the
opt-in TOU pilot.
With one exception, the incremental peak period impact among households who
received the Enhanced E&O treatment compared to households that did not was not
statistically significant. In other words, the additional messaging did not increase peak
period impacts. The exception was CARE/FERA customers in the moderate climate
region who had an incremental increase in load impacts equal to about 0.6%.
The offer to high bill volatility, low income customers to enroll on the Level Pay Plan as a
way of managing volatility in bills across months and seasons was only taken up by a
very small number of customers.
SECTION 1 EXECUTIVE SUMMARY
Default Time-of-Use Pricing Pilot Interim Evaluation 5
Overall, the load impacts were generally in the expected range established during the default
pilot design planning stages. The opt-in pilot was designed in a way to be more reflective of opt-
out enrollment conditions by using the “pay-to-play” recruitment strategy. However, it was still
expected that load impacts would be lower under default conditions due to potentially lower
customer awareness rates, and the unavoidable customer self-selection bias of an opt-in
recruitment strategy where engaged customers are more likely to enroll.
1.2.2 Structural Bill Impacts
Structural bill impacts were estimated for summer, winter and the year as a whole. Key findings
include the following:
Rate 4 and Rate 5 have very similar distributions of structural benefiters, non-benefiters,
and customers in the neutral bill impact category of ±$3/month.
A majority of customers are neither structural benefiters nor non-benefiters on an annual
basis. Over 30% of non-CARE/FERA customers are structural non-benefiters while
fewer than 20% of CARE/FERA customers fall into the same category. However, the
CARE/FERA group does not include customers in the hot climate region where bill
increases under the TOU rates are more likely to occur.
Over 50% of customers in the hot climate region and Climate Zone 10 are structural
non-benefiters on an annual basis. In the summer months, about 80% of customers in
these regions are structural non benefiters while about 15% fall into the neutral category.
Roughly 40% and 60% of CARE/FERA customers in the moderate and cool climate
regions, respectively, are neither structural benefiters nor non-benefiters in the summer
months.
In the winter months, between 25% and 30% of non-CARE/FERA customers in all
climate regions would save money on TOU rates. This outcome is expected because
SCE’s OAT is not seasonally differentiated. The TOU rates are seasonally differentiated
with higher prices during the summer and lower prices during the winter.
The structural bill impacts were generally as expected for customers transitioning from a non-
seasonally differentiated OAT to a seasonally differentiated TOU rate with higher peak period
prices in the summer and lower peak period prices in the winter. On average, a large portion of
customers are structural non-benefiters in the summer, but many are able to offset the higher
priced summer months with lower bills in the winter to reach the neutral category on an annual
basis.
1.2.3 Customer Attrition
Customer participation rates were tracked separately for the pre-enrollment period and the post
enrollment period. During the pre-enrollment period, customers selected to participate in the
pilot could opt-out of the pilot and stay on their current rate, select an alternative TOU rate, or
take no action and be enrolled on the assigned TOU pilot rate.
During the post enrollment period customer attrition is driven by three very different factors. One
is customers who move, referred to as customer churn. Another is customers who become
ineligible as a result of factors such as installing solar, going onto medical baseline, or switching
SECTION 1 EXECUTIVE SUMMARY
Default Time-of-Use Pricing Pilot Interim Evaluation 6
to service from a Community Choice Aggregator (CCA). The final factor is customers who
consciously opt out of the rate because they are unhappy being on a TOU rate.
Key findings concerning customer attrition include the following:
When the pre-enrollment opt-out decision is defined as selecting the OAT rather than the
offered default rate, the difference in opt-out rates between Rates 4 and 5 were very
small and not statistically significant. However, when the opt-out decision is defined as
choosing either the OAT or the alternative TOU rate, the opt-out rate was about 5%
higher (one percentage point) for Rate 4 than for Rate 5. This finding, along with the fact
that more customers offered Rate 4 chose Rate 5 than vice versa, indicates that the
average customer has a small but statistically significant preference for Rate 5 over Rate
4.
Customers presented with loss aversion messaging were slightly more likely to opt out
before enrollment compared to those who received messaging focused on an
opportunity to save money on TOU. This difference was statistically significant.
There was no difference in pre-enrollment opt-out rates between customers who
received a monthly rate comparison and those who received a seasonal rate
comparison. Though, it should be noted that a total annual bill comparison was also
presented to both informational treatment groups.
Post-enrollment opt-out rates were very small and fell between 0.7% and 1.0% for
CARE/FERA and non-CARE/FERA customers in all climate regions. This indicates the
vast majority of customers stay on the rate once they are enrolled on a TOU rate.
Customers on Rate 4 were statistically significantly more likely to opt out post-
enrollment. Again, it is possible the longer peak period was less desirable for some
customers. However, the difference was very small (1.3% vs. 1.2%).
The analysis of opt-out rates shows a small but statistically significant preference for Rate 5,
with its shorter peak period but higher peak price, over Rate 4. There was also a slight
advantage for the “Opportunity to Save” messaging over the “Loss Aversion” message. There
were no observed differences in opt-out rates between customers receiving seasonal versus
monthly structural bill information. In most instances, the pre-enrollment opt-out rate was
roughly 20%, but once customers enrolled on the rate, very few left.
Default Time-of-Use Pricing Pilot Interim Evaluation 7
2 Introduction
In Decision 15-07-001, the California Public Utilities Commission (CPUC or the Commission)
ordered California’s three investor owned utilities (IOUs) to conduct certain “pilot” programs
and studies of residential Time-of-Use (TOU) electric rate designs (TOU Pilots and Studies)
beginning in 2016, and to file applications no later than January 1, 2018 proposing default TOU
rates for residential electric customers. The IOUs were also directed to form a working group
(TOU Working Group) to address issues regarding the TOU pilots and to hire one or more
qualified independent consultants to assist with the design and implementation of the TOU
Pilots and Studies. The TOU Working Group (WG) was comprised of 37 entities and included
almost 100 people. Nexant, Inc. was engaged as the independent consultant.
Although the primary focus of the TOU pilots was to provide insights that would guide default
implementation, customers were not allowed to be defaulted onto TOU rates prior to January
2018. As such, in 2016, the IOUs implemented pilots based on opt-in enrollment. The pilots,
based on a “pay-to-play” randomized control trial, were designed in a way intended to be more
reflective of opt-out enrollment conditions. The pilot design and results from these pilots are
documented in a number of reports and insights from these pilots were used to guide the design
of the default pilots that are the focus of this evaluation.1
In late 2016, Nexant worked with the TOU Working Group to develop designs for the default
pilots. The design report2 was used as input to Advice Letter filings by SCE and the two other
IOUs. On December 16, 2016 SCE submitted Advice Letter 3531-E3 detailing the proposal for
the default TOU pilot. At the request of the CPUC, and in response to the Office of Ratepayer
Advocates protest, SCE submitted Advice Letter 3531-E-A4 on February 24, 2017 as a
supplemental filing to provide additional information on the original Proposed Default Time-of-
Use (TOU) Pilot plan. The CPUC issued Resolution E-48475 on May 12, 2017 approving the
1 George, S., Sullivan, M., Potter, J., & Savage, A. (2015). Time-of-Use Pricing Opt-in Pilot Plan. Nexant, Inc. (hereafter referred to
as the TOU Pilot Design Report).
SCE: Advice Letter 3335-E; PG&E: Advice Letter 4764-E; and SDG&E: Advice Letter 2835-E.
SCE: Resolution E-4761; PG&E: Resolution E-4762; and SDG&E: Resolution E-4769.
The First Interim Report can be found here: http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=6442453144 Additional related documents on the CPUC website can be found here: http://www.cpuc.ca.gov/General.aspx?id=12154
The Second Interim Report is contained in two volumes, one authored by Nexant covering the load and bill impact analysis and the second, authored by Research Into Action covering the second survey. The Nexant report can be found at the following link: http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=6442455573 The RIA report can be found at: http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=6442455572
The Final Report can be found here: http://www.cpuc.ca.gov/WorkArea/DownloadAsset.aspx?id=6442457172 Additional related documents on the CPUC website can be found here: http://www.cpuc.ca.gov/General.aspx?id=12154
2 https://www1.sce.com/NR/sc3/tm2/pdf/3531-E.pdf (See Appendix A, starting on Page 86 of the document)
3 https://www1.sce.com/NR/sc3/tm2/pdf/3531-E.pdf
4 https://www1.sce.com/NR/sc3/tm2/pdf/3531-E-A.pdf
5 http://docs.cpuc.ca.gov/PublishedDocs/Published/G000/M183/K366/183366304.PDF
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 8
pilot plans contained in Advice Letters 3531-E and 3531-E-A and established that SCE’s default
pilot will gather information on the following objectives:
1. SCE’s operational readiness to default large numbers of customers onto TOU rates over
a short time. Relevant metrics include call volume, billing exception processing,
database capabilities, tracking systems, rate change and bill processing, system
enhancements, and bill protection processing.
2. The impact of different marketing, education and outreach (ME&O) strategies on
awareness of rate options, opt-out rates, engagement with the TOU rate and customer
perceptions while on a TOU rate. Specific ME&O options examined included variation in
the type of structural bill information provided in conjunction with the default notifications,
two messaging strategies, and different format and content for welcome package
materials.
3. The average peak and off-peak change in energy usage by customers enrolled on each
default rate (referred to as rates 4 and 5 to reflect differences in the start time for the
peak period, 4 PM versus 5 PM).
4. The bill impacts for customers enrolled onto each rate.
5. The opt-out rate for customers defaulted onto each rate under each notification
treatment.
6. The impact of options such as level payment plans (LPP) on customer retention on each
rate as well as on load and bill impacts and customer perceptions while on their default
TOU rate.
An assessment of operational readiness— objective 1— is not included in this evaluation and
survey-related metrics such as awareness, customer satisfaction, and others—objective 2— are
being addressed through a separate contract with a survey firm. This evaluation report focuses
primarily on estimating load and bill impacts and opt-out rates for various treatments –
objectives 3 through 6.
The remainder of this section summarizes the pilot design and the specific rate and other
treatments included in the pilot. Section 3 provides an overview of the analysis methods that
were used to estimate load and bill impacts and to analyze opt-out rates. Sections 4, 5 and 6
present the analysis results for load impacts, bill impacts and opt-out rates, respectively. Finally,
key findings for objectives 3 through 6 above are presented in Section 7.
2.1 Experimental Design A key objective of any pilot or experiment is to establish a causal link between the experimental
treatments (e.g., TOU rates, messaging strategy, etc.) and the outcomes of interest (e.g.,
load impacts, changes in bills, customer satisfaction, etc.). The best way to do this is through
strict adherence to a rigorous experimental design that isolates the treatments of interest from
other factors that might influence impacts of interest. A randomized encourage design (RED)
was used to meet this standard. With an RED, different randomly selected samples of
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 9
customers are offered different experimental treatments (in this case, a TOU rate or different
content or messaging in the recruitment materials) and another random group of customers is
not offered anything (e.g., the control group). Some who are offered the treatment take it and
some do not. Because each sample is a statistical clone of the other due to the random
selection (especially in this case where sample sizes are quite large), comparing the behavior of
the encouraged group with that of the control group allows for an unbiased assessment of the
impact of the treatment. This analysis requires a two-step process in order to isolate the impact
of the encouragement (e.g., the offer of a treatment) from the treatment itself (e.g., enrolling on
the TOU rate), as explained in Section 3.
In this pilot, three random samples of customers of roughly 200,000 households each were
drawn. One group was offered default Rate 4, one was offered Rate 5 (the rate offers were the
encouragement), and the third was not offered any new rate option. Within each of the two
encouraged groups, random samples comprised of roughly 50,000 households received
different structural bill comparisons and messaging content as explained more fully in Section
2.2. Because of the large sample sizes and the random selection process, each sample is a
statistical clone of the others. As such, any observed differences across the groups in metrics of
interest (e.g., load impacts, bill impacts, opt-out rates, etc.) can be attributable to the impact of
the treatment offered to each sample or to random error in the sampling process. With such
large samples, random error is quite small, which means that the experiment has a very high
degree of internal validity.
A comparison of pretreatment load shapes and other observable characteristics available for all
customers shows how well the random sampling was done at SCE. Figure 2-1 shows a
comparison of customer characteristics between each rate treatment group and the control
group and Figure 2-2 and Figure 2-3 show a comparison of average weekday load shapes for
the rate treatment and control groups based on pretreatment data. As can be seen, there is no
observable difference in any of these characteristics between the treatment and control groups,
which ensures that the load and bill impacts and opt-out rates reported in subsequent sections
of this report are unbiased and have a very high degree of internal validity.
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 10
Figure 2-1: Control Group Demographics Validation
Figure 2-2: Control Group Load Profile Validation – Rate 4
0%
10%
20%
30%
40%
50%
60%
70%
Co
nne
cte
d
Co
nts
tra
ined
Dis
enga
ged
Gre
en
Elit
e
Pra
gm
atists
Oth
er
No
n H
isp
anic
No
Sp
anis
h
Pre
fers
Sp
anis
h
Both
Un
kn
ow
n
EB
PP
EB
PP
WP
PT
R
SD
P
My A
cco
unt
Sin
gle
Fam
ily
MyA
ccou
nt
Persona Language DR Other
Perc
en
t o
f C
usto
me
rs
Rate 4 Rate 5 Control
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1 2 3 4 5 6 7 8 9 101112131415161718192021222324
kW
Hour Ending
Treatment Control
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 11
Figure 2-3: Control Group Load Profile Validation – Rate 5
The pilot population includes customers in four climate regions: hot, moderate, cool and
California’s Climate Zone 10. Climate Zone 10 was included as part of the moderate region in
SCE’s Opt-In TOU pilot, but was found to have weather conditions more similar to the zones in
the hot region. Customers enrolled in the California Alternate Rates for Energy (CARE) and
Family Electric Rate Assistance (FERA) programs in the moderate and cool climate region were
eligible for default TOU but CARE/FERA customers in the hot climate region and Climate Zone
10 were excluded from default notification.
2.2 Pilot Treatments Table 2-1 summarizes the default notification treatments that were tested in the pilot.
Approximately 400,000 residential customers received default notifications. As seen in the table,
an equal number of customers were defaulted onto each of two different TOU rates. In addition,
half of the customers on each rate received structural bill information comparing their bills on the
default rate and on the otherwise applicable tariff (OAT) on a seasonal basis and the other half
received structural bill comparisons on a monthly basis. Finally, half of all customers received a
message that focused on managing bills by avoiding usage during high priced peak periods (a
“loss aversion” message) while the other half received a message that focused on managing
bills by using more electricity during low priced, off-peak periods (the “opportunity” message
group). The primary objectives of these tests were to determine:
If opt-out rates differed between Rates 4 and 5;
If opt-out rates differed with the content of the notification information concerning rate
comparisons and the nature of the message employed;
What the load and bill impacts were for Rates 4 and 5 and whether they differed by a
statistically significant amount in light of the differences in the price ratios and rate
periods.
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1 2 3 4 5 6 7 8 9 101112131415161718192021222324
kW
Hour Ending
Treatment Control
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 12
Table 2-1: Default Notification Treatments
Notification Cell
Rate Rate
Comparison Messaging Sample Size
1
4
Monthly Loss Aversion 49,998
2 Opportunity to Save 49,999
3 Seasonal
Loss Aversion 50,000
4 Opportunity to Save 50,000
5
5
Monthly Loss Aversion 50,000
6 Opportunity to Save 49,998
7 Seasonal
Loss Aversion 50,000
8 Opportunity to Save 49,999
Figure 2-4 and Figure 2-5 show the timing of the rate periods for Rates 4 and 5 and the prices in
each period. As seen, Rate 4 is a two-period rate in summer and a three-period rate in winter.
Rates and rate periods are the same on weekdays and weekends. The peak period in both
summer and winter is from 4 to 9 PM. In winter, there is a super off-peak period from 8 AM to 4
PM. The peak-to-off-peak price ratio in summer is 1.8:1 for usage above the baseline quantity.
In winter, the peak and off-peak prices are very similar, but super off-peak prices are about 65%
lower than peak-period prices. The structure of Rate 5 is similar to that of Rate 4 except that
the peak period is shorter (5 to 8 PM) and peak-period prices are higher, as are the peak-to-off-
peak price ratios.
Figure 2-4: Default Pilot Rate 4
Figure 2-5: Default Pilot Rate 5
Figure 2-6 and Figure 2-7 show examples of two default notification letters. Figure 2-6 is a
notification for a customer receiving an offer for Rate 4. In this case, it includes the “loss
aversion” message and a seasonal structural bill comparison. Figure 2-7 is a notification for a
customer receiving an offer for Rate 5 along with the “opportunity to save” messaging and an
annual structural bill comparison. The loss aversion or opportunity messaging can be found in
the box to the left of the orange circle with the letter “A”. The seasonal or monthly bill
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Summer
Winter
Summer
Winter
Day Type Season
Hour Ending
Weekday
Weekend
Off-Peak (22¢) Peak (40¢)
Off-Peak (22¢) Mid-Peak (26¢)
Off-Peak (27¢) Super Off-Peak (16¢)
Off-Peak (27¢) Super Off-Peak (16¢)
Mid-Peak (28¢)
Mid-Peak (28¢)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Summer
Winter
Summer
Winter
Day Type Season
Hour Ending
WeekendOff-Peak (27¢) Super Off-Peak (16¢)
WeekdayOff-Peak (27¢)
Peak (47¢)Off-Peak (22¢)
Mid-Peak (29¢)
Mid-Peak (28¢)Off-Peak (22¢)
Super Off-Peak (16¢) Mid-Peak (29¢)
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 13
comparisons are included on the second page of the letter, on a business reply card labeled “B.”
Finally, rate information can be found on the third page, labeled “C.”
Figure 2-6: Default Notification – Rate 4 – Seasonal Rate Comparison – Loss Aversion
A
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 14
B
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 15
C
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 16
Figure 2-7: Default Notification – Rate 5 – Monthly Rate Comparison – Opportunity to Save
A
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 17
B
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 18
Table 2-2 summarizes the various treatments that were examined after customers enrolled on
the new TOU rates and the samples sizes for each treatment group. The total sample for the
post-default treatments is estimated to equal roughly 310,000 customers. The drop of 90,000
from the 400,000 customers who received default notifications is attributable to normal customer
churn, changes in eligibility (e.g., medical baseline, etc.), customers who enrolled on a TOU rate
other than the one offered in the default notice, and customers who chose to stay on the OAT.
The enrolled population on each of the default rates was divided equally into those slated to
receive basic or enhanced welcome packets and ongoing education and outreach (E&O)
communication, and then segmented further into two groups, those deemed to be most
C
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 19
impacted by bill volatility and those who are not. The segment of customers impacted by bill
volatility was considered to be income-constrained customers who were expected to experience
increased seasonal bill differentials under the default TOU rate. The criteria for inclusion in this
segment are CARE or FERA enrolled customers for whom the (absolute) difference in average
summer and average winter bills is higher under their default TOU rate than under the OAT.
Approximately 17% of all enrolled customers were identified as impacted by bill volatility. As
seen in Table 2-2, this segment of customers is further divided into two equal groups, with one
group receiving information on SCE’s Level Payment Plan (LPP) as a means of managing
month-to-month bill volatility.
Table 2-2: Post-Enrollment Treatments
Aftercare Cell
Rate Communication Impacted by Bill
Volatility LPP
Promotion Sample
Size
1
4
Enhanced E&O
Impacted by Bill Volatility
LPP Promotion 6,448
2 No Promotion 6,448
3 Not Impacted No Promotion 64,245
4
Basic E&O
Impacted by Bill Volatility
LPP Promotion 6,420
5 No Promotion 6,418
6 Not Impacted No Promotion 64,245
7
5
Enhanced E&O
Impacted by Bill Volatility
LPP Promotion 6,646
8 No Promotion 6,644
9 Not Impacted No Promotion 65,311
10
Basic E&O
Impacted by Bill Volatility
LPP Promotion 6,705
11 No Promotion 6,703
12 Not Impacted No Promotion 65,195
Figure 2-8 and Figure 2-9 show the basic and enhanced welcome packets. One difference
between the two welcome packets concerns formatting, with the basic welcome packet being a
simple business letter and the enhanced packet being more like a glossy brochure. The
enhanced packet also contains more information about available programs and tools for
managing energy use and bills under TOU rates, such as incentives for smart thermostats,
Budget Assistant (a usage alert tool), and online tools accessible through My Account, among
others.
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 20
Figure 2-8: Basic Welcome Packet Format and Content
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 21
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 22
Figure 2-9: Enhanced Welcome Packet Format and Content
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 23
SECTION 2 INTRODUCTION
Default Time-of-Use Pricing Pilot Interim Evaluation 24
Default Time-of-Use Pricing Pilot Interim Evaluation 25
3 Methodology
As discussed in Section 2, this report provides load impacts for the initial summer period of the
pilot (June through September, 2018), and structural bill impacts for each of the two rate
treatments tested at SCE for various customer segments and climate regions. The incremental
load impacts for the enhanced E&O treatment were also tested. Opt-out rates for various rate
and notification treatments are also reported in Section 6. This section summarizes the
methodological approaches used to estimate the metrics of interest for each pilot treatment. The
discussion is organized into three broad sections summarizing the approach for estimating load
impacts, structural bill impacts, and opt-out rates.
3.1 Load Impacts The estimation of load impacts by rate period and changes in annual and seasonal energy use
for each pilot rate are key pilot objectives. Also of interest is how load impacts vary across
climate regions and customer segments (e.g., non-CARE/FERA customers and CARE/FERA
customers) for two of the four climate regions, since CARE/FERA customers could not be
defaulted in the two hot climate regions. The approach used to estimate load impacts is
summarized below.
As discussed above, the pilot involves a randomized encouragement experimental design. With
a RED structure involving a single rate treatment of interest (for simplicity), the study sample is
randomly divided into two groups. One group is offered the treatment and the other is not. The
group offered the treatment is referred to as the encouraged group and the group not offered
the treatment is referred to as the control group. Some people in the encouraged group will
accept the treatment and others will not. With a RED, impacts for those who accept the
treatment offer are estimated through a two-step process. In the first step, loads by time period
for the encouraged group are subtracted from loads for the control group. As stated above, the
encouraged group includes both those who accept the encouragement (that is, those who enroll
on the new rate) and those who do not. The estimated load impact based on these two groups
of customers is referred to as the intention-to-treat (ITT) effect. In the second analysis step, the
ITT estimate is divided by the percent of the encouraged group who take up the treatment offer.
This value represents the impact for those who took the treatment (referred to as the impact of
the treatment on the treated).6 A conceptual overview of the RED design and analysis for
estimating load impacts is shown in Figure 3-1.
6 This second stage calculation relies on an assumption that decliners are not influenced by the fact that they received an offer. If,
for example, decliners shifted load simply because they received an offer to go on a new rate, load impact estimates for non-decliners would be biased upward.
SECTION 3 METHODOLOGY
Default Time-of-Use Pricing Pilot Interim Evaluation 26
Figure 3-1: Design and Analysis Schematic for a RED Experiment
For the pilot, the first stage ITT impact was estimated using what is called a difference-in-
differences (DiD) analysis. This method estimates impacts by subtracting treatment customers’
loads (or in this first stage, the encouraged customers’ loads) from control customers’ loads in
each hour or time period after the treatments are in place and subtracts from this value the
difference in loads between treatment and control customers for the same time period in the
pretreatment period. Subtracting any difference between treatment and control customers prior
to the treatment going into effect adjusts for any difference between the two groups that might
occur due to random chance.
The DiD calculation can be done arithmetically using simple averages or can be done using
regression analysis. Customer fixed effects regression analysis allows each customer’s mean
usage to be modeled separately, which reduces the standard error of the impact estimates
without changing their magnitude. Additionally, regression software allows for the calculation of
standard errors, confidence intervals, and significance tests for load impact estimates
that correctly account for the correlation in customer loads over time.7 Implementing a DiD
through simple arithmetic would yield the same point estimate but it would not generate
confidence intervals.
A typical regression specification for estimating impacts is shown below:
𝑘𝑊𝑖,𝑡 = 𝛼𝑖 + 𝛿treat𝑖 + 𝛾post𝑡 + 𝛽(treatpost)𝑖,𝑡 + 𝑣𝑖 + 휀𝑖,𝑡
In the above equation, the variable kWi,t equals electricity usage during the time period of
interest, which might be each hour of the day, peak or off-peak periods, daily usage or some
7 More accurately, they account for the correlation in regression errors within customers over time.
Exclusions
Utility
Population
Target
Population
Randomized
into Control and Treatment
Groups
Treatment
Group
Control
Group
Control
Group
Accepters
(X%)
Treatment
Group Placed on Program
Pre-
treatment
Post-
treatment
Intent-to-Treat-
Impact (Differences-in-
DifferenceskWht
post
kWhcpre
Decliners
(100-X%)
Treatment Effect
on the Treated
Divide intent-to-
treat by percent of treatment group
who accepted treatment (x%)
kWhtpre
kWhcpost
SECTION 3 METHODOLOGY
Default Time-of-Use Pricing Pilot Interim Evaluation 27
other period. The index i refers to customers and the index t refers to the time period of interest.
The estimating database would contain electricity usage data during both the pretreatment and
post-treatment periods for both treatment (encouraged) and control group customers. The
variable treat is equal to 1 for treatment customers and 0 for control customers, while the
variable post is equal to 1 for days after the TOU rate has been implemented and a value of 0
for days during the pretreatment period. The treat post term is the interaction of treat and post
and its coefficient β is a difference-in-differences estimator of the treatment effect that makes
use of the pretreatment data. The primary parameter of interest is β, which provides the
estimated demand impact during the relevant period. The parameter ai is equal to mean usage
for each customer for the relevant time period (e.g., hourly, peak period, etc.). The vi term is the
customer fixed effects variable that controls for unobserved factors that are time-invariant and
unique to each customer.
Customer attrition is an important factor to address in the load impact analysis. Customer
attrition stems from four factors; customers who move (referred to as churn); customers who
become ineligible after enrolling in the pilot; customers who opted out before the pilot began,
and customers who dropped off the rate after enrollment because they were unhappy being on
the TOU rate. Customer churn and changes in eligibility should be the same for both treatment
and control customers. As such, dropping customers from both treatment and control groups
due to churn and changes in eligibility does not introduce selection effects.
The majority of load impact estimates reported in Section 4 are based on a comparison of loads
between each treatment group and the control group. Estimates for customer segments and
climate regions are developed by first partitioning the treatment and control groups into samples
for each climate region and/or customer segment of interest and then applying the analysis
method outlined above to the partitioned data. An exception to this approach occurs when
examining the incremental impact of the enhanced E&O options summarized in Section 2.2. For
this analysis, data for the two treatment groups (e.g., standard and enhanced E&O treatments)
are combined. The standard treatment acts as a control group for the enhanced treatment and
the estimated impact represents the incremental impact of the enhanced treatment
The load impact estimates reported here conform to the requirements for ex post evaluation of
non-event based demand response resources as indicated in California’s Demand Response
Load Impact Protocols.8 These protocols require that load impacts in each hour be developed
for the average weekday and monthly system peak days for each month of the year. Although
not explicitly required by the protocols, load impacts for the average weekend day are also
developed for each month of the year given that the TOU rates are also effective on the
weekends. As this is an ex post evaluation, average weekday impacts are based on the
observed customer load pooled across the weekdays in each month, and similarly for weekend
days. Monthly system peak day impacts are estimated based on loads that occur on the
historical monthly system peak days. Weather normalized results, such as those conducted for
demand response ex ante load impacts, are not currently in scope for this evaluation. Load
impacts are presented in both nominal (kWh) and proportional (%) terms.
8 http://www.calmac.org/events/FinalDecision_AttachementA.pdf
SECTION 3 METHODOLOGY
Default Time-of-Use Pricing Pilot Interim Evaluation 28
Figure 3-2 displays an image from an Excel spreadsheet containing the output that is produced
for each rate treatment, customer segment, climate region, day type, and month covered by this
interim analysis. These Excel spreadsheets are available upon request through the CPUC. Pull
down menus in the upper left hand corner of the spreadsheet allow users to select different
customer segments, climate regions, day types (e.g., weekdays, weekends, monthly peak day)
and time period (individual months or the average of June, July, August and September). In this
written report, tables and graphs are presented that report estimated load impacts by treatment,
rate period, customer segment, and day type for the summer period.
As discussed in Section 2 the experimental design and sampling were constructed so that load
impacts and other metrics can be reported for selected customer segments and climate regions.
For the segments around which the pilots were designed, load impacts are estimated using the
model represented in the equation above for the data partitioned by segment (for both treatment
and control customers). These estimates are internally valid by virtue of the RED design and
DiD analysis.
Figure 3-2: Average Hourly Load Impact Estimates for Rate 4
control_n treat_treated referencetreat_kwimpact lower_90upper_90pct_impact period referencetreat_kwimpact lower_90upper_90pct_impact
PeriodControl
Customers
Treatment
Customers
Ref.
kW
Treat
kW
Impact
kW
%
ImpactHour Period
Ref.
kW
Treat
kW
Impact
kW
%
Impact
Rate Rate 4 Peak 184,208 148,571 1.41 1.39 0.02 0.02 0.02 1.5% 1 Off-Peak 0.76 0.77 0.00 -0.01 0.00 -0.5%
Segment All Mid-Peak N/A N/A N/A N/A N/A N/A N/A N/A 2 Off-Peak 0.67 0.67 0.00 -0.01 0.00 -0.6%
Time Period Summer Off-Peak 184,208 148,571 0.85 0.85 0.00 0.00 0.00 0.1% 3 Off-Peak 0.61 0.61 0.00 0.00 0.00 -0.5%
Day Type Average Weekday Super Off-Peak N/A N/A N/A N/A N/A N/A N/A N/A 4 Off-Peak 0.57 0.57 0.00 0.00 0.00 -0.4%
day Day (kWh) 184,208 148,571 23.14 23.02 0.12 0.11 0.13 0.5% 5 Off-Peak 0.55 0.55 0.00 0.00 0.00 -0.4%
6 Off-Peak 0.56 0.56 0.00 0.00 0.00 -0.5%
7 Off-Peak 0.59 0.60 0.00 0.00 0.00 -0.5%
8 Off-Peak 0.62 0.62 0.00 0.00 0.00 0.0%
9 Off-Peak 0.65 0.65 0.00 0.00 0.00 0.1%
10 Off-Peak 0.72 0.72 0.00 0.00 0.00 0.1%
11 Off-Peak 0.82 0.81 0.00 0.00 0.00 0.2%
12 Off-Peak 0.92 0.92 0.00 0.00 0.00 0.1%
13 Off-Peak 1.04 1.04 0.00 0.00 0.01 0.3%
14 Off-Peak 1.15 1.15 0.01 0.00 0.01 0.5%
15 Off-Peak 1.25 1.25 0.01 0.01 0.01 0.7%
16 Off-Peak 1.35 1.34 0.01 0.01 0.01 0.9%
17 Peak 1.43 1.41 0.02 0.02 0.02 1.5%
18 Peak 1.47 1.44 0.02 0.02 0.03 1.7%
19 Peak 1.44 1.42 0.02 0.02 0.03 1.7%
20 Peak 1.37 1.35 0.02 0.02 0.02 1.6%
21 Peak 1.33 1.32 0.02 0.01 0.02 1.2%
22 Off-Peak 1.25 1.24 0.00 0.00 0.01 0.4%
23 Off-Peak 1.09 1.09 0.00 0.00 0.00 -0.2%
24 Off-Peak 0.91 0.92 0.00 -0.01 0.00 -0.4%
90% Conf.
Interval
90% Conf.
Interval
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
kW
Hour Ending
Ref. kW Treat kW Impact kW 90% Conf. Interval
SECTION 3 METHODOLOGY
Default Time-of-Use Pricing Pilot Interim Evaluation 29
3.2 Structural Bill Impacts The impact of TOU rates on customers’ bills is an important metric of interest to stakeholders,
and a primary objective of the evaluation. When customers are transitioned to TOU rates, their
bills can change in two ways. The first is due simply to the change in the pricing structure,
holding behavior constant. The second is due to changes in behavior as a result of the
difference in price signals. The first change is known as a structural bill impact, and can be
computed based on usage data prior to customers enrolling on the new rate. Factoring in the
impact of behavior change in response to the new prices requires analysis of post-enrollment
loads for both treatment and control customers in order to control for changes that might be due
to factors other than differences in prices. Because bill impacts can vary rather significantly
across seasons, and since this report is based only on usage for the summer season, this
evaluation does not present behavioral impacts or total bill impacts. Those will be presented
after customers have been on the new tariffs for a full year.
Structural bill impacts were estimated for the summer, winter, and annual time periods using
pretreatment data for the treatment group for each rate and relevant customer segment. Annual
impacts are based on monthly bill estimates from January to December 2017. This time period
was selected to ensure that customer energy use was as close to the present time as possible,
but wasn’t significantly influenced by SCE’s communications with customers about the pilot.
Summer impacts are based on June through September 2017 and winter impacts are based on
January through May and October through December 2017.
Average monthly bills for each treatment group customer on the OAT and TOU rate were
provided by SCE. The difference in bills on the TOU rate and the OAT will identify whether a
customer was a structural benefiter or non-benefiter, as shown in the equation below:
𝑆𝑡𝑟𝑢𝑐𝑡𝑢𝑟𝑎𝑙 𝐵𝑖𝑙𝑙 𝐼𝑚𝑝𝑎𝑐𝑡
= (𝑏𝑖𝑙𝑙 𝑐𝑎𝑙𝑐𝑢𝑙𝑎𝑡𝑒𝑑 𝑤𝑖𝑡ℎ 𝑝𝑟𝑒𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑢𝑠𝑎𝑔𝑒 𝑜𝑛 𝑇𝑂𝑈 𝑟𝑎𝑡𝑒)
− (𝑏𝑖𝑙𝑙 𝑐𝑎𝑙𝑐𝑢𝑙𝑎𝑡𝑒𝑑 𝑤𝑖𝑡ℎ 𝑝𝑟𝑒𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑢𝑠𝑎𝑔𝑒 𝑜𝑛 𝑂𝐴𝑇)
Many customers experienced structural bill impacts that were close to zero. As such, it could
appear that a large share of customers were structural benefiters or non-benefiters even when
bill impacts for a large number of customers are quite small. To address this, a neutral category
of +/- $3 per month was defined. The neutral category helps ensure that the assignment to the
structural benefiter or non-benefiter category is more meaningful and not overly influenced by
customers who would experience a difference in bills of only a few dollars.
The final results from the structural benefiter / non-benefiter analysis are presented in column
graphs and shown as percentages for the each season and on an annual basis. An example is
shown in Figure 3-3. For each rate and relevant segment, the percentage of customers who are
non-benefiters, neutral (+/- $3), or benefiters based on their average monthly bills for the time
period of interest are shown as individual columns. The three columns within each rate and
segment combination total 100%, thus showing the distribution of structural benefiters and non-
benefiters for each rate and segment of interest.
SECTION 3 METHODOLOGY
Default Time-of-Use Pricing Pilot Interim Evaluation 30
Figure 3-3: Example of Structural Bill Impact Results (Annual, Rate 4)
3.3 Opt-Out Analysis Analysis of customer opt-out rates provides useful insights concerning relative customer
preferences among the rate options and may also help predict what opt-out rates might be
under full scale roll out of default TOU pricing. Comparisons of pre-enrollment opt-out rates
across rate options are indicators of the relative preferences of customers for each rate option
and comparisons across notification content and messaging within a rate option can provide
insights about the relative effectiveness of these notification alternatives. Comparisons across
customer segments and climate regions reflect the influence of these factors on customer
acceptance.
In this report, customer attrition was measured in two key time periods: pre-enrollment and post-
enrollment. A series of t-tests were used to determine if there were statistically significant
differences in pairwise comparisons of pre-enrollment opt-out rates across the two TOU options,
differences in the structural bill comparison information provided as part of the notification (e.g.,
seasonal versus annual), and the type of messaging presented in the pre-enrollment
informational communications. T-tests were also used to identify any statistically significant
differences in post-enrollment opt-out rates by rate, CARE/FERA status, climate region, and
post-enrollment treatment. A t-test is a statistical test that is used to determine if the mean
values of two populations differ – in this case the mean opt-out rate for the notification cells was
examined. If they key output of the t-test, the p-value, is below 0.100 then the difference in opt-
out rates is determined to be statistically significant at the 90% confidence level.
AllAll - Non-
CARE/FERA
Moderate &Cool -
CARE/FERA
Non-Benefiter 31% 34% 19%
Neutral 62% 57% 79%
Benefiter 7% 9% 1%
0%10%20%30%40%50%60%70%80%90%
100%
Perc
en
t o
f C
usto
mers
Non-Benefiter Neutral Benefiter
Default Time-of-Use Pricing Pilot Interim Evaluation 31
4 Load Impacts
This report section summarizes the load impacts for the two rate treatments tested by SCE.
Load impacts were estimated for the peak and off-peak periods and for average hourly and daily
energy use for the following rates, customer segments, and climate regions:
For all customers on each rate for the pilot as a whole and for all customers in each
climate region (hot, moderate, cool, and Climate Zone 10)
Non-CARE/FERA customers on each rate for the pilot as a whole and across climate
regions (hot, moderate, cool, and Climate Zone 10) and CARE/FERA customers in the
moderate and cool climate regions.
As discussed above, it’s imperative that comparisons across regions and climate zones are
cognizant of the differences in the mix of customers across regions. That is, because
CARE/FERA customers are not included in the two hot climate regions, comparisons of load
impacts across the two hot and two cooler regions reflect not only differences due to climate but
also differences in the mix of customers, with both CARE/FERA and non-CARE/FERA
customers in the moderate and cool regions and only non-CARE/FERA customers in the two
hot regions. Similarly, comparisons across customer segments for the service territory as a
whole do not just reflect differences in behavior between CARE/FERA and non-CARE/FERA
customers but also differences in the mix of customers across climate regions. The all utility
impacts are representative of what SCE can expect at the service territory level for full roll out of
the rates, because CARE/FERA customers will not be defaulted in the hot climate regions for
full roll out. But it is not appropriate to claim that a difference of, say, 50% between CARE/FERA
and non-CARE/FERA customers at the service territory level accurately reflects a difference in
behavior between the two groups of customers, all other factors held constant. In addition to the
above, Nexant estimated incremental load impacts for customers that received the Enhanced
(high-touch) ME&O treatment for each rate and for each climate region.
Load impacts are reported here for each rate period for the average weekday, average
weekend, and average monthly peak day for the summer months of June through September
2018. Impacts are reported for each rate, climate region and customer segment summarized
above.
Underlying the values presented in the report are electronic tables that contain estimates for
each hour of the day for each day type, segment, and climate region for the summer; and for
each month separately. These values are contained in Excel spreadsheets that are available
upon request through the CPUC. Figure 4-1 shows an example of the content of these
electronic tables for SCE Rate 4 for all eligible customers in the service territory. Pull down
menus in the upper left hand corner allow users to select different customer segments, climate
regions, day types (e.g., weekdays, weekends, monthly peak day) and time periods (individual
months or seasons).
The remainder of this section is organized by rate treatment—load impacts are presented for
each relevant customer segment and climate region for each of the two rates. Following this
discussion, incremental impacts of enhanced E&O over the standard E&O communication are
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 32
presented. Finally, comparisons of load impacts across the two TOU rates are made for the
common hours (5 PM to 8 PM) that are shared across rates.
Figure 4-1: Example of Content of Electronic Tables Underlying Load Impacts Summarized in this Report (SCE Rate 4, Average Summer 2018 Weekday, All Customers)
4.1 Summary of Pilot Rates Figures 2-4 and 2-5 in Section 2 summarized the rate periods and prices for Rates 4 and 5.
Importantly, the prices shown in those figures and discussed below do not reflect the baseline
credit of 7¢/kWh that applies to each rate.
Rate 4 has two rate periods on summer weekdays and three on winter weekdays. The peak and
mid-peak period on Rate 4 is the same all year long and runs from 4 PM to 9 PM. The peak to
off-peak price ratio (ignoring the baseline credit) is 1.8 to 1 in summer and mid-peak to super
off-peak ratio is 1.75 to 1 in winter. Customers on SCE’s Rate 4 pay super off-peak prices on
weekends in the winter. In summer, off-peak prices are in effect on weekends from 9 PM to 4
PM, which is the time-period covered by the combination of off-peak and super off-peak prices
during winter.
SCE’s Rate 5 has two rate periods on summer weekdays and three on winter weekdays, the
same structure as Rate 4. Compared with Rate 4, Rate 5 has a much shorter peak period but a
slightly higher peak price in summer months (47¢/kWh for Rate 5 versus 40¢/kWh for Rate 4)
and slightly high mid-peak price in winter months (29¢/kWh for Rate 5 versus 28¢/kWh for Rate
4). The peak period runs from 5 PM to 8 PM. Rate 5 also features a super off-peak price of
roughly 16¢/kWh between 8 AM and 5 PM on weekends during winter. The ratio of peak to off-
peak prices in the summer is roughly 2.1 to 1. In winter, the mid-peak to super off-peak price
ratio is roughly 1.8 to 1. On weekends, customers pay the off-peak price between 8 PM and 8
AM and the super off-peak price during the same overnight hours as on weekdays, from 8 AM
to 5 PM. For the two rates, the summer season covers the months of June through September.
The winter season is October through May.
4.2 Rate 4 Figure 4-2 shows the average peak period load reduction in absolute terms for Rate 4 for SCE’s
service territory as a whole and for each climate region. The lines bisecting the top of each bar
in the figure show the 90% confidence band for each estimate. If the confidence band includes
control_n treat_treated referencetreat_kwimpact lower_90upper_90pct_impact period referencetreat_kwimpact lower_90upper_90pct_impact
PeriodControl
Customers
Treatment
Customers
Ref.
kW
Treat
kW
Impact
kW
%
ImpactHour Period
Ref.
kW
Treat
kW
Impact
kW
%
Impact
Rate Rate 4 Peak 184,208 148,571 1.41 1.39 0.02 0.02 0.02 1.5% 1 Off-Peak 0.76 0.77 0.00 -0.01 0.00 -0.5%
Segment All Mid-Peak N/A N/A N/A N/A N/A N/A N/A N/A 2 Off-Peak 0.67 0.67 0.00 -0.01 0.00 -0.6%
Time Period Summer Off-Peak 184,208 148,571 0.85 0.85 0.00 0.00 0.00 0.1% 3 Off-Peak 0.61 0.61 0.00 0.00 0.00 -0.5%
Day Type Average Weekday Super Off-Peak N/A N/A N/A N/A N/A N/A N/A N/A 4 Off-Peak 0.57 0.57 0.00 0.00 0.00 -0.4%
day Day (kWh) 184,208 148,571 23.14 23.02 0.12 0.11 0.13 0.5% 5 Off-Peak 0.55 0.55 0.00 0.00 0.00 -0.4%
6 Off-Peak 0.56 0.56 0.00 0.00 0.00 -0.5%
7 Off-Peak 0.59 0.60 0.00 0.00 0.00 -0.5%
8 Off-Peak 0.62 0.62 0.00 0.00 0.00 0.0%
9 Off-Peak 0.65 0.65 0.00 0.00 0.00 0.1%
10 Off-Peak 0.72 0.72 0.00 0.00 0.00 0.1%
11 Off-Peak 0.82 0.81 0.00 0.00 0.00 0.2%
12 Off-Peak 0.92 0.92 0.00 0.00 0.00 0.1%
13 Off-Peak 1.04 1.04 0.00 0.00 0.01 0.3%
14 Off-Peak 1.15 1.15 0.01 0.00 0.01 0.5%
15 Off-Peak 1.25 1.25 0.01 0.01 0.01 0.7%
16 Off-Peak 1.35 1.34 0.01 0.01 0.01 0.9%
17 Peak 1.43 1.41 0.02 0.02 0.02 1.5%
18 Peak 1.47 1.44 0.02 0.02 0.03 1.7%
19 Peak 1.44 1.42 0.02 0.02 0.03 1.7%
20 Peak 1.37 1.35 0.02 0.02 0.02 1.6%
21 Peak 1.33 1.32 0.02 0.01 0.02 1.2%
22 Off-Peak 1.25 1.24 0.00 0.00 0.01 0.4%
23 Off-Peak 1.09 1.09 0.00 0.00 0.00 -0.2%
24 Off-Peak 0.91 0.92 0.00 -0.01 0.00 -0.4%
90% Conf.
Interval
90% Conf.
Interval
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
kW
Hour Ending
Ref. kW Treat kW Impact kW 90% Conf. Interval
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 33
0, it means that the estimated load impact is not statistically different from 0 at the 90% level of
confidence. If the confidence bands for two bars do not overlap, it means that the observed
difference in the load impacts is statistically significant. If they do overlap, it does not necessarily
mean that the difference is not statistically significant.9 In these cases, t-tests were calculated to
determine whether the difference is statistically significant.10 Bars with blue and green stripes
indicate that the segment includes a combination of CARE/FERA customers and non-
CARE/FERA customers, while solid green bars represent segments that are non-CARE/FERA
only. Solid blue bars represent segments that are CARE/FERA customers only. However, it is
important to note that the “All” category includes non-CARE/FERA customers from all climate
regions but CARE/FERA customers only from the moderate and cool climate regions. As a
result, the “All” estimates cannot be directly compared to the “Moderate” and “Cool” estimates.
Figure 4-2: Average Peak Period Load Impacts for SCE Rate 4 by Climate Region (Positive values represent load reductions)
As seen in Figure 4-2, the average peak-period load impact for the service territory as a whole
and for each climate region is statistically significant at the 90% level of confidence. On
average, default pilot participants across SCE’s service territory on Rate 4 reduced peak-period
electricity use by 1.5%, or 0.02 kW, across the five-hour peak period from 4 PM to 9 PM.
Keeping in mind that differences across regions reflect both differences in climate and the
presence or absence of CARE/FERA customers, the average peak-period load reduction
ranges from a high of 2.0% and 0.04 kW in the hot climate region to a low of about 1.5% and
0.02 kW in the cool climate region. The difference in load impacts between the moderate and
cool climate regions is small but statistically significant while the difference in impacts in Climate
Zone 10 and the hot region are not statistically significant.
9
For further discussion of this topic, see https://www.cscu.cornell.edu/news/statnews/stnews73.pdf.
10 The test was applied at the 90% confidence level which means that a t-value exceeding 1.65 indicates statistical significance.
All Hot Zone10 Moderate Cool
Percent Impact 1.5% 2.0% 1.8% 1.3% 1.5%
Absolute Impact 0.02 0.04 0.04 0.02 0.02
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07
Ab
so
lute
kW
Im
pa
ct
Du
rin
g P
ea
k P
eri
od
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 34
Table 4-1 shows the average percent and absolute hourly load impacts for each period for
weekdays, weekends, and for the average monthly system peak day for the SCE service
territory as a whole and for the participant population in each climate region. The percent
reduction equals the load impact in absolute terms (kW) divided by the reference load. Shaded
cells in the table contain load impact estimates that are not statistically significant at the 90%
confidence level. The percentage and absolute values in the first row of Table 4-1, which
represent the load impacts in the peak period on the average weekday, equal the values shown
in Figure 4-2, discussed above.
The reference loads shown in Table 4-1 represent estimates of what customers on the TOU rate
would have used if they had not responded to the price signals contained in the TOU tariff. As
seen in the table, average hourly usage during the peak period is roughly 1.41 kW for the
service territory as a whole, and around 0.96 kW over the 24 hour average weekday. In the hot
climate region and Climate Zone 10, average usage in the peak period is greater at 2.06 kW
and 2.13 kW, respectively. Average usage in the moderate climate region is 1.63 kW and in the
cool region it is 1.07 kW, which is roughly half what it is in the hot region. However, the cool
climate region includes CARE/FERA customers while the hot climate region does not.
The monthly system peak day estimates represent the average across the four weekdays, one
in each summer month, when SCE’s system peaked in 2018. Peak period reference loads are
higher on these days than on the average weekday. For the service territory as a whole, the
percent reduction in monthly system peak day peak period loads (1.4%) is slightly lower than
the load reduction on the average weekday (1.5%); however, the absolute load reduction (0.03
kW) is slightly higher than on the average weekday (0.02 kW). Customers had small but
statistically significant daily usage decreases on the average weekend even though off-peak
prices were in effect for the majority of weekend hours and mid-peak prices were in effect for
the remaining hours.
Table 4-1: Average Hourly Load Impacts by Climate Region Rate Period and Day Type for SCE Rate 4
(Positive values represent load reductions, negative values represent load increases)
* A shaded cell indicates estimate is not statistically significant
Ref. kWImpact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
Impact
Peak 4 PM to 9 PM 1.41 0.02 1.5% 2.06 0.04 2.0% 2.13 0.04 1.8% 1.63 0.02 1.3% 1.07 0.02 1.5%
Off-Peak 9 PM to 4 PM 0.85 0.00 0.1% 1.21 0.01 0.8% 1.17 0.00 -0.2% 0.92 0.00 -0.2% 0.70 0.00 0.2%
Day All Hours 0.96 0.00 0.5% 1.38 0.02 1.2% 1.37 0.01 0.5% 1.07 0.00 0.3% 0.78 0.00 0.6%
Mid-Peak 4 PM to 9 PM 1.37 0.02 1.3% 1.99 0.03 1.5% 2.07 0.03 1.5% 1.57 0.02 1.1% 1.05 0.01 1.3%
Off-Peak 9 PM to 4 PM 0.87 0.00 0.1% 1.22 0.01 0.7% 1.21 0.00 -0.2% 0.94 0.00 -0.1% 0.72 0.00 0.2%
Day All Hours 0.97 0.00 0.5% 1.38 0.01 1.0% 1.39 0.00 0.3% 1.07 0.00 0.2% 0.79 0.00 0.5%
Peak 4 PM to 9 PM 1.85 0.03 1.4% 2.29 0.03 1.3% 2.93 0.06 1.9% 2.24 0.03 1.2% 1.37 0.02 1.3%
Off-Peak 9 PM to 4 PM 1.01 0.00 0.1% 1.29 0.01 0.9% 1.51 0.00 -0.1% 1.14 0.00 0.0% 0.81 0.00 0.1%
Day All Hours 1.19 0.01 0.6% 1.50 0.02 1.0% 1.81 0.01 0.6% 1.37 0.01 0.4% 0.92 0.00 0.5%
Monthly
System
Peak
Rate 4
Day Type Period Hours
All Hot Moderate CoolZone10
Average
Weekday
Average
Weekend
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 35
Figure 4-3 shows the absolute peak period load impacts for Rate 4 for CARE/FERA and non-
CARE/FERA customers for the service territory as a whole and for each climate region. Non-
CARE/FERA segments are shaded with green while CARE/FERA segments are shaded in blue.
In the cool climate region, both the percent and absolute load impacts in the peak period differ
by a statistically significant amount and impacts are smaller for CARE/FERA customers than for
non-CARE/FERA customers. In the moderate climate region, non-CARE/FERA customers
produced larger percentage load reductions compared to non-CARE/FERA customers, but
there is no statistically significant difference in the percentage or absolute impacts between the
two customer segments. There is a statistically significant difference in load impacts between
CARE/FERA and non-CARE/FERA customers at the service territory level but this comparison
reflects both potential differences in behavior across the two segments as well as the fact that
the non-CARE/FERA estimate includes customers in the hotter climate regions where absolute
load impacts are typically larger. As such, this is not a valid comparison if the objective is to
reflect only behavioral differences between the two customer segments.
Figure 4-3: Average Peak Period Impacts for SCE Rate 4 by Climate Region & CARE/FERA Status
(Positive values represent load reductions)
Table 4-2 shows the estimated load impacts for each day type for the different rate period for
the service territory as a whole and by climate region for non-CARE/FERA customers, and
Table 4-3 shows the same segment values for CARE/FERA customers. For the service territory
as a whole, non-CARE/FERA customers have average peak-period reference loads that are
larger than CARE/FERA customers (1.52 kW for non-CARE/FERA and 0.97 kW for
CARE/FERA), however the CARE/FERA segment only includes customers in the moderate and
cool climate regions. Non-CARE/FERA customers have larger average usage rates across all
climate regions and for daily electricity usage on average summer weekdays, weekends, and on
monthly system peak days.
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
CARE/FERA
AllModerate
& CoolHot Zone10 Moderate Cool
Percent Impact 1.6% 1.2% 2.0% 1.8% 1.2% 1.5% 1.7% 1.0%
Absolute Impact 0.02 0.01 0.04 0.04 0.02 0.02 0.02 0.01
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07
Ab
so
lute
kW
Im
pa
ct
Du
rin
g P
ea
k P
eri
od
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 36
For the majority of customer segments and climate regions, there was a small but statistically
significant reduction in daily electricity consumption. Put differently, the observed reduction in
peak-period energy use was not completely offset by load shifting to non-peak time periods.
Indeed, most segments and climate regions showed a small reduction in usage in the off-peak
period rather than an increase which would be observed if the amount of load shifting was
significant. CARE/FERA customers in the moderate and cool climate regions decreased
average daily usage on weekdays by 0.7%, whereas non-CARE/FERA customers across all
climate regions decreased their usage by 0.5%. On monthly system peak days, non-
CARE/FERA customers reduced daily electricity use by 0.6% and CARE/FERA decreased their
overall usage by 0.5%. CARE/FERA customers displayed larger average load reductions than
non-CARE/FERA customers on off-peak periods overall and for every climate region on
average weekdays and average weekends.
Table 4-2: Average Hourly Load Impacts by Rate Period and Day Type for SCE Rate 4 by Climate Region -- Non-CARE/FERA Customers
(Positive values represent load reductions, negative values represent load increases)
* A shaded cell indicates estimate is not statistically significant
Ref. kWImpact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
Impact
Peak 4 PM to 9 PM 1.52 0.02 1.6% 2.06 0.04 2.0% 2.13 0.04 1.8% 1.77 0.02 1.2% 1.15 0.02 1.7%
Off-Peak 9 PM to 4 PM 0.91 0.00 0.0% 1.21 0.01 0.8% 1.17 0.00 -0.2% 0.99 0.00 -0.5% 0.75 0.00 0.1%
Day All Hours 1.04 0.00 0.5% 1.38 0.02 1.2% 1.37 0.01 0.5% 1.16 0.00 0.1% 0.83 0.00 0.6%
Mid-Peak 4 PM to 9 PM 1.49 0.02 1.3% 1.99 0.03 1.5% 2.07 0.03 1.5% 1.72 0.02 0.9% 1.13 0.02 1.4%
Off-Peak 9 PM to 4 PM 0.93 0.00 0.0% 1.22 0.01 0.7% 1.21 0.00 -0.2% 1.01 0.00 -0.4% 0.76 0.00 0.2%
Day All Hours 1.04 0.00 0.4% 1.38 0.01 1.0% 1.39 0.00 0.3% 1.16 0.00 0.0% 0.84 0.00 0.5%
Peak 4 PM to 9 PM 2.02 0.03 1.4% 2.29 0.03 1.3% 2.93 0.06 1.9% 2.46 0.03 1.1% 1.49 0.02 1.3%
Off-Peak 9 PM to 4 PM 1.09 0.00 0.2% 1.29 0.01 0.9% 1.51 0.00 -0.1% 1.23 0.00 -0.1% 0.86 0.00 0.2%
Day All Hours 1.28 0.01 0.6% 1.50 0.02 1.0% 1.81 0.01 0.6% 1.49 0.00 0.3% 0.99 0.01 0.5%
Monthly
System
Peak
Rate 4
Day Type Period Hours
All - Non-CARE/FERA Hot - Non-CARE/FERAModerate - Non-
CARE/FERACool - Non-CARE/FERA
Zone10 - Non-
CARE/FERA
Average
Weekday
Average
Weekend
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 37
Table 4-3: Average Hourly Load Impacts by Rate Period and Day Type for SCE Rate 4 by Climate Region -- CARE/FERA Customers
(Positive values represent load reductions, negative values represent load increases)
* A shaded cell indicates estimate is not statistically significant
4.3 Rate 5 SCE’s Rate 5 has two rate periods on summer weekdays, and two rate periods on summer
weekends, the same structure as Rate 4. Rate 5 peak period prices are higher than for Rate 4
but the peak period is only three hours, from 5 PM to 8 PM, whereas the Rate 4 peak period is
five hours, from 4 PM to 9 PM. The Rate 5 peak price is 47.0¢/kWh for non-CARE/FERA
customers and the off-peak price of 22¢/kWh on summer weekdays from hours 8 PM to 5 PM,
which is the same price as the off-peak price for Rate 4.
Figure 4-4 shows the peak period load reductions on average weekdays for Rate 5. All load
reductions are statistically significant at the 90% confidence level. The load reductions for the
SCE territory as a whole, 2.0% or 0.03 kW are larger than those for Rate 4 (1.5% or 0.02 kW).
The difference in average hourly peak period load reductions is statistically significant in both
absolute and percentage terms. Load impacts were greatest in the Climate Zone 10 region
(2.4% or 0.05 kW) although there is no statistically significant difference in absolute load
impacts between the hot zone and Climate Zone 10. On the other hand, the difference in the
absolute load impacts for all customers in the moderate and cool regions is statistically
significant. Indeed, the absolute load reduction in the moderate region is twice as large as in the
cool region, although the difference in the percentage impacts is not as great.
Ref. kWImpact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
Impact
Peak 4 PM to 9 PM 0.97 0.01 1.2% N/A N/A N/A N/A N/A N/A 1.23 0.02 1.5% 0.83 0.01 1.0%
Off-Peak 9 PM to 4 PM 0.62 0.00 0.5% N/A N/A N/A N/A N/A N/A 0.72 0.01 0.9% 0.57 0.00 0.3%
Day All Hours 0.69 0.01 0.7% N/A N/A N/A N/A N/A N/A 0.83 0.01 1.1% 0.62 0.00 0.5%
Mid-Peak 4 PM to 9 PM 0.92 0.01 1.3% N/A N/A N/A N/A N/A N/A 1.18 0.02 1.8% 0.79 0.01 1.1%
Off-Peak 9 PM to 4 PM 0.63 0.00 0.5% N/A N/A N/A N/A N/A N/A 0.74 0.01 0.9% 0.58 0.00 0.3%
Day All Hours 0.69 0.01 0.8% N/A N/A N/A N/A N/A N/A 0.83 0.01 1.1% 0.62 0.00 0.5%
Peak 4 PM to 9 PM 1.24 0.02 1.4% N/A N/A N/A N/A N/A N/A 1.66 0.02 1.5% 1.01 0.01 1.4%
Off-Peak 9 PM to 4 PM 0.73 0.00 0.1% N/A N/A N/A N/A N/A N/A 0.89 0.00 0.5% 0.64 0.00 -0.1%
Day All Hours 0.83 0.00 0.5% N/A N/A N/A N/A N/A N/A 1.05 0.01 0.8% 0.72 0.00 0.3%
Monthly
System
Peak
Rate 4
Day Type Period Hours
Moderate & Cool -
CARE/FERAHot - CARE/FERA Moderate - CARE/FERA Cool - CARE/FERAZone10 - CARE/FERA
Average
Weekday
Average
Weekend
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 38
Figure 4-4: Average Peak Period Load Impacts for SCE Rate 5 by Climate Region (Positive values represent load reductions)
Table 4-4 presents estimates of load impacts for all relevant rate periods and day types for Rate
5 at the aggregate and climate region level. Average reference load usage was 1.43 kW at the
full pilot level during the peak time on an average weekday. The highest demand estimates
were observed in Climate Zone 10 on monthly system peak days during the peak period with a
reference load of 2.98 kW.
The Climate Zone 10 and hot climate regions had largest percentage reductions for average
weekday (2.4% and 2.3%) respectively (but these segments do not include CARE/FERA
customers). The cool climate region had the lowest load impacts and average load usage during
the peak for average weekdays, average weekends, and monthly system peak days. The
average reduction in daily electricity use was statistically significant overall and in each climate
region for every day type.
All Hot Zone10 Moderate Cool
Percent Impact 2.0% 2.3% 2.4% 2.2% 1.8%
Absolute Impact 0.03 0.05 0.05 0.04 0.02
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07A
bs
olu
te k
W I
mp
ac
t D
uri
ng
Pe
ak
Pe
rio
d
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 39
Table 4-4: Average Hourly Load Impacts by Climate Region, Rate Period and Day Type for SCE Rate 5
(Positive values represent load reductions, negative values represent load increases)
* A shaded cell indicates estimate is not statistically significant
Figure 4-5 shows the peak period load reductions on weekdays for non-CARE/FERA and
CARE/FERA customers. As noted with Rate 4, there are no CARE/FERA customers in the hot
or Climate Zone 10 regions. In both the moderate and cool climate regions, non-CARE/FERA
load reductions are larger than CARE/FERA load reductions in both absolute and percentage
terms. These differences are statistically significant in absolute terms in both climate regions
and in percentage terms in the cool climate region. Indeed, the absolute load reductions are
nearly twice as large for non-CARE/FERA customers compared with CARE/FERA customers in
these two climate regions.
Figure 4-5: Average Peak Period Impacts for SCE Rate 5 by Climate Region & CARE/FERA Status
(Positive values represent load reductions)
Ref. kWImpact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
Impact
Peak 5 PM to 8 PM 1.43 0.03 2.0% 2.11 0.05 2.3% 2.17 0.05 2.4% 1.65 0.04 2.2% 1.08 0.02 1.8%
Off-Peak 8 PM to 5 PM 0.90 0.00 0.3% 1.28 0.00 0.2% 1.26 0.00 0.2% 0.98 0.00 0.4% 0.74 0.00 0.3%
Day All Hours 0.96 0.01 0.0% 1.38 0.01 0.0% 1.37 0.01 0.0% 1.07 0.01 0.0% 0.78 0.00 0.0%
Mid-Peak 5 PM to 8 PM 1.38 0.02 1.6% 2.03 0.04 1.7% 2.10 0.04 1.7% 1.59 0.03 1.9% 1.05 0.01 1.3%
Off-Peak 8 PM to 5 PM 0.91 0.00 0.3% 1.29 0.00 0.0% 1.28 0.00 0.0% 1.00 0.00 0.5% 0.75 0.00 0.2%
Day All Hours 0.97 0.00 0.0% 1.38 0.00 0.0% 1.39 0.00 0.0% 1.07 0.01 0.0% 0.79 0.00 0.0%
Peak 5 PM to 8 PM 1.88 0.04 2.0% 2.33 0.05 2.1% 2.98 0.07 2.2% 2.28 0.06 2.4% 1.38 0.02 1.7%
Off-Peak 8 PM to 5 PM 1.09 0.00 0.3% 1.38 0.00 0.3% 1.64 0.00 0.2% 1.24 0.00 0.4% 0.86 0.00 0.2%
Day All Hours 1.19 0.01 0.0% 1.50 0.01 0.0% 1.81 0.01 0.0% 1.37 0.01 0.0% 0.92 0.00 0.0%
Average
Weekday
Average
Weekend
Monthly
System
Peak Day
Rate 5
Day Type Period Hours
All Hot Moderate CoolZone10
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
CARE/FERA
AllModerate
& CoolHot Zone10 Moderate Cool
Percent Impact 2.1% 1.5% 2.3% 2.4% 2.2% 1.9% 1.9% 1.2%
Absolute Impact 0.03 0.02 0.05 0.05 0.04 0.02 0.02 0.01
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07
Ab
so
lute
kW
Im
pa
ct
Du
rin
g P
ea
k P
eri
od
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 40
Table 4-5 and Table 4-6 show the load impacts for each rate period and day type for Rate 5 at
the aggregate level and across climate regions. Non-CARE/FERA customers had higher
average load and load reductions during peak times across all climate regions on average
weekdays, weekends and monthly system peak days. An interesting finding is that the load
impacts on all off-peak periods were greater for the CARE/FERA group than the non-
CARE/FERA group across the four climate regions and the different day types. No values are
reported for the hot and Climate Zone 10 regions for CARE/FERA customers as the pilot didn’t
include these populations.
Non-CARE/FERA customers had statistically significant reductions in average daily demand
across most day types in each climate region except the cool climate region. The greatest daily
reductions occurred in the hot climate region and Climate Zone 10. On the average weekday,
these customers reduced their average demand by 0.6%. CARE/FERA customers also had
average daily demand reductions, generally equal to less than 0.1% but statistically significant.
Table 4-5: Average Hourly Load Impacts by Rate Period and Day Type for SCE Rate 5 by Climate Region -- Non-CARE/FERA Customers
(Positive values represent load reductions, negative values represent load increases)
* A shaded cell indicates estimate is not statistically significant
Ref. kWImpact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
Impact
Peak 5 PM to 8 PM 1.55 0.03 2.1% 2.11 0.05 2.3% 2.17 0.05 2.4% 1.81 0.04 2.2% 1.16 0.02 1.9%
Off-Peak 8 PM to 5 PM 0.96 0.00 0.2% 1.28 0.00 0.2% 1.26 0.00 0.2% 1.06 0.00 0.2% 0.78 0.00 0.2%
Day All Hours 1.04 0.01 0.0% 1.38 0.01 0.6% 1.37 0.01 0.6% 1.16 0.01 0.0% 0.83 0.00 0.0%
Mid-Peak 5 PM to 8 PM 1.50 0.02 1.6% 2.03 0.04 1.7% 2.10 0.04 1.7% 1.74 0.03 1.8% 1.13 0.02 1.3%
Off-Peak 8 PM to 5 PM 0.98 0.00 0.2% 1.29 0.00 0.0% 1.28 0.00 0.0% 1.08 0.00 0.3% 0.80 0.00 0.2%
Day All Hours 1.04 0.00 0.0% 1.38 0.00 0.3% 1.39 0.00 0.3% 1.16 0.01 0.0% 0.84 0.00 0.0%
Peak 5 PM to 8 PM 2.05 0.04 2.2% 2.33 0.05 2.1% 2.98 0.07 2.2% 2.50 0.07 2.7% 1.51 0.03 1.7%
Off-Peak 8 PM to 5 PM 1.17 0.00 0.2% 1.38 0.00 0.3% 1.64 0.00 0.2% 1.34 0.00 0.2% 0.92 0.00 0.2%
Day All Hours 1.28 0.01 0.0% 1.50 0.01 0.7% 1.81 0.01 0.6% 1.49 0.01 0.0% 0.99 0.00 0.0%
Average
Weekday
Average
Weekend
Monthly
System
Peak Day
Rate 5
Day Type Period Hours
All - Non-CARE/FERA Hot - Non-CARE/FERAModerate - Non-
CARE/FERACool - Non-CARE/FERA
Zone10 - Non-
CARE/FERA
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 41
Table 4-6: Average Hourly Load Impacts by Rate Period and Day Type for SCE Rate 5 by Climate Region -- CARE/FERA Customers
(Positive values represent load reductions, negative values represent load increases)
* A shaded cell indicates estimate is not statistically significant
4.4 Post-enrollment Treatments
4.4.1 Enhanced E&O
SCE varied the education and outreach provided to participants who were on the two TOU rates. Half of the pilot participants on each rate received what SCE describes as enhanced education and outreach which had different formatting and content as summarized in Section 2.2. Figure 4-6 shows the average incremental impact attributable to the enhanced education and outreach at the aggregate level and for each climate region for Rate 4, while Figure 4-7 shows the average incremental impacts at the aggregate level and for each climate region for Rate 5. Positive values in the figure indicate an incremental increase in load reductions (e.g., load reductions are larger with enhanced education) while a negative value means load reductions were smaller for the enhanced education group relative to the less frequent communication. As seen, incremental impacts were not statistically significant in any of the climate regions.
Ref. kWImpact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
ImpactRef. kW
Impact
kW
%
Impact
Peak 5 PM to 8 PM 0.97 0.02 1.5% N/A N/A N/A N/A N/A N/A 1.25 0.02 1.9% 0.83 0.01 1.2%
Off-Peak 8 PM to 5 PM 0.65 0.00 0.8% N/A N/A N/A N/A N/A N/A 0.77 0.01 1.1% 0.59 0.00 0.5%
Day All Hours 0.69 0.01 0.0% N/A N/A N/A N/A N/A N/A 0.83 0.01 0.1% 0.62 0.00 0.0%
Mid-Peak 5 PM to 8 PM 0.92 0.01 1.5% N/A N/A N/A N/A N/A N/A 1.18 0.02 2.0% 0.79 0.01 1.1%
Off-Peak 8 PM to 5 PM 0.66 0.00 0.7% N/A N/A N/A N/A N/A N/A 0.78 0.01 1.1% 0.60 0.00 0.4%
Day All Hours 0.69 0.01 0.0% N/A N/A N/A N/A N/A N/A 0.83 0.01 0.1% 0.62 0.00 0.0%
Peak 5 PM to 8 PM 1.25 0.02 1.3% N/A N/A N/A N/A N/A N/A 1.68 0.02 1.3% 1.01 0.01 1.3%
Off-Peak 8 PM to 5 PM 0.77 0.00 0.6% N/A N/A N/A N/A N/A N/A 0.96 0.01 0.9% 0.67 0.00 0.4%
Day All Hours 0.83 0.01 0.0% N/A N/A N/A N/A N/A N/A 1.05 0.01 0.0% 0.72 0.00 0.0%
Average
Weekday
Average
Weekend
Monthly
System
Peak Day
Rate 5
Day Type Period Hours
Moderate & Cool -
CARE/FERAHot - CARE/FERA Moderate - CARE/FERA Cool - CARE/FERAZone10 - CARE/FERA
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 42
Figure 4-6: Rate 4 Incremental Load Impacts from Enhanced E&O Treatment
by Climate Region (Positive values represent larger load reductions for Enhanced E&O customers relative
to Basic E&O Customers)
Figure 4-7: Rate 5 Incremental Load Impacts from Enhanced E&O Treatment by Climate Region
(Positive values represent larger load reductions for Enhanced E&O customers relative to Basic E&O customers)
All Hot Zone10 Moderate Cool
Percent Impact 0.0% 0.1% -0.1% 0.1% 0.0%
Absolute Impact 0.00 0.00 0.00 0.00 0.00
-0.015
-0.010
-0.005
0.000
0.005
0.010
Inc
rem
en
tal k
W I
mp
ac
t D
uri
ng
Peak P
eri
od
All Hot Zone10 Moderate Cool
Percent Impact 0.0% -0.1% 0.1% 0.0% 0.1%
Absolute Impact 0.00 0.00 0.00 0.00 0.00
-0.015
-0.010
-0.005
0.000
0.005
0.010
In
cre
me
nta
l k
W Im
pa
ct
Du
rin
g P
ea
k P
eri
od
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 43
Figure 4-8 and Figure 4-9 display the average incremental peak period impact attributable to the
enhanced education and outreach by CARE/FERA status for each climate region for Rate 4 and
Rate 5, respectively. With one exception, there are no discernible differences in impacts
between the enhanced and basic groups as the estimates shown are both negative and positive
with almost no segments being statistically significant. The exception is the CARE/FERA
moderate climate region segment where the enhanced education group experienced significant
incremental load reductions relative to the non-enhanced group in both Rate 4 and Rate 5.
Although the incremental savings were statistically significant, they were small (0.01 kW or
0.7%).
Figure 4-8: Rate 4 Incremental Peak Period Load Impacts from Enhanced E&O Treatment by Climate Region & CARE/FERA Status
(Positive values represent larger load reductions for Enhanced E&O customers relative to Basic E&O customers)
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
CARE/FERA
AllModerate
& CoolHot Zone10 Moderate Cool
Percent Impact 0.0% 0.2% 0.1% -0.1% 0.0% 0.7% 0.1% -0.2%
Absolute Impact 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00
-0.010
-0.005
0.000
0.005
0.010
0.015
Inc
rem
en
tal k
W I
mp
ac
t D
uri
ng
Peak P
eri
od
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 44
Figure 4-9: Rate 5 Incremental Load Impacts from Enhanced E&O Treatment by Climate Region & CARE/FERA Status
(Positive values represent larger load reductions for Enhanced E&O customers relative to Basic E&O customers)
4.4.2 Level Payment Plan
As discussed in Section 2, the enrolled population on each of the default rates was segmented
into two groups, those deemed to be most impacted by bill volatility and those who are not. The
segment of customers impacted by bill volatility was considered to be income-constrained
customers who were expected to experience increased seasonal bill differentials under the
default TOU rate. This segment of customers was further divided into two equal groups, with
one group receiving information on SCE’s Level Payment Plan (LPP) as a means of managing
month-to-month bill volatility.
The Pilot plan called for estimating the incremental enrollments in LPP that occurred as a result
of the additional messaging and, if enrollment was large enough, to determine if load impacts
differed between customers who were and were not on the LPP. However, among the group of
approximately 52,000 pilot treatment customers who were deemed most impacted by bill
volatility, only 40 enrolled in LPP after the launch of the pilot. As such, participation is not large
enough to determine any differences in load impacts between LPP and non-LPP participants.
4.5 Comparison Across Rates Figure 4-10 compares the load impacts for the two rates tested by SCE for the common set of
peak-period hours from 5 PM to 8 PM for the entire summer of 2018. Using a common set of
hours reduces differences in impacts across rates that might be due to differences in the
number of hours included in the peak period or the timing of those hours. The hours from 5 PM
to 8 PM define the peak period for SCE’s Rate 5. Rate 4 has a five hour peak period, from 4 PM
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERA
Non-CARE/FERA
CARE/FERA
AllModerate
& CoolHot Zone10 Moderate Cool
Percent Impact 0.0% 0.3% -0.1% 0.1% -0.2% 0.5% 0.1% 0.1%
Absolute Impact 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00
-0.015
-0.010
-0.005
0.000
0.005
0.010
0.015
Inc
rem
en
tal k
W I
mp
ac
t D
uri
ng
Pe
ak
Pe
rio
d
Non-CARE/FERA & CARE/FERA Non-CARE/FERA Only CARE/FERA Only
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 45
to 9 PM and both tariffs have two rate periods in summer. The shorter duration of Rate 5 is
offset by the higher peak price. Both Rate 4 and Rate 5 have the same baseline credit.
Customers on Rate 5, which had a shorter peak period with a higher peak period price,
produced larger average load reductions than Rate 4 customers in every climate region during
the common hours from 5 Pm to 8 PM, although not all differences were statistically significant.
The largest difference was in the moderate climate region, where Rate 5 customers had
absolute load reductions that were nearly twice as large as those provided by Rate 4 customers.
There was also a statistically significant difference in load reductions in Climate Zone 10,
although not in the hot or cool climate regions.
Figure 4-10: Average Impacts from 5 PM to 8 PM Across Rates
Figure 4-11 presents the average daily kWh impacts for each rate during the summer 2018
period. Daily load reductions were roughly twice as large for Rate 5 when compared to Rate 4 in
the moderate climate region but the opposite was true in the hot climate region, where the daily
reduction for Rate 4 was roughly twice as large as for Rate 5. The differences in the cool climate
region and Climate Zone 10 are not statistically significant. For the service territory as a whole,
Rate 5 had a slightly larger, statistically significant load reduction compared with Rate 4.
Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5
All Hot Zone10 Moderate Cool
Percent Impact 1.7% 2.0% 2.1% 2.3% 1.9% 2.4% 1.4% 2.2% 1.6% 1.8%
Absolute Impact 0.02 0.03 0.04 0.05 0.04 0.05 0.02 0.04 0.02 0.02
0.00
0.01
0.02
0.03
0.04
0.05
0.06
0.07
kW
Im
pac
t fr
om
5 t
o 8
PM
SECTION 4 LOAD IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 46
Figure 4-11: Average Daily kWh Impacts Across Rates
Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5
All Hot Zone10 Moderate Cool
Percent Impact 0.5% 0.6% 1.2% 0.6% 0.5% 0.6% 0.3% 0.7% 0.6% 0.5%
Absolute Impact 0.12 0.14 0.38 0.19 0.15 0.20 0.08 0.19 0.10 0.10
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50D
ail
y k
Wh
Im
pac
t
Default Time-of-Use Pricing Pilot Interim Evaluation 47
5 Structural Bill Impacts
This section summarizes the structural bill impact estimates for the two rate treatments tested
by SCE. As discussed in Section 3.2, the impact of TOU rates on customers’ bills is an
important metric of interest to stakeholders, and a primary objective of the evaluation. Because
bill impacts can vary rather significantly across seasons, and since this report is based only on
usage for the summer season, this evaluation does not present behavioral impacts or total bill
impacts. Those will be presented after customers have been on the new tariffs for a full year.
Structural bill impacts were estimated for the average month in summer, winter, and for the
entire pre-treatment year. The proportions of structural benefiters, non-benefiters and customers
in the neutral category (e.g., ±$3/month) are presented for each rate for CARE/FERA and non-
CARE/FERA customers by climate region and for the pilot as a whole.
5.1 Rate 4 The structural benefiter analysis was conducted for the summer, winter and annual periods
using pretreatment data from the treatment group for each rate and relevant customer segment.
Annual impacts were based on monthly bill estimates from January 2017 through December
2017. Summer impacts were based on June 2017 through September 2017 and winter impacts
were based on January 2017 through May 2017 and October 2017 through December 2017.
Monthly bills for each treatment customer on their OAT and default TOU rate were provided by
SCE. The difference in bills based on the TOU rate and the OAT determines if a customer is a
structural benefiter, a structural non-benefiter, or falls in a neutral range defined as having a
structural bill impact between ±$3.11
Results from the structural benefiter / non-benefiter analysis are presented in column graphs
and shown as percentages for the summer season, winter season, and on an annual basis. For
each rate and relevant segment, the percentage of customers who are non-benefiter, neutral
(+/- $3), or benefiters based on their average monthly bills for the time period of interest are
shown as individual columns. The three columns within each rate and segment combination
total to 100%, thus showing the distribution of structural benefiters and non-benefiters for each
rate and segment of interest.
Figure 5-1 presents the outcome of the structural benefiter analysis for Rate 4 at the annual
aggregate level across all climate regions for all customers as well as for CARE/FERA and non-
CARE/FERA groups. Figure 5-2 and Figure 5-3 present the same data for the summer and
winter rate periods.
At the annual level, slightly more than 60% of all customers were in the neutral column while
almost 80% of CARE/FERA consumers in the moderate and cool climate regions had neutral
bill impacts. Non-CARE/FERA customers across all climate regions had the greatest share of
non-benefiters (34%), although a majority of customers were either benefiters or neutral. The
observed difference in the distribution of bill impacts across customers segments is due in part
11
See Section 3.2 for additional details on the methodology.
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 48
to differences in the distribution of customer segments across climate regions. The CARE/FERA
distributions represent the moderate and cool regions only where the percent of customers that
are neutral or structural benefiters is higher whereas the non-CARE/FERA distributions are
influenced by the two hotter regions where the proportion of structural non-benefiters is higher.
Figure 5-1: Rate 4 – Annual Structural Benefiter / Non-Benefiter Analysis by CARE/FERA Status12
As seen in Figure 5-2, nearly all participating customers are either structural non-benefiters or in
the neutral category in summer. Over 60% of customers in the non-CARE/FERA segment are
non-benefiters, while about 50% of CARE/FERA customers in the moderate and cool climate
regions are non-benefiters and 50% are neutral.
12
The percentage values in the tables may not add up to 100% due to rounding in this and the following figures.
AllAll - Non-
CARE/FERA
Moderate& Cool -
CARE/FERA
Non-Benefiter 31% 34% 19%
Neutral 62% 57% 79%
Benefiter 7% 9% 1%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 49
Figure 5-2: Rate 4 – Summer Structural Benefiter / Non-Benefiter Analysis by CARE/FERA Status
The winter season had the highest rates of neutral customers as well as structural benefiters.
About 26% of non-CARE/FERA customers are structural benefiters in the winter season. About
90% of CARE/FERA customers are neutral during the same time period. Across the three time
periods, a slightly higher proportion of CARE/FERA customers are structural non-benefiters
than non-CARE/FERA customers.
Figure 5-3: Rate 4 – Winter Structural Benefiter / Non-Benefiter Analysis by CARE/FERA Status
Figure 5-4, Figure 5-5, and Figure 5-6 present the outcomes of the structural benefiter analysis
for the Rate 4 CARE/FERA status level by climate regions for the annual, summer, and winter
AllAll - Non-
CARE/FERA
Moderate& Cool -
CARE/FERA
Non-Benefiter 61% 65% 49%
Neutral 36% 32% 51%
Benefiter 3% 3% 0%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
AllAll - Non-
CARE/FERA
Moderate& Cool -
CARE/FERA
Non-Benefiter 9% 10% 4%
Neutral 69% 64% 90%
Benefiter 22% 26% 6%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 50
time periods, respectively. The findings are consistent with the aggregate results as nearly all
customers are structural non-benefiters or neutral in the summer season for every climate
region. The CARE/FERA group also had fewer non-benefiters and more participants in the
neutral category compared to non-CARE/FERA in the moderate and cool climate regions
throughout the year.
The proportion of structural non-benefiters increases as the climate regions move from cool to
hot and Climate Zone 10 as seen in the annual and summer season (Figure 5-4 and Figure
5-5). The hot region and Climate Zone 10 had the highest number of structural benefiters during
the winter season as seen in Figure 5-6. The winter season for Rates 4 and 5 had three rate
periods compared to the two rate periods for the summer season. Both Rates 4 and 5 had a
super off-peak period during the day where the price was $16 ¢/kWh.
Figure 5-4: Rate 4 – Annual Structural Benefiter / Non-Benefiter Analysis by Climate Region & CARE/FERA Status
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Non-Benefiter 51% 56% 41% 29% 21% 14%
Neutral 43% 37% 49% 70% 69% 84%
Benefiter 7% 6% 10% 1% 10% 1%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 51
Figure 5-5: Rate 4 – Summer Structural Benefiter / Non-Benefiter Analysis by Climate Region & CARE/FERA Status
Figure 5-6: Rate 4 – Winter Structural Benefiter / Non-Benefiter Analysis by Climate Region & CARE/FERA Status
5.2 Rate 5 Figure 5-7 presents the outcome of the annual structural benefiter analysis for Rate 5 at the
aggregate level across climate regions while Figure 5-8 and Figure 5-9 contain the same data
for the summer and winter seasons. SCE’s Rate 5 differs from Rate 4 in several ways. Both
rates have two rate periods on summer weekdays; however the Rate 5 peak period is only three
hours, from 5 to 8 PM, compared to five hours on Rate 4. Additionally, the peak period price is
greater on Rate 5 (47 ¢/kWh versus $40 ¢/kWh on Rate 4). In spite of these differences, overall,
the general pattern of structural benefiters, non-benefiters, and neutrals is very similar between
Rate 4 and Rate 5. Nearly all customers are structural non-benefiters in the summer season,
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Non-Benefiter 80% 85% 74% 61% 51% 42%
Neutral 18% 13% 23% 38% 45% 58%
Benefiter 2% 2% 3% 0% 4% 0%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Non-Benefiter 17% 16% 10% 4% 7% 4%
Neutral 54% 57% 62% 90% 69% 91%
Benefiter 30% 27% 28% 6% 24% 6%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 52
and there is a higher proportion of structural non-benefiters among non-CARE/FERA customers
compared to CARE/FERA customers across the seasons and annually.
Figure 5-7: Rate 5 – Annual Structural Benefiter / Non-Benefiter Analysis by CARE/FERA Status
Figure 5-8: Rate 5 – Summer Structural Benefiter / Non-Benefiter Analysis by CARE/FERA Status
AllAll - Non-
CARE/FERA
Moderate& Cool -
CARE/FERA
Non-Benefiter 31% 34% 20%
Neutral 62% 57% 79%
Benefiter 7% 9% 1%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
AllAll - Non-
CARE/FERA
Moderate& Cool -
CARE/FERA
Non-Benefiter 63% 66% 49%
Neutral 35% 31% 50%
Benefiter 2% 3% 0%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 53
Figure 5-9: Rate 5 – Winter Structural Benefiter / Non-Benefiter Analysis by CARE/FERA Status
Figure 5-10, Figure 5-11, and Figure 5-12 display the annual, summer and winter structural
benefiter distributions for the CARE/FERA and non-CARE/FERA groups across the different
climate regions. Overall, the distributions of structural benefiters and non-benefiters for the Rate
5 climate regions are similar to the estimates found for Rate 4.
The CARE/FERA group also had fewer non-benefiters and more participants in the neutral
category compared to non-CARE/FERA in the moderate and cool climate regions throughout
the year. The hot and Climate Zone 10 regions have larger proportions of non-benefiters at the
summer and annual levels compared to the cool and moderate climate regions. The hottest
regions also have the largest number of benefiters during the winter months.
AllAll - Non-
CARE/FERA
Moderate& Cool -
CARE/FERA
Non-Benefiter 10% 11% 5%
Neutral 67% 62% 88%
Benefiter 23% 27% 7%
0%10%20%30%40%50%60%70%80%90%
100%P
erc
en
t o
f C
us
tom
ers
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 54
Figure 5-10: Rate 5 – Annual Structural Benefiter / Non-Benefiter Analysis by Climate Region & CARE/FERA Status
Figure 5-11: Rate 5 – Summer Structural Benefiter / Non-Benefiter Analysis by Climate Region & CARE/FERA Status
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Non-Benefiter 51% 55% 40% 29% 21% 15%
Neutral 42% 39% 50% 69% 69% 84%
Benefiter 6% 6% 10% 2% 10% 1%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Non-Benefiter 81% 86% 74% 62% 53% 43%
Neutral 17% 13% 23% 38% 44% 57%
Benefiter 2% 1% 3% 0% 3% 0%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 55
Figure 5-12: Rate 5 – Winter Structural Benefiter / Non-Benefiter Analysis by Climate Region & CARE/FERA Status
5.3 Comparison Across Rates Figure 5-13, Figure 5-14, and Figure 5-15 present the results of the structural bill impact
analysis for each rate side-by-side. The two rates have very similar distributions of benefiters,
non-benefiters, and neutral customers.
Figure 5-13: Annual Structural Benefiter / Non-Benefiter Analysis by Rate
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Non-Benefiter 19% 17% 12% 6% 8% 5%
Neutral 52% 54% 59% 86% 67% 89%
Benefiter 29% 29% 29% 8% 24% 6%
0%10%20%30%40%50%60%70%80%90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5
AllAll - Non-
CARE/FERAModerate & Cool
- CARE/FERA
Non-Benefiter 31% 31% 34% 34% 19% 20%
Neutral 62% 62% 57% 57% 79% 79%
Benefiter 7% 7% 9% 9% 1% 1%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
SECTION 5 STRUCTURAL BILL IMPACTS
Default Time-of-Use Pricing Pilot Interim Evaluation 56
Figure 5-14: Summer Structural Benefiter / Non-Benefiter Analysis by Rate
Figure 5-15: Winter Structural Benefiter / Non-Benefiter Analysis by Rate
Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5
AllAll - Non-
CARE/FERAModerate & Cool
- CARE/FERA
Non-Benefiter 61% 63% 65% 66% 49% 49%
Neutral 36% 35% 32% 31% 51% 50%
Benefiter 3% 2% 3% 3% 0% 0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%P
erc
en
t o
f C
us
tom
ers
Non-Benefiter Neutral Benefiter
Rate 4 Rate 5 Rate 4 Rate 5 Rate 4 Rate 5
AllAll - Non-
CARE/FERAModerate & Cool
- CARE/FERA
Non-Benefiter 9% 10% 10% 11% 4% 5%
Neutral 69% 67% 64% 62% 90% 88%
Benefiter 22% 23% 26% 27% 6% 7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Pe
rce
nt
of
Cu
sto
me
rs
Non-Benefiter Neutral Benefiter
Default Time-of-Use Pricing Pilot Interim Evaluation 57
6 Customer Attrition
This section summarizes customer attrition and opt-out rates for each rate and informational
treatment tested by SCE. As discussed in Section 3.3, an analysis of customer opt-out rates can
provide useful insights concerning relative customer preferences among the rates. Comparing
opt-out rates across notification treatment options reveals whether seasonal or monthly bill
impact information or alternative messaging impacts customer acceptance of the rate offer.
Findings for these and related metrics are discussed in this section, which is organized around
pre- and post-enrollment periods.
6.1 Pre-enrollment Opt-outs During the pre-enrollment notification period, customers could take one of the following actions:
Opt out to their OAT;
Select a different TOU rate option than the one they were scheduled to be defaulted
onto (including switching from Rate 4 to Rate 5 or vice versa);
Do nothing, which would enroll them on the default rate that was offered to them.
As discussed in Section 3, these decisions can be examined using pairwise comparisons across
rates and other treatment options. Pairwise comparisons are discussed in Section 6.1.1.
6.1.1 Pairwise Comparisons
Figure 6-1 shows the percent of customers who chose to remain on the OAT for each rate
option and customer segment. As seen, for this metric, there is very little difference in opt-out
rates between Rates 4 and 5. For the service territory as a whole, the opt-out rate for Rate 4
was 18.4% and the value for Rate 5 was 18.2%. The observed difference between the two is
not statistically significant. The difference in opt-out rates for CARE/FERA and non-CARE/FERA
customers are similarly small and not statistically significant. Using this metric, there does not
appear to be a preference for one rate over the other. However, Figure 6-2 tells a somewhat
different story about customer preferences. The opt-out rates in this figure include customers
who rejected the default rate option in favor of either the OAT or the alternative TOU rate. Using
this metric, more customers rejected Rate 4 than Rate 5. While the difference in these opt-out
rates was relatively small, the one percentage point difference equals roughly a 5% difference in
opt-out rates across two Rate options and it is statistically significant for all three groups shown
in Figure 6-2.
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 58
Figure 6-1: Pre-Enrollment Opt-Out Rates by CARE/FERA Status (Customers Choosing the OAT Rather than the Default Rate)
Figure 6-2: Pre-Enrollment Opt-Out Rates by CARE/FERA Status (Customers Choosing the OAT or an Alternative TOU Rate Rather than the Default Rate)
Figure 6-3 and Figure 6-4 show the pre-enrollment opt-out rates by default pilot rate, climate
region, and CARE/FERA status for the two types of opt-out actions (e.g., opt out to the OAT or
opt out to the OAT plus the alternative TOU rate). As seen, opt-out rates are lowest in the cool
climate region and highest in the two hot regions under both definitions of opt-out. For non-
CARE/FERA customers, there is nearly a five percentage point difference, or a difference of
almost 30%, in opt-out rates between the cool zone and the two hot climate regions for both
rates using the OAT opt-out definition. Using the other opt-out definition, the difference across
regions was slightly less but still quite significant. Given that customers in the hot regions are
more likely to see bill increases on the TOU rate compared to the OAT than are customers in
AllAll - Non-
CARE/FERAModerate & Cool -
CARE/FERA
Rate 4 18.4% 18.6% 17.8%
Rate 5 18.2% 18.4% 17.5%
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%%
of
Cu
sto
mers
wh
o O
pte
d O
ut
Rate 4 Rate 5
AllAll - Non-
CARE/FERAModerate & Cool -
CARE/FERA
Rate 4 19.3% 19.4% 18.9%
Rate 5 18.5% 18.6% 17.8%
0%
5%
10%
15%
20%
25%
% o
f C
usto
mers
wh
o O
pte
d O
ut
Rate 4 Rate 5
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 59
the cool region, it would appear that the structural bill comparison information provided as part
of the notifications was read by many and had the intended effect of alerting customers in the
hotter regions that the new rate could lead to bill increases.
Figure 6-3: Pre-Enrollment Opt-Out Rates by Climate Region and CARE/FERA Status (Customers Choosing the OAT Rather than the Default Rate)
Figure 6-4: Pre-Enrollment Opt-Out Rates by Climate Region and CARE/FERA Status (Customers Choosing the OAT or an Alternative TOU Rate Rather than the Default Rate)
Another important question is whether the granularity of the structural bill information provided
or the type of messaging had a material impact on opt-out rates. Table 6-1summarizes the eight
different notification treatment cells that were tested in the pilot. The cells differ with respect to
the default rate offered (Rate 4 or Rate 5), the granularity of the bill comparisons presented
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Rate 4 21.1% 21.3% 20.0% 20.0% 16.8% 16.6%
Rate 5 21.4% 21.0% 19.8% 19.0% 16.5% 16.7%
0%
5%
10%
15%
20%
25%
% o
f C
usto
mers
wh
o O
pte
d O
ut
Rate 4 Rate 5
Non-CARE/FERA
Non-CARE/FERA
Non-CARE/FERA
CARE/FERANon-
CARE/FERACARE/FERA
Hot Zone10 Moderate Cool
Rate 4 21.7% 21.9% 20.7% 20.9% 17.7% 17.9%
Rate 5 21.7% 21.1% 20.1% 19.3% 16.8% 17.1%
0%
5%
10%
15%
20%
25%
% o
f C
usto
mers
wh
o O
pte
d O
ut
Rate 4 Rate 5
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 60
(monthly or seasonal), and the messaging type (loss aversion versus opportunity to save). Each
cell contained approximately 50,000 customers.
Table 6-1: Default Notification Cells
Notification Cell
Rate Rate
Comparison Messaging
1
4
Monthly Loss Aversion
2 Opportunity to Save
3 Seasonal
Loss Aversion
4 Opportunity to Save
5
5
Monthly Loss Aversion
6 Opportunity to Save
7 Seasonal
Loss Aversion
8 Opportunity to Save
Table 6-2 and the following two tables report statistics for three types of actions. The column
labeled “Chose Alternate TOU Rate” refers to customers offered Rate 4 who chose Rate 5 or
who were offered Rate 5 and chose Rate 4. The column labeled “Opted Out” refers to
customers who did not accept either the rate offered or the alternative pilot rate. Finally, to
answer the question “How many customers did not accept the rate they were offered?” the
percentages reported in the previous two columns can be added together. Those percentages
are shown in the final two columns labeled “Opted Out of Default Rate.” Cells shaded in gray
indicated that there is not a statistically significant different between treatment cells at the 90%
confidence level (a p-value greater than 0.100)
Table 6-2 shows the difference in opt-out rates between two messages included in default pilot
notification materials. Customers presented with a loss aversion message were slightly more
likely to opt out compared to customers receiving a message framing TOU as an opportunity to
save money on their electricity bills. This result occurred for both opt-out definitions. The
difference was statistically significant for most comparisons. For example, about 18.8%
customers in the “loss aversion” cells opted out to the OAT while 17.9% in the “opportunity” cells
opted out to the OAT. The p-value for these t-tests was less than 0.100, indicating that the
difference is statistically significant.
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 61
Table 6-2: Effect of Messaging on Pre-enrollment Opt-outs
(Loss Aversion vs. Opportunity to Save)
Comparison Notification
Cell Messaging
Treatment Customers
Chose Alternate TOU
Rate
Opted Out to OAT
Opted Out of Default Rate
% P-
value % P-value % P-value
1 vs. 2 1 Loss Aversion 49,998 1.0%
0.012 18.9%
0.004 19.9%
0.001 2 Opportunity 49,999 0.8% 18.2% 19.0%
3 vs. 4 3 Loss Aversion 50,000 0.9%
0.339 18.9%
0.000 19.8%
0.000 4 Opportunity 50,000 0.8% 17.7% 18.6%
5 vs. 6 5 Loss Aversion 50,000 0.3%
0.470 18.7%
0.000 19.0%
0.000 6 Opportunity 49,998 0.3% 17.8% 18.0%
7 vs. 8 7 Loss Aversion 50,000 0.3%
0.047 18.6%
0.000 18.9%
0.000 8 Opportunity 49,999 0.2% 17.7% 18.0%
* A shaded cell indicates estimate is not statistically significant
Table 6-3 compares pre-enrollment opt-out rates for customers who received monthly rate
comparisons and those who received a seasonal rate comparison in their default notification
materials. In all instances, the differences in opt-out rates were very small and were not
statistically significant except for one instance (the “opportunity message” for Rate 4). These
results indicate that the granularity of rate comparisons did not affect the pre-enrollment opt-out
rates.
Table 6-3: Effect of Granularity of Rate Comparison Information
on Pre-enrollment Opt-outs
(Monthly vs. Seasonal)
Comparison Notification
Cell
Granularity of Rate
Comparison
Treatment Customers
Chose Alternate TOU
Rate
Opted Out to OAT
Opted Out of Default Rate
% P-
value %
P-value
% P-
value
1 vs. 3 1 Monthly 49,998 1.0%
0.321 18.9%
0.977 19.9%
0.834 3 Seasonal 50,000 0.9% 18.9% 19.8%
2 vs. 4 2 Monthly 49,999 0.8%
0.575 18.2%
0.044 19.0%
0.063 4 Seasonal 50,000 0.8% 17.7% 18.6%
5 vs. 7 5 Monthly 50,000 0.3%
0.524 18.7%
0.703 19.0%
0.765 7 Seasonal 50,000 0.3% 18.6% 18.9%
6 vs. 8 6 Monthly 49,998 0.3%
0.530 17.8%
0.828 18.0%
0.772 8 Seasonal 49,999 0.2% 17.7% 18.0%
* A shaded cell indicates estimate is not statistically significant
Finally, Table 6-4 shows the difference in opt-out rates between customers offered Rate 4 and
Rate 5 for each of the notification treatment options. Each pairwise comparison holds
differences in notification options constant and compares the opt-out rates for each tariff. As
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 62
discussed above, based on the opt-out definition structured around the OAT, there is very little
difference between the values for each rate. However, when switching from the default rate to
the alternative rate is factored into the definition, there are statistically significant differences
with the opt-out rate for Rate 4 being higher than for Rate 5. This is because many more
customers offered Rate 4 chose Rate 5 than vice versa, as seen in the column labeled “Chose
Alternate TOU Rate.” The percent of customers who were offered Rate 4 but chose the
alternate rate was about 0.9% while the percent of customers who were offered Rate 5 and
chose the alternate rate was closer to 0.3%. Although the peak period price is greater on Rate
5, the peak period is shorter, which may appeal to customers. This difference was statistically
significant in each pairwise comparison presented in the table below.
Table 6-4: Effect of Rate Offered on Pre-enrollment Opt-outs
Comparison Notification
Cell Rate
Treatment Customers
Chose Alternate TOU
Rate
Opted Out to OAT
Opted Out of Default Rate
% P-
value %
P-value
% P-
value
1 vs. 5 1 Rate 4 49,998 1.0%
0.000 18.9%
0.389 19.9%
0.000 5 Rate 5 50,000 0.3% 18.7% 19.0%
2 vs. 6 2 Rate 4 49,999 0.8%
0.000 18.2%
0.078 19.0%
0.000 6 Rate 5 49,998 0.3% 17.8% 18.0%
3 vs. 7 3 Rate 4 50,000 0.9%
0.000 18.9%
0.204 19.8%
0.000 7 Rate 5 50,000 0.3% 18.6% 18.9%
4 vs. 8 4 Rate 4 50,000 0.8%
0.000 17.7%
0.972 18.6%
0.017 8 Rate 5 49,999 0.2% 17.7% 18.0%
* A shaded cell indicates estimate is not statistically significant
6.2 Post-enrollment Opt-Outs Post-enrollment opt-out rates were very small during the period following enrollment through the
end of the summer (September 2018). Cumulative opt-out rates are presented for the post-
enrollment period for each climate region and CARE/FERA status in Figure 6-5, Figure 6-6, and
Figure 6-7. Generally any difference in cumulative opt-out rates between segments occurred
during the pre-treatment period. Post-enrollment opt-out rates for all customer segments were
between 0.7% and 1.0%. Post enrollment opt-out rates are lowest in the cool climate region and
highest in the hot region. Within the moderate climate region, Rate 5 customers show a slightly
lower opt-out rate than Rate 4 customers.
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 63
Figure 6-5: Cumulative Opt-Out Rates for Hot and Zone 10 Climate Regions13
Figure 6-6: Cumulative Opt-Out Rates for Moderate Climate Region
13
Opt-out rates here present customers who opted out to the OAT, not those who opted out into the alternate rate.
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
Mar2018
Apr2018
May2018
Jun2018
Jul 2018 Aug2018
Sep2018
% o
f C
usto
mers
wh
o O
pte
d O
ut
Rate 4 Hot Rate 4 Zone10 Rate 5 Hot Rate 5 Zone10
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
Mar2018
Apr2018
May2018
Jun2018
Jul 2018 Aug2018
Sep2018
% o
f C
usto
mers
wh
o O
pte
d O
ut
Rate 4 Moderate - Non-CARE/FERA Rate 4 Moderate - CARE/FERA
Rate 5 Moderate - Non-CARE/FERA Rate 5 Moderate - CARE/FERA
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 64
Figure 6-7: Cumulative Opt-Out Rates for Cool Climate Region
Also of interest are post-enrollment opt-out rates by aftercare treatment cell. Table 6-5
summarizes the various treatments that were examined after customers enrolled on the new
TOU rates and the sample sizes for each treatment group.
The enrolled population on each of the default rates was divided equally into those slated to
receive basic or enhanced welcome packets and ongoing education and outreach (E&O)
communication and then segmented further into two groups, those deemed to be most impacted
by bill volatility and those who are not. The segment of customers impacted by bill volatility was
considered to be income-constrained customers who experience increased seasonal bill
differentials under the default TOU rate. As seen in Table 6-5, this segment of customers is
further divided into two equal groups, with one group receiving information on SCE’s Level
Payment Plan (LPP) as a means of managing month-to-month bill volatility.
Table 6-5: Post-Enrollment Treatments
Aftercare Cell
Rate Communication Impacted by Bill
Volatility LPP
Promotion Sample
Size
1
4
Enhanced E&O
Impacted by Bill Volatility
LPP Promotion 6,448
2 No Promotion 6,448
3 Not Impacted No Promotion 64,245
4
Basic E&O
Impacted by Bill Volatility
LPP Promotion 6,420
5 No Promotion 6,418
6 Not Impacted No Promotion 64,245
7
5
Enhanced E&O
Impacted by Bill Volatility
LPP Promotion 6,646
8 No Promotion 6,644
9 Not Impacted No Promotion 65,311
10
Basic E&O
Impacted by Bill Volatility
LPP Promotion 6,705
11 No Promotion 6,703
12 Not Impacted No Promotion 65,195
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
Mar2018
Apr2018
May2018
Jun2018
Jul 2018 Aug2018
Sep2018
% o
f C
usto
mers
wh
o O
pte
d O
ut
Rate 4 Cool - Non-CARE/FERA Rate 4 Cool - CARE/FERA
Rate 5 Cool - Non-CARE/FERA Rate 5 Cool - CARE/FERA
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 65
Figure 6-8 shows cumulative post-enrollment opt-out rates for the various aftercare treatment
cells and Table 6-6 shows similar information along with the results of a series of t-tests. Cells
highlighted in gray indicate that the difference in opt-out rates within that comparison is not
statistically significant. While the opt-out rate for customers impacted by bill volatility who
received the LPP offer is greater than the rate for those who did not, this difference is not
statistically significant at the 90% confidence level for both rates combined and separately. The
same is true between customers who received the basic E&O versus those who received the
enhanced E&O. The only statistically significant difference in opt-out rates occurred between
Rate 4 and Rate 5. However, the actual percentage point difference is incredibly small (1.34%
versus 1.19%).
Figure 6-8: Cumulative Post-Enrollment Opt-Out Rates by Aftercare Treatment
LPP Offer No OfferBasicE&O
EnhancedE&O
Rate 4 Rate 5
Impacted by BillVolatility
E&O Type Rate
Opt-Out Rate 1.28% 1.17% 1.29% 1.24% 1.34% 1.19%
0.0%
0.2%
0.4%
0.6%
0.8%
1.0%
1.2%
1.4%
1.6%
% o
f E
nro
lled
Cu
sto
mers
w
ho
Op
ted
Ou
t
SECTION 6 CUSTOMER ATTRITION
Default Time-of-Use Pricing Pilot Interim Evaluation 66
Table 6-6: Cumulative Post-Enrollment Opt-Out Rates by Aftercare Treatment
Rate Comparison Aftercare Treatment
Number of Customers
Post-enrollment
Opt-Out Rate14
P-Value
Both Rates
Impacted by Bill Volatility
LPP Offer 25,790 1.28%15 0.276
No Offer 25,726 1.17%
E&O Type Basic E&O 153,363 1.29%
0.252 Enhanced E&O 153,352 1.24%
Rate Rate 4 152,432 1.34%
0.000 Rate 5 154,283 1.19%
Rate 4
Impacted by Bill Volatility
LPP Offer 12,722 1.30% 0.545
No Offer 12,704 1.22%
E&O Type Basic E&O 76,204 1.35%
0.767 Enhanced E&O 76,228 1.33%
Rate 5
Impacted by Bill Volatility
LPP Offer 13,068 1.25% 0.348
No Offer 13,022 1.13%
E&O Type Basic E&O 77,159 1.23%
0.176 Enhanced E&O 77,124 1.15%
* A shaded cell indicates estimate is not statistically significant
14
An additional decimal point is included to avoid confusion that could result from rounding errors.
15 Unlike the previous sections, these percentages are based on the enrolled population, not the notified population. This leads to
higher opt-out rate estimates.
Default Time-of-Use Pricing Pilot Interim Evaluation 67
7 Key Findings
The first summer of SCE’s default TOU pilot summarized above has produced a large amount
of information that will help guide SCE’s approach to implementation of default TOU pricing.
However, it must be kept in mind that these load impact findings are based on only the summer
months. Load impacts will differ significantly during winter months and the actions of TOU pilot
participants may be quite different over the course of a full year.
Differences in load and bill impacts and opt-out rates across customer segments at the service
territory level reflect not just differences across segments, but also differences in the mix of
customers across climate regions. CARE/FERA customers in the hot climate region and Climate
Zone 10 were not allowed to be enrolled on TOU tariffs using default recruitment. As such,
comparisons across the two hot and two more moderate regions not only reflect differences in
climate but also differences in the mix of customers. These differences must be kept in mind
when making comparisons across segments and climate regions.
7.1 Load Impacts Key findings pertaining to load impacts from the SCE pilots include:
On average, default customers on both Rates 4 and 5 produced small but statistically
significant, peak-period load reductions. Peak period load reductions averaged roughly
1.5% for Rate 4 and 2.0% for Rate 5.
Load reductions for the common hours shared by the two rates (5 to 8 PM) were greater
for Rate 5 than for Rate 4, likely because of the higher peak period price per kWh. It’s
also possible the shorter peak period of Rate 5 allowed for greater flexibility in customer
response to the price signal. The difference was statistically significant for the territory as
a whole, the moderate climate region, and Climate Zone 10.
Statistically significant but small reductions in daily electricity use were found for both
rates and in all climate regions. It appears that the average customer in SCE’s service
territory was more likely to reduce overall usage during the peak period rather than shift
usage to off-peak hours.
The pattern of load reductions across climate regions in absolute terms was consistent
between the two rates but was slightly different in percentage terms. Absolute peak
period load reductions were largest in Climate Zone 10 and the hot climate region
regions, but these segments did not include CARE/FERA customers. Absolute impacts
were smallest in the cool climate region, which included CARE/FERA and non-
CARE/FERA customers.
In the moderate and cool climate regions, non-CARE/FERA customers typically had
statistically significantly greater peak period impacts compared to CARE/FERA
customers. One exception was households in the moderate climate region on Rate 4,
where the difference was not statistically significant. This finding is consistent with the
opt-in TOU pilot.
SECTION 7 KEY FINDINGS
Default Time-of-Use Pricing Pilot Interim Evaluation 68
With one exception, the incremental peak period impact among households who
received the Enhanced E&O treatment compared to households that did not was not
statistically significant. In other words, the additional messaging did not increase peak
period impacts. The exception was CARE/FERA customers in the moderate climate
region who had an incremental increase in load impacts equal to about 0.6%.
The offer to high bill volatility, low income customers to enroll on the Level Pay Plan as a
way of managing volatility in bills across months and seasons was only taken up by a
very small number of customers.
7.2 Structural Bill Impacts Key findings pertaining to bill impacts include:
Rate 4 and Rate 5 have very similar distributions of structural benefiters, non-benefiters,
and customers in the neutral bill impact category of ±$3/month.
A majority of customers are neither structural benefiters nor non-benefiters on an annual
basis. Over 30% of non-CARE/FERA customers are structural non-benefiters while
fewer than 20% of CARE/FERA customers fall into the same category. However, the
CARE/FERA group does not include customers in the hot climate region where bill
increases under the TOU rates are more likely to occur.
Over 50% of customers in the hot climate region and Climate Zone 10 are structural
non-benefiters on an annual basis. In the summer months, about 80% of customers in
these regions are structural non benefiters while about 15% fall into the neutral category.
Roughly 40% and 60% of CARE/FERA customers in the moderate and cool climate
regions, respectively, are neither structural benefiters nor non-benefiters in the summer
months.
In the winter months, between 25% and 30% of non-CARE/FERA customers in all
climate regions would save money on TOU rates. This outcome is expected because
SCE’s OAT is not seasonally differentiated. The TOU rates are seasonally differentiated
with higher prices during the summer and lower prices during the winter.
7.3 Customer Attrition Key findings pertaining to the opt-out analysis include:
When the pre-enrollment opt-out decision is defined as selecting the OAT rather than the
offered default rate, the difference in opt-out rates between Rates 4 and 5 were very
small and not statistically significant. However, when the opt-out decision is defined as
choosing either the OAT or the alternative TOU rate, the opt-out rate was about 5%
higher (one percentage point) for Rate 4 than for Rate 5. This finding, along with the fact
that more customers offered Rate 4 chose Rate 5 than vice versa, indicates that the
average customer has a small but statistically significant preference for Rate 5 over Rate
4.
Customers presented with loss aversion messaging were slightly more likely to opt out
before enrollment compared to those who received messaging focused on an
opportunity to save money on TOU. This difference was statistically significant.
SECTION 7 KEY FINDINGS
Default Time-of-Use Pricing Pilot Interim Evaluation 69
There was no difference in pre-enrollment opt-out rates between customers who
received a monthly rate comparison and those who received a seasonal rate
comparison. Though, it should be noted that a total annual bill comparison was also
presented to both informational treatment groups.
Post-enrollment opt-out rates were very small and fell between 0.7% and 1.0% for
CARE/FERA and non-CARE/FERA customers in all climate regions. This indicates the
vast majority of customers stay on the rate once they are enrolled on a TOU rate.
Customers on Rate 4 were statistically significantly more likely to opt out post-
enrollment. Again, it is possible the longer peak period was less desirable for some
customers. However, the difference was very small (1.3% vs. 1.2%).
7.4 A Note About Comparing Default and Opt-in Results If comparisons are made between results from this default pilot and the prior opt-in pilot, it is
important to note a few important considerations:
The first summer for the opt-in pilot covered July through September, while the default
pilot estimates presented in this report include June through September. The omission of
June, which is often a cooler month, from the opt-in pilot could affect the size of the
impacts from the first summer.
The peak period for Rate 1 in the opt-in pilot was from 2 PM to 8 PM whereas, the peak
period for Rate 4 in the default pilot is from 4 PM to 9 PM. Rate 2 in the opt-in pilot has
the same peak period hours, 5 PM to 8 PM, as Rate 5 in the default pilot.
The peak period prices and price ratios also changed between the opt-in and default
pilot. The summer peak period price for Rate 1 was $0.35 during the longer peak period
under the opt-in pilot compared to $0.41 under the shorter peak period for Rate 4 in the
default pilot. The peak to super-off-peak ratio for Rate 1 was 1.5:1 while the peak to off-
peak ratio for Rate 4 is 1.8:1. The summer peak period price for Rate 2 in the opt-in pilot
($0.54 ¢/kWh) was higher than for Rate 5 in the default pilot ($0.49 ¢/kWh). The peak to
super-off-peak ratio for Rate 2 was 3.1:1 while the peak to off-peak ratio for Rate 5 is
2.1:1.
The opt-In pilot included CARE/FERA customers in each climate region whereas the
default pilot does not include CARE/FERA customers in the hot climate zone or in
Climate Zone 10.
Climate Zone 10 was included in the Moderate climate region in the opt-in pilot.
In summary, the months included in the evaluation, peak period hours, prices, and inclusion of
CARE/FERA customers all changed between the opt-in and default pilots. Therefore, the
differences observed between the pilots are not solely a difference in customer response to opt-
in versus default enrollment strategies.
Headquarters
101 2nd Street, Suite 1000
San Francisco CA 94105-3651
Tel: (415) 369-1000
Fax: (415) 369-9700
www.nexant.com