Data-Driven Enrollment: Getting to Know Your Consumers
-
Upload
enroll-america -
Category
Healthcare
-
view
1.056 -
download
2
Transcript of Data-Driven Enrollment: Getting to Know Your Consumers
© 2016 Enroll America | StateOfEnrollment.org
Ed Coleman, National Analytics and Data Director
Molly Warren, Senior Policy Analyst | 05.13.16
Data Driven Enrollment
© 2016 Enroll America | StateOfEnrollment.org 2
1. What we mean by Data Driven Enrollment2. How we Learn What we Learn3. Findings from OE3
1. Consumer Survey Findings2. Increasing Appointment Attendance – Using Text
messages3. Consumer Preferences – Digging into appointment data
from the Connector4. Email Findings5. Fine Message Test Findings
4. Some Planned Future Work5. Discussion + Q&A
Agenda for Today’s Discussion
© 2016 Enroll America | StateOfEnrollment.org
DEFINING DATA DRIVEN ENROLLMENT
© 2016 Enroll America | StateOfEnrollment.org
HOW WE LEARN…
© 2016 Enroll America | StateOfEnrollment.org
Two Ways we Learn
• Start with Data: Take a data set and look to see what kinds of data you have.
• Look for Trends: For example, we have zip code data from the appointment data in the Connector so can look for trends by geography.
Observational learning
• Start with a Question: During OE2, 41% of people failed to attend connector appointment. How can we increase attendance?
• Design a Test: The test should be specifically designed to answer the question
Experimental learning
© 2016 Enroll America | StateOfEnrollment.org
CONSUMER SURVEYObservational Learning:
© 2016 Enroll America | StateOfEnrollment.org
• Fielded an analytics survey of EA’s email list• Generated nearly 3000 responses, nearly 4 times as many responses
as past surveys• Allowed us to group consumers into 7 buckets based on
• 2015 insurance status• Whether or not they shopped for insurance in OE3• 2016 insurance status
• Survey asked about • Health care use and health status• Experience shopping for insurance
• Note: The EA email list leans older and poorer is not a representative sample of the general public or all consumers.
Consumer Survey Details
© 2016 Enroll America | StateOfEnrollment.org 8
Non Shopping Reasons
Overall Shop Unin Non Shop Unin Shop Exit Non Shop Exit0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Sha
re o
f Res
pond
ents
Who
Fou
nd T
hese
Issu
es to
be
a B
arrie
r to
Get
ting
Insu
ranc
e
“What were the biggest reasons why you chose to not sign up for health insurance in 2016?”
© 2016 Enroll America | StateOfEnrollment.org 9
Consumer Confidence
Shop New
Shop Renew
Shop Unin
Shop Exit
Non Shop Unin
Autorenew
Non Shop Exit
Overall
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
15%
19%
38%
36%
17%
8%
23%
20%
52%
50%
24%
33%
23%
43%
26%
43%
33%
32%
37%
31%
61%
49%
52%
37%
Not Confident Somewhat Confident Very Confident
“How confident are you that you made the best decision for you?”
© 2016 Enroll America | StateOfEnrollment.org
Additional Information and Confidence“What other information or help would have made you feel more confident about your choice?”
Overa
ll
Autoren
ew
Non S
hop E
xit
Non S
hop U
nin
Shop E
xit
Shop N
ew
Shop R
enew
Shop U
nin0%
10%
20%
30%
40%
50%
60%
70%
Yearly Costs Info Coverage InfoInfo on How Insurance Works In-Person Help from an Insurance AgentIn-Person Help from a Non-Profit Communication from an Insurance CompanyOther
Shar
e of
Res
pond
ents
Who
Wou
ld F
ind
Mor
e In
form
atio
n A
bout
the
Follo
win
g H
elpf
ul
© 2016 Enroll America | StateOfEnrollment.org 11
Use of In-Person Assistance
Overall Shop Exit Shop New Shop Renew Shop Unin0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
Per
cent
of R
espo
nden
ts W
ho U
sed
The
Giv
en
Res
ourc
e
“Did you use any in-person assistance to help choose a health insurance plan?”
© 2016 Enroll America | StateOfEnrollment.org
Confidence after using Assistance
Not confident Somewhat confident Very confident0%
20%
40%
60%
80%
100%
8%
40%
53%
How confident are you that you chose the best health insurance option for you?
Series1
Response
Per
cent
of R
espo
nses
© 2016 Enroll America | StateOfEnrollment.org
Appointment Satisfaction
I was confident in my assister’s knowledge about how health insurance works.
After my appointment, I had a clear understanding of the health insurance options available to me.
I was confident in my assister's knowledge about the health insurance plans available to me.
Attending my assister appointment was a good use of my time.
I would recommend in-person assistance to a close friend.
My assister made me feel comfortable during my appointment and understood my personal situation.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
12%
12%
11%
6%
7%
6%
34%
34%
31%
24%
24%
26%
41%
41%
48%
57%
58%
60%
Please rate the following statements about your appointment
Strongly disagree Disagree Neutral Agree Strongly agree
© 2016 Enroll America | StateOfEnrollment.org
INCREASING ATTENDANCE RATES
Experimental Learning: In-Person Assistance
© 2016 Enroll America | StateOfEnrollment.org 15
Increasing Appointment Attendance
• We noticed by looking at the data from OE2 that appointment attendance rates were not great (41% no-show rate)
• Question: How do we lower the no-show rate / increase the appointment attendance rate?
• Answer: Randomly test different types of reminder messages and see if any of them resulted in lower no-show rates
© 2016 Enroll America | StateOfEnrollment.org 16
Reminder Messages
1. Standard Message / Control (no additional motivational language): “Remember! Your appointment for free, expert help with your health insurance application is coming up.”
2. Empathy: “Hey there, we know money’s tight. Get all your questions answered at your appointment for free, expert help with your health insurance application.”
3. Bandwagon: “Hey there, lots of people in your neighborhood are getting free, expert help with their health insurance applications. Don’t miss out!”
4. Positive identity: “Thanks for taking charge of your health and your wallet! Remember your appointment for free, expert help with your health insurance application.”
© 2016 Enroll America | StateOfEnrollment.org 17
Attendance Rate by Message
Control Empathy Bandwagon Positive identity0%
5%
10%
15%
20%
25%
30%
35%
29.4%
25.5% 26.2%
30.3%
Connector appointment flake rate by text message treatment group (lower bars are better)
Treatment group
13.4%
© 2016 Enroll America | StateOfEnrollment.org 18
No Show Rate with Personalization
Control Bandwagon Empathy0%
5%
10%
15%
20%
25%
30%
35%33%
27%25%25%
24%
26%
Connector appointment no show rates with and without first name personalization (lower bars are better)
Without first nameWith first name
24%
10.5%
© 2016 Enroll America | StateOfEnrollment.org
CONSUMER PREFERENCES – THE CONNECTOR
Observational Learning:
© 2016 Enroll America | StateOfEnrollment.org
A Scheduling Tool for Consumers
© 2016 Enroll America | StateOfEnrollment.org
Growth in Connector from OE2 to OE3
© 2016 Enroll America | StateOfEnrollment.org
Common & Popular Appointment Times in OE3
Percent of appointments filled by day of week and time of day
Number of attended appointments by day of week and time of day
CO
MM
ON
POPU
LAR
© 2016 Enroll America | StateOfEnrollment.org
Appointment Uptake by Day in OE2 & OE3
sun mon tue wed thu fri sat0%
5%
10%
15%
20%
17%
12% 12%11% 11%
10%
20%
Day of the Week
% of Appts Filled
sun mon tue wed thu fri sat0%
5%
10%
15%
20%
9%10% 11%
10% 9%9%
14%
Day of the Week
Saturday appointments were the most popular in both OE2 and OE3
Percent of Appointments Filled – OE2 Percent of Appointments Filled – OE3
© 2016 Enroll America | StateOfEnrollment.org
Attendance Rates by Appointment Type in OE3
© 2016 Enroll America | StateOfEnrollment.org
Distance Traveled to Appointments in OE2 & OE3
% of Appointments0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
37%
4%7%
9% 8%6%
10%
7%
12%
<1 1-2 2-3 3-4 4-5 5-6 6-8 8-10 >10
Miles Traveled to Appointment - OE2
% of Appointments0%
5%
10%
15%
20%
25%
30%
35%
40%
45%41%
3%
6% 7% 6% 6%
10%
6%
15%
<1 1-2 2-3 3-4 4-5 5-6 6-8 8-10 > 10
Miles Traveled to Appointment- OE3
On average, consumers traveled 4.6 miles to their appointment in OE3
© 2016 Enroll America | StateOfEnrollment.org
No Show Rate by Appointment Distance in OE3
<1 1-2 2-3 3-4 4-5 5-6 6-8 8-10 >100%
5%
10%
15%
20%
25%
30%
35%
40%
32%
36%33% 34%
31% 32%29%
27%
22%
Miles Traveled to Appointment
No S
how
rate
© 2016 Enroll America | StateOfEnrollment.org
More in Recently Released Issue Brief!
© 2016 Enroll America | StateOfEnrollment.org
EMAIL FINDINGSExperimental Learning:
© 2016 Enroll America | StateOfEnrollment.org 29
Email Engagement Rate by Source
0%
4%
8%
12%
16%
20%
4.6%
18%
Email open rate
0.0%
1.0%
2.0%
3.0%
0.27%
3.3%
Email click rate
All emails
Field-collected emails
© 2016 Enroll America | StateOfEnrollment.org
Email A/B Testing - Timing
Adding “monthly” in the subject line resulted in 23% fewer unique opens.
(statistically significant).
Adding “monthly” in the body of the email did not affect the number of
clicks among those who opened the email.
Sent November 15, 2015
Unique opens
Unique clicks
Clicks among opens
Unsubscribes
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Lower monthly Obamacare costs Lower Obamacare costs
****
****
***
Significance levels: * p < .05, ** p < .005, *** p < .0005, **** p<.00005
A B
© 2016 Enroll America | StateOfEnrollment.org
FINE MESSAGE TESTINGExperimental Learning:
© 2016 Enroll America | StateOfEnrollment.org
Fine Message Testing on Calculator
• Test was designed and embedded on the Get Covered Calculator
• Users receive estimated eligibility and monthly insurance cost
• During the test, users randomly assigned to receive one of four messages on the fine.
© 2016 Enroll America | StateOfEnrollment.org
Enrollment by Cost Estimate & Test Group in OE3
Total $99 or less $100-199 $200-299 $300-399 More than $400
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%ControlSimpleExplainDetailed
Calculator cost estimate
"Enroll online" click rate
© 2016 Enroll America | StateOfEnrollment.org
Enrollment by Fine Estimate & Test Group in OE3
$695 $700-1000 $1000-1500 More than $15000%
5%
10%
15%
20%
25%
30%
35%N = 8755
N = 1716
N = 4579
N = 3534
No detailed fine amount
Detailed fine amount
Estimated Fine Amount
"Enroll online" click rate
© 2016 Enroll America | StateOfEnrollment.org
• Consumers in the control group (which received no message about the fine) were slightly more likely to click “Enroll Online” than those who received a message on the fine with their calculator results.
• The finding does not mean we should stop talking about the fine.
• Combined with previous fine test results and other research, such as the Perry Undem surveys, this finding suggests that language on the fine is important to motivate certain consumers at certain times, but not universally.
• Enroll America recommends messaging the benefits of enrollment to motivate consumers, such as drawing attention to low cost premiums and availability of financial help, rather than focusing only on the penalty of not enrolling.
Main Takeaways
© 2016 Enroll America | StateOfEnrollment.org
FUTURE WORK
© 2016 Enroll America | StateOfEnrollment.org
• Looking for connections between earned media and in-person assistance interest
• Looking at consumer behavior though our new (in OE3) Plan Explorer tool
Future work
© 2016 Enroll America | StateOfEnrollment.org
Q + A + DISCUSSION