Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in...
Transcript of Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in...
Impact Analytics in PracticeCheryl Berinato, VP Market Research, Macy’s
Karen Hudzinski, Manager Market Research, Macy’sPeter Kriss, Lead Research Scientist, Medallia
2
Customer Experience at Macy’s
Macy’s My Customer Engagement Program measures customer experience across main touchpoints…
‘Test and Learn’ is not new to Macy’s, but the magnitude and speed of this process have increased.
3
How We Analyze Impact TodayWhat?
Standard monthly analysesAd hoc projects
When?Monthly reporting cadence
YoY primary benchmark
Who?Key CX stakeholders:
Macy’s Stores & macys.com, Furniture & Mattress, Wedding & Gift Registry,
Macy’s Credit Customer Service, Macy’s Backstage, Bloomingdale’s
Omnichannel & Outlet
What works well?Collaboration from stakeholdersData as yardstick of initiative successEngagement from all areas of the organization
What are the challenges?Limited sample size for smaller programsMultiple data sources within MCY & BLM
4
What About Your Organization?
Who are the primary owners of improvement initiatives?
a) Frontline leadersb) Mid-level operational leadersc) Senior operational leaders of each functiond) A centralized teame) Other
What level of knowledge does the CX team usually have about specific initiatives?
a) No insight into what they areb) Aware but only at high levelc) Knows the details well but not
involved in executiond) Directly involved in execution
5
What About Your Organization?
Compare and contrast your organization with others at your table:
Who evaluates whether an initiative is successful?
What types of success metrics are most critical?
What level of analytical rigor is expected?
Choose a notable similarity or difference to share with the room
6
Our Impact Analytics Journey
2015 2016 Today
“Innovation at Scale” Research SciencePartner Program
Results of Prototype
7
A Collaborative Prototype ProcessWeekly conversations with Peter to discuss:
Results and implications of tests run throughout the weekApproaches to dig deeper into noteworthy results
Different or new initiatives to testBarriers to tests we wanted to run but could not
8
Fine Jewelry & Watch Pilot – Impact on NPSPurposeto compare group impacted vs. group not impacted over time to measure what that impact is
Impact scoredifference between the impact and comparison groups, taking into account how much they differed before the pilot
NPS improvementof 24% among jewelry/watch customers attributable to the pilot
+ ##.##
+##.####.##
9
Fine Jewelry & Watch Pilot – Impact on Spend
+ ##.##
+##.####.##Average jewelry/watch
purchase size saw a double-digit percentage increase attributable to the pilot
10
Top Doors Initiative – Impact on NPS
NPS improvement of 1.8%among all customers who purchased in Top Doors pilot stores attributed to the changes
+ ##.##
+#.##
#.##
11
How Action Impact is Valuable
Conceptually valuable
Different lens, fresh perspective
Convenient platform for advanced analyses
Practically valuable
Embedded into Medallia portal
Visual representation
Level of protection
12
Implications and Learnings
Implications for the organization
“Test and learn” phase is not just a phase
Innovation is a necessity
Implications for Customer Engagement team
Addition to quarterly, seasonal, or annual recaps
Quick pulse on pre/post analyses
Indicate when customers begin to give credit for improvements
13
What We Learned About theImpact Analysis Process
Iterative and exploratory Opportunity to go deeper
14
What We Learned About the Potential Value
Prototype Client X
It quickly and easily answers a lot of the questions that we probably spend way too much time trying to figure out the answers to.
“”
Prototype Client Z
It helps us on our own, without assistance from our global analytics team, get a quick and accurate read on the impact … to help us make better decisions about what to do next.
“”
Making analysis easier is good…
…but the key value is supporting decisions
15
Quantifying the impact of initiatives is a strategic imperative
(and it’s never been easier)
16
Q&A