Analytics Academy 2015 Presentations
271
WHY ARE WE TALKING ABOUT ANALYTICS TODAY?
-
Upload
harvardcomms -
Category
Marketing
-
view
257 -
download
1
Transcript of Analytics Academy 2015 Presentations
WHY ARE WE TALKING
ABOUT ANALYTICS
TODAY?
Photo: Tobias Seiderer, Flickr
Photo: Scott Akerman, Flickr
INSIGHTS ACTIONS
YOU GET WHAT
YOU MEASURE
“I’m gonna write me a new
minivan this afternoon!”
WE DO ALL THIS AGAINST
A BACKDROP OF A
WORLD
OF DATAMax Hancock for the Noun Project
Data for storytelling
Thank you.
Enjoy the day!
We Are What We Measure
Steve Mulder
16
17
18
What we measure becomeswhat we reward
19
What we reward becomeswhat we doandwhat we are
20
What are you measuring now?
21
In the world of NPR…
Broadcast radio listening
Digital analytics
Donations and membership
Underwriting revenue
Important things to measure, but do they tell the whole story of what you are trying to do and be?
Are there things you’re doing less of simply because they’re not as easy to measure?
We tend to measure what is familiar and easy to count
22
Community impact
More informed public, influence on culture, policy change, etc.
Audience insights
Listener attitudes, unmet needs, changing behaviors
Digital
Less familiar to many people in our organizations
Constantly changing and hard to keep up with
Overwhelming amount of data at our fingertips
It’s harder to measure what is less quantifiable, newer, changing, or difficult to track
23
Using data effectively
is a lot about mindset
24
25
26
27
Trap #1We measure the wrong thing
28
Trap #2We make decisions by blindly following data
29
30
Trap #3We become optimizers instead of free thinkers
31
Optimization via A/B or usability testing
The danger of optimizing to a local maximum
Local maximum
32
33
1. Data is the platform from which we leap
34
For NPR, ratings and analytics data tell us about listening
35
Third-party research validates trends in media fragmentation and digital adoption
36
Channels proliferate, presenting both opportunity and challenge
“Radio isn’t going away, it’s going everywhere.”
37
Our own research and personas reveal changing consumer behaviors & opportunities
38
It’s not about building things that users know to ask for
39
New digital interfaces in familiar locations (connected car)
40
Experiments with new listening experiences: NPR One
Exploring experiences that meet needs users can’t yet articulate
41
2. Data helps us know how high we’ve jumped
and whether to keep going
42
The Facebook experiment: Local station stories geo-targeted on NPR’s Facebook stream
43
Local stories saw consistently higher engagement and grew local audience
44
Pivot and double-down: We created a workflow tool so we can scale this offering
45
and more about
continuous learning
3. It’s less about achieving goals
46
Yes, this sounds like Lean Startup thinking
Assumption/hypothesis
Minimum Viable Product
Analytics, research, testing
47
Designing choice in the new NPR One app
Original design: Give users more control/choices
Hypothesis to test: Fewer immediate choices + simplicity = longer listening
Results: Listening time up 8%
48
Walking the narrow path
49
Ingredients of an analytics culture
50
Bring data to life, make it memorable and actionable
51
Bring data to life to make it memorable and actionable
52
Emphasize what matters most
53
With great power comes great complexity. Don’t let analytics feel like this.
54
55
Thanks for the metric. Is that good or bad?
Context is the engine of meaning
56
Context can turn data into insight and action
57
Context over time, against averages, or against custom cohorts
58
Embed data in regular processes and habits
59
Embed data in regular processes and habits
60
Data without analysis and storytelling is like food without taste
61
Fewer dashboards,
more insights!
62
Putting Data to Work
How Digital Analytics Informs Operations
Analytics Academy
March 19, 2015
Overview
What is your business goal?How does digital communications
support that goal?
What are the goals of digital strategy?
What can you learn about user behavior to
determine if you are achieving that goal?
How can you use that knowledge to improve
your strategy?
Digital Recruitment Strategy
WEB & SOCIAL
Strategies for inbound traffic
Goals of the Website
• Help prospective applicants discover the
Harvard experience
• Hear authentic current student voices
• Understand the application process and
successfully complete the application
• Access information about application
process and admission decision
Analyzing Web Traffic Google Analytics Monthly Report
Metrics Report Interpret
Interpreting the Metrics
• Seasonality - The Admissions cycle affects traffic to the site. By understanding that traffic we can plan for server capacity and uptime during our most critical time.
Regular Decision Early Decision
Understand Traffic Sources
• Traffic Acquisition - How does it change over time?
January 1, 2014
January 1, 2015
Analytics-driven Content
• Landing pages: identify
where users are
entering your site, other
than the homepage
• Highlight key messages
• Make content easily
accessible across the
site
Understand Traffic Sources
• Mobile Traffic - Mobile traffic has grown from 15% in 2014 to 20.5% of all traffic in one year on smartphones alone. On EA decision day, mobile traffic was nearly one quarter of all traffic.
Mobile Strategy
• Optimize for mobile.
Assume users will view
on mobile first
–Responsive design. All
content must work on
mobile– web, email,
tour
–Test early and often
on many devices,
browsers
Social Media & Google
AnalyticsAcquisition > Social > Users Flow
Social Media Many sources, constant change
• What can we measure?
We can’t we measure?
• Aggregating social data
from various platforms
• Measuring engagement at
the post level to inform
content strategy
Social Analytics
Hootsuite
TwitterCounter
Iconosquare
Tumblr
YouVisit
Informing Social Media Content
• Our audience are 16-18 years old.
• We are monitoring usage to go where they
are, rather than asking them to come to
us.
EMAIL & OUR SEARCH LIST
Strategies for outbound communication
Goals of Outreach
• Search list are highest priority users–
actively recruiting these prospective
applicants
• Targeted outreach focused on students
from diverse socioeconomic and cultural
backgrounds
• Drive traffic to our website, engage with
our students, start an application
Measuring Email Effectiveness
• Engagement is measured in
open rates, clickthroughs
• Deliverability, bounces, quality
of list
• Use UTM codes to link back
to website
• Send through Slate,
integrated with admissions
database. More on that later!
Google URL Builder
● utm_source: Identify the advertiser, site, publication, etc. that is
sending traffic to your property, for example: google, citysearch,
newsletter4, billboard.
● utm_medium: The advertising or marketing medium, for example: cpc,
banner, email newsletter.
● utm_campaign: The individual campaign name, slogan, promo code,
etc. for a product.
● utm_term: Identify paid search keywords.
Screenshot of Email Metrics
Email Report
Email Content Optimize based on learning
• 70% of emails opened on mobile. Design for mobile– Responsive design
– Length of message
– Placement of links
• Know when to send based on audience –3-5pm for our audience
• Test subject lines– A/B testing
SLATE (DATABASE OR CRM)
converting prospects to applicants
CRM & Admissions Database
• Content strategy is based in goals
for recruitment & yield. CRM tracks
progress towards that goal.
• Also makes online reading of
applications possible
• Implemented this year- opportunity
to reevaluate strategies and better
analyze our data!
Student’s Path in App Cycle
• Applicant data is centralized throughout all
stages of the process– as a prospect,
inquiry, and applicant.
Prospect Inquiry Applicant
Getting Specific with our
MessageRecruitment communications for targeted
prospective students
Low socio-economic
status
seniors
Hispanic, African
American, Asian
American, Native
American
Low Application
schools
Low SES Juniors
HFAI UMRP HCC
First in the
family to graduate
from a four-year
college
First
Gen
Measuring Digital Recruitment
Track offline interactions and aggregate data in Slate
• Interactions with current students
• History of engagement is kept in the applicant record
• Query and pull reports based on interactions
Measuring Digital Recruitment
• Widgets, queries and reports demonstrate how
recruitment leads to applications.
15797 17173
18498
20132
2018 2019
Searched
+8.71%
Not Searched
+8.83%
Number of Applicants to the
Class of 2018 and 2019
Hit or miss?
How to tell, and what to do next
What does ‘success’ mean?● Metrics paired to business goals
o Pageviews (reach)
o Time on site (attention)
o Email signups (loyalty, audience building)
● Anecdotal feedbacko internal (local leadership, faculty)
o external (media, influencers, awards)
What does ‘failure’ mean?
● Failed to meet expectations
● Elicited negative responses
● Opportunity cost
From the Harvard channels
From the Harvard channels
Campaign checklistUsing reporting from start to finish
1. Decide what’s worth sharing
2. Build consistent tracking methodology
3. Share prepared content
4. Implement analytics collection process
5. Put analytics through feedback loop
6. Evaluate effectiveness, determine
takeaways
Deciding what to share — and how to share it
Editorial decisions
*Editorial calendar
*Trending topics (Twitter, FB, Iconosquare)
Google trends
Topsy
Encore Alert and similar services
Format
Platform updates
Mobile proliferation
*Current research (Sharing Fast and Slow)
Choose content: Editorial
Choose content: FormatSharing Fast and Slow
104
POST PRIMED FOR SHARING/REACH POST PRIMED FOR INTEREST/CLICKS
200 clicks to story176 retweets, 13 new followers
Build and execute consistent
reporting framework
Put analytics through feedback loop
Evaluate effectiveness, determine takeaways
The art and science of effective social
media in campaign messaging.
Painting By Numbers
Usable Knowledge #ThroughEducation
Launch/learning/measurement Launch/learning/measurement
Usable Knowledge #ThroughEducation
Launch/learning/measurement Launch/learning/measurement
CAMPAIGN PRIORITY (DEAN) CAMPAIGN ENGAGEMENT
Usable Knowledge: A History
Strategy
Strategy
Validate and experiment for hypothesis: Does social deliver?
Case Study
(Experiment)
Strategy
Strategy
Strategy
Social
-Toiling
-Training
-Tweeting
-Timing
(OBSERVE)
-Practice
-Partner
-Performance
Social is like Moneyball
Sabermetrics and scouting
Small tweaks
Primetime!
Platform?
Passion.
#ThroughEducation
Strategy
Social
-Toiling
-Training
-Tweeting
-Timing
(OBSERVE)
-Practice
-Partner
-Performance
National trending
#ThroughEducation
@HGSE Impressions over the
past month – focus on 9/19
Momentum…
Brad Paint By Numbers
Et cetera
thanks.
Copyright © President & Fellows of Harvard College
Behind the Retweet: Learning from Social Analytics
Harvard Analytics Academy, March 19, 2015
Robert Bochnak, Assistant Director, Alumni Marketing and Communications, Harvard Business
School, Harvard Analytics Academy, March 19, 2015
-Assistant Director of Alumni Marketing and Communications at the Harvard Business School where I manage the office’s social media channels
-Have helped generate over 18,000 unique
alumni interactions via Twitter since January
2013
-Graduate of UMass-Amherst and the Tufts
University Graduate School of Arts and
Sciences
-Author, Social Media Matters blog,
http://robertbochnak.wordpress.com/
-Superhero sidekick of Aniceta
15
0
A little bit about me….
Our Approach: Twitter
15
1
Identify
EngageLeverage
Our Approach: Identify HBS alumni who are on
Twitter and add them to the appropriate list.
15
2
Our Approach: Engage alumni through retweets, original
tweets promoting their work, tweets sharing events, tweets
connecting alumni with each other, tweets to alumni based
on their interests, etc.
15
3
Our Approach: Leverage the relationships we have
developed with alumni so they participate in discussions.
15
4
But before all this engagement could happen we
needed to collect data on our alumni….
So we decided that each alumni interaction we had
on Twitter would be tracked on an excel
spreadsheet. This sheet would include information
drawn from the Twitter profile and alumni directory
file of each alumnus/a and would include
information such as city and state of residence,
interest, year of graduation, and section.
15
5
Data Collection
15
6
Data Use
With this data—as earlier slides conveyed—we’re able to send targeted tweets to alumni based on their interests.
15
7
Reporting and disseminating the data
-Each interaction with an alumnus/a is “scored” on
the tracking spreadsheet: 1-2 “touches” a month
equals “minimally engaged”; 2-4 “touches” a
month equals “moderately engaged”; and 5 or
more “touches” equals “highly engaged.”
-Interactions are retweets, favorited tweets, or
original tweets by individual alumni.
15
8
Reporting and disseminating the data cont.
15
9
Reporting and disseminating the data cont.
-We also measure the connections we help make
16
0
Reporting and disseminating the data cont.
-Twitter-based interactions are included in monthly reports.
16
1
Reporting and disseminating the data cont.
16
2
Reporting and disseminating the data cont.
16
3
Thank You!
Questions?
16
4
YOUR GUIDE TO EMAIL METRICS
DETOX
EMAIL METRICS
Which camp do you fall into?
DOES YOUR
DASHBOARD
NEED A
DASHBOARD?
WHAT
DASHBOARD?
BEWARE OF METRICS BLOAT
○ Numbers replace your business goals
○ Confusion among team members
○ Lost alignment within organization
○ Spread thin aggregating metrics
○ Metrics no longer used to inform business
decisions
HOW DO YOU TREAT
METRICS BLOAT?
a. Invest in more fancy tools
b. Wither and die
c. Other
METRICS ARE
PEOPLE, TOOShow some empathy, would you?
COMMUNICATION
IS ABOUT
BUILDING HUMAN
RELATIONSHIPS
“How are you going to build that
relationship?
PROVIDE VALUE
Events○ Workshops
○ Lectures
○ Screenings
Interesting content○ Articles
○ Online videos
○ Digests
Service○ Online application
○ Customer support
Stewardship○ Funds
○ Volunteer opportunities
MEASURE VALUE
Your business goals should align with how
you provide value.
SEPARATE EMAIL
METRICS FROM
SYSTEMS
THINK ABOUT METRICS IN
TERMS OF PEOPLE AND VALUE
EXAMPLE
EVENTS
Place your screenshot here
EVENTS
attendanceevent
invitationregistration
Place your screenshot here
PUBLISHING CONTENT
PUBLISHING CONTENT
sharenewsletter read
STEWARDSHIP
Place your screenshot here
STEWARDSHIP
donationappeal awareness
Place your screenshot here
SERVICE
SERVICE
inquirypromotion awareness
ONCE YOU FIND
YOUR BEACON...
START SIMPLE
Email Attendees
December invitation 25
January invitation 30
February invitation 5
March invitation 25
BUILD TRENDLINES
ASK QUESTIONS
○ What happened in February?
○ Was the creative different?
○ Was the email sent at a different time?
○ Did the email go to the wrong list?
CHOOSE YOUR ADVENTURE
○ Wait for more ‘beacon’ data
○ Tweak your strategy (one variable at a
time)
○ Follow the trail with more analytics
CATHERINE’S
GUIDETo Email Metrics Detox
1. IDENTIFY BUSINESS GOALS
○ What value do you offer?
○ How are you building a relationship with your
audience?
2. CONNECT YOUR COMMS
STRATEGY TO YOUR GOALS
What is the path to your end goal?
3. TRACK THE END GOAL
Use the trendline as a proxy for success.
4. WORK BACKWARDS
○ Ask questions about anomalies.
○ Dig in deeper, if you must.
4. TWEAK
One variable at a time!
CATHERINE’S EMAIL
METRICS DETOX PLAN
1. Identify your
business goals.
2. Connect your
communications
strategy to your
goals.
3. Track the end goal.
4. Work backwards. 5. Tweak
Tracking the Life of an Event
Using Analytics
Event Life Cycle
Define Success and Plan
Accordingly
Measures of Success• Registration
• Revenue
• Attendee satisfaction
• Target demographic representation
• Smooth event flow
• Numbers of first-time attendees
• Social amplification
• Smooth registration process
• Email engagement
• Social connections during/after event
Communications Strategy
• Who is your audience?
• What channels?
• How frequently?
• How long is your campaign?
• What is your content strategy?
Stay Nimble
Establish a Benchmark
Save the Date
1st Invite 2nd Invite
Save the Date
1st Invite 2nd Invite
Your
Harvard:
Mexico
Your
Harvard:
China
Tell the Whole Story
Dive Into Attendee Data
• Registration
• Revenue
• Attendee satisfaction
• Target demographics
• Effectively applying learnings from past events
• Smooth event flow
• Numbers of first-time attendees
• Social amplification
• Smooth registration process
• Email engagement
• Social connections during/after event
Measure Success
Escape Excel Hell: Make Your Reports
Memorable and Actionable
Hi!I’m Mike Petroff
• Digital Content Strategist
• Loves GIFs (but not emojis)
• Twitter: @mikepetroff
• New Dad (excuse any puns)
IS THIS YOU?
IS THIS YOU?
IS THIS YOU?
IS THIS YOU?
GOOD
LUCK!
I’ve been there.
I’ve tried.
I’ve been there.
I’ve tried.
I’ve failed.
If you build it, they will come?
If you build it, they will come?
Data alone is not
inherently
helpful.A brief history
lesson.
Data and maps.
London Underground Tube Map
▣1908
Source: datavizblog.com/
London Underground Tube Map
▣1908
Underground lines were
laid out geographically,
often superimposed over
the roadway of a city map.
Source: datavizblog.com/
London Underground Tube Map
▣Meet Henry Beck
an English engineering draftsman.
London Underground Tube Map
▣1933
Source: datavizblog.com/
London Underground Tube Map
▣1933
Beck believed that
passengers riding the
trains were not too
bothered about the
geographical accuracy,
but were more interested
in how to get from one
station to another, and
where to change.
Source: datavizblog.com/
Format matters.What is the intent of the reader?
Bring focus.
Bring focus.
But…We’re buried
in raw data.
How do we bring
focus to what’s
important?
‘’From 2013 to 2020, the
digital universe will grow by
a factor of 10 – from 4.4
trillion gigabytes to 44
trillion. It more than
doubles every two years.
Source: emc.com
‘’Most companies estimate
that they are analyzing a
mere 12% of the data they
have, according to a recent
study.
Source: Forrester Research
Fix the reporting
process.Find what
matters.
The Failed Reporting Process
collect
report
The Failed Reporting Process
The Successful Reporting Process
collect
analyze
present
Collect
▣ Good data in, good data out
▣ Track consistently
▣ Share the collection process, and make it transparent
Analyze
▣ Provide insights
▣ Highlight opportunities
▣ Identify things that can be changed
Present
▣ Bring focus to the the most important thing
▣ Find the best delivery method
▣ Be consistent, but be memorable
SimplifyShow the goal’s performance.
Simplify
Simplify
Brand ExamplesAnnual Reports
Kickstarter
Gates Foundation
InsightsShow the reason it matters.
Daily Gazette report
What: Quick view of daily Gazette
email performance
Who: Senior leaders, Gazette
editors, Media Relations, Digital
Strategy
Why: Identify what resonates with
readers, engagement trends
Place your screenshot here
Daily Gazette report
Not very
helpful.
Daily Gazette report
Better, but
room to
grow.
Weekly Gazette web
traffic
What: Google Analytics dashboard
Who: Senior leaders, Gazette,
Media Relations, Digital Strategy
Why: Identify overall traffic, explain
sources of traffic, anomalies, review
owned traffic performance, social
impact
Weekly Gazette web traffic
Identifies not only top overall stories, but
highlights traffic generated from owned
sources like Harvard.edu, Daily Gazette
email, and Harvard’s social channels.
Weekly Gazette web traffic
▣ A written summary of select
referral traffic, including
influencers on social media
▣ Identifies media pickup and
impact on traffic
Monthly report
What: Combine metrics across
various digital platforms
Who: Senior leaders, HPAC, AA&D
Why: Analyze trends, large spikes
and dips, measure against previous
month/year
Monthly report
▣ Compare against previous
month and previous year, use
conditional formatting
Monthly report
▣ Provide most important findings and with each
report; explain changes and future opportunities
How to get started
▣ Define the goal and the audience
▣ Identify metrics that measure the goal
▣ Collect metrics consistently
▣ Simplify to show performance against the goal
▣ Present insights that are memorable and lead to
action
YOU control the context.
