NYC Data Driven Business Meetup - 2.7.17
25
olark.com Measuring Chat Success Presented by Karl Pawlewicz, Voice of Olark February 7, 2017 @karlpawlewicz | @Olark
-
Upload
karl-pawlewicz -
Category
Technology
-
view
70 -
download
0
Transcript of NYC Data Driven Business Meetup - 2.7.17
olark.com
Measuring Chat Success
Presented by Karl Pawlewicz, Voice of OlarkFebruary 7, 2017
@karlpawlewicz | @Olark
olark.com
AgendaWhat does it mean to be a Data Driven Business?What's in the Olark data stack?How are we organized to make data accessible?What kind of data does chat (or a similar customer comms channel) provide?Real world examples of data driven decisionsWhere can you go online or IRL for more insight on data best practices?Q&A
olark.com
What does 'data driven' mean?Leadership/CEO
"Data has been an essential part of business since the existence of business. The main difference is now it's easier to get both good and bad data. To be data driven is not to ignore hunches and intuition, but to figure out the best method to validate your ideas, and to act on it. "Data drive decision making reduces uncertainty, but does not remove it. Without data, we are just guessing, and hoping for the best.""Assess our performance at setting and achieving objectives."Help product managers make decisions to determine where to focus their efforts, and understand the results of our efforts in the past, and help frame the revenue impact.""Deciding where to invest in growth, I use information from inside and outside of the organization to make informed decisions on the potential impact of various options."
olark.com
What does 'data driven' mean?Marketing
"Using data to make better team decisions, I can see which campaigns are working (or not), and easily calculate ROI - not just for dollars spent, but also for the time and resources we're investing into each activity. Being data-driven, especially in marketing, makes it easy to think strategically, plan better, help our own team understand how their contributions affect our goals, and show the company as a whole how we're impacting the bottom line."
olark.com
What does 'data driven' mean?Product/Research
"The way I see it, data is most useful when it helps you identify "obvious" and inarguable truths i.e. "Wow what a huge difference in performance, we should definitely do that thing" or "Wow that's clearly a bad idea, it got dominated by this other thing." Data helps provide definite answers for smaller, simpler problems. But sometimes, for bigger and more difficult decisions that seem to be 50/50 even after you've seen all the data, those are the ones where great intuition come in and (after a certain point) no amount of additional data would help. Steve Jobs is a common example of someone with great product insights. No amount of data would have been able to definitively tell him either way that a new device like the iPad would be a huge success. However, I'm sure data allowed Apple to become pretty confident in taking that risk."
olark.com
What does 'data driven' mean?Engineering
"- project requirements include identifying / collecting / analyzing key metrics on which to base decisions
- project deliverables include some report on those metrics
More generally, projects are seen as experiments, and therefore are not complete without the data and the analysis."
olark.com
What's in the Olark data stack?
Customer data:LookerSegmentAmplitudeGoogle Analytics
Marketing data:HubspotUnbounceSEMRushMozGoogle Analytics
Third Party APIsMonkey LearnBuiltwithAlexaFirstOfficer
olark.com
How are we organized?Team size:
Data department of one consults with all departments to understand common and team-specific needs.
Important aspects of this system: One central warehouse so that all team members are using the same data. It's kept up
to date and is known to be high-integrity.Data accessibility in incredibly important, but having standard definitions that everyone
on the team knows and can trust is as important. If multiple team leads are building reports, it helps to have the data team define and implement the metric definitions so that everyone is computing the same thing, and not people reporting metrics with the same name differently.
olark.com
Data from chat?Quantitative:
Chat volumeHow's my team doing RevenueWhere are people chatting on my siteWhat percentage of people chatWhat countries are visitors chatting from
Qualitative:Product feedbackWhat are people saying about my company (sentiment)What are people saying about my competitorsWhich chats lead to sales
olark.com
Data from chat?Quantitative - Volume
olark.com
Data from chat?Quantitative - How's my team doing
olark.com
Data from chat?Quantitative - Revenue
olark.com
Data from chat?Quantitative - Where are people chatting on my site
olark.com
Data from chat?Quantitative - What percent of people chat
olark.com
Data from chat?Quantitative - Where country are visitors chatting from
olark.com
Data from chat?Qualitative - Product feedback
olark.com
Data from chat?Qualitative - How are people comparing me to my competitors?
olark.com
Data Driven Decisions IRLWhat do you want chat to do?
Improve customer happiness?How to measure: Chat reports, NPS,
Increase conversions?How to measure: Number of conversations, Number of sign-ups
Increase sales?How to measure: Revenue reports, CRM results
Product feedback?How to measure: Chat transcripts
Marketing insight?How to measure: Chat transcripts
olark.com
Data Driven Decisions IRLOnePageCRM -Alan O'Rourke at OnePageCRM:
Manually compared new customers in their CRM with Olark logs to see who talked to customer success before signing up.
New customers who chatted to customer success before signing up were almost twice the size of our average customer.
His two person live chat team brought in an additional $24,000 in new business in over 12 months (not including renewals).
240 hours on chat, earning $100 every hour - an average that out performed most of their other marketing channels.
Based on this data, Alan made the decision to hire another person to staff chat on OnePageCRM's marketing site in a timezone they weren't already covering.
SOURCE: https://blog.olark.com/how-to-measure-live-chat-success-with-analytics
olark.com
Upper90Soccer.com -Ben Jata of Upper90"Should I add more people on chat?"
Conversion Rate increased 11% when chatting
Full-time chat specialist generated $40K in revenue in roughly 6 months
Jan 16
Feb 16
Mar 16
Apr 16
May 16
Jun 16
Jul 16
Aug 16
Sept 16
Oct 16
Nov 16
Dec 16
OLARK
Total chats 5 4 10 7 7 154 328 407 391 416 431 428
Great chats 2 1 4 0 1 24 45 54 82 48 46 10
Chats to review 0 0 0 1 0 3 5 0 6 4 3 1
Average chats per day 1.67 1.33 2.5 1.4 1.75 8.56 13.12 15.65 17.77 16.64 17.96 22.53
Median initial response time (seconds) 16 13 30 18 24 28 25 24 19 19 18 23
Offline messages 23 54 46 36 38 55 59 55 64 55 73 64
Ratings (5 stars max)
Overall chat 4.5 5 4.75 3 5 4.85 4.63 4.78 4.75 4.8 4.86 4.55
Responsiveness 5 5 5 4 5 4.83 4.67 4.84 4.83 4.74 4.89 4.91
Knowledge 5 5 4.75 5 5 4.64 4.57 4.74 4.67 4.67 4.71 4.55
Friendliness 3 5 5 3 4 4.79 4.7 4.8 4.89 4.7 4.84 4.91
olark.com
Data Driven Decisions IRLAcquireConvert.com -Giles Adam Thomas of AcquireConvert:
Use the same language your customers use to increase conversionsCopy transcripts to a spreadsheetEvaluate for commonly used words and or phrasesInsert those phrases into your landing page and CTA copy"We saw a 176.33% conversion rate increase through this copy change alone."
SOURCE: https://blog.olark.com/how-to-use-transcripts-to-increase-conversion-rates-and-profits
olark.com
Data Driven Decisions IRLDisruptive Advertising -Jacob Baadsgaard of Disruptive Advertising:
Tested four 'unexpected' greeters on the homepage for their business, a PPC ad agency:“What’s your best marriage advice?”“If your AdWords account was an animal, what would it be?”“Who’s your #1 competitor?”“What are you struggling with right now?”
Winning test was _____________Also tested on on a client who sells promo products for lanyards:
“How many lanyards are you looking for?”“Hi! I’m Brad! Let me know if you need any help!”“What event do you need the lanyards for?”
The Greeter that won by a landslide was “How many lanyards are you looking for?”
Using the Olark Greeter in clever ways on clients' landing pages increased conversion rates by as much as 37 percent!
SOURCE: https://blog.olark.com/how-clever-greeters-increase-conversion-rates
olark.com
Want to learn more?1. Pandas, scipy, scikit-learn (python libraries) are all well documented2. reddit.com/r/datasets/ is a fun place to find datasets to teach yourself things3. UMcoursera course starting soon: www.coursera.org/learn/python-data-analysis4. Books:
a. https://www.amazon.com/Python-Machine-Learning-Sebastian-Raschka/dp/1783555130/
b. https://www.amazon.com/Data-Science-Business-Data-Analytic-Thinking/dp/1449361323/
olark.com
One last thought on data people...
1) Domain ExpertYou need someone with domain expertise on you data team. The particular domain obviously depends your data teams role. Here, it's mostly company health and customer behavior. If your data team is focused more on your product (e.g. fitness monitor company has data team to do something interesting with your heartrate time series) then people you hire should have some exposure to that or a similar domain and exhibit the ability to learn about it independently.
2) ResearcherDesign experiments, execute them, make sure the results are statistically valid. Compromise between business goals and timelines and confidence (not something you have to do in academic science). Doubt everything.
3) Computer ScientistMaking computers do stuff. Know how to optimize queries, use efficient computation patterns. Maintain production code, design and or use complex algorithms. If you're team does a lot of ML, this expertise can run very deep (not the case here). A person should be able to read about how an algorithm works, understand how to use it, it's limitations, red flags, etc. It's a bad sign if someone just williy-nilly applies techniques without checking that the premise of the technique holds, and assumes all results are valid.
4) System AdministratorI manage our data warehouse. Many tasks are similar to that of a database administrator. I need to make sure our data is secure, the databases are optimized for the types of queries we're doing, they're up, the data is correct, design the schema, etc. Security is pretty easy, since it's not customer facing and there are relatively few users. This may not be the case at larger companies. Eng ops also helps and hand holds this a bit, same with keeping the database up and running. Verifying and cleaning the data is awful and time consuming. "data munging" is the thing that every data person spends more time doing than they would like.
SOURCE: https://www.aapor.org/AAPOR_Main/media/Task-Force-Reports/BigDataTaskForceReport_FINAL_2_12_15_b.pdf
