Text analytics summit - West 2014

SELLING TEXT ANALYTICS TO YOUR BOSS

Nov 2014

Intro – speaker, the talk & quick intro of the audience.

Quick summary of challenges with Text Analytics

Role play (what mistakes are done today?)

The elevator pitch

Proposed tactical approach

References for reading materials

CONTENTS

Intended for Knowledge Sharing only 2

INTRO


About me:

"As Director, Analytics for the Digital Developed Markets department at Visa Inc., I am responsible for helping the Leadership & Stakeholders with actionable insights derived from Analytics. The business questions span the whole spectrum across the Product, Marketing, Sales and Relationship. We leverage any of the various options, i.e., Strategic analysis, Advanced Analytics, Text Analytics or Mining depending on the problem being solved."

The talk:

Content will range start with why does Text Analytics need a special session on convincing boss, followed by a role play summarizing current mistakes, a sample elevator pitch for your boss and a proposed execution plan. The content is tailored for Mid to Senior Level Managers trying to convince Leaders/Executives/Heads. It doesn’t provide any technical details –methodologies, tools, vendors or hardware investments.

*Disclaimer: Participation in this summit is purely on personal basis and not representing VISA in any form or matter. The talk is based on learnings from work across industries and firms. Care has been taken to ensure no proprietary or work related info of any of my the firms I have worked with is used in any materials.




The elevator pitch



CONTENTS


TOUGHEST ELEVATOR PITCH TODAY


Don't you know I'm human too?

WHY DO THEY SAY NO?


PERCEPTION ISSUES – too niche, too specific problem solving.

COMPLEX & CHALLENGING – variety of data sources, structure, time consuming

RoI UNCLEAR

LACK OF TRAINED RESOURCES

MY CURRENT SUITE OF ANALYTICS IS ENOUGH

ARE ALL THOSE STATEMENTS TRUE?


PERCEPTION ISSUES – too niche, too specific problem solving.

COMPLEX & CHALLENGING – variety of data sources, structure, time consuming

RoI UNCLEAR

LACK OF TRAINED RESOURCES

MY CURRENT SUITE OF ANALYTICS IS ENOUGH

MYTH

TRUE

TRUE

TRUE

FALSE




The elevator pitch



CONTENTS



How is it done today, more or less

https://www.youtube.com/watch?v=BUsiI47ol6g

https://www.youtube.com/watch?v=BUsiI47ol6g




The elevator pitch



CONTENTS


NECESSARY COMPONENTS IN AN ELEVATOR PITCH


30 sec summary of the idea, customer & value proposition

What is it? How does it help, aka, what will s/he get out of it? How it works? Target Customers What do you need? Why this and nothing else (USP & SWOT Analysis) How will you grow this?

Get your boss hooked,

Once hooked, the detailed explanation…

THE ELEVATOR PITCH


Text Analytics is a “Mind Reading” Tool. Can tell us how Consumers perceive our products and what their needs are. It complements the other Analytical

techniques to answer the why questions.

We will use it to help Brad (a Product Manager) prioritize Password reset experience by showing him the magnitude of bad user reactions. It will be a

quick survey followed by Text Analytics.

We have identified an $1M+ Opportunity in Password reset experience via Text Analytics.

WHAT IS IT AND HOW IT HELPS?


Text mining/Analytics is the process of deriving high-quality information from text with the goal of turning text into data for analysis, via application of natural language processing (NLP) and analytical methods.

Source: http://en.wikipedia.org/wiki/Text_mininghttp://en.wikipedia.org/wiki/Text_mining#Text_mining_and_text_analytics

Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output.

e.g. frequency distributions, pattern recognition, tagging/annotation, information extraction, data mining techniques including link and association analysis, visualization, and predictive analytics.

http://en.wikipedia.org/wiki/Text_mining

http://en.wikipedia.org/wiki/Text_mining#Text_mining_and_text_analytics

A TYPICAL EXAMPLE OF FINDINGS OR INSIGHTS…


Overall NPS: 70% (Promoters: 80%; Detractors: 10%)

What are the Promoters happy about?*

60% of the users love the simplicity of use of Mobile App 40% feel the recommendations are relevant 30% like the two level authentication feature

What are the Detractors are unhappy about?*

10% hate the password reset experience 8% feel that password reset link takes too long to reach their email inbox 5% feel that the text updates don’t provide sufficient information

What new features did the users ask for?

Monthly reminders Functionality for the receivers to confirm the payment FX conversion change alerts

*depends on the how the questions were framed in the surveys

A TYPICAL IMPACT SIZING…


KPIs

Monthly unique users 20,000,000

#Page Views per user 40

#Total Page Views from Users 800,000,000

Quantifying the problem

#users trying to reset Password 2,000,000

Users from the survey who hate the current experience (consistent and all detractors

cited this as the reason)100%

Assuming everyone who tries to reset password hates the experience, so users at

risk2,000,000

We would be left with 720,000,000

Conservative Realistic Aggressive

Impact Sizing ($)

By changing the experience, we decrease detractors by

50% 80% 100%

and save these many Page Views 40,000,000 64,000,000 80,000,000

At a CPM of $2 (Cost per 1000 impressions), we improved Ad Revenue by

$80,000 $128,000 $160,000

Annual Revenue impact $960,000 $1,536,000 $1,920,000

Resetting the current Password reset experience can save at least 1 MM annually…

HIGH LEVEL STEPS INVOLVED…


Objective

Translation to Analytical Framework

Data Collection and Preparation

Analysis, Validation & Verification

Actionable insights and impact sizing

A/B Testing

Rollouts

• Understand the core needs, expected outcomes, generate hypotheses and know if it’s an Unstructured or Structured problem

• Consult with User/Market Research team on the applicability of the text input to your problem statement and how/where to get the data from.

• Decide on Analytical methodology, e.g., Word bubble, Link, SVM

• Data Collection: Structured (Surveys, Call Center logs) or Unstructured (Reviews, Social media comments, Blogs, Articles)

• Data Preparation: Crowd sourcing for tagging, Cleaning (Stemming, Stop words, normalization, adjectives & entities, preposition removal, usage patterns, n-grams)

• Data Transformations: TF, IDF, Medians, Max, transformations, Dim reductions, etc.• Marrying with internal data: Activity, Clickstream, Demo or Geo, Product.

• Execution of Analysis on one sample and validation on another. • Verification of insights with other established processes, checking for reliability.

• Generate recommendations and impact sizing (current problem and other areas)

• Champion vs. Challenger testing of the impact.

• Rollouts – New product, feature, changes, etc.

1

2

3

4

5

6

7

Hmmm, what’s this?

HIGH LEVEL STEPS INVOLVED…


Objective

Translation to Analytical Framework

Data Collection and Preparation

Analysis, Validation & Verification

Actionable insights and impact sizing

A/B Testing

Rollouts

1

2

3

4

5

6

7

Analyst & Stakeholder

Analyst, Researcher, Data Instrumentation, & Data Manager,

Developer, Data Scientist

Analyst, Data Manager, Data Scientist

Analyst, Data Scientist, Stakeholder and SME, Researcher

Analyst, Stakeholder, Leader

Analyst, A/B Testing, Stakeholder, Developer

Stakeholder, Leadership & Executives

Yeah, but who does it?

Knowing what is the key need

Right framework, right data, right questions

Most challenging, time consuming, and tricky phase (smarts, patience &

determination needed)

Next most challenging – iterations on methodology/data to get to a acceptable

lift

Realistic sizing and allied applications of insights

Buy-ins

Corporate Strategy shifts

Easy?

POSSIBLE SOLUTIONS SET BY STAKEHOLDERS …


MARKETING

PRODUCT

CUSTOMER SUPPORT

• SOV, Brand awareness and Perception• Sentiment in Media coverage

• Competitive and industry benchmarking

FUNCTIONS USE CASES

STRATEGY

RISK

• Consumer Pain points – Experience, Product stability, use cases, etc.

• Machine logs analysis for user interactions studies

• Informs Analytics of Consumer Sentiment, reactions, expectations, pain-points and helps in tailoring better

actions

• Market and User Research• Competitive and industry benchmarking

• Gaming behavior and hacker communities for new use cases

Others (Legal, HR, Finance)

• Contract verifications, Backgrounds, etc.• Resume parsing, Career Planning, Talent Management, etc.

WHAT DO YOU NEED?


Executive support must.

Stakeholder buy-in and strong advocacy and support.

Clearly defined and communicated objectives mapped to company initiatives

Clearly defined and accepted success criteria for Text Analytics Initiatives

Tools & Technologies:• Non-Coders: KNIME, Rapidminer• Semi-Coders: SAS, Angoss & Polyanalyst• Full on Coders (Advanced Users): R and Python

Type of Talent: Closely tied to business needs. However one advanced programmer (Python, R) and an analyst are needs. 50% time of a Researcher and 20% time of Stakeholder customer needed.

Partnership with User/Market researcher team needed to frame right questions and make right readings from the data.

Partnership with Data Instrumentation and Data Managers to collect, prepare, execute and ramp up this process.

USP & SWOT…


STRENGHTS

• More aha and surprises• Gut>Emotions>Metrics

• Consumers feel respected and connect better when you talk to them

THREAT

• Big data technologies need to plan for ever changing text data & privacy

concerns (Volume, variety, velocity and veracity)

• More devices, more challenges

OPPORTUNITIES

• Cross functional team to standardize the data collection and preparation• Buy-ins on success criteria

• Train the analysts• Evolving NLP techniques, vendors and

market buy-ins

Closest to truth! Insights from metrics based analysis is Analyst’s reasoning and Insights from Text Analytics are Voice of Consumers!

WEAKNESSES

• Cost and complexity of execution*• RoI – various ways to look at it

• Not create and forget (nuances during automation)

• Lack of resources

*Unique data collection changes (low response rates of surveys), internationalization expansions, etc.

HIGH LEVEL THUMBRULES OF WHEN TO DO VS. NOT…


Minimum and reliable sample size (>=5% or 10K+ users are leaving some active or passive feedback somewhere).

Share of Voice, Brand awareness and perceptions have stabilized and product has been live for atleast a year for users to get familiar with it.

High Level analysis reveals correlation between Portfolio KPIs and relational NPS/SOV/call center metrics/Brand metrics.

BI, User Research, Analytics, A/B Testing have all been properly invested and strengthened in that order. Text Analytics and Mining usually go together into helping ramped up business. These two are also dependent on the previously mentioned practices for success and show most value when all others feel or are not able to answer the questions.

100% accuracy isn’t required but a strong and reliable directional insight is the need of the hour.




The elevator pitch



CONTENTS


THE THREE STAGES OF EXECUTION


PICK

PROVE

SELL

• Stakeholder discussions to find out pressing questions

• Prioritize – Requester; urgency; impact; efforts• Choose “highest PR potential” problem for POC

• Create action plan – data collection, cleansing, methodology, timelines, expected outcome template, success criteria.

• SWAT team – Stakeholder rep, Analyst & Dev or Data Scientist

• Check-ins & documentation of what worked and did not, do’s/don’ts, challenges & nuances.

• Insights communication & Impact estimation.• Champion vs. Challenger measurement.

• Highlight victories – underdog story, winning against the odds, challenges faced, etc.

• Ramp plans: hiring, cost, time, areas where it can be used

• Branding – Internal, and if possible, external too, make it ‘cool’ and desirable.

Best to go prepared to

you boss

Boss buy-in would be

really good

Boss & Stakeholders

primary doers. Should go up

to Execs.

STILL NOT CONVINCED?


Not surprising, every change agent has it tough… Just do it (forgiveness is better than permission),

Unsure of ability to pull off or too much to do already? No time or people or tool! freelancers/contractors, vendors, POCs either yourself by learning it or informal partnership with a data scientist or a coder in the company

Cost concerns: innovation labs, done by university students under NDA Run a company innovation contest

Your boss unsure if Stakeholders will like it? Get it done and sell to Stakeholders boss Brownbags or other team meetings

Your boss just doesn’t want to do it: Create a Watsapp! …satisfaction of getting it done and value add is huge enough reward.


REFERENCES


SAMPLE EXECUTIVE SUMMARY


Objective

• Set up Text Analytics infrastructure to transform unstructured consumer inputs into actionable insights

Methodology• POC to analyze the utility for our business and recommend a go/no-go decision

Key Findings• Things worked• That didn’t work• Who are the largest customers

Recommendations• Go/No go – Yes, but with as a cross functional team and clear success criteria• How big an opportunity?


Some interesting applications of Text Mining/Analytics,

Project Dreamcatcher: http://www.slate.com/articles/news_and_politics/victory_lab/2012/01/project_dreamcatcher_how_cutting_edge_text_analytics_can_help_the_obama_campaign_determine_voters_hopes_and_fears_.html

J.K.Rowling caught http://blogs.wsj.com/speakeasy/2013/07/16/the-science-that-uncovered-j-k-rowlings-literary-hocus-pocus/

Movers and Shakers I follow in this space:Seth Grimeshttp://altaplana.com/TAS08-TextAnalyticsForDummies.pdfhttp://www.slideshare.net/SethGrimeshttp://altaplana.com/grimes.html

Junling Huhttp://www.aboutdm.com/

Intended for Knowledge Sharing only 27Intended for Knowledge Sharing only 27

SOME ADDITIONAL STUFF TO READ ABOUT…

*No endorsements or claims of validity/reliability. Used it myself and wanted to share with you.

http://www.slate.com/articles/news_and_politics/victory_lab/2012/01/project_dreamcatcher_how_cutting_edge_text_analytics_can_help_the_obama_campaign_determine_voters_hopes_and_fears_.html

http://blogs.wsj.com/speakeasy/2013/07/16/the-science-that-uncovered-j-k-rowlings-literary-hocus-pocus/

http://altaplana.com/TAS08-TextAnalyticsForDummies.pdf

http://www.slideshare.net/SethGrimes

http://altaplana.com/grimes.html

http://www.aboutdm.com/


Good foundation on Text Mining from Statisticahttp://loyaltysquare.com/text_mining.phphttp://www.statsoft.com/Textbook/Text-Mining/button/3http://www.nltk.org/book/

KNIME use cases (Google search sufficient for Angoss, Rapidminer, SAS whitepapers yields): http://www.meetup.com/Bay-Area-KNIME-Users/events/199339222/comments/418637362/?itemTypeToken=COMMENT&a=uc1_rd&read=1&_af_eid=199339222&_af=event

Text Mining Video tutorials by Rapidminerhttps://www.youtube.com/watch?v=hpvda_Rfg3s&list=PLeE4eVNDyo0VKTTiPg0zlX_OYiWII-sow

Text Mining & Analytics on Courseerahttps://www.coursera.org/specialization/datamining/20https://www.coursera.org/course/textanalytics

On Udemy (Text Mining for Bloggers)https://www.udemy.com/text-mining-for-bloggers/


IF YOU WANT TO LEARN YOURSELF…

*No endorsements or claims of validity/reliability. Used it myself and wanted to share with you.

http://loyaltysquare.com/text_mining.php

http://www.statsoft.com/Textbook/Text-Mining/button/3

http://www.nltk.org/book/

http://www.meetup.com/Bay-Area-KNIME-Users/events/199339222/comments/418637362/?itemTypeToken=COMMENT&a=uc1_rd&read=1&_af_eid=199339222&_af=event

https://www.youtube.com/watch?v=hpvda_Rfg3s&list=PLeE4eVNDyo0VKTTiPg0zlX_OYiWII-sow

https://www.coursera.org/specialization/datamining/20

https://www.coursera.org/course/textanalytics

https://www.udemy.com/text-mining-for-bloggers/


Here is my contact, reach out may be

On Linkedinhttps://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a

*Linkedin has all the other contact details – Email and Phone, etc.

Slidesharehttp://www.slideshare.net/RamkumarRavichandran/

Blog coming soon…


QUESTIONS/COMMENTS

https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a

http://www.slideshare.net/RamkumarRavichandran/

Text analytics summit - West 2014

Data & Analytics

Transcript of Text analytics summit - West 2014