PROPRIETARY AND CONFIDENTIAL, MARITZ COPYRIGHT 2009 1 Integrating Survey and Social Media Research...

PROPRIETARY AND CONFIDENTIAL, MARITZ COPYRIGHT 2009 1

Integrating Survey and Social Media Research

Understanding the Voice of the Customer (VOC) in mobile devices using

Maritz Product Launch Monitor study.

Michael House

Vice President, Text Analytics

Maritz Research

Sentiment Symposium

April 12, 2011


Today’s Session

1. Briefly review how organizations struggle when it comes to “putting the voice of the customer (VOC) to work”

2. Discuss a best practice in VOC integration, a Uniform Set of Customer Experience Categories, that enables comparing and contrasting multiple VOC sources

3. Examine preliminary results from the PLM study

A. Sentiment of Sources

B. A Look at Two Brands: Blackberry and HTC

4. Wrap Up


1The struggle to “Put the Voice of the Customer (VOC) to Work”


Assessing Organizations Success in Using VOC

• Maritz conducted the Maritz Voice of the Customer Practices & Challenges Survey* in 2009 with managers– ‘Blue Chip’ companies

– 131 managers responded

– Multiple industry sectors

* Presented by Dr. D. Randall BrandtVice President, Customer Experience & LoyaltyMaritz Research

American Marketing AssociationMay 12 2009


Where Organizations Still Struggle

• Integrating multiple sources of VOC data to define priorities for improvement

• Demonstrating the link between customer and financial metrics

• Linking the VOC to internal operational and service metrics

• Integrating the VOC and the Voice of Employees

• Clarifying survey-based action items so that their owners know what to do or fix

• Pinpoint practices or business processes that must be improved

• Communicating action plans to address VOC-driven issues

• Getting managers/partners to act on VOC-driven issues


In a Nutshell...

• Managers desire to know the positive and negative aspects of a product or service to drive improvements and marketing activities

• But most organizations are not where they want to be when it comes to integrating and deploying insights drawn from the Voice of the Customer

Integrating

VOC Data

Link VOC toBusiness Results

Analyze

VOC Data

Customer survey metrics

Capture VOC Data


2Ideas for Putting the Voice of the Customer to Work


Our Vision for an Integrated CEM/CGM Offering

• Customer experience sources today can broadly be grouped into:

– Primary customer research and other ‘internal’ sources– Public media, including social media

• Each source has its biases and limitations. None have an exclusive license to the “truth”. The best, most robust picture emerges from the integration of all sources.

• This integration should be designed into a customer experience measurement and management process (Integrated Architecture):

– Built upon a holistic understanding of the business process and how those processes produce measureable business results (Blueprinting and Linkage)

– Supported by a common framework for combining findings across category – a Uniform Set of Customer Experience Categories


Considerations Important in Using VOC Information Sources

Social Media Survey Data

Cannot be sure where the author fits – a customer, an intender, awareness?

We know the respondent: they can be associated with specific locations, employees, products, transactions…

Data is ‘opt in’ and needs to be interpreted in that light.

Sample is ‘representative’ and provides a balanced snapshot.

Market “coverage” is very high. Surveys generally have far fewer observations.

Data is available continuously with dynamic updating of information, charts…

Data tends to be historical.

Documents contents are highly variable.

Documents, or comments, have focused content

Text data can be analyzed using a common structure to identify important insights!


Bottom Line…

Maritz perspective is social media monitoring coupled with survey research can help managers and marketers maximize success and make well-founded, informed decisions.


3

A Look at VOC Sources Using a Uniform Set of Customer Experience Categories


The Smartphone Product Launch Monitor

• Maritz monitored online media and surveyed over 3,000 US consumers in the evolving Smartphone market from November and December 2010

• Maritz and evolve24, a Maritz Research company, used a combination of:– Social media identification (looking for mentions of

interest in or buying Smartphone's on Twitter or Facebook)

– Surveys done among consumer panels• Topics include awareness, consideration, and

product likes/dislikes of Smartphone's among those who recently purchased or plan to in the near future

• Devices included those currently available, as well as those announced, but not yet released

• We developed a list of common categories using social media and our expertise in the market to compare text results


The Survey Open End Questions – Driving Sentiment into the Comments

• “You said that you eliminated [Smartphone] from those that you had been thinking about.” (Model Considered)– “What did you like about it that made you consider it in

the first place?”

– “And what didn’t you like about it that made you eliminate it from your list?”

• “What model did you purchase/would you buy?” (Model Purchased)– “What do you like about it that makes you choose this

model?”

– “Are there things that you don’t like about this model?”

For these comments and social media, we examine the volume and sentiment of responses by categories.


We Also Asked Which Categories were Most Important in the Consideration Phase

Smartphone price Price of voice plan Price of data service plan Wireless coverage quality Phone call sound quality Picture-taking capabilities/camera Dropped Calls Video-taking capabilities Music-playing services and capabilities Video-playing services and capabilities Availability of video games Look and feel of the device The quality of the applications loaded The keypad

Brand name Operating system Ease of use The number of available applications

loaded Technical performance Ability to synchronize Office applications Quality of the available technical support After sales service Battery life The screen resolution GPS capabilities The size of the screen Ability to access e-mail The app store that comes with the device

“Among the list below, please indicate the item you think is most important to your choice of Smartphone.”

We look at the percentage of respondents.


3A A Look at Sentiment…


Comment Sentiment

• For survey questions, focusing on dislikes and likes of Smartphone's yields expected sentiment

• Overall average across the four questions is 0.63– For Model Purchased, the average

is slightly higher at 0.66.

• This approach can be valuable in online, mobile, and mail survey modalities to identify both positive and negative information

Survey Sentiment (0.0 to 1.0)

Likes Dislikes

Model Purchased .94 .37

Model Considered .81 .41

Average Sentiment

% Topics Above

Average

% Topics Below

Average

.55 70% 30%

• For social media, we extract sentiment at the topic level

• Average is moderately lower than survey likes/dislikes average


3B

Brands:BlackberryHTC


Approach

• For the ‘likes/dislikes’ question, we identified the top 5 categories assigned by brand– In all cases, the 5 categories added up to more than

70% of the total number of assignments.

• For the survey quantitative approach, 10 or more categories are needed to cover that number of respondents

• We also look at the top 5 categories by brand for social media


3BBrands:Blackberry


BlackBerry Model Purchased LikesPercentage of Respondents Assigned

37%

22%

19%

15%

14%

The keypad

Ease of use

Price of device

Look & feel of device

Ability to access e-mail

Ca

teg

ori

es

BlackBerry Model Considered LikesPercentage of Respondents Assigned

40%

27%

19%

13%

10%

The keypad

Look & feel ofdevice


Ease of use

Brand name

Ca

teg

ori

es

• 4 out of 5 categories are consistent

• User interface stands out:• Ease of use• Keypad• Look & feel

• E-mail also consistent


BlackBerry Model Considered DislikesPercentage of Respondents Assigned

33%

29%

15%

15%

13%


Price of device

Ease of use

Number available appsloaded

Technical performance

Ca

teg

ori

es

BlackBerry Model Purchased DislikesPercentage of Respondents Assigned

44%

29%

16%

15%

10%


The keypad

The size of the screen

Technicalperformance

Price of device

Ca

teg

ori

es

• Fewer categories are consistent

• The user interface also stands out:• Look & feel• Screen size• Keypad• Ease of use

• Some concern about price and performance


From the Quantitative Perspective,

BlackBerry Most Important Purchase Reason Percentages

10%

9%

9%

9%

8%

7%

6%

6%

5%

5%

5%

Price of Device

Coverage quality

Ease of use

The keypad

Price of data service plan


Ability to synchronize Office apps


Call sound quality

Operating system

Price of voice plan

Cat

ego

ries

• Also see the user interface may be an issue, but doesn’t stand out:• Keypad• Ease of use

• Price shows up more

• E-mail, while not on top, does show up


BlackBerry Social Media Top Five Categories

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Oct Nov-Dec Jan

Time Period

E-mail

Ease of use

Picture-taking capabilities

Battery life

App store with device

BTW: This increase demonstrates the need for monitoring social media to catch escalating topics…

• For Nov/Dec, the survey timeframe, we see ‘ease of use’ is the only user interface category – it may be getting lost in the mix

• E-mail is consistent with what customers ‘like’ about Blackberry• The discussion of other features point to a more ‘functional’ focus


Large number of tweets referring to getting ‘free’ products…


For Blackberry…

• User interface shows up as an important area in what consumers like/dislike in the survey

• UI does show up in the most important consideration, but not a top issue

• In social media, less emphasis on UI and more focus on features – may be related to less ‘hands on’ experience


3BBrands:HTC


HTC Model Considered LikesPercentage of Respondents Assigned

21%

20%

18%

16%

16%

Look & feel ofdevice

Operating system

Price of Device

Technicalperformance

The size of thescreen

Ca

teg

ori

es

HTC Model Purchased LikesPercentage of Respondents Assigned

41%

23%

18%

16%

15%




Price of Device

Picture-takingcapabilities

Ca

teg

ori

es

• 4 of 5 categories consistent

• The user interface shows up:• Look & feel• Screen size

• Performance is important• Technical• OS

• Price of device is also important


HTC Model Considered DislikesPercentage of Respondents Assigned

39%

25%

14%

10%

8%

Price of Device


The keypad

Battery life


Ca

teg

ori

es

HTC Model Purchased DislikesPercentage of Respondents Assigned

33%

33%

17%

14%

5%


Battery life

The keypad

Price of Device


Ca

teg

ori

es

• Very consistent results

• UI is still a concern:• Look & feel• Keypad• Size of screen

• Battery life surfaces

• Price of device is a concern


HTC Most Important Purchase Reason Percentages

12%

12%

10%

9%

8%

8%

7%

5%

5%

3%


Operating system

Price of Device

Coverage quality

Ease of use

Call sound quality

Price of data service plan

Price of voice plan


Dropped Calls

Ca

teg

ori

es

• Performance surfaces as a top issue (technical, OS)• Price does shows up more• UI and battery life do not show up at all


HTC Social Media Top Five Categories

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Oct Nov-Dec Jan

Time Period

Operating system

Picture-taking capabilities

App store with device

Battery life

E-mail

Same issue with HTC as Blackberry…

• UI does not surface• Performance shows up in OS• Battery life is consistent with comments in survey,

but not the checklist


For HTC…

• User interface shows up as an important area in what consumers like/dislike only in the survey comments

• OS may be a core area for HTC – shows up in all three sources

• Battery life surfaces in comments and social media, and may be an area to look into


4 Wrap UP


Summary…

• Uniform Common Categories provides a basis for comparing information from multiple text sources

• Survey data is a balanced approach for discerning topic areas impacting products and services

• Asking likes/dislikes helps drive sentiment into comments to get a balanced view– A consideration for ‘rating websites’?

• Social media measured dynamically is critical for identifying escalating topics and measuring effects of actions taken

• All three sources add value and insight into the ‘picture’ for a given Smartphone


Questions

??

PROPRIETARY AND CONFIDENTIAL, MARITZ COPYRIGHT 2009 1 Integrating Survey and Social Media Research...

Documents

Transcript of PROPRIETARY AND CONFIDENTIAL, MARITZ COPYRIGHT 2009 1 Integrating Survey and Social Media Research...