Post on 26-Jun-2015
description
www.le.ac.uk
Dr Ruth Page, Professor Jeremy LevesleyUniversity of Leicester
rep22@le.ac.uk, @ruthtweetpagejl1@le.ac.uk
Measuring customer care talk in Twitter
Overview
• Linguistic methods for quantitative analysis– Semantic Differential– Corpus linguistics– Discourse Analysis
• Why might this be useful?– Identifying distinctive patterns in communication– Customer care training
• How metrics can be turned into indices
Data sets
Page data• Data – 177,735 tweets• 100 publically available
accounts– 40 companies– 30 celebrities– 30 ‘ordinary’ accounts
• Gathered in 2010 and 2012– Hashtags (Page 2012)– Apologies (Page 2014)
Precise data• BT Care
– 4014 tweets– 69,976 words
• HSBC UK Help– 3882 tweets– 78,375 words
Methods: Scraping
• Data capture (Page)– Bespoke python code that worked with the
Twitter API to scrape all public posts from named accounts
– Automatically sorted tweets• Updates• Addressed messages (starting with @username)• Retweets
• Converted files to plain text
Methods: Sampling• travel:
– @bluejet, @luxorlv, @southwestair, @british_airways, @londonmidland, @connectbyhertz, @carnivalcruise
• entertainment: – @directv, @marvel, @travelchannel, @tvguide
• food: – @sainsburys, @waitrose, @tastidlite, @popeyeschicken, @starbucks,
@dunkindonuts, @wholefoods, @uktesco, @dunkindonuts• technology:
– @emccorp, @itunesmusic, @dellcares, @costcomcares• finance:
– @hoover, @hrblock, @zappos, @wachovia, @intuit• sport:
– @chargers, @chicagobulls• retail:
– @selfridges, @americanapparel, @karenmillen, @reiss, @marksandspencer, @rubbermaid, @johnlewisretail.
Frequency distribution
Frequency
Cumulative frequency
End-to-begin cumulative frequency
Frequency distribution
Semantic differential
• Semantic differential consists in three value:– Evaluation (Good – Bad)– Activity (Active – Passive)– Potency (Strong – Weak)
• For each word from the 1,000 most frequently used words these three values are measured.
• Each value belongs to interval [-4.6, 4.6]
Visualization
• Right figure depicts the distribution of tweets in the space of the first three principal components calculated for the first 150 words
• We can see dense cone and small cluster outside the cone
Visualization
• We calculate two subsets of the first 150 words which realize an 80% covering of tweets.
• We calculate three subsets of the first 150 words which realize a 70% covering of tweets.
Visualization
• Right figures depict the distribution of tweets in the space of the first three principal components calculated for the first 80% covering
• We can see three clusters
layered structure
• Right figures depict the distribution of tweets in the space of the first three principal components calculated for the first 70% covering
• We can see Layered structure
Visualization
• Right figures depict the distribution of tweets in the space of the first three principal components calculated for the second 70% covering
• We can see layered structure
Question 1
• What kinds of messages do different groups of Twitter members post to their accounts?
• Methods– Quantifying the number of each type of post
INSIGHT: Distribution of tweet types (2010)All groups favoured updates, with celebrities most of allTwitter is an environment for ‘broadcasting’ one-to-many messages‘Conversational’ one-to-one messages were less
INSIGHT: Distribution of tweet types (2012)Corporate tweeting behaviour changed and becomes more ‘conversational’What’s distinctive about the corporate addressed messages?
Using Corpus-based methods
• Corpus – a definition – Collection of representative texts– Machine readable form
• Concordancing tools– Antconc (Laurence 2014) - Freeware– Wordsmith Tools, Wmatrix – Proprietary
• Search and sort lexical strings• Compare with other corpora
Corpus linguistics: Basic steps
Start by examining….1.Frequency of words2.Keywords in Context (KWIC)3.Collocations
Clusters of words that repeatedly occur together
Is the frequency pattern specific to this dataset?
Keyness list• ‘Keyness’
– Statistical over-use of words– I compared the corporate
addressed messages with all tweets in my dataset
• INSIGHT:– The items in the keyness list
cluster together and are typically found in apologies
HSBC, BT Care and ‘Page’ data
• INSIGHT• HSBC UK Help and BT
Care apologise even more than the companies in my dataset!
Why are apologies so important?
• Twitter is a public environment where customers can complain
• Damage to the company’s reputation
• Apologies need to rebuild reputation and re-establish rapport between company and customer
Example: KWIC list for HSBC’s sorry
Methods: Discourse Analysis
• Manually extracted all examples of apologies from data (c.1200 egs)
• Coded manually in Excel• Features identified by
other researchers interested in apologies
• Other communicative features
• Formulae which indicate the apology• Problem restated in the apology• Explanation or account• Offer of repair• Greeting• Naming• Additional questions or instructions• Emoticons and conversational
features (discourse markers)
Do companies repeat the problem or not?Companies tend to avoid repeating the problem in their apology.This enables them to preserve their reputation, but it can appear impersonal.BT Care typically uses ‘vague language’ to avoid restating the problemHSBC UK Help typically restates the problem
Do companies explain why the problem occurred?Companies do not often explain why a problem occurred. But when they do, it typically downplays their role in the offence that prompted the complaint.The effect can be to mitigate damage to reputation
Do companies make an ‘offer of repair’?Offering recompense to the customer can be a way to rebuild reputation and re-establish rapport with the customer.It’s not always appropriate though, and depends on the company in question.
Personalising the message
• Signatures– 37% of apologies by
companies– 0 of apologies by
ordinary accounts– 100% by HSBC Help UK– 0 by BT Care
• Name of customer– 19% of apologies by
companies– 11% of apologies by
ordinary accounts– 69% by HSCB Help UK– 0 by BT Care
‘Rapport’ talk: Greetings and Emoticons HSBC BT Care
Hi 2399 2815
Hello 230 14
Good afternoon 344 0
Good evening 306 0
Good morning 357 14
3636 2844
INSIGHTThe style of an apology can be more or less formal. More conversational features like greetings and signals of emotional response like emoticons can be used to project rapport with the customer.
In my data, 19% of companies and none of the ordinary accounts used greetings. Six percent of the companies and 25% of the ordinary accounts used emoticons.
HSBC uses ‘rapport’ features more often than BT Care (figures given per million words)
HSBC BT Care
:) 2118 343
:( 77 29
;-) 89 0
:-) 115 572399 429
Does the customer need to respond again?Companies may not be able to respond completely to the complaint in Twitter. They may need more information or ask a third party to respond.This is risky as it means that the communication chain can break down leading to greater customer dissatisfaction.
Comparison of questions on the HSBC Help and BT Care accounts
HSBC UK Help• 31% of all company tweets
contained a punctuated question
• 15% of the questions checked if the customer had been in touch
• 10% asked if further help was needed
BT Care• 44% of all company tweets
contained a punctuated question
• No questions asked if the customer had been in touch or needed further help
Summary of BT Care and HSBCHSBC UK Help
• Risk reputation by restating the problem
• Build rapport by– Personalising their tweets by
always signing off and frequently use the customer’s name
– Use greetings and emoticons
• Use follow up questions to check customers’ needs
BT Care• Protect reputation by rarely
restating the problem and use explanations to defer blame
• Limited rapport– Rarely if ever sign off and never
use the customer’s names– Rarely use conversational
features
• Never use follow up questions to close the apology
Application for clients
• What makes an apology good PR?– Different factors– Not all linguistic (e.g. timeliness)– Different people will value different aspects
• Successful customer care is not mechanistic• But analysis can identify areas of need and
then training can be developed to improve practice