Advanced Speech and Language Technology for Complex...
Transcript of Advanced Speech and Language Technology for Complex...
Advanced Speech and Language Technology for Complex
Customer Care Automation and Self-Service
Roberto PieracciniChief Technology Officer
SpeechCycle, Inc.26 Broadway, 11th Floor
New York, NY [email protected]
What is SpeechCycle?
SpeechCycle is the leading provider of 3rd Generation speech applications for digital service providers.
Started in 2001; located in NYC.Rapidly growing software company (about 70 people)Telephone based automatic spoken dialog systems for complex customer care.Deployment models: on-demand and on-premise managed service.Processing millions of complex support calls every month for the largest cable and telecommunication operators in the US and Australia.Full automation rates up to 40%Experts in speech recognition, speech science, software engineering, advanced voice interaction design, and the strategic value of speech systems in the contact center.
A brief history of commercial spoken dialog systems
GENERATIONFIRST SECOND THIRD
Time Period 1994-2001 2000-2005 2004-todayType of
Application Informational Transactional Problem Solving
ExamplesPackage Tracking, Flight Status
Banking, Stock Trading, Train Reservation
Customer Care, Technical Support, Help Desk.
Architecture Proprietary Static VoiceXML Dynamic VoiceXMLComplexity
(Number of DMs) 10 100 1000
Interaction Turns A couple 5-10 More than 10
Dialog Modality directed
directed + natural language (SLU) + mixed initiative
directed + natural language (SLU) + intelligent mixed initiative
LangTech 2008 4February 29, 2008
The Problem
• Cost of customer care is on a steep increasing curve; this is especially true in the Digital Service Provider arena– Devices and services are becoming more and more complex—more
things can go wrong– Number of customer grows year by year– …and so the number of agents needed to provide quality support– Outsourcing and offshoring reached a point of diminishing returns
• Quality of customer care is on a decreasing curve– Long waiting queues– New products and services offered every so often– Agents are not always up to date– Turnover makes agent training difficult and costly– Difficult to maintain consistent quality of service– Infrastructures are not always up to date
LangTech 2008 5February 29, 2008
…been there
The switchboards were something to behold, with many, many operators sitting in long rows plugging countless plugs into countless jacks. The cost of adding new subscribers had risen to the point unforeseen in the earlier days, and that cost was continuing to rise, not in a direct, but in a geometric ratio. One large city general manager wrote that he could see the day coming soon when he would go broke merely by adding a few more subscribers.
AT&T, Early 1900s
LangTech 2008 6February 29, 2008
Customer Care AutomationValue Proposition
• Value for the customer (the provider)– Reduced costs
• Value for the final user (the subscriber)– No waiting in a queue– Consistent quality of customer care
• Value for the technology vendor (i.e. us)– Revenue - Profit
LangTech 2008 7February 29, 2008
Technical Support Automation is Difficult
• Acquisition of knowledge• Keeping up to date with new products and services• Emotional state of callers• Problem identification• Caller mental model• Instructing non technical savvy callers• Uncontrolled events• Challenging acoustic environment• Cultural barriers against speaking to machines• Callers do not trust automated systems can help them• Callers are not cooperative
LangTech 2008 8February 29, 2008
SpeechCycle’s Approach to Automated Technical Support
• Speech recognition over the telephone with sophisticated Voice User Interface
• Detect call reason using advanced natural language technology• Ask simple questions when needed• Don’t ask questions...if possible. Integrate with provider’s
customer information systems• Instruct caller to follow simple diagnostic and troubleshooting
steps if needed.• Perform diagnostic and troubleshooting steps automatically if
possible. Integrate with diagnostic systems. • Automated reporting and call performance classification• Use log data to monitor and continuously improve automation and
caller experience– Speech– Language– Logic
LangTech 2008 9February 29, 2008
The Doctor Analogy
• Know your patients even before they talk– Medical records
• Let your patients talk– What’s your problem?
• TAKE THE INITIATIVE• Then ask simple clarification questions
– Does it hurt when you laugh?• Take measurements, run tests• Prescribe cure• If it does not work:
– Try something else• If it does not work:
– Send to a specialist with an updated medical record.
LangTech 2008 10February 29, 2008
The Doctor Analogy applied toAutomated Technical Support
• Know your callers even before they talk– Integration with Customer Account DB
• Let callers talk– Problem identification with natural language
• TAKE THE INITIATIVE• Then ask simple clarification questions
– Directed dialog• Take measurements, run tests
– Integration with diagnostic tools• Prescribe cure
– Step by step resolution• Problem solved?
– How did we do?• If it did not work:
– Try something else• If it did not work:
– Escalate to human agent
LangTech 2008 11February 29, 2008
The long term vision for automated technical support
Please describe your problemMy internet connection is slowSit back and relax. I will fix it for you and call you back when I am done!
SPECH ONLY
FULLY INTEGRATED
Customer account information
Home device informationDiagnostic tools
Network state
IMMERSIVECALLER
EXPERIENCE
LangTech 2008 12February 29, 2008 Confidential
The Continuous Improvement Cycle
Call-Flow Design
Call-Flow TestingPrompts and Grammars Management
Real-time VUI optimization and learning
Reporting
Call Data Analysis
Natural Language grammar development, tuning and testing
Speech Recognition Performance analysis
Dynamic Content Refresh
DATA
LangTech 2008 13February 29, 2008
Handling complexity
CanYouSeeModem
yes
no
WaitGetBrand continue ExpertPrompt
~UserOption.help~InteractionError
MistookRouter
true
ExpertPrompt
nobrand
NeedMakeModel
~exit
~exit
CallerConfirmedCabledataModem=true
TopEnd_CompareModemVars
TopEnd_MakeModel
success
CustomerAccount_Success
SupportedModem
SupportedOhMyModem
client
cox
!!C/C/C!!
Unsupported
Rpt_OutOfScopeExit=UnsupportedModem
HandleTransferOkay
~exit
TopEnd_VerifyModem
CustomerHasMultipleModems
true
ModemHasBatteryBackup
true
ModemHasBatteryBackup
true
TopEnd_NoBrandModem
WaitUnplugModem
ModemProximity
continue ~UserOption.help
yes
no WaitIdentify
~UserOption.help
~InteractionError
continue ~UserOption.help
WaitUnplugModem_deux
continue
StillNeedHelpIdentifyingno
yes
ProximityBad
~exit
yes
no
noModemProximity_deux
CantReachOrFindPowerCord_deux
otherModemUnplugProblem=true
other
~UserOption~InteractionError
Thanks
ProximityBad Escalate ~exit
reach
find
CanYouReachModemPowerCord
yes
WaitUnplugModem_deux
continueno
yes
Rpt_WantAgent=true
~InteractionError
NeedHelpIdentifying_2
HelpUnplugModemNMUnplugModem
MistookRouter
true
WaitUnplugModem_2
~InteractionError
~UserOption.help continue
OtherProblemCapture_2
TransferQueue=normal
HandleTransferOkay
ThxAndMoreInfoBlurb
ThxAndMoreInfoBlurb
ThxAndMoreInfoBlurb
HelpUnplugModem
CycleModemRecently_1
yes
no
~InteractionError
WaitUnplugModemAndCoax
continue
FirstAction=CameBackPrompt
~UserOption.operator
ThxAndMoreInfoBlurb_1
CallerUnpluggedCoax=true
~InteractionError
WaitGetPenForVoIPSteps
AskIfShouldReadVoIPStepsWithoutPen
~UserOption.help
Good
continue
yes
no
VoIPStep1of4
~InteractionError
AskIfShouldRepeatAllVoIPSteps
~UserOption.helpno
~InteractionError
VoIPStep2of4
~InteractionError.NoInput
VoIPStep3of4
~InteractionError.NoInput
VoIPStep4of4
~InteractionError.NoInput
yes
~UserOption.repeat
VoIPStepsEpilogue
VoIPStepsRepeatPreamble
NB: For CIP4.0 compatibility, in VoIPStepNofN activities, moved RepeatPrompts definitions to HelpPrompts; in all questions, changed RepeatThat grammar use to “help”; removed RepeatPrompts (5158.wav) from
AskIfShouldReadVoIPStepsWithoutPen, and (5164.wav) from AskIfShouldRepeatAllVoIPSteps. We should undo these changes when CIP4.1 is ubiquitous.
HasRouter
true
~InteractionError.NoInput
heardVoIPStepsheardVoIPSteps
true true
~InteractionError.NoInput
VoIPStepsPreamble_router
VoIPStep2of4_router
VoIPStepsPreamble
VoIPStep1of4
VoIPStep3of4
~InteractionError.NoInput
VoIPStep4of4
~InteractionError.NoInput~InteractionError.NoInputabort
operator
Motorola
Linksys
Toshiba
yesno
no
yes
yes
no
no
yes
yes
no
no
yes
yes
no
yes
no
no
yes
yes
yesno
ConfirmZoom noyes
ConfirmNetgear noyes
ConfirmCom21
ConfirmWebstar
ConfirmDLink
ConfirmAmbit
ConfirmBestdata
ConfirmCybersurfer
ConfirmToshiba
ConfirmTerayon
ConfirmRCA
ConfirmMotorola
ConfirmLinksys
NeedMakeModel
NeedMakeModel
NeedMakeModel
NeedMakeModel
NeedMakeModel
NeedMakeModel
NeedMakeModel
ConfirmCMXno
yes
netgear
linksys
motorola
rca
NeedMakeModelModemGroupNameBest dataBestdata
Zoom
CMX
DLink
Scientific Atlanta
Com 21
3Com
Netgear
Ambit
Cybersurfer
RCARCA NEW
rca
Terayonno
IsCMX_1 yes
no
modemtype=cmx LeftThree
modemtype=3com BottomTwo
ConfirmArris yes
noArris
modemtype
linksys netgear
WaitLinksysHaveCableLight
~UserOption.help
DoesLinksysHaveCableLight
no
continue
MistookRouter=true
continue
ThanksForChecking
WaitNetgearHaveStatusLight
continue ~UserOption.help
DoesNetgearHaveStatusLightyes
no
Mistook Router!
modemtype
netgear
LinksysTooNetgearToo
UnplugThatModem
JustAMoment
ComputerCount=multiple
lightsAgainVisit=net
MistookRouter=true
ComputerCount=multiple
modemtype
linksys
lightsAgainVisit=net
continue
yes
RouterBrand=linksys RouterBrand=netgear
If caller mistook router, we identify modem properly, then
power it down.
WaitGoFindModem_3 WaitGoFindModem_4
CallerConfirmedCabledataModemtrue
dlink
WaitDLinkHaveStatusLight
continue ~UserOption.help
DoesDLinkHaveStatusLight
yes
no
RouterBrand=dlink
WaitGoFindModem_5
continue
MistookRouter=true
ComputerCount=multiple
modemtype
dlink
lightsAgainVisit=net
DLinkToo
TopEnd_ModemDown
TopEnd_GetModem TopEnd_GetModem TopEnd_GetModem
- Hundreds of pages of call-flow (300-400 for typical call flow)
- Thousands of nodes
- Several thousands of prompts
- Hundreds of grammars
Call Flow as Software
Use software engineering principles to create and maintain
large call-flows
- Hierarchical structure- Reuse, libraries- Object oriented principles- Configurability- Source control
Call Flow as Software
Use software engineering principles to create and maintain
large call-flows
- Hierarchical structure- Reuse, libraries- Object oriented principles- Configurability- Source control
LangTech 2008 14February 29, 2008
Symptom Identification with Statistical Spoken Language Modeling (SSLU)
SSLU toolsInternet Symptoms
0.00%2.00%4.00%6.00%8.00%
10.00%12.00%14.00%
inte
rnet
slow
secu
rity
oper
ator
intern
etse
tup
inte
rnet
cant
conn
ect
vide
o
mod
emlogo
n
inte
rnet
appo
intm
ent
hom
enet
wor
k
emailset
upinac
cess
iblesite
s
serv
ice
cable
ques
tion
billin
gvo
ip
emailcan
tsen
dorrec
ipad
dres
sinte
rnet
inte
rmitten
t
emailser
vice
chan
geac
coun
t
softw
are
multip
lesy
mto
ms
pare
ntalco
ntro
l
cras
hfro
zenc
ompu
tye
s
help
emailspa
mpo
pups
canc
elfirew
all
repe
at
emailw
ebm
ail
37 Unique Symptoms225,000 utterances+ 3,000 utterances/week
37 Unique Symptoms225,000 utterances+ 3,000 utterances/week
Video Symptoms
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
14.00%
16.00%
18.00%
Serv
iceN
oSer
vice
Cha
nnelMissing
BoxN
otW
orking
Pictur
eNoP
icture
Intern
et
Pictur
eSno
wyF
uzzy
Line
Prob
lem
Pictur
ePoo
rQua
lity
Ope
rator
Rem
oteN
otW
orking
Appo
intm
ent
Pictur
eMultip
lePicS
ymptom
HDTV
Pictur
eOther
Cha
nnelOther
BoxO
ther
DVR
Ord
erPa
yPer
View
Other
Serv
iceO
ther
BoxM
oxi
Rem
oteV
ague
Soun
dNoS
ound
BoxW
ontT
urnO
n
Ord
erOnD
eman
dOther
TVNoC
ableTV
Guide
NotW
orking
Pictur
eBlack
Rem
oteO
ther
Cha
nnelCan
tCha
nge
TVOther
Rem
oteP
rogr
amMyR
emote
Acco
untB
illing
Serv
iceM
ultip
leOut
Serv
iceV
ague
Acco
untS
ervice
Cha
nge
Ord
erOther
BoxV
ague
Soun
dPoo
rQua
lity
Serv
iceO
utag
e
PINPa
sswor
dPro
blem
Ord
erOnD
eman
dVag
ue
Acco
untO
ther
Soun
dVag
ue
Soun
dOther
Pictur
eVag
ue
Pictur
ePixelated
Pictur
eFro
zen
Phon
e
Ord
erVa
gue
Aske
dToC
all
TVW
ontT
urnO
n
Cha
nnelVa
gue
Pictur
eOther
Color
s
Acco
untV
ague
Ord
erPa
yPer
View
Vagu
e
Ord
erPI
NPa
sswor
d
Ord
erOnD
eman
dError
Erro
rOther
Clock
BoxM
ultip
leBo
xSym
ptom
s
74 Unique Symptoms210,000 utterances+ 2,000 utterances/week
74 Unique Symptoms210,000 utterances+ 2,000 utterances/week
LangTech 2008 15February 29, 2008
Contender: Exploring Alternative Strategies
Cumulative average automation rate
14.00%
15.00%
16.00%
17.00%
18.00%
19.00%
20.00%
21.00%
10/26 10/27 10/28 10/29 10/30 10/31 11/1 11/2 11/3 11/4 11/5 11/6 11/7
WITH EXAMPLESOFFER CHOICESORIGINAL
0
5000
10000
15000
20000
25000
10/26 10/27 10/28 10/29 10/30 10/31 11/1 11/2 11/3 11/4 11/5 11/6 11/7
Number of calls
LangTech 2008 16February 29, 2008
Conclusions
• The third generation of dialog systems is here• Problem Solving: not just form filling or transactions• The doctor analogy• The power of backend integration and data • Call-flow as software• Symptom Identification and SSLU• Exploration and Learning