© 2004 Oracle Corporation and Vocalocity, Inc. All Rights Reserved 1 The Role of the Internet in...

© 2004 Oracle Corporation and Vocalocity, Inc. All Rights Reserved 1

The Role of the Internet in the Evolution of Speech and Telephony Applications

Curt TuckeyDirector, Voice Laboratory

Ken RehorChief Architect


A bit of background…


LaptopWeb Server

Database

HTTP RequestHTTP Request

HTTP Response(Device Markup)HTTP Response(Device Markup)

Internet Service

Provider Or Internal

Network

Internet application model (Part I)

InternetInternet


The Model-View-Controller Pattern

• MVC (e.g. MAWL, Bell Labs, 1995)–Data adapters, other actions (e.g. Enterprise

javabeans)–Controller (e.g., java servlet)–View (e.g. JSP and taglib templates)


Make no little plans; they have no magic to stir men's blood and probably will themselves not be realized.

Make big plans; aim high in hope and work, remembering that a noble, logical diagram once recorded will not die.

-- Daniel Burnham


Phone Web – August 1995


Make no little plans; they have no magic to stir men's blood and probably will themselves not be realized.

Make big plans; aim high in hope and work, remembering that a noble, logical diagram once recorded will not die.

-- Daniel Burnham


Landline or

mobile phone

Web Server

Database

Bringing the internet model to voice (Part I)

Service ProviderService Provider

Voice Gateway

Phone Network

PSTN or WirelessPSTN or Wireless

HTTP RequestHTTP Request

HTTP Response(Device Markup)HTTP Response(Device Markup)


Evolution of Architectures for Automated Telephone Services

PSTNEnd User

Internet

EndUser

•Service logic•Transaction server interface

ApplicationServer

VoiceXML GatewayVoiceXML Gateway

•Voice & Telephony functions•ASR, TTS, DTMF•Audio play/record•Telephony interface

•VoiceXML clientVoiceXML clientIVR Platform

•Voice & Telephony functions•ASR, TTS, DTMF•Audio play/record•Telephony interface

•Service logic•Transaction server interface

Service logic

Transactioninterface


VoiceXML Heritage

J. C. Ramming D. Ladd 2/96C. Tuckey 11/98

B. D. LucasL. Boyer

P. Danielsen

K. G. Rehor

J. Ferrans

G. KaramN. Klarlund

PML

VoxML

2000

1995

SpeechML

Bell LabsPML Phone

Web

C. Tuckey 7/00

M. Benedikt

PML

PML


Economic Impact

IT WAS ARGUABLY THE GREATEST misallocation of capital in recent history. In the late 1990s, telecoms firms spent billions building new data networks....

Yet innovation in telecoms has not stopped. And it may be telephone services that help struggling telecoms firms to claw their way out of their slump.

The real impetus behind better voice applications, however, is the recent agreement on an industry-wide standard. Beginning in 1999, and pushed by such leading telecoms firms as AT&T, IBM, Lucent and Motorola, the industry has come up with a lingua franca for voice applications called VoiceXML....

VoiceXML could yet rescue telecoms carriers from their folly in stringing so much optical fibre around the world. It is ironic that it should be old-fashioned voice that lightens the darkness in fibre.

From The Economist, 12/12/2002

Copyright © 2004 Oracle Corporation and Vocalocity, Inc. All Rights Reserved.

VoiceXMLand

Phone Web Architecture

Goal: Leverage Web Architecture

• Languages

• Protocols

• Architecture


Visual vs. Voice markup

Web app components• HTML – Structure

– Layout, input declaration, transitions, etc.

• Images• Text• Scripts

Voice Web app components• VoiceXML – Structure

– Dialog flow, input declaration, transitions, etc.

• Audio files• Text (for TTS)• Scripts


<form>

<field name="main_menu"> <prompt> <audio src="welcome.wav"> Welcome to Acme. You can choose sales, repair, or order status.</audio> </prompt> <grammar src="main_menu.grxml"/> </field>

<help> You can say sales, repair, or order status. </help> <noinput> You must say something. </noinput> <nomatch> I didn't understand you. Please try again. </nomatch>

<block> <submit next="http://acme.com/route... " method="get"/> </block>

</form>

VoiceXML example with error handling

main.vxmlNote: Code simplified for demonstration purposes…


Internet or

Intranet

Web user

HTTP

Application(web) server

• Application logic• Content and data• Transaction processing• Database interface

<html>

Web Application Architecture

• Images• Audio files• Scripts

HTTP


Internet or

Intranet

Phone user

Web user

HTTP

HTTP



<html>

<vxml>

VoiceXMLgateway

PSTN

Phone / Web Application Architecture

• Grammars• Audio files• Scripts

• Images• Audio files• Scripts

HTTP


.wav

<grxml>

Internet

PSTN

Caller

Customer service, please…

HTTP

Webserver

<vxml>

AS

R

TT

SA

udio

DT

MF

Te

lep

ho

ny

VoiceXMLinterpreter

middleware

VoiceXMLgateway

Welcome toAcme products

…

Voice Application Architecture and Components

OA

&M


Internet /Intranet



HTTP

Application Backend Architecture

Database(content)

Transaction Server

Web service

Intranet /Internet

.wav

<grxml>

<vxml>


Key Points

• Architecture leverages all things "internet"– Languages, protocols, servers, developers, etc.

• Separation of concerns– Application logic / database vs. telephony / speech resources– Enables new business models

• Voice ASP

• Prepackaged applications

• URL (application) associated with phone number– Calling party or Called party– Share resources among many applications (VoiceASP)

• High-level languages, specific to domain / task– Simplify development and maintenance

The Voice Ecosystem


Business models

• Web ecosystem• Voice ecosystem• App components, Packaged apps• App development shops

• Voice Hosting / VoiceASP• 'voice web' changed the proprietary IVR market

– Hardware to software evolution

Standards enable the ecosystem


Voice Application Components and Considerations

• Application Server• Business Logic• Content and Content Management• Transaction Server Interface• Networking

• Caching• Load Balancing• Failover / Redundancy• Security• Applications and System Monitoring• Billing• Provisioning

Voice Standards Evolution


Standards: When

“The Apocalypse of the Two Elephants” –(source: Andrew S. Tanenbaum “Computer Networks” / David Clark)


Phone/Voice Markup Language Evolution

• 1995: PML v1– <prompt>, <collect>

• 1996/97: PML v2– "interpreted" HTML, "IVR mode"– macros for localized/specialized control

• 1998: VoxML

• 1998: SpeechML

• 1999/2000: VoiceXML


Voice / Web Standards

• W3C Voice Browser Working Group– VoiceXML 2.0, 2.1, & beyond– Speech Recognition– Speech Synthesis

• W3C Multimodal Interaction Working Group– Multimodal architecture– External/network event mechanism– Natural Language Semantics / EMMA

• VoiceXML Forum– Certification programs: platforms and developers– Education– Marketing

• IETF– Internet protocols– SIP, MRCP


Road to VoiceXML 2.0

VoiceXML 1.0specification

VoiceXML 2.0standard

PML, VoxML, SpeechML

Member submissions& change requests


Other Standards

• Offshoots from VoiceXML: W3C Speech Interface Framework

• GrXML• SSML• SI• EMMA• CCXML• …


Proliferation of VoiceXML


Who uses VoiceXML?

• Wireless carriers– Voice Dialing– Info portals

• Enterprises– Financial Services– CRM / Customer Self-Service– Intranet

• Telecommunication Services– Network prompters– Directory Assistance– Personalized features

• Travel– American Airlines, United Airlines, Orbitz, Song– 511 in many states (California, Utah, Virginia, etc.)– GM / OnStar


Wireless Carriers

Voice dialing, Information portals, Customer service

• AT&T Wireless #121 (Tellme and Comverse)• Cingular (BeVocal)• Verizon Wireless (HeyAnita)• SprintPCS (Nuance)• USCellular (BeVocal)

• Telecom Italia Mobile (iTIM/Loquendo)• Orange (UK)• T-Mobile (US, Germany)


Enterprise Apps

• Financial Services– Merrill Lynch, E*Trade, Schwab

• Customer self-service (internal and external)– Name/Address change (NetByTel, Voxeo)

• Time, Inc. magazines– Mortgage pre-approval (NetByTel, Nuance)

• ABNM AMRO, Countrywide– Medicare claims status and processing

• Empire Blue Cross– Intranet universal access

• SBC "HR Speak"

• Unified messaging, productivity– Oracle Collaboration Suite– Siemens OpenScape


Telecom Apps

• Directory Assistance– AT&T Toll-Free Directory Assistance (1-800-555-1212)

– Telecom Italia ("12" and "1412")

• Findme/Advanced Calling/etc.– AT&T CallVantage– Z-Tel Personal Voice Assistant– Webly

• Customer Care– Bell Canada "Emily/Emilie"


Everywhere

• Europe– UK (Lastminute.com)– Italy (TrenItalia Railway Timetable and Fares) – Germany: T-Mobile InfoTalk– Spain: Vodafone– Portugal (Telisma)– Monaco (Monaco Telecom)– Ericsson "Talking Intranet" (Germany, Austria, Switzerland)– France (Loquendo CRM app for perfume company)

• South Africa– CRM

• Asia– Japan: Voizi– Singapore– Korea– China


What's next?

Evolving standards for different needs

Example: VoiceXML telephony vs. CCXML


Call processing: <transfer>

• Blind– Go somewhere but

don't return

• Bridge– Add on another party,

resume execution when done talking


<form id="xfer">

<block> <prompt> Calling Riley. Please wait. </prompt> </block>

<transfer name="mycall" dest="tel:+1-555-123-4567" >

</transfer>

</form>


• Blind transfer

Note: Code simplified for demonstration purposes…


<form id="xfer"> <block> <prompt> Calling Riley. Please wait. </prompt> </block>

<transfer name="mycall" dest="tel:+1-555-123-4567" bridge="true" >

</transfer></form>



• Bridge transfer



<transfer name="mycall" dest="tel:+1-555-123-4567" bridge="true" > <prompt> Say cancel at any time to disconnect this call.</prompt> <grammar src="cancel.grxml" type="application/srgs+xml"/>

</transfer></form>



• Bridge transfer with cancel feature



<transfer name="mycall" dest="tel:+1-555-123-4567" bridge="true" transferaudio="music.wav" connecttimeout="60s" > <prompt> Say cancel at any time to disconnect this call.</prompt> <grammar src="cancel.grxml" type="application/srgs+xml"/>

<filled> <assign name="mydur" expr="mycall$.duration"/> <if cond="mycall == 'busy'"> <prompt> Riley's line is busy. Try back later. </prompt> <elseif cond="mycall == 'noanswer'"/> <prompt> Riley didn't answer the phone. Please call back another time. </prompt> </if> </filled>

</transfer></form>




CCXML

• Markup language for 3rd party call control• Connection management

– Individual call legs– Conferencing– Event handling

• W3C Last Call Working Drafthttp://www.w3.org/TR/2004/WD-ccxml-20040430/


PSTN

customer

3rd Party Call Control (SIP and RTP)

VoIPGateway

ConnectionControl

[CCXML]

Voicedialog

[VoiceXML]

signalling

audio

signalling

ApplicationLogic

(HTTP)

• Call control application manages connections• When to answer• Where to route ‘call’ or media• Answer and disconnect supervision


PSTN

customer

3rd Party Call Control

VoIPGateway

ConnectionControl

[CCXML]

Voicedialog

[VoiceXML]

signalling

audio

signalling

RoutingApplication

Logic

(HTTP)

• Call control application manages connections• When to answer• Where to route ‘call’ or media• Answer and disconnect supervision

VUIApplication

Logic

(HTTP)

?xml version="1.0" encoding="UTF-8"?><ccxml version="1.0">

 <var name="state0" expr="'init'"/> <eventprocessor statevariable="state0">

 <transition state="'init'" event="connection.ALERTING"> <accept/> </transition>

 <transition state="'init'" event="connection.CONNECTED" name="evt"> <log expr="'Houston, we have liftoff.'"/>

<dialogstart src="'gimme.vxml'"/> <assign name="state0" expr="'dialogActive'" /> </transition>

Call processing using CCXML

<transition state="'init'" event="connection.CONNECTED" name="evt"> <log expr="'Houston, we have liftoff.'"/>

<dialogstart src="'gimme.vxml'"/> <assign name="state0" expr="'dialogActive'" /> </transition>

 <transition state="'dialogActive'" event="dialog.exit" name="evt"> <log expr="'Houston, the dialog returned [' + evt.values.input + ']'" /> <exit /> </transition>

<transition event="connection.DISCONNECTED" name="evt"> <exit/> </transition>

 <transition event="error.*" name="evt"> <log expr="'Houston, we have a problem: (' + evt.reason + ')'"/> <exit/> </transition> </eventhandler> </ccxml>

Distributed Media Processing and Control


Internet

PSTN

HTTP

Webserver

A

udio

DT

MF

Te

lep

ho

ny

VoiceXMLinterpreter

middleware

VoiceXMLgateway

Media Resource Control Protocol (MRCP)

OA

&M

ASR TTS

MRCP

• Lightweight protocol for distributed media processing• IETF draft

Internet

audio video…

© 2004 Oracle Corporation and Vocalocity, Inc. All Rights Reserved 1 The Role of the Internet in...

Documents

Transcript of © 2004 Oracle Corporation and Vocalocity, Inc. All Rights Reserved 1 The Role of the Internet in...