Spoken Dialogue Systems and the Spoken Dialogue Systems and the GALAXY ArchitectureGALAXY Architecture
29 October 200029 October 2000
Advanced Technology Laboratories
1 Federal Street • A&E Building 2W • Camden, New Jersey 08102
Jerry FrankeJerry FrankeSenior Member, Engineering StaffSenior Member, Engineering [email protected] [email protected] 856.338.3341
September 99
Talk OutlineTalk Outline
• Spoken Language DevelopmentSpoken Language Development
• GALAXY II SystemGALAXY II System
• SUMMIT (speech recognition)SUMMIT (speech recognition)
• TINA (natural language parsing)TINA (natural language parsing)
• GENESIS (natural language generation)GENESIS (natural language generation)
• Turn ManagementTurn Management
• GALAXY II DomainsGALAXY II Domains
• Overview/DemonstrationOverview/Demonstration
September 99
Spoken Language DevelopmentSpoken Language Development
• UniversitiesUniversities
– MIT, CMU, ColoradoMIT, CMU, Colorado
• Basic research labsBasic research labs
– ATT, SRIATT, SRI
• Software developersSoftware developers
– Nuance, SpeechWorksNuance, SpeechWorks
• Domain developersDomain developers
– Lockheed Martin - ATLLockheed Martin - ATL
September 99
GALAXY II SystemGALAXY II System
• Developed by MIT Spoken Language Systems groupDeveloped by MIT Spoken Language Systems group
• Multiple servers performing parts of the dialogue processMultiple servers performing parts of the dialogue process
SpeechRecognition
LanguageUnderstanding
TurnManagement
LanguageGeneration
SpeechSynthesis
Sound
Sound
SUMMIT TINA GENESIS
September 99
SUMMIT (speech recognition)SUMMIT (speech recognition)
• Three elements: vocabulary, language models, acoustic modelsThree elements: vocabulary, language models, acoustic models
• Pause words stripped outPause words stripped out
• AM: segment-based models and boundary-based diphone AM: segment-based models and boundary-based diphone modelsmodels
• LM: Forward Viterbi search with a class bigram model, followed LM: Forward Viterbi search with a class bigram model, followed by a backward A* search with a class trigram modelby a backward A* search with a class trigram model
• Produces N-best list or word graph of possible utterancesProduces N-best list or word graph of possible utterances
• Models trained on domain corpusModels trained on domain corpus
• Models achieve speaker-independenceModels achieve speaker-independence
September 99
TINA (natural language parsing)TINA (natural language parsing)
• Selects from N-best list depending on grammatical parseSelects from N-best list depending on grammatical parse
• Grammars reflect both syntactic and semantic structureGrammars reflect both syntactic and semantic structure
• Result is a semantic frameResult is a semantic frame
• Example:Example:
““Where is the library in Swain Hall?”Where is the library in Swain Hall?”
Clause: LOCATEClause: LOCATE
Topic: PUBLIC-BUILDINGTopic: PUBLIC-BUILDING
Quantifier: DEFQuantifier: DEF
Name: libraryName: library
Predicate: INPredicate: IN
Topic: HALLTopic: HALL
Name: SwainName: Swain
September 99
GENESIS (natural language generation)GENESIS (natural language generation)
• Processes semantic framesProcesses semantic frames
• Embeds semantic frame components into context-dependent Embeds semantic frame components into context-dependent message templatesmessage templates
• Two types of output:Two types of output:
– natural language messagesnatural language messages– messages are sent to some speech synthesis modulemessages are sent to some speech synthesis module
– possibility of output in multiple languagespossibility of output in multiple languages
– keyword-value pairskeyword-value pairs– useful structure for the turn management backenduseful structure for the turn management backend
• Can be used to map between (translate) languagesCan be used to map between (translate) languages
September 99
Turn ManagementTurn Management
• Manages the system’s part of the dialogueManages the system’s part of the dialogue
• Fuses current utterance with dialogue history for full contextFuses current utterance with dialogue history for full context
• Five main tasks:Five main tasks:
– Answer user’s requests (information retrieval)Answer user’s requests (information retrieval)
– Initiate sub-dialogues to clarify the user’s requestInitiate sub-dialogues to clarify the user’s request
– Track progress through the dialogueTrack progress through the dialogue
– Control response to the userControl response to the user
– Provide assistance in using the system when neededProvide assistance in using the system when needed
September 99
GALAXY II DomainsGALAXY II Domains
From MIT:From MIT:
– Jupiter - weather forecastsJupiter - weather forecasts
– Pegasus - airline schedulingPegasus - airline scheduling
– Voyager - Cambridge, Massachusetts city guideVoyager - Cambridge, Massachusetts city guide
– Dinex - Boston restaurant guideDinex - Boston restaurant guide
– Wheels - automobile classified adsWheels - automobile classified ads
– Mercury - airline flight bookingMercury - airline flight booking
From Lockheed Martin - ATL:From Lockheed Martin - ATL:
– DARPA Communicator - airline flight, hotel, car rental bookingDARPA Communicator - airline flight, hotel, car rental booking
– DARPA LCS-Marine/Marine Small Unit Logistics - supply DARPA LCS-Marine/Marine Small Unit Logistics - supply requestrequest
– DARPA LCS-Army - data collection during equipment testsDARPA LCS-Army - data collection during equipment tests
September 99
Overview/DemonstrationOverview/Demonstration
• Booking airline flights (round trip and one way)Booking airline flights (round trip and one way)
• Hotel, car rentalHotel, car rental
• Uses user profile to fill in some information about reservationUses user profile to fill in some information about reservation
• Uses some real-world knowledgeUses some real-world knowledge
• Information retrieval via mobile agentsInformation retrieval via mobile agents
September 99
Try It YourselfTry It Yourself
• CMU 1-412-268-1084CMU 1-412-268-1084
• MIT 1-877-527-8255MIT 1-877-527-8255
• Colorado http://communicator.colorado.edu/Colorado http://communicator.colorado.edu/
Top Related