M. Liu, T. Stanley, J. Baca and J. Picone Intelligent Electronic Systems Center for Advanced...
-
Upload
miles-jones -
Category
Documents
-
view
212 -
download
0
Transcript of M. Liu, T. Stanley, J. Baca and J. Picone Intelligent Electronic Systems Center for Advanced...
M. Liu, T. Stanley, J. Baca and J. PiconeIntelligent Electronic Systems
Center for Advanced Vehicular SystemsMississippi State University
URL:http://www.cavs.msstate.edu/hse/ies/publications/conferences/ieee_secon/2006/hlt_system/
A Robust Architecture for Human Language
Technology Systems
SECON 2006 Page 2 of 14
● Speech recognition and synthesis
● Natural language processing
● Machine translation
Human Language Technology
Overview of Human Language technologies
SECON 2006 Page 3 of 14
Architecture for Human Language Technology Systems
monolithic system distributed systems
Speech recognitionInformation
retrieval
●●
●
decompose into function
components
Development of HLT systems:
● Monolithic fashion
● Distributed system
Natural language understanding
SECON 2006 Page 4 of 14
Galaxy (DARPA) Communicator
• Hub
• Servers
• System initialization
Galaxy Communicator is an open source architecture for constructing dialogue systems.
Its target is to provide the next generation of intelligent conversational interfaces to distributed information. The goal is to support the creation of speech-enabled interfaces that scale gracefully across modalities, from speech-only to interfaces that include graphics, maps, pointing and gesture.
How It Works
SECON 2006 Page 5 of 14
Advantages VS Disadvantages
• Frequent deadlocks in the
communication between
servers
• Need for automated recovery
from server failures
• Lack of a common user
interface for all the applications
• Development time
• Complex inter-process communication
• Plug-and-play approach
• Automating server startup
• The need for robustness to
error and improved debugging
capabilities
• A common interface to allow
users to select among
applications.
SECON 2006 Page 6 of 14
Automated server management
● Multiple Applications
Though the communicator process
monitor provides a good interface
to start and terminate servers, it
requires manual monitoring.
● Process Manager
module
Automatically starts and controls all
server processes in the prototype
system architecture.
SECON 2006 Page 7 of 14
Architectural Enhancements – Process Manager
ProcessManager
Client Side Server Side
Speech Analysis
Hub
Signal Detector
Data RecorderSpeech
Recognition
Hub
Signal DetectorData Recorder
Speech Recognition
SECON 2006 Page 8 of 14
Common Application Interface
A Single Screen Interface
Once the user selects an application,
the Demo Selector loads and
displays the user interface needed
for the specific application.
• Requirement of a common interface
• Demo Selector module
• Process Manager module
Demo Selector interface
SECON 2006 Page 9 of 14
Architectural Enhancements – Robustness Improvements
State Machine architecture
• Servers redesign
• Each state gets a particular
message.
• Trap inter-process communication
errors
Handshaking
• A simple protocol
• Sends a signal
• Waits for an acknowledgement
• Communicator frame
Wait_for_Audio_
Ready state
Data_Transfer
State
End_Of_Utterance
State
Initializationstate
Audio Ready
state
Data_Transfer
State
End_Of_Utterance
State
Initializationstate
Audio_Ready_Ack
state
Audio_Ready
Audio_Ready_
Ack
Data
Data_Ack
End_Points
End_Of_Utterance
End_Of_Utterance
Ack
Speech Analysis Client Signal Detector
SECON 2006 Page 10 of 14
Evaluation
Experiment 1:comparing the results obtained by testing utterances from the extended pilot database on the original and the enhanced architecture.
Three experiments were conducted to measure the
quantitative improvements in the robustness of the system.
SECON 2006 Page 11 of 14
Evaluation
Experiment 2
Scenario sample:(Dialog system Application) Imagine you are in a big city to attend a conference. Once the conference proceedings are over for the day, you want to visit some sites of interest. You don’t have a map with you and have no idea about the layout of the city. Use the system to plan your trip.
Procedure:Five users were asked to engage in 24 usage scenarios using the original and the enhanced architecture. After a 10-minute practice to get familiar with the functionality of the system, the user performed the scenarios. The entire experiment took approximately 1 hour 30 minutes. The user was asked to cease testing if there was a system failure or he/she exceeded the allotted time of 30 minutes.
SECON 2006 Page 12 of 14
Evaluation
* E stands for enhanced architecture; O stands for original architecture.
Results:
SECON 2006 Page 13 of 14
Evaluation
Experiment 3
SECON 2006 Page 14 of 14
Future work
• Further experiments should be conducted to obtain
additional measures of the robust improvements due
to the enhanced architecture.
• Adding other state of the art application to our
existing HLT package.
• Enhance the Process Manager to create and
manage server processes on different host machines
SECON 2006 Page 15 of 14
Reference
• J. Aberdeen, B. George and S. Bayer, “Galaxy Communicator,” SourceForge.net, Open Source Technology Group, VA Software, Fremont, California, December 2005, (http://sourceforge.net/projects/communicator).
• K. Hacioglu and B. Pellom, “A Distributed Architecture for Robust Automatic Speech Recognition,” Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 1234-1234, Hong Kong, April 2003.
• J. Baca, F. Zheng, H. Gao and J. Picone, “Dialog Systems for Automotive Environments,” Proceedings of the European Conference on Speech Communication and Technology (EUROSPEECH), Geneva, Switzerland, pp. 1929‑1932, September 2003.
• Fredrik Olsson. 2002. ``A requirement analysis for an open set of human language technology tasks''. In Proceedings of Workshop on Portability Issues in Human Language Technologies held in conjunction with the Third International Conference on Language Resources and Evaluation (LREC 2002), Las Palmas, Spain, June.
• W. Ward and B. Pellom, “The CU Communicator System,” Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop, Keystone, Colorado, USA, pp. 1234‑1234, December 1999.
• http://www.speechrecognition.philips.com/index.asp?id=506• http://www.virage.com/home/• http://www.cavs.msstate.edu/hse/ies/projects/dialog/