Post on 14-Jan-2016
description
Automatic Spoken Document Processing for Retrieval and Browsing
Zahra Ahmadi
Outline
• Motivation • Typical speech retrieval system• Works done• Dealing with OOV • Improvements
2
Motivation
• Ever-increasing computing power and connectivity bandwidth, together with falling storage costs resulting in an overwhelming amount of data of various types
• Information search and retrieval is a key application area • Less attention to speech search
• As data availability increases, lack of adequate technology for processing spoken documents becomes the limiting factor to large-scale access to spoken content
• Automatic approaches for indexing and searching spoken document collections are very desirable
3
Typical Speech Retrieval System
• Two primary processing stage: ▫ Offline process of audio content to generate index▫ Query searches via interface and system’s retrieval based
on indexes • ASR is the core component of speech retrieval system
4
SDR Challenges
• Primary difficulties due to limitations of ASR technology:▫ Highly spontaneous, unprepared speech▫ Topic-specific or person-specific vocabulary & language
usage▫ Unknown content and topics potentially lacking support in
general language model▫ Wide variety of accents and speaking styles▫ OOVs in queries▫ Infrequent query terms, which are most useful for retrieval
5
Prominent Approaches
• Many of prominent research efforts: SDR-TREC in 1999-2000
• Significant recent contributions on wide variety of speech sources: ▫ SpeechBot: audio from public web sites▫ SCANMail: voice mail▫ Oral history interviews▫ SpeechFind: National Gallery of the Spoken Word (NGSW)
consisting of speeches, news broadcasts, and recordings that are of significant historical content
6
TREC-SDR: A Success Story
• About 550 hours of broadcast news • Segmented manually into 21,574 stories of 250 words on
average• Evaluation of ASR systems tuned to broadcast news domain:
15-20% WER• Preexisting approximate manual transcriptions had WER of
14.5% for video and 7.5% for radio broadcasts • Accuracy evaluation: by human assessors search queries
• Retrieval performance was flat with respect to ASR WER (1-best) variations in the range of 15-30% (robust to recognition errors)
• No severe degradation in retrieval performance when evaluating with ASR outputs in comparison with approximate manual transcriptions
7
TREC-SDR Robustness Results
8
Shortcomings of TREC-SDR
• Speech recognizers tuned heavily for domain:▫ Lead to very good ASR performance▫ Unrealistic to expect 10–15% WER especially when
decoding speech is mismatched to the training data▫ Common to observe WER of 30-50%
• Very low OOV rates: ▫ Typically below 1%▫ Query-side OOV (Q-OOV) was very low as well
By Q-OOV rate close to 15%, severe degradation in MAP performance (50% relative, from 44 to 22) occurs
9
Dealing with OOV Query Words
• Most common: represent both query and spoken document using subword units:▫ Linguistically:
Phone: completely solves OOV problem, low performance Syllable: stable acoustically, poor language model Morpheme: hard to distinguish acoustically Stem-ending: acceptable OOV, distinguishable segment
(agglutinative lang.)
▫ Data driven: Multigram: non-overlapping, variable-length, phone
subsequences with some predefined maximum length Particle: found greedy to max. leave-one-out likelihood of
bigram LM Morph: based on minimum description length
10
Dealing with OOV Query Words (cont)
• Advocates tighter integration of ASR and IR:▫ Index phone n-grams appearing in ASR N-best lists▫ Focused on broadcast news thus benefiting from good ASR
performance
• Combination of word and subword level indexing: ▫ word-level indexing and querying is still more accurate▫ abundance of word-spotting false-positives in subword
retrieval▫ somewhat masked by the MAP measure
11
Effects of Using Different Methods
12
Dealing with OOV Query Words (cont)
• Building inverted index from ASR lattice:▫ Storing full connectivity information in lattice ▫ Retrieval is performed by looking up strings of units▫ Allows for exact calculation of n-gram expected counts but
more general proximity information is hard to calculate
• Query expansion: ▫ Expand to similar in-vocabulary phrases▫ Phone confusion matrix: acoustic confusion between words▫ Stemming ▫ Semantic similarity
Use of more than just one-best information (N-best lists or lattices) significantly improves retrieval accuracy
13
Long Spoken Communications
• Important to locate the relevant portion• Achievable by segmenting documents into topics and
locating topics • Spoken Utterance Retrieval (SUR): where segments are
short or when consist of short utterances• SUR goal is to find all utterances containing query• Applications: browsing broadcast news, telephone
conversations, teleconferences, and lectures• NIST STD 2006 Evaluation :
▫ Locating exact occurrence of query in large heterogeneous speech archives
▫ Notable technique with significant improvements: setting detection thresholds in a term-specific fashion to maximize ATWV metric
14
Spoken Document Understanding & Organization
Keywords in spoken document to
understand subject matters
Automatically extracting key information of
events in segmented short
paragraphsAutomatically
segmented into short
paragraphs with some
central concept
Automatically generating
summery for each
segmented short
paragraphs
Automatically generating title for each
short paragraph
Automatically analyzing subject
topics of segmented paragraphs,
clustering with topic labels, organizing
hierarchical presentation
15
Finally…
• Use of audio content and text metadata jointly can improve retrieval performance
• Conjunction of subword and word-based methods improves performance
• Need to universal ASR which controls variance in WER across narrow domains as SDR poses new challenges for the core ASR
• Cross-Language SDR: Assumes queries and target spoken documents are not in the same language ▫ Bilingual performance was lower than English monolingual run▫ However, the degree of degraded performance was shown to
depend on the translation resources used.▫ Extension of TREC collections by manually translating short
topics into five European languages: Dutch, French, German, Italian, Spanish
16
17
References
• C. Chelba, T.J. Hazen, M. Saraclar. “Retrieval and Browsing of Spoken Content”. IEEE Signal Processing Magazine, May 2008.
• L. Lee, B. Chen. “Spoken Document Understanding and Organization”. IEEE Signal Processing Magazine, September 2005.
• J. Garofolo, G. Auzanne, and E. Voorhees. “The TREC Spoken Document Retrieval Track: A Success Story”. Proc. Recherche d’Informations Assiste par Ordinateur: Content Based Multimedia Information Access Conf., 2000.
• L. Begeja, D. Gibbon, et. Al. “A System for Searching and Browsing Spoken Communications”. 2004.
• S. Parlak, M. saraclar. “Spoken Term Detection for Turkish Broadcast NEWS”. ICASSP 2008.
• N. Bertoldi, M. Federico. “Cross-Language Spoken Document Retrieval on the TREC SDR Collection”. Springer, pp.476-481, 2003.
• C. Chelba, T.J. Hazen. “Automatic Spoken Document Processing for Retrieval and Browsing”. Tutorial slides, NAACL 2006.
• …
18