David van Leeuwen, Stephan Raaijmakers, Wessel Kraaij
AMI community of interest meeting
Automatic segmentation of meeting recordings
Automatic segmentation of meeting recordings2
TNO is active in five core areas
Facts & Figures:- Annual turnover: 553 Mio euro- Employees: 5100
Automatic segmentation of meeting recordings3
A unique Dutch ICT innovation centre
About TNO ICT• Established: 1 January 2003 • Bundling of former KPN Research with TNO’s ICT
related departments• One of the largest ICT knowledge centres in Europe
Features and unique selling points• Independent • Frontrunner• Multidisciplinary:
• Conceptual and hands-on• Technical, economical and sociological• In-depth Telecom and IT expertise
Key figures• Annual turnover: EUR 40 Mio• 375 professionals• 10 high-quality patents per year• Locations in Delft, Groningen and Enschede
Automatic segmentation of meeting recordings4
Indexing meeting recordings
• Characteristics of meetings:• Lack of structure• Low information density• Rich in non-verbal cues
• Challenge:• identify segments• annotate segments
• TNO focus:• robust features• multi-level segmentation• low tech requirements
Automatic segmentation of meeting recordings5
Application scenario
• Enabling step: building a browsable meeting recording archive:• multimodal analysis:• video analysis (e.g. motion zones), • speech analysis (e.g. diarization, laughter detection) • transcript analysis (e.g. topic segmentation, summarization,
sentiment analysis).
• Usage Scenario: searching/filtering interesting segments, • As soon as meeting segments have been detected and annotated,
they can be exploited for any a search or summary generation application.
• E.g. all positive comments on the company’s new flagship product expressed by marketing consultant “Joe. ”.
Automatic segmentation of meeting recordings6
TNO proposition
• TNO seeks participation of COI members for a mini-project for the application of multi-level segmentation and annotation of meeting (or lecture) recordings.
• A feasibility study of the application of TNO technology for CoI member product line.
• Technologies ready for evaluation in a mini-project:• Speaker diarization: who spoke when (practical when
speakerphones or central microphones are used)• Topic segmentation: would like to perform a test on an
archive with real data.• Sentiment classifcation: state of the art labeling performance,
would like to perform a test on meeting data• Motion zone classification: finding hot spots
Automatic segmentation of meeting recordings7
Speaker Diarization
• Answers the question Who spoke When?
• No prior information from participants required• no training necessary• but absolute identity therefore not resolved
• we can use speaker recognition technology for this
• Useful for finding out• who talks most• who interacts with whom• who says important things (using transcript)• …
Automatic segmentation of meeting recordings8
TNO solution for Speaker Diarization
• Technology• requires unobtrusive distant microphones• uses acoustic properties of the voice• uses direction of signal
• if multiple microphones are available
• Performance• is evaluated in NIST Rich Transcription benchmark
evaluations• is among the best performing teams• good co-operation with these teams
Automatic segmentation of meeting recordings9
Hot spot segmentation
• Motion: pixels different from background estimation• Motion is measured in zones• Per person 3 zones:
• head• hands• close-up camera
• Motion gives indication about speaker activity
• Gesture activity• Head movement• Cue for ‘hot spots’
Automatic segmentation of meeting recordings10
Feature browser: motion zones
Hot spot?
Automatic segmentation of meeting recordings11
Subjectivity in meeting transcripts
• Focal point of TNO ICT: sentiment analysis in texts• Determine if texts (like movie reviews) are positive, negative
or neutral (global sentiment classification; see paper)• Determine local sentiment (phrase level)• Find subjective and objective statements
• For AMI: apply subjectivity detection to speech transcripts• Align subjectivity with hot spot information and speaker
segmentation
• Integrated browser for multimodal sentiment cues:• Motion (gestures)• Speaker information• Subjectivity information
Automatic segmentation of meeting recordings12
Topic segmentation
• Automated division of texts (like meeting transcripts) into separate topics• Main topics• Fine-grained subtopic structure• A Machine Learning problem: learn on basis of segmented texts
• AMI data is hard• Low interannotator agreement• Highly technical and overlapping vocabulary
• TNO: two approaches• An approach based on Conditional Random Fields, using sequential
(contextual) information, optimized for standard error metrics• An SVM-based approach, optimized for a new and better error
metric• Both approaches significantly outperform the baseline LCSEG
algorithm, a well-known and quite good algorithm
Automatic segmentation of meeting recordings13
Meeting transcript (ground truth)
• ==========:1• so um the thing we have to know is you already know what we're going to do , you also read
what this the things or , not yet , okay . so um , yeah , it has to be original , trendy , user-friendly that's what we're going to design . uh first we have uh uh three steps of uh making the the remote control . fir the first thing is th the functional design , that's very important . we have to look what the needs are , the effects of the functional design , and and how the mm the the remote control works , so that's where we're going to look in the functional design , it's for the f next meeting .
• ==========:2• yes . • the the second thing is the conceptual design , that's what it that's uh the spe the
specifications of the components and the properties and the specifications of the user interface . and we have to look what uh the market is doing for what kind of uh remote controls are in the market . and the third thing is uh the detailed design um and that's exa yeah , you know what it is , it's exactly how it looks and whatever . okay so uh no , this is a these are two smartboards , with the uh f uh s an introduction of that one .
• and you already saw you know all that that you here can put uh things in the the red project
uh map . folder , okay .
Top Related