Application of Topic Segmentation in Audiovisual Information Retrieval
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video...
-
Upload
maria-eskevich -
Category
Technology
-
view
129 -
download
2
description
Transcript of Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video...
![Page 1: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/1.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Comparing Retrieval Effectiveness ofAlternative Content Segmentation Methods
for Internet Video Search
Maria Eskevich1, Gareth J.F. Jones1, Martha Larson2
Christian Wartena2,3
Robin Aly4, Thijs Verschoor4, Roeland Ordelman4
1 Centre for Digital Video Processing, Centre for Next Generation LocalisationSchool of Computing, Dublin City University, Dublin, Ireland
2 Delft University of Technology, Delft, The Netherlands3 Univ. of Applied Sciences and Arts Hannover
4 University of Twente, The Netherlands
![Page 2: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/2.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Outline
I MediaEval 2011 Rich Speech Retrieval TaskI 3 participant groups methodsI Results and examplesI ConclusionI Future Work: Brave New Task at MediaEval 2012
![Page 3: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/3.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
![Page 4: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/4.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
![Page 5: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/5.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
![Page 6: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/6.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1
6=Meaning 1 6=
Transcript 2
Meaning 2
Conventional retrieval
![Page 7: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/7.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1
6=
Meaning 1
6=
Transcript 2Meaning 2
Conventional retrieval
![Page 8: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/8.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1 6=Meaning 1 6=
Transcript 2Meaning 2
Conventional retrieval
![Page 9: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/9.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1 6=Meaning 1 6=
Transcript 2Meaning 2
Conventional retrieval
![Page 10: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/10.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1 =Meaning 1 6=
Speech act 1 6=
Transcript 2Meaning 2
Speech act 2
Extended speech retrieval (find jump-in points)
![Page 11: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/11.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1 =Meaning 1 6=Speech act 1 6=
Transcript 2Meaning 2Speech act 2
Extended speech retrieval (find jump-in points)
![Page 12: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/12.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Task Goal:I Information to be found - combination of required
audio and visual content, and speaker’s intention
Transcript 1 =Meaning 1 6=Speech act 1 6=
Transcript 2Meaning 2Speech act 2
Extended speech retrieval (find jump-in points)
![Page 13: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/13.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 14: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/14.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:
I Videos from Internet video sharing platform blip.tv(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)
I Automatic Speech Recognition (ASR) transcript providedby LIMSI and Vocapia Research
I Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 15: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/15.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)
I Automatic Speech Recognition (ASR) transcript providedby LIMSI and Vocapia Research
I Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 16: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/16.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia Research
I Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 17: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/17.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploader
I 50 user-generated short web style queries collected viacrowdsourcing, associated with following speech act types:
I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 18: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/18.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:
I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 19: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/19.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 20: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/20.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:
I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 21: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/21.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech act
I Accurate transcript of the labeled speech act
![Page 22: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/22.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2011Rich Speech Retrieval (RSR) Task
I Data provided to task participants:I Videos from Internet video sharing platform blip.tv
(ME10WWW dataset: testset: 1727 episodes, ca. 300 hs)I Automatic Speech Recognition (ASR) transcript provided
by LIMSI and Vocapia ResearchI Metadata manually added by the uploaderI 50 user-generated short web style queries collected via
crowdsourcing, associated with following speech act types:I ’expressives’: apology (1), opinion (21)I ’assertives’: definition (17)I ’directives’: warning (6)I ’commissives’: promise (5)
I Data available for results assessment:I Time of the relevant item for the labeled speech actI Accurate transcript of the labeled speech act
![Page 23: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/23.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Evaluation Metrics
![Page 24: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/24.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Evaluation Metrics
![Page 25: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/25.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Evaluation Metrics
![Page 26: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/26.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Evaluation Metrics
![Page 27: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/27.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Evaluation Metrics
I Mean Reciprocal Rank (MRR):
RR =1
RANKI Mean Generalized Average Precision (mGAP):
GAP =1
RANK. PENALTY
![Page 28: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/28.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Evaluation Metrics
I Mean Reciprocal Rank (MRR):
RR =1
RANKI Mean Generalized Average Precision (mGAP):
GAP =1
RANK. PENALTY
![Page 29: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/29.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the words
I Segmentation with sliding window:
![Page 30: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/30.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
![Page 31: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/31.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
![Page 32: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/32.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
![Page 33: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/33.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
I Retrieval using BM25, BM25F - for use of metadataI Post-processing:
![Page 34: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/34.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
I Retrieval using BM25, BM25F - for use of metadata
I Post-processing:
![Page 35: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/35.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
I Retrieval using BM25, BM25F - for use of metadataI Post-processing:
![Page 36: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/36.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
I Retrieval using BM25, BM25F - for use of metadataI Post-processing:
![Page 37: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/37.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
I Retrieval using BM25, BM25F - for use of metadataI Post-processing:
![Page 38: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/38.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 1: Sliding Window (SW)
I Tag and lemmatize the wordsI Segmentation with sliding window:
I Retrieval using BM25, BM25F - for use of metadataI Post-processing:
![Page 39: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/39.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 2: Speech Segments (Sp)
I Segmentation based on silence points and changes ofspeakers
I Search engine used: PFTijah
![Page 40: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/40.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 2: Speech Segments (Sp)
I Segmentation based on silence points and changes ofspeakers
I Search engine used: PFTijah
![Page 41: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/41.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 2: Speech Segments (Sp)
I Segmentation based on silence points and changes ofspeakers
I Search engine used: PFTijah
![Page 42: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/42.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 3: Lexical cohesion (LC)
I Segmentation:I into lexically coherent segments, using 2 algorithms:
C99 and TextTilingI additional segment boundaries for silences > 0.5 sec
I SMART IR system with language modeling
![Page 43: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/43.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Approach 3: Lexical cohesion (LC)
I Segmentation:I into lexically coherent segments, using 2 algorithms:
C99 and TextTilingI additional segment boundaries for silences > 0.5 sec
I SMART IR system with language modeling
![Page 44: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/44.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
RSR Results: MRR and mGAP
RunName WindowSize60 30 10
MRR mGAP MRR mGAP MRR mGAPSW asr sh 0.37 0.32 0.32 0.27 0.19 0.19
SW asr meta sh 0.35 0.29 0.30 0.25 0.14 0.14SW meta 0.20 0.15 0.18 0.13 0.06 0.06
Sp asr 0.34 0.27 0.27 0.22 0.16 0.16Sp asr meta 0.34 0.25 0.26 0.21 0.15 0.15
Sp meta 0.18 0.14 0.14 0.11 0.07 0.07LC asr tt 0.36 0.25 0.29 0.18 0.09 0.09
LC asr meta tt 0.39 0.28 0.30 0.20 0.14 0.14LC meta 0.18 0.11 0.09 0.07 0.03 0.03
![Page 45: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/45.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
RSR Results: MRR and mGAP
RunName WindowSize60 30 10
MRR mGAP MRR mGAP MRR mGAPSW asr sh 0.37 0.32 0.32 0.27 0.19 0.19
SW asr meta sh 0.35 0.29 0.30 0.25 0.14 0.14SW meta 0.20 0.15 0.18 0.13 0.06 0.06
Sp asr 0.34 0.27 0.27 0.22 0.16 0.16Sp asr meta 0.34 0.25 0.26 0.21 0.15 0.15
Sp meta 0.18 0.14 0.14 0.11 0.07 0.07LC asr tt 0.36 0.25 0.29 0.18 0.09 0.09
LC asr meta tt 0.39 0.28 0.30 0.20 0.14 0.14LC meta 0.18 0.11 0.09 0.07 0.03 0.03
![Page 46: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/46.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation Methods
Segment:I 100 % Recall of the relevant contentI High Precision (30, 56 %) of the relevant contentI Topic consistency
![Page 47: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/47.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation Methods
![Page 48: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/48.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation Methods
![Page 49: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/49.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation MethodsExample 1
![Page 50: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/50.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation MethodsExample 1
![Page 51: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/51.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation MethodsExample 1
![Page 52: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/52.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation MethodsExample 1
![Page 53: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/53.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Relationship BetweenRetrieval Effectiveness and Segmentation MethodsExample 2
![Page 54: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/54.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Overlap of query wordswith ASR transcript or Metadata. Example 3
Segments on the same topicare retrieved in top of the list;
Use of metadata for segmentscontaining relevant content:
I decrease the rank, ifsegment > 1 topic
I does not affect the rank, ifsegment = 1 topic
![Page 55: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/55.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Overlap of query wordswith ASR transcript or Metadata. Example 3
Segments on the same topicare retrieved in top of the list;
Use of metadata for segmentscontaining relevant content:
I decrease the rank, ifsegment > 1 topic
I does not affect the rank, ifsegment = 1 topic
![Page 56: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/56.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Overlap of query wordswith ASR transcript or Metadata. Example 3
Segments on the same topicare retrieved in top of the list;
Use of metadata for segmentscontaining relevant content:
I decrease the rank, ifsegment > 1 topic
I does not affect the rank, ifsegment = 1 topic
![Page 57: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/57.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Overlap of query wordswith ASR transcript or Metadata. Example 4
Segment:I High Recall (83, 100 %)I Precision = 23 %I Several topics covered− > Use of metadata increase the rank (Rank = 1)
![Page 58: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/58.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Overlap of query wordswith ASR transcript or Metadata. Example 4
Segment:I High Recall (83, 100 %)I Precision = 23 %I Several topics covered
− > Use of metadata increase the rank (Rank = 1)
![Page 59: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/59.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Overlap of query wordswith ASR transcript or Metadata. Example 4
Segment:I High Recall (83, 100 %)I Precision = 23 %I Several topics covered− > Use of metadata increase the rank (Rank = 1)
![Page 60: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/60.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Conclusions and Future Work
I Segmentation plays significant role in retrieving relevantcontent
I High recall and precision of the relevant content within thesegment lead to good segment ranking.
I Related metadata is useful to improve ranking of thesegment with high recall and non relevant content.
− > Current exploration of segmentation methods to generatesegments that have high recall and precision of the relevantcontent for the query
I Due to small size of the query set no general conclusionson the difference based on the speech act type
![Page 61: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/61.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Conclusions and Future Work
I Segmentation plays significant role in retrieving relevantcontent
I High recall and precision of the relevant content within thesegment lead to good segment ranking.
I Related metadata is useful to improve ranking of thesegment with high recall and non relevant content.
− > Current exploration of segmentation methods to generatesegments that have high recall and precision of the relevantcontent for the query
I Due to small size of the query set no general conclusionson the difference based on the speech act type
![Page 62: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/62.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Conclusions and Future Work
I Segmentation plays significant role in retrieving relevantcontent
I High recall and precision of the relevant content within thesegment lead to good segment ranking.
I Related metadata is useful to improve ranking of thesegment with high recall and non relevant content.
− > Current exploration of segmentation methods to generatesegments that have high recall and precision of the relevantcontent for the query
I Due to small size of the query set no general conclusionson the difference based on the speech act type
![Page 63: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/63.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Conclusions and Future Work
I Segmentation plays significant role in retrieving relevantcontent
I High recall and precision of the relevant content within thesegment lead to good segment ranking.
I Related metadata is useful to improve ranking of thesegment with high recall and non relevant content.
− > Current exploration of segmentation methods to generatesegments that have high recall and precision of the relevantcontent for the query
I Due to small size of the query set no general conclusionson the difference based on the speech act type
![Page 64: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/64.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
Conclusions and Future Work
I Segmentation plays significant role in retrieving relevantcontent
I High recall and precision of the relevant content within thesegment lead to good segment ranking.
I Related metadata is useful to improve ranking of thesegment with high recall and non relevant content.
− > Current exploration of segmentation methods to generatesegments that have high recall and precision of the relevantcontent for the query
I Due to small size of the query set no general conclusionson the difference based on the speech act type
![Page 65: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/65.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2012 Brave New Task:Search and Hyperlinking
I Use Scenario:I A user is searching for a known segment in a video
collection.I Furthermore, because the information in the segment might
not be sufficient for his information need, s/he wants tohave links to other related video segments, which may helpto satisfy information need related to this video.
I Sub-tasks:I Search: finding suitable video segments based on a short
natural language query,I Linking: defining links to other relevant video segments in
the collection.
![Page 66: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/66.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2012 Brave New Task:Search and Hyperlinking
I Use Scenario:I A user is searching for a known segment in a video
collection.I Furthermore, because the information in the segment might
not be sufficient for his information need, s/he wants tohave links to other related video segments, which may helpto satisfy information need related to this video.
I Sub-tasks:
I Search: finding suitable video segments based on a shortnatural language query,
I Linking: defining links to other relevant video segments inthe collection.
![Page 67: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/67.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2012 Brave New Task:Search and Hyperlinking
I Use Scenario:I A user is searching for a known segment in a video
collection.I Furthermore, because the information in the segment might
not be sufficient for his information need, s/he wants tohave links to other related video segments, which may helpto satisfy information need related to this video.
I Sub-tasks:I Search: finding suitable video segments based on a short
natural language query,
I Linking: defining links to other relevant video segments inthe collection.
![Page 68: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/68.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2012 Brave New Task:Search and Hyperlinking
I Use Scenario:I A user is searching for a known segment in a video
collection.I Furthermore, because the information in the segment might
not be sufficient for his information need, s/he wants tohave links to other related video segments, which may helpto satisfy information need related to this video.
I Sub-tasks:I Search: finding suitable video segments based on a short
natural language query,I Linking: defining links to other relevant video segments in
the collection.
![Page 69: Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search](https://reader036.fdocuments.us/reader036/viewer/2022062514/5591a8961a28ab384e8b45b5/html5/thumbnails/69.jpg)
Comparing Retrieval Effectiveness of Alternative Content Segmentation Methods for Internet Video Search
ediaEval 2012
Thank you for your attention!
Welcome to MediaEval 2012! http://multimediaeval.org