Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese...
-
date post
20-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese...
![Page 1: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/1.jpg)
Supervised by Prof. LYU, Rung Tsong MichaelSupervised by Prof. LYU, Rung Tsong Michael
Department of Computer Science & Engineering
The Chinese University of Hong Kong
Prepared by: Chan Pik Wah, Pat
Ngai Cheuk Han, Table
LYU0102LYU0102
XML for InteroperableXML for Interoperable Digital Video Library Digital Video Library
![Page 2: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/2.jpg)
OutlineOutline
Project OverviewProject Overview Extraction TechniquesExtraction Techniques
Video Optical Character Recognition (VOCR) Video Optical Character Recognition (VOCR) Scene Change DetectionScene Change Detection
StorageStorage XMLXML Knowledge EnrichmentKnowledge Enrichment
ImplementationImplementation Tasks in next semesterTasks in next semester
![Page 3: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/3.jpg)
MotivationsMotivations
Rapid increase in Rapid increase in the usage of the usage of multimedia multimedia informationinformation
New approach: New approach: DIGITAL VIDEO DIGITAL VIDEO LIBRARYLIBRARY
Project Outline
![Page 4: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/4.jpg)
MotivationsMotivations
Little attention paying on video Little attention paying on video information extraction and storageinformation extraction and storage
Scalability of the system in terms of Scalability of the system in terms of adding new extraction componentsadding new extraction components
Lack of a generic framework for Lack of a generic framework for presentation and visualization of presentation and visualization of video informationvideo information
Project Outline
![Page 5: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/5.jpg)
TargetsTargets
Provide an open architecture that Provide an open architecture that can integrate different digital video can integrate different digital video library functionslibrary functions
Increase the reusability of the Increase the reusability of the information extracted from videosinformation extracted from videos
Deliver and present the video to Deliver and present the video to multiple computing platformmultiple computing platform
Project Outline
![Page 6: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/6.jpg)
Ways to achieveWays to achieve
Modal concept of the digital video library Modal concept of the digital video library functions functions
Collaborating the video information Collaborating the video information processing moduleprocessing module
Using XML for storageUsing XML for storage Universal formatUniversal format Flexible, scalableFlexible, scalable Present in different waysPresent in different ways Easy to search based on particular tags Easy to search based on particular tags
Generic framework for presentation and Generic framework for presentation and visualization of video informationvisualization of video information
Project Outline
![Page 7: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/7.jpg)
Overview of our projectOverview of our project
Project Outline
![Page 8: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/8.jpg)
AchievementsAchievements Implement two of the video information Implement two of the video information
extraction techniquesextraction techniques Video Optical Character DetectionVideo Optical Character Detection Scene Changes DetectionScene Changes Detection
Store the extracted information as XMLStore the extracted information as XML Build an XML editor in the tool for editingBuild an XML editor in the tool for editing Do knowledge enrichment base on the Do knowledge enrichment base on the
information extractedinformation extracted
Project Outline
![Page 9: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/9.jpg)
Extraction TechniquesExtraction Techniques
Text Detection
Camera Motion
Face Detection
Scene Changes
WordRelevance
Audio Level
Extraction Techniques
![Page 10: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/10.jpg)
Video OCR for Digital Video OCR for Digital News News
Help to locate topics by extracting the Help to locate topics by extracting the words in the captionswords in the captions
News captions provide vital search News captions provide vital search information of the videoinformation of the video
Video OCR results extracted the keywords Video OCR results extracted the keywords on the frameson the frames
The results can be used together with the The results can be used together with the words extracted from the transcript for words extracted from the transcript for indexingindexing
Extraction Techniques
![Page 11: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/11.jpg)
Video OCR for Digital Video OCR for Digital NewsNews
Extraction Techniques
![Page 12: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/12.jpg)
Scene changeScene change
Detection TechniqueDetection Technique Effective method for segmenting a Effective method for segmenting a
video sequence into significant video sequence into significant componentscomponents
Extraction Techniques
![Page 13: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/13.jpg)
Existing MethodExisting Method
Image difference methodImage difference method Histogram Difference MethodHistogram Difference Method Histogram Difference Method using Histogram Difference Method using
DC Coefficient ImageDC Coefficient Image Our Algorithm & ImplementationOur Algorithm & Implementation Histogram difference method Histogram difference method
with dynamic thresholdwith dynamic threshold
Extraction Techniques
![Page 14: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/14.jpg)
Build and compared the histogram Build and compared the histogram with the pervious scenewith the pervious scene
Calculate the histogram differenceCalculate the histogram difference If (total difference) > thresholdIf (total difference) > threshold
=> scene change=> scene change Use the first frame as key frame Use the first frame as key frame
Our Algorithm & Implementation
Extraction Techniques
![Page 15: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/15.jpg)
XMLXML
Extensible Markup LanguageExtensible Markup Language W3CW3C Create its own mark-up language for Create its own mark-up language for
describing the contentsdescribing the contents
Storage
![Page 16: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/16.jpg)
AdvantagesAdvantages of using XML of using XML
Platform and system independentPlatform and system independent Create your own tag Create your own tag Adopt UnicodeAdopt Unicode Universal formatUniversal format ScalableScalable
Storage
![Page 17: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/17.jpg)
XML schemaXML schema
Storage
![Page 18: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/18.jpg)
XML ParserXML Parser
A parser is an A parser is an interface between interface between an XML document an XML document and the application and the application programprogram
Document Object Document Object Model (DOM)Model (DOM)
Storage
![Page 19: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/19.jpg)
How to present XMLHow to present XML
Tree model becomes Tree model becomes very similar to an very similar to an XML schemaXML schema
Represented as Represented as nodes that show nodes that show element/attribute element/attribute names or the text names or the text content and their content and their relative places relative places within the XMLwithin the XML
Storage
![Page 20: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/20.jpg)
Content creation in Content creation in digital video librarydigital video library
Collaborating different video Collaborating different video information extraction techniques, information extraction techniques, mainlymainly Knowledge Cross-referencingKnowledge Cross-referencing Knowledge EnrichmentKnowledge Enrichment
Access to video by contentAccess to video by content Communicate information trends Communicate information trends
across time, spaceacross time, space Provide fast and effective searchingProvide fast and effective searching
Storage
![Page 21: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/21.jpg)
Knowledge EnrichmentKnowledge Enrichment
Geographic information Geographic information Extract geographic names of countries Extract geographic names of countries
and cities from text recognized from the and cities from text recognized from the video OCR or speech recognition video OCR or speech recognition
Knowledge from geographic naming Knowledge from geographic naming database enrich the informationdatabase enrich the information
Allow query or browse for events at a Allow query or browse for events at a particular location or within some particular location or within some “distance” of that location“distance” of that location
Storage
![Page 22: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/22.jpg)
Our ImplementationOur Implementation
Use a known set of places along with Use a known set of places along with their spatial coordinates and some their spatial coordinates and some additional information for knowledge additional information for knowledge enrichment enrichment
Use the XML file as the source Use the XML file as the source material to be processedmaterial to be processed
Try to extract names of major cities Try to extract names of major cities by processing the text in the sourceby processing the text in the source
Storage
![Page 23: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/23.jpg)
Geographic naming Geographic naming databasedatabase
An XML file with An XML file with the following the following format is usedformat is used
For each city:For each city: City IDCity ID Name of cityName of city Name of countryName of country LongitudeLongitude LatitudeLatitude
Storage
![Page 24: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/24.jpg)
The updated XML fileThe updated XML file
Storage
![Page 25: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/25.jpg)
Knowledge enrichment Knowledge enrichment component in our toolcomponent in our tool
Extract and Extract and list out all the list out all the cities cities mentioned in mentioned in the videothe video
Allow user to Allow user to select any of select any of them to look them to look for further for further information of information of that citythat city
Storage
![Page 26: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/26.jpg)
Program PlatformProgram Platform
Microsoft Visual C++Microsoft Visual C++®® Object-OrientedObject-Oriented Faster MFC applications Faster MFC applications Composite Controls Composite Controls ActiveX ActiveX
Microsoft Microsoft ®® DirectShow DirectShow ®®
Component object Model (COM) Component object Model (COM) High-quality capture and playback of High-quality capture and playback of
multimedia streams multimedia streams
Implementation
![Page 27: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/27.jpg)
Video PlayerVideo Player
Implementation
![Page 28: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/28.jpg)
ControlControl
filter graph managerfilter graph manager Dialog Box create with the class Dialog Box create with the class CFormViewCFormView
Implementation
![Page 29: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/29.jpg)
Scene Change & VOCDScene Change & VOCD
• CScrollView
• CMenu
• Add the extracted information to XML
Implementation
![Page 30: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/30.jpg)
XML EditorXML Editor
TreeViewTreeView XML read by parserXML read by parser Tag in XMLTag in XML Node in TreeNode in Tree
Implementation
![Page 31: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/31.jpg)
Knowledge EnrichmentKnowledge Enrichment
Dialog Box create as class Dialog Box create as class CFormView CFormView Read the databaseRead the database Compare with the XML generatedCompare with the XML generated
Implementation
![Page 32: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/32.jpg)
Problems & SolutionsProblems & Solutions
Implementation
![Page 33: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/33.jpg)
Problems & SolutionsProblems & Solutions Multi-modal tool --- Integrate all the Component Multi-modal tool --- Integrate all the Component Docking Window is used Docking Window is used Flexible & Efficient to add different new modals Flexible & Efficient to add different new modals CSizingControlBarCSizingControlBar
Implementation
![Page 34: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/34.jpg)
Problems & SolutionsProblems & Solutions
Implementation
![Page 35: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/35.jpg)
DemoDemo
![Page 36: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/36.jpg)
Tasks in Next SemesterTasks in Next Semester
Focus on using XML to do Focus on using XML to do multimedia presentationmultimedia presentation
Style Sheet (XSLT) not suitable for Style Sheet (XSLT) not suitable for multimedia document generationmultimedia document generation
New format on multimedia New format on multimedia presentationpresentation
SMILSMIL
Future
![Page 37: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/37.jpg)
Tasks in Next SemesterTasks in Next Semester
Time-based multimedia content Time-based multimedia content Capable to synchronize the playback Capable to synchronize the playback
of all multimedia elements of all multimedia elements Transform XML generated into SMIL Transform XML generated into SMIL
format for presentationformat for presentation Design a style sheet suitable for Design a style sheet suitable for
multimedia document generationmultimedia document generation
Future
![Page 38: Supervised by Prof. LYU, Rung Tsong Michael Department of Computer Science & Engineering The Chinese University of Hong Kong Prepared by: Chan Pik Wah,](https://reader035.fdocuments.us/reader035/viewer/2022062516/56649d445503460f94a216d2/html5/thumbnails/38.jpg)
Q & AQ & A