CMU SCS 15-826: Multimedia Databases and Data Mining Lecture #25: Multimedia indexing C. Faloutsos.
MMBD - Multimedia Databases
-
Upload
rahmivolkan -
Category
Software
-
view
570 -
download
5
Transcript of MMBD - Multimedia Databases
MULTIMEDIA DATABASESAND MPEG7
Rahmi Volkan Başar
Department of Computer Engineering
METU
May, 2013
Multimedia Databases
• Introduction
• Capabilities of DB Types
• Search on MMDB
• Multimedia Content Description
• Research Fields
Multimedia Data
• Text: using a standard language (SGML, HTML)
• Graphics: encoded in CGM, postscript
• Images: bitmap, JPEG, MPEG
• Video: sequenced image data at specified rates
• Audio: recordings in a string of bits in digitized form
Database vs Multimedia Database
• Databases– well structured data organization
– efficient storage of large amounts of data
– querying
– transactional support for concurrent users
– numbers, strings
• Multimedia Databases– large content
– different structures
– not easily searched/queried
Use Cases
• Repositories: central location for data maintained by DBMS, organized in storage levels
• Presentations: delivery of audio and video data, temporarily stored, ‘VCR-like functionality’
• Collaborative: complex design, analyzing data
Capabilities
• Relational Databases
– Atomic / Tables
– Data relation – Common Foreign Keys
– Record: Content – No meta information
– A predefined set of domains for columns
• Hard to extend
• BLOB data type exist
Capabilities
• Object Oriented Databases
– Schema is “Class”
– All data is “Object”
– References
– New data types
• Easy. New class is a new data type.– Appropriate for multimedia data
Capabilities
• Object Relational Databases
– In addition to RDBMS
• Object references
• New types– Multimedia
– MMDBMS
• Extensible ORDBMSs
Search
• Collection of data. How to search?
– Any standards?
– Workarounds?
• Search: Retrieve similar images…
– Fast, Correct
• Content-based
– New techniques?
Search
• Content Based Retrieval Facilities
– Supported by MMDBMS
• Organize and Manage accordingly
– Compare based on a number of features
• Shape/Color/Texture
• Meta-Data?
– Always.
Content Based Retrieval
• Accurate representation of the multimedia objects in the database
– For accuracy and efficiency
– Combination: Different features
• Similarity Search
– High-dimensional feature vectors
• Special multi-dimensional indexing structures
• Dimension reduction methods.
Multimedia Content Description Standard: MPEG-7
• Influential XML based multimedia meta-data standard• Description of the storage media:
– Format, Image Size, Audio Quality, Video Frames etc.
• Creation and production information:– Creation date and location, title, genre, etc.
• Content semantic description:– Events, concepts, objects, etc.
• Content structural description:– Shot and key frames with color, texture and motion
features, etc.
• Metadata about the description:– Author, version, creation date, etc.
MPEG-7
• Expression of multimedia data
• Missing: Search for Implicit Data
– The meaning of the structure: Not expressed
– Ex. A video: length, format, name, dates etc.
• Gender: Documentary, Interview, Movie
• Theme: Science, Sports, Horror
• No consideration on search engines
MPEG-7
• Search:
– XPath, XQuery
– Semantic Views Query Language
Simple MPEG7 Example<Mpeg7>
<Description xsi:type="SemanticDescriptionType">
<Semantics>
<Label>
<Name> Car </Name>
</Label>
<Definition>
<FreeTextAnnotation>
Four wheel motorized vehicle
</FreeTextAnnotation>
</Definition>
<MediaOccurrence>
<MediaLocator>
<MediaUri> image.jpg </MediaUri>
</MediaLocator>
</MediaOccurrence>
</Semantics>
</Description>
</Mpeg7>
MPEG7 Details
• Standardizes 3 parts:
– Description tools
• Descriptors (D)
• Description Schemes (DS).
– Description Definition Language (DDL)
• To specify these schemes
– System tools
MPEG7 Details
• Descriptors (D)– Representation of a feature
• Syntactic and Semantic
– Low-level audio or visual features • Color, motion, texture etc
– Audiovisual content • Location, time etc
• Objects can be described– Several descriptors.
MPEG7 Details
• Description Schemes (DS) describe
– Specification of the relations
• Between Descriptors
• Between Description Schemes
– Relations can be structural and semantics
– High-level audiovisual (AV) features
• Regions, segments, events etc
MPEG7 Details
• Description Definition Language
– Based on XML
• Defines the structural relations between descriptors– Creation and modification of description schemes
– Creation of new descriptors.
MPEG7 Details
• System Tools
– Deal with Descriptor management
• Binarization
• Synchronization
• Transport
• Storage
MPEG7 Details - Overview
MPEG7 Details
• Next Slide
– Description of a Video Segment
MPEG7 Details
• How to extract semantics?
– i.e. Intelligent Information Retrieval
– Drawback of the standard
– Ontology help required:
• Domain Specific Ontology (Football, Location)
• Automatically extract information
• Use for a better search result
Research Fields
• Design: still in research
• Queries: techniques need to be modified
• Rest:
– Modeling: complex objects, wide range of types
– Storage: representation, compression, buffering during I/O, mapping
– Performance: physical limitations, parallel processing
• Thank you!
• Questions?
References
• Wikipedia: Various Pages• Computer Science and Engineering Department
Resources:– University of Notre Dame– Northumbria University– Carnegie Mellon University– Boston College– Simon Fraser University– Georgia Institute of Technology
• Interview with A. Anil Sinaci