The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27,...
-
Upload
emery-fleming -
Category
Documents
-
view
218 -
download
0
Transcript of The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27,...
![Page 1: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/1.jpg)
The MPEG-7 Standard- A Brief Tutorial -
Ali Tabatabai
Sony US Research Laboratories
February 27, 2001
![Page 2: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/2.jpg)
2
Outline
Objectives of the MPEG-7 Standard
Main Elements of MPEG-7
Scope of MPEG-7
MPEG-7 Application Areas
MPEG 7’s relation with other standards
![Page 3: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/3.jpg)
3
Why do we need MPEG-7 ?
N e e d
• Content Management
• Fast & Accurate Access
• Personalized Content Production and Consumption
• Automation
+
Support for Advanced Query
•Visual•Audio•Sketch
![Page 4: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/4.jpg)
4
MPEG: A Brief History (1)
MPEG: Moving Picture Experts Group
ISO / IEC/JTC1/SC29/WG11
A Working Group of ISO/IEC in charge of the Development of Standards for
Coded Representation of Digital Audio and Video
Established in 1988
![Page 5: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/5.jpg)
5
MPEG: A Brief History (2)
MPEG-1: Interactive CD and MP3 11 / 1992
MPEG-2: DTV, STB, DVD 11 / 1994
MPEG-4: Web and Mobility ver1: 09 /1998
ver2: 11
/1999
MPEG-7: ??? 08 / 2001
MPEG-21: Multimedia Framework 11 / 2001
![Page 6: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/6.jpg)
6
MPEG-7: What Is It ?
THE MPEG 7 THE MPEG 7 STANDARDSTANDARD THE MPEG 7 THE MPEG 7 STANDARDSTANDARD
Content Description of Various Audio Visual
Information
IS NOT a COMPRESSION Standardsimilar to MPEG-1/2/4 or their
Extension
IS NOT a STANDARD forFEATURE
EXTRACTION/MATCHING
Types of Audio Visual Information• Audio, Speech• Moving video, still pictures, graphics• Information on how objects are combined in scenes
Types of Audio Visual Information• Audio, Speech• Moving video, still pictures, graphics• Information on how objects are combined in scenes
![Page 7: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/7.jpg)
MPEG-7: Application Areas Storage and retrieval of audiovisual databases (image, film,
radio archives) Broadcast media selection (radio, TV programs) Surveillance (traffic control, surface transportation, production
chains) E-commerce and Tele-shopping (searching for clothes /
patterns) Remote sensing (cartography, ecology, natural resources
management) Entertainment (searching for a game, for a karaoke) Cultural services (museums, art galleries) Journalism (searching for events, persons) Personalized news service on Internet (push media filtering) Intelligent multimedia presentations Educational applications Bio-medical applications
![Page 8: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/8.jpg)
8
MPEG-7Description Scope for AV Content
Description Granularity Low-level
High-level
Form
Access
Classification
Link
Context
![Page 9: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/9.jpg)
9
MPEG-7: Main Elements
Descriptors (D)
syntax and semantics of each feature representation
Description Schemes (DS)
structure and semantics of the relationships between
components
Description Definition Language (DDL)
creation of new DS’s
modification/extension of existing DS’s
![Page 10: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/10.jpg)
10
MPEG-7: Major Functionalities
Systems (ISO / IEC 15938 -
1)
Description Definition Language (ISO / IEC 15938
- 2)
Visual (ISO / IEC 15938 -
3)
Audio (ISO / IEC 15938 -
4)
Multimedia Description Schemes (ISO / IEC 15938
- 5)
Reference Software (ISO / IEC 15938 -
6)
![Page 11: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/11.jpg)
11
DS1
DS2 DS2
DS3 D1 D3 D2 Systems
0001100
<Object> <Label/> <Definition/>
. .
</Object>
DDL
Instantiation
MPEG-7: Main Elements (2)
![Page 12: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/12.jpg)
12
MPEG-7: Systems
It defines tools to:
manage and protect intellectual property
synchronize between content and description
provide for efficient storage and transport
![Page 13: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/13.jpg)
13
MPEG-7: DDL and its Components
Description Definition Language:
Creation of the Ds and DS’s: XML Schema & MPEG-7 Extensions
Instantiation of XML
XML Schema:
Data types
Simple and Complex types
Elements, attributes
Inheritance, Abstract types
MPEG-7 extensions:
Array and Matrix data type
![Page 14: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/14.jpg)
14
MPEG-7: Audio
Sound Effects
Music Instrument Timbre
Spoken Content
Melody Contour
![Page 15: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/15.jpg)
15
MPEG-7: Visual (1)
Color quantization, dominant, scalable, color-structure, layout,
GoF/GoP
Texture
Shape region-based, contour-based, 3D
Motion camera motion, motion trajectory, parametric motion,
motion activity
![Page 16: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/16.jpg)
16
MPEG-7: Visual (2)
Localization spatio temporal
Others face recognition
![Page 17: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/17.jpg)
17
MPEG-7: Basic Visual Structures
Grid Layout
2D-3D Multiple View
Time Series
Spatial 2D Coordinates
Temporal Interpolation
![Page 18: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/18.jpg)
18
Low level Audio Visual descriptors
• Color • Camera motion• Motion activity• Mosaic
• Color • Motion trajectory• Parametric motion• Spatio-temporal
shape
• Color • Shape• Position• Texture
Video segments Still regions
Moving regions Audio segments
• Spoken content • Spectral
characterization• Music: timbre,
melody
![Page 19: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/19.jpg)
19
MPEG-7: MMDS Basic Elements
Datatype &structures
Link & medialocalization Basic DSs
Basic elementsBasic elements
Schematools
Time, Duration, Medialocators Language
Annotation,Person, Place
Root, Top-level elements, Packages
![Page 20: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/20.jpg)
20
MPEG 7: Content Management and Description
Content descriptionContent description
Content managementContent management
Creation &production
Media ContentUsage
Conceptualaspects
Structuralaspects
Title, Creator, Creation location & date, Purpose,
Classification, Genre, Review, Parental guidance,
etc. (Author generated)
Format, Coding, Instances, Identification, Transcoding
Hint, etc.(Several instances)
Rights holder, Access rights, Usage Record, Financial aspects,
etc. (Evolution)
Datatype &structures
Link & medialocalization Basic DSs
Basic elementsBasic elements
Schematools
Viewpoint of the structure: Segments• Spatial / temporal structure• Audio, video low-level Ds
• Elementary semantic information.
Viewpoint of conceptual notions• Events, objects, abstract concepts, and
their relation
![Page 21: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/21.jpg)
21
Foreground
Example of Segment trees
SR1: Creation, Usage meta
information Media description Textual annotation Color histogram, Texture
SR2: Shape Color Histogram Textual annotation
SR6: Color Histogram Textual annotation
SR5: Shape Textual annotation
SR4: Shape Color Histogram Textual annotation
SR3: Shape Color Histogram Textual annotation
Background
![Page 22: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/22.jpg)
22
Segment Tree
Shot1 Shot2 Shot3
Segment 1
Sub-segment 1
Sub-segment 2
Sub-segment 3
Sub-segment 4
segment 2
Segment 3
Segment 4
Segment 5
Segment 6
Segment 7
Semantic DS (Events)
• Introduction
• Summary
• Program logo
• Studio
• Overview
• News Presenter
• News Items
• International
• Clinton Case
• Pope in Cuba
• National
• Twins
• Sports
• Closing
TimeAxis
![Page 23: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/23.jpg)
23
MPEG 7: Navigation and Access
Navigation &Navigation &AccessAccess
Summary
Variation
Content descriptionContent description
Content managementContent management
Creation &production
Media ContentUsage
Conceptualaspects
Structuralaspects
Datatype &structures
Link & medialocalization Basic DSs
Basic elementsBasic elements
Schematools
Efficient support of: discovery, browsing, navigation, visualization
Substitution of the original contentAdaptation to terminal, network, or
user preferences
![Page 24: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/24.jpg)
24
MPEG 7: Hierarchical summary
A-VData
HierarchicalSummary
HighlightLevel HighlightLevel
HighlightSegment
HighlightSegment
HighlightSegment
HighlightSegment
HighlightSegment
HighlightSegment
HighlightSegment
![Page 25: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/25.jpg)
25
MPEG 7: Sequential summary
SequentialSummary
A-VData
TextProperty
TextProperty
FrameProperty
FrameProperty
FrameProperty
SoundProperty
SoundProperty
SoundProperty
![Page 26: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/26.jpg)
26
MPEG 7: Variation
VIDEO IMAGE TEXT AUDIO
Modality
Fidelity
Source
Variation
A
IH
GFE
DCB
Universal Multimedia Access Adapt delivery to network and terminal characteristics (QoS)
![Page 27: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/27.jpg)
27
MPEG-7: Content Organization
Navigation &Navigation &AccessAccess
Summary
Variation
AnalyticModel
Content organizationContent organization
Content descriptionContent description
Content managementContent management
Creation &production
Media ContentUsage
Conceptualaspects
Structuralaspects
Description and organization of collection of documents
Collection &Classification
Datatype &structures
Link & medialocalization Basic DSs
Basic elementsBasic elements
Schematools
![Page 28: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/28.jpg)
28
MPEG-7: Collection
![Page 29: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/29.jpg)
29
MPEG 7: User Interaction
Navigation &Navigation &AccessAccess
Summary
Variation
AnalyticModel
Content organizationContent organization
Content descriptionContent description
Content managementContent management
Creation &production
Media ContentUsage
Conceptualaspects
Structuralaspects
Collection &Classification
UserUserInteractionInteraction
User preferences
Datatype &structures
Link & medialocalization Basic DSs
Basic elementsBasic elements
Schematools
User identification and preferences:Filtering, search and browsing
![Page 30: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/30.jpg)
30
MPEG-7 Its Relation with other standards
AHG on “Metadata harmonization”:
SMPTE: Metadata dictionary, KLV encoding
Dublin Core Metadata Initiative
European Broadcast Union
AHG on TV AnyTime Application
Large number of Liaisons:
SMPTE
Dublin Core
W3C (XML Schema)
etc.
![Page 31: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/31.jpg)
Competition:• Individual work• Definition scope and r
MPEG-7: TimeLine - The Work Plan
Call for proposalsCall for proposals
20012000199919981996
Working draftWorking draft Committee draftCommittee draft
Draft internationalstandard
Draft internationalstandard
Final committeedraft
Final committeedraft
InternationalstandardInternationalstandard
Divergence Convergence
![Page 32: The MPEG-7 Standard - A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001.](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e625503460f94b5e4e8/html5/thumbnails/32.jpg)
32
Conclusions on AV Content Description and MPEG-7
MPEG-7:
AV content description for interoperable
application
Description Definition Language:
XML Schema (flexibility) + Binary version
(efficiency)
Description Schemes:
Library of description tools
Covers a wide range of generic needs