MPEG-7 Interoperability Use Case

Motivation MPEG-7: set of standardized tools for

describing multimedia content at different abstraction levels

Implemented in the form of XML Schemas

Lack of explicit semantics: intended semantics in form of documentation

Lack of precise semantics: ambiguities resulting from flexibility in structuring the descriptions

Motivation (cont’d) Attach formal semantics to mpeg7

by translating it to an ontology The lack of precise and explicit

semantics leaves room for different interpretations

interoperability issues raised among the developed mpeg7 ontologies

Examples I Semantically identically metadata

can be represented in multiple ways

E.g. annotating an image depicting ‘Zidane scoring against England’

Possible ways: using free annotation, keywords, structured annotations etc.

Examples I (cont’d)<FreeTextAnnotation

xml:lang="en">Zinedine Zidane scoring against England

</FreeTextAnnotation> <KeywordAnnotation

xml:lang="en"><Keyword>Zinedine</Keyword><Keyword>Zidan</Keyword><Keyword>scoring</Keyword><Keyword>England</Keyword><Keyword>goal</Keyword>

</KeywordAnnotation>

<StructuredAnnotation>

<Who>

<Name xml:lang="en">Zinedine Zidane</Name>

</Who>

<WhatAction>

<Name xml:lang="en">

Zinedine Zidane scoring against England.

</Name>

</WhatAction>

</StructuredAnnotation>

Example II The image representation in itself as a

multimedia entity can be performed in multiple ways as well

Possible ways: using the Stillregion DS or even the VideoSegment DS: no formal semantics to prevent one from selecting to interpret an image as a still region or as a frame of a video sequence

Example III Semantically different metadata can be

represented using the same description tool

E.g., the Multimedia Description Schemes:VideoSegment DS can be used to represent a single frame in a video, an arbitrary sequence of frames, or the full video sequence

Three different semantic concepts: frame, shot, video that can’t be discriminated based on the provided XML Schemas

Example IV Loose (intended) semantics of MPEG-7

e.g. with respect to the intended meaning of the structure decomposition schemes

E.g., not part-whole semantics: having an image annotated as ‘Zidane scoring’ and a second image, having a segment annotated the same, a single query wouldn’t be adequate to retrieve both

Approaches to building an MPEG-7 ontology The Hunter approach Initially a DAML+OIL enriched RDFS

ontology, subsequently translated into OWL Intended to be used under a core ontology

(ABC) to enable harmonization with domain ontologies

Decomposition captured using subclass constructs

However no axioms included (e.g. an image consists of a set of spatial, and not temporal, segments)

Approaches to building an MPEG-7 ontology (cont’d) The SMART Web project approach The MultimediaContent and

Segment classes along with corresponding properties implement the recursive nature of multimedia content decomposition

Partial axioms

Approaches to building an MPEG-7 ontology (cont’d) Tsinaraki’s approach Translate the MPEG-7 MDS into

OWL to serve as a core ontology for the integration of domain specific knowledge into the MPEG-7 semantic metadata

Domain ontologies need to be ‘mapped’ to the MPEG-7 schemas

The Hunter’s and Tsinaraki’s approaches in practice Use Case: Annotate and image of Zidane scoring

:image01 rdf:type mpeg7:Image:goal01 rdf:type s:Goal:scoring01 rdf:type s:Scoring

:image01 mpeg7:depicts :goal01:goal01 abc:hasAction :scoring01:scoring01 abc:hasAgent s:_b1s:_b1 :hasName :“Zinedine Zidane”

mpeg7:Image rdfs:subclass mpeg7:MultimediaContentmpeg7:MultimediaContent rdfs:subclass abc:Manifestations:Scoring rdfs:subclass abc:Actions:Goal rdfs:subclass abc:Event :region01 rdf:type mpeg7:StillRegion

(Hunter’s approach)

(cont’d)

:Zidane rdf:type mpeg7:PlayerObjectmpeg7:PlayerObject owl:subclass mpeg7:AgentObjectTypempeg7: AgentObjectType rdfs:subclass

mpeg7:SemanticType:Zgoal rdf:type s:Goals:Goal rdf:type mpeg7:EventTypempeg7: EventType rdfs:subclass mpeg7:SemanticType:Zidane mpeg7:Relation :Zgoal

(Tsinaraki’s approach)

Possible solutions Define semantic mappings between the

various mpeg7 ontologies (exploit the potential of using rules on top of ontologies and reason on them)

Built a core multimedia ontology (exploit perhaps a foundational ontology’s axiomatization to ensure not only explicit but precise semantics as well)

Use of ‘semantically enabled’ profiles (again formal ways to specify the semantics constraints are required)

Thank you!