A Multimedia Service with MPEG-7 Metadata and Context Semantics

Lehrstuhl Informatik 5(Informationssysteme)

Prof. Dr. M. JarkeI5-CKKh-0309-1

WMM 2009

Yiwei Cao Ralf Klamma

Maziar Khodaei

A Multimedia Service with MPEG-7 Metadata and Context Semantics

Yiwei Cao, Ralf Klamma, and Maziar Khodaei

Informatik 5 (Information Systems), RWTH Aachen University

20.03.2009Toulouse, France

The 9th Workshop on Multimedia Metadata (WMM‘09)



WMM 2009


Maziar Khodaei

Agenda

Motivation and Scenario System Design of CA3M Service

(Context-aware Mobile Multimedia Service) A MPEG-7ToRDF Converter Implementation and Evaluation Conclusions and Outlook

CA3M

Mobility Multimedia

Uncertainty



WMM 2009


Maziar Khodaei

Motivation Context [Dey 1999]

– Interaction between users und applications– Context awareness: location, time, device, community, activities

Context-aware Mobile Multimedia Systems– Presentation of relevant information: right information in the right

format to the right community at right time and place– Multimedia adaptation of 3D models, videos, audios, images, texts

Data Uncertainty in Location Based Services (LBS) [Henrickson 2004]

– Inconsistencies between models und real world– Inconsistencies between local environment models– Representation: unknown; ambiguous; imprecise; erroneous



WMM 2009


Maziar Khodaei

Car Race ScenarioContext-Aware Multimedia Sharing

Communities produce and consume media at different places, times, etc. Semantic mapping of context data on semantic base types needed

– GPS data -> Tribunes, Curves, etc. (Places)– Time data -> Lap, Start, Finish, Box stop, etc. (Events)– Racing car data –> Ferrari, McLaren-Mercedes, BMW-Sauber Community, etc. (Agents)

Multimedia adaption needed because of different mobile devices



WMM 2009


Maziar Khodaei

Concepts of CA3M Community-Aware Static Multimedia

Adaptation based on Clustering Algorithms [KSCa06]

Context-Aware Adaptation based on Semantic Einrichment of Multimedia Information [CKHJ08]

How to support MPEG-7/MPEG-21 Semantic Enrichment for further Semantic Processing?

How to systematically handle Uncertain Information in furtherSemantic Processing?

Context-AwareAdaptation

System(OWL)

[CKHJ08, MDM‘08]

Media AdaptationSystem

(MPEG-7/21)

[KSCa06, ECTEL‘06]

MPEG-7ToRDF

Context Reasoner[CKHJ08, MDM‘08]

Uncertainty Management



WMM 2009


Maziar Khodaei

Selection of MPEG7ToRDF Converter Technologies

MPEG-7 Ontology [García et al., 2006]

– Developed by Roberto Garcia Gonzales within Rhizomik – Based on MPEG-7 as of 2001– In OWL-Full-Format

Alternatives

TBD: MPEG-21 Conversion

Multimedia Ontologies Format Mapping

MPEG-7 Upper DMS [Hunter, 2001] OWL-Full -

COMM [Bloehdorn et al., 2005] OWL-DL -

aceMedia [ace, 2005] RDFS-DS -

MPEG-7 Ontologie (Rhizomik)[García et al., 2006]

OWL-Full XSD2OWLXML2RDF



WMM 2009


Maziar Khodaei

The MPEG7ToRDF Converter The MPEG-7 Ontology based on MPEG-7 Schema as of 2004 Mapping MPEG-7 XML Schema to MPEG-7 Ontology in OWL Mapping MPEG-7 XML Documents to RDF


Context Reasoner [CKHJ08]

MPEG-7ToRDF

Media Adaptation

System(MPEG-7/21)

[KSCa06]

Context-Aware

AdaptationSystem(OWL)

[CKHJ08]

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:mpeg7="http://manet.informatik.rwth-aachen.de/ ~khodaei/mpeg7-v2.owl#"> <mpeg7:SemanticPlaceType rdf:about="http://manet.informatik.rwth- aachen.de/~khodaei/place.xml#SemanticPlaceType_ admin20080227090806191"> <mpeg7:Place rdf:parseType="Resource"> <mpeg7:GeographicPosition rdf:parseType="Resource"> <mpeg7:Point rdf:parseType="Resource"> <mpeg7:longitude>7.438253814591</mpeg7:longitude> <mpeg7:latitude>43.774566243600006</mpeg7:latitude> <mpeg7:altitude>29</mpeg7:altitude> </mpeg7:Point> </mpeg7:GeographicPosition> </mpeg7:Place> <mpeg7:Label rdf:parseType="Resource"> <mpeg7:Definition rdf:parseType="Resource">curve-1</mpeg7:Definition> <mpeg7:Name>43:46.47397N, 7:26.29523E, 29m</mpeg7:Name> </mpeg7:Label> </mpeg7:SemanticPlaceType> </rdf:RDF>



WMM 2009


Maziar Khodaei

Context Reasoner

Reasoning on context information SPARQL

– RDF Query Language– Ontology Reasoning Language– Implementation using the Jena Framework



MPEG-7ToRDF

Media Adaptation

System(MPEG-7/21)

[KSCa06]

Context-Aware


[CKHJ08]

PREFIX mpeg7: <http://manet.informatik.rwth-aachen.de/~khodaei/mpeg7-v2.owl#>PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>SELECT ?agent ?placeWHERE {

?agent mpeg7:name “FirstName LastName"?agent mpeg7:hasGeographicPosition ?loc1?loc1 mpeg7:longitude ?lon1?loc1 mpeg7:latitude ?lat1?place mpeg7:hasGeographicPosition ?loc2

?place mpeg7:definition “curve-n”?loc2 mpeg7:longitude ?lon2?loc2 mpeg7:latitude ?lat2FILTER Distance(?loc1, ?loc2) <= distance

}



WMM 2009


Maziar Khodaei

Uncertainty Management Clustering Algorithms based on User Preferences

– Calculation Euclidean Distance of Communities– Clusters are recalculated when new users come

Rating/Ranking as User Query Feedback



MPEG-7ToRDF

Media Adaptation

System(MPEG-7/21)

[KSCa06]

Context-Aware


[CKHJ08]



WMM 2009


Maziar Khodaei

Evaluation Evaluating within the DFG UMIC Excellence Cluster 2006-2011

– Concepts and demonstrators for smart, mobile, broadband, low-cost systems will be developed which support the demanding applications of the next-decade mobile Internet

– Mobile Multimedia Information for– Cultural Heritage Management (Virtual Campfire)– Virtual City Guides– Immersive Games

– Semantic Enrichment of Metadata (MPEG-7, MPEG-21, …) for Location-Based Services

– Systematic Evaluation of Services with MoBSOS [RKSp08] Process

– Converting MPEG-7 / MPEG-21 Files into OWL/RDF– SPARQL Queries: Context Search on Semantic Base Types



WMM 2009


Maziar Khodaei

Conclusions

CA3M - A mobile context-aware multimedia service – Integration into existing LAS architecture as a web service– Enhancement of information interoperability – Evaluation within UMIC Cluster of Excellence

Technical Improvements– Generating MPEG-7 ontology based on the 2004 MPEG-7 Schema– Realizing a converter mapping XML document to OWL/RDF– Integrated reasoning on multimedia and context information– Semantic query processing for multimedia information– Systematic handling of uncertainty for location based services

(rating/ranking, clustering)



WMM 2009


Maziar Khodaei

Outlook Data Management

– Improvement of SPARQL queries for large amount of data– Performing SPARQL queries on relational databases– Field tests with mobile communities

Uncertainty Management– Improvement of context reasoning– Handling uncertainty with probabilistic methods

– Fuzzy logic– Bayesian network

Applications & User-Generated Content– Mobile Storytelling– Cultural Heritage Management – Services on various mobile devices e.g. iPhone, Google G1

A Multimedia Service with MPEG-7 Metadata and Context Semantics

Technology

Transcript of A Multimedia Service with MPEG-7 Metadata and Context Semantics