Media Objects in TimeA Multimedia Streaming System

Björn Feustel, Thomas Schmidt

{feustel,schmidt}@fhtw-berlin.deFH für Technik und Wirtschaft Berlin

Computer Centre

Agenda

Introductory Ideas

Media Object Model

MIR – A Media Information Repository

Media Objects in Time Synchronized Media Streams Compound Flow Model Wavelet Web Video

Conclusions and Outlook

Ideas

Networked multimedia revolutionizes teaching as did the invention of blackboard and chalk:

Online aid in the lecture hall– Teachers load multimedia presentation

objects to supplement lectures

Telelearning at home– Students recall from and work with

full web applications

We face an urgent demand for a heterogeneous multitude of multimedia applications

Educational Task

Provide an Environment for easy implemen-tation of specific educational applicationssupporting:

Free definition of information layers General media handling incl. Streaming Non-linear information structuring Time-dependent presentation Event-based interactions

Time in Teaching

Perception is a time-dependent process: The Notion of Time is vital to Teaching

– A timeline provides speed and (some) linear orderings (as trad. teachers do)

– Time control processes determine performance

Multimedia introduces Time-based Material:– Audio, video, online data processing

– Synchronisation tasks arise

Process Synchronous Scalable Media Streams

Modeling Media Objects in Time

Temporal and spatial alignment rises a new level of complexity:• Temporal media placement needs playscripts

• Authoring challenging in geometry+time state space

Individual object preservation : pure video-type streaming ansatz ?

Fascinating interaction potentials within analytically preserved object model allows for scene hopping

Underlying Logic of an Object Model needed!

Media Object Model

• Media Object (MOB)– Neutral container of MOBs or data– Arbitrary annotations from OO classes– Reusable, self-consistent component– Mob composition open to semantics

• Active Document Structures– Flexible object reference model– Event-type actions imposable along

referential structures

Media Information Repository

Fundamenting Multimedia Applicationswww.rz.fhtw-berlin.de/MIR

Media Information Repository

• Media Object Database– General storage of multimedia data– Provides flexible information structuring– Supports arbitrary annotations

• Mir Authoring Environment– Database editing in the Web– Application specific interface

• Mir Architecture– General/adaptable access for Clients– Pluggable application subservers

Media Object Database

MIR Authoring

MIRArchitecture

Media Objects in Time

Synchronized Media Streams

Accurate scheme for temporal and spatial placement of presentation component:

• Timeless objects s.a. text, images, etc.

• Time-streamed media s.a. video, audio, etc.

• Any composition of the above

are combined in a

synchronous Web stream for Browser Display

Compound Flow Model

Intuitively structured object logic:Structuring implements temporal & spatial

inclusion relation Reusable complex presentation components Media Objects as central composition entities include Playscripts Time based flow generator Suitable for light weight client implementation

MobIT Media Objects

Universal container for MOBs or media elements

Object reference list + Playlist Local co-ordinate system in time and space Congruence of structure and state space:

Referential self-consistency Object structure with temporal and spatial

meaning (inclusion)

Scalability and Parametrizability

Media

Object

Structure

Flow Generator

Object resolution and time linearization

Linearization suitable for dynamic change

Wavelet Web Video

Joint Work with H. Cycon and Group JAVA implementation of real-time

Wavelet PACC Codec Scalable Video Streamer at low bandwidth (QCIF at 12 fps < 64 kbit/s) CIF (352 x 288 pixels) at 5 fps performs in real-time within an applet on a 500 MHz Pentium Machine

Architecture and Implementation

• 3-tiered Open Multimedia Architecture• Intelligent Multimedia Database System • Additional lightweight XML-Datastore• Subserver Instances for Streaming Media

or Data Processing• Wavelet compressed Web Video • Interfaces for additional Media Types or

Pluggable Subservers • Pure Java Implementation for Universal Use

Conclusions & Outlook

Universal Media Database. Application defined Annotations.

Active Document Structuring.

Reusable Presentation Components.

Web Authoring.

Time Synchronization for General Types of

Media.

Scalable Video streams in Wavelet compression.

Interaction Model under way.

Virtual Design

Knowledge Market Place