SAMBITS: Interactive Digital TV and Rich Media Streaming, based on DVB-MHP

© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 1

SAMBITS:SAMBITS: Interactive DigitalInteractive Digital TV and TV and Rich Media Rich Media Streaming, based on DVB-MHPStreaming, based on DVB-MHP

Limitations of A/V-Streaming on the Internet today SAMBITS:

System System architecturearchitecture Combination of broadcast and Internet delivery Combination of broadcast and Internet delivery System proposal for Rich Media Streaming, using System proposal for Rich Media Streaming, using

MPEG-4 and MPEG-7 in DVB-MHPMPEG-4 and MPEG-7 in DVB-MHP Multimedia Studio SystemMultimedia Studio System: New production tools: New production tools Multimedia User TerminalMultimedia User Terminal SAMBITS Demonstration ScenariosSAMBITS Demonstration Scenarios

Eurovision Songcontest 2001Eurovision Songcontest 2001BBC‘s Big AlBBC‘s Big Al

Gerhard StollInstitut für Rundfunktechnik, München, Germany

ITU-Workshop on Multimedia Convergence, Geneva, 14 March, 2002


18%

82%

July 1998

The growths of A/V-streaminThe growths of A/V-streaming @g @ Internet: Internet: InterestInterest in in Internet-Streaming in Internet-Streaming in the USthe US

Source: edison media research, Arbitron, 2001

Nearly half of the Internet users in the US was listening to Internet-Radio already

35%

65%

January 2000

January 200110 %: during last week

21%: during last month56%

44%

10%: during last month

27%

73%

July 1999

4%: during last week

36%

20%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Listen Audio Online Watch Video Online

22% ofUS citizens

areAudio

Streamies

12% ofUS citizens

areVideo

Streamies


TV and Internet:TV and Internet:Convergence of Broadcast & Telecom networksConvergence of Broadcast & Telecom networks

CoreCoreTransportTransport

ServiceServiceProviderProvider

UserUserTerminalTerminal

AccessAccess

Internet

Internet / Telecom Services

Core NetworkIP, ATM, SDH,

WDM

NodeNode

POTS/ISDNxDSL/fibrePMP/UMTS

POTS/ISDNxDSL/fibrePMP/UMTS

BroadcastNetwork

Broadcaster

HeadendHeadend

HFCLMDSHFC

LMDS


Broadcast@Internet: Broadcast@Internet: Progress with Seven-League Boots or just a Hype?Progress with Seven-League Boots or just a Hype?

Streaming Audio/Video on the Internet gets more and more attention,

but

Streaming of “events“ does not work properly today, because Internet is not a “broadcast network“ Multicasting is not implemented widely Serving millions of viewers is still expensive for the content

provider QoS on the Internet is not guaranteed

What happened during the EBU‘s Webcast of the Eurovision Song Contest 2000 & 2001? Please have a look!


InternetTV: InternetTV: Convergence of TV and InternetConvergence of TV and Internet

A successful web-experience with live-streams of events needs an intelligent management of - Internet (unicast) and - Broadcast (multicast) channels- Links between the two channels

A system is needed, which combines the broadcast and Internet delivery can manage both channels produces and tailors the content for both channels provides an easy to use multimedia terminal to the end-

user

Solution:


S A M B I T SS A M B I T S

System for Advanced

Multimedia Broadcast &

InformationTechnology

Services

Project Coordination:


Internet@DTV or DTV@Internet:Internet@DTV or DTV@Internet:„The value chain“„The value chain“

Content owner/ provider

Content owner/ provider

Broadcaster/ TV Company

Broadcaster/ TV Company

USERUSER

Set-top boxSet-top boxNetwork

operator

Networkoperator

Dig. Platformservice provider

Dig. Platformservice provider

Content owner

/ provider

Content owner

/ providerPortalPortal

EPGEPG

SoftwareSoftware

CACA

ISPISP TelcoTelco PCPC

DTV

Internet

ContentCreation

ContentCreation

PackagingAgg.tion

Navigation

PackagingAgg.tion

Navigation

Multiplex, Customer

interface

Multiplex, Customer

interfaceDeliveryDelivery

Usage, Consumer

device

Usage, Consumer

device

Internet@DTV or DTV@Internet : How to cross the DTV bridge?Internet@DTV or DTV@Internet : How to cross the DTV bridge?

Involvement


SAMBITS: SAMBITS: providing a perfect TV and Web providing a perfect TV and Web experience for the viewerexperience for the viewer

Combines MPEG-2, MPEG-4, MPEG-7, DVB-MHP and the Internet for high-quality interactive multimedia broadcasting, including Rich Media Streaming

Develops tools for integration of broadcast and internet value chain

Enables interactive personalised information through broadcast and internet channels

Objectives:


SAMBITS:SAMBITS:Concept of complete System and Reference ChainConcept of complete System and Reference Chain

BroadcastService

BroadcastService

InternetService

InternetService

Broadcast ChannelDVB-S, DVB-C, DVB-T


Internet ChannelPSTN/ISDN,

Cable (Coax & Fiber)Satellite



SAMBITSIntegratedReceiverDecoder


Bro

adca

st N

etw

ork

Ad

apte

rIn

tern

et N

etw

ork

Ad

apte

r

Inte

rnet

In

terf

ace

Mo

du

leB

road

cast

In

terf

ace

Mo

du

le

ForwardInteraction

path

ReturnInteraction

path SAMBITSUser Terminal

(STB)

Content Producer Content Provider Consumer

Local interaction

Local interaction

BroadcastService

BroadcastService

InternetService

InternetService









Bro

adca

st N

etw

ork

Ad

apte

rIn

tern

et N

etw

ork

Ad

apte

r

Inte

rnet

In

terf

ace

Mo

du

leB

road

cast

In

terf

ace

Mo

du

le

ForwardInteraction

path

ReturnInteraction

path SAMBITSUser Terminal

(STB)

Content Producer Content Provider Consumer

Local interaction

Local interaction


SAMBITS:SAMBITS:Systems for Advanced Multimedia Broadcast and IT ServicesSystems for Advanced Multimedia Broadcast and IT Services

Main Directions of the ProjectMain Directions of the Projectdeveloped a multimedia studio system (example

implementation) for real-time and offline content manipulation, annotation/event characterisation and insertion/integration

developed a consumer type multimedia terminal interactive multimedia services based on MPEG-4 and

MPEG-7 including synchronising broadcast and Internet content

demonstrated integrated (DVB/Internet) services using a consumer type terminalEurovision Song Contest 2001 Scientific program: Story of “Big Al”

Project duration: 1 January 2000 – 31 December 2001


SAMBITS:SAMBITS: Concept of Interactive Concept of Interactive Programme Production Sub-SystemProgramme Production Sub-System

Content integration

Contentcreation/editing

Content display and pseudo- consumer

interaction

Raw A/V

Storage

Off-lineProducer

Internet

Broadcastcontent

Internetcontent


Interactive Programme Production SystemInteractive Programme Production System

ContentCapture

Tools

ContentCreation

Tools

EncodingDecoding

Tools

IndexingTools

Database& Search

Tools

ProgrammeCreation &

VisualisationTools

ContentCapture

ContentCapture

MHPWorkshop:Jpad Pro

MHPWorkshop:Jpad Pro

AdobePremiere

5.1

AdobePremiere

5.1

3DStudio

Max 3.0

3DStudio

Max 3.0

VRML toMPEG-4Scene

Translator

VRML toMPEG-4Scene

Translator

HTMLContent Creation

HTMLContent Creation

MPEG-2A/V

Codecs

MPEG-2A/V

Codecs

MPEG-4AV

Codec

MPEG-4AV

Codec

MPEG-7Codec

MPEG-7Codec

HighLevelIndex

HighLevelIndex

SegmentLevelIndex

SegmentLevelIndex

ImageSearch

Tool

ImageSearch

Tool

ProgrammeCreation

ProgrammeCreation

ProgrammeVisualisation

ProgrammeVisualisation


SAMBITS: Studio and Server SystemsSAMBITS: Studio and Server Systems

aa

ISP

InternetService

BroadcastService

On-lineService

ProvisionSubsystem

BroadcastIntermediate

Storage

WebIntermediate

Storage

Live MPEG-2 TS

TSPL

Live A/V

MPEG-2 TS

ServiceProvider

ContentProducer

InteractiveProgrammeProductionSubsystem

Off-LineService

ProvisionSubsystem

MPEG-2 PES

MPEG-4 ES

MPEG-4 File

MPEG-7

Webcontent

ContentDatabase

WebAccess

RawA/V

Content


Online Service Provision Sub-system Online Service Provision Sub-system Hardware and Software ComponentsHardware and Software Components


SAMBITS:SAMBITS:Broadcast MechanismsBroadcast Mechanisms

Mu

ltip

lex

ing

Traditional content

Multi-protocolencapsulated

real time events

MPEG-4 files,MPEG-7 files,

and web contentin object carousel

MPEG-4 streamsin PES packets

Added value contentsynchronised with traditional DTV service

Added valuecontent

MPEG-2 A/Vmaterial

MPEG-2/DVBtransport

stream


Composition of DVB-Transport stream and Composition of DVB-Transport stream and Internet deliveryInternet delivery

MPEG-2 AV stream

MPEG-7 fileannotation of main program

repeated transmission

MPEG-4 AV stream (synchronisable to MPEG-2)

MPEG-4 files2D/3D scenes

MPEG-7 filesassociated to MPEG-4 files

HTML pages


MPEG-7 filesassociated to MPEG-4 files

description of segmentURLs to related material

segment 1 segment 2 segment N

DV

B-M

UX

Inte

rne

t


HTML pages

HTML pages

main

dig

ital

TV

pro

gram


User feedback / requests


ad

ditio

na

lm

ultim

ed

ia c

on

tent

retu

rnc

ha

nn

el


Relationship to Key Action IV:Relationship to Key Action IV:Essential Technologies and InfrastructuresEssential Technologies and Infrastructures

SAMBITS as major European contributor to MPEG-7 Standard for interaction in combined broadcast and Internet applications Contributions to “Multimedia Content Description Interface” Participation in Core Experiment (CE) and Experimentation

Model (XM) development Creation of a production & postproduction studio system to

enter and edit metadata-based High Level -semantic- and Low Level -syntactic- MPEG-7 descriptions

Development of a multimedia receiver terminal with metadata based search functionality and user interaction

Large-scale shared virtual and augmented environments


SAMBITS:SAMBITS:Contributions to MPEG-7 standardisationContributions to MPEG-7 standardisation

Transmission/Storage Medium

IP MP4Delivery

Layer

Multiplex

MPEG-2 ATM ...

Multiplexed Streams

MultiplexMultiplex

Schemastreams

Descriptionstreams

CompressionLayer

Elementary Streams

Multimediastreams

UpstreamData

Application

APIs

Defines Describe

Reconstruction

DescriptionDecoder

SchemaDecoder

BiM/ TextualParsing

BiM/ TextualDecoding


SAMBITS and MPEG-7 StandardSAMBITS and MPEG-7 Standard

Generation of Descriptions Structure (e.g. Segments) Content (low level and high level)

Searching and Filtering Support of Program Production Preparation for Delivery

Use on studio site:

Searching Content oriented search

(e.g. classification and structured annotation)User Interface Management

Providing information about related material Triggering user interface events, based on

scene change and new related material

Use on terminal site:


SAMBITS Consumer Terminal:SAMBITS Consumer Terminal:FunctionalityFunctionality

Enable consumers to receive multimedia broadcasting (MPEG-2, MPEG-4, MPEG-7)

Enable user to retrieve additional content from the Internet (HTML)

let users interact with multimedia content consumers have a back-channel available via Internet based on MPEG-7 meta-data user can search and

retrieve content consumers can create their individual reception

experience


SAMBITS Multimedia Terminal:SAMBITS Multimedia Terminal:Types of content through broadcast channelTypes of content through broadcast channel

MHP Section FilterMHP Section Filter

Packet Elementary StreamPacket Elementary Stream SectionsSections

DVB-Board DriverDVB-Board Driver

CableCable SatelliteSatellite

MPEG-2 Transport StreamMPEG-2 Transport Stream

MPEG-2A/V

Streams

MPEG-2A/V

Streams

MPEG-4A/V

Streams

MPEG-4A/V

Streams

MPEG-4BIFS

Streams

MPEG-4BIFS

Streams

MPEG-4Files

MPEG-4Files

MPEG-7Files &

Streams

MPEG-7Files &

Streams

HTMLPages

HTMLPages

MHPProgram

Data

MHPProgram

Data

DVB-Board DriverDVB-Board Driver

SoftwareLayer

LogicalChannels

PhysicalNetwork


SAMBITS Multimedia Terminal:SAMBITS Multimedia Terminal:Types of content through Internet Types of content through Internet && local storage local storage

Internet ClientInternet Client

EthernetEthernet

InternetInternet

MPEG-4A/V Files

& Streams

MPEG-4A/V Files

& Streams

MPEG-7Metadata

MPEG-7Metadata

HTMLPages

HTMLPages

MP

G-4

File

sM

PG

-4Fi

les

MP

EG

-7Fi

les

MP

EG

-7Fi

les

HTM

LP

ages

HTM

LP

ages

MH

PP

rogr

am D

ata

MH

PP

rogr

am D

ata

Local DiskLocal Disk

MP

EG

-2P

ES

MP

EG

-2P

ES

Extended MHP Storage ManagerExtended MHP Storage Manager SoftwareLayer

LogicalChannels

PhysicalNetwork


System outline of a conventional System outline of a conventional set-top boxset-top box


The SAMBITS terminal can be interpreted as an The SAMBITS terminal can be interpreted as an extension of a standard compliant MHP STB.extension of a standard compliant MHP STB.

conventional STBMHP (IRT)

MHP compliantSTB

conventionalSTB

SAMBITSterminal

MPEG-7 MPEG-4

MHP (IRT)

Fujitsu-Siemens


SAMBITS: System outline of the SAMBITS: System outline of the multimedia consumer terminalmultimedia consumer terminal


Interaction of applications with the differentInteraction of applications with the differentmodules of DVB-MHPmodules of DVB-MHP

Application

User InteractionStorage Media

Control Remote Program

TCP/IP

Others overUDP/IP

DSM-CCServiceInformation

MPEG-2Section

Filter

TunerControl

CAControl

MediaControl

Graphics

Tuner

CA

Demux

Remote Control keyboard

API

MediaDecoder

+

AudioOutput

VideoOutput

InteractionChannel

Network


Answer: SAMBITS!Answer: SAMBITS!Systems for Advanced Multimedia Broadcast and IT ServicesSystems for Advanced Multimedia Broadcast and IT Services

Interactive Digital TV, based on MPEG-2 System as transport medium for broadcast and individually addressed traffic in connection with return channel

MPEG-4 synchronised services to MPEG-2 main program

MPEG-4 and MPEG-7 based push servicesLocal and full, i.e. remote interactivity in user terminalHandover/roaming of network functionalitiesCompatibility with the standardised DVB API:

The Multimedia Platform (DVB-MHP)

Features:

SAMBITS: Interactive Digital TV and Rich Media Streaming, based on DVB-MHP

Documents

Transcript of SAMBITS: Interactive Digital TV and Rich Media Streaming, based on DVB-MHP