Introduction and Architectures 23/8 - 2004 INF5070 – Media Storage and Distribution Systems:

Introduction and Introduction and ArchitecturesArchitectures

23/8 - 2004

INF5070 – Media Storage and Distribution Systems:

2004 Carsten Griwodz & Pål Halvorsen

INF5070 – media storage and distribution systems

Overview

Intro about the course multimedia applications and challenges

Architectures Media (Video) on Demand Machine internals Video server structures Examples

INF5070:The Course



Lecturers

Carsten Griwodz email: griff @ ifi

Pål Halvorsen email: paalh @ ifi



Content

Network

Network

Network

Network

architectures

file systems

resource scheduling

media data

protocols

topologies

distribution



Content System architectures

(server and system designs)

Media data (wire and file formats, codecs)

Media characteristics and user behavior (processing multimedia data, user interactivity)

Resource management (CPU and memory management)



Content Protocols with and without Quality of Service

(QoS) (specific and generic QoS approaches)

Storage systems (management of multimedia files, retrieval)

Distribution(use of caches and proxy servers)

Adaptation, mobility and Peer-to-Peer (various clients, different amount of resources)



Content Student assignment

(will be presented more in-depth later):

for example: TCP friendliness variations New TCP implementations Network emulators Comparison of Linux shedulers File system benchmarking (differnet OSes and file systems) Comparison of open source video servers (Helix vs. Darwin) Comparison of methods for network performance montoring ...

write a report and present to the class at the end of the course



Goals Media servers and distribution system

media characteristics architectures system support protocols distribution mechanisms …

Be able to evaluate any combination of these mechanisms, e.g., data placement on disk and memory caching

algorithm video popularity and proxy caching algorithm …



Exam

Prerequisite: approved presentation of student assignment

Oral exam (??/12-2004): all transparencies from lectures content of own student assignment

Applications and Challenges



Applications Multimedia enriches the user interface giving new

applications

Broadcast – server is only VCR substitute (e.g., MTV Europe)

Media-on-Demand (MoD) Network-based, user controlled distribution of media content

Video-on-Demand (VoD) (e.g., Bell Atlantic ~1000 users, 700 choices)

News-on-Demand (NoD) (e.g., CNN, BBC web today, BAD quality) Learning-on-Demand (LoD) (e.g., the OMODIS project)

Virtual worlds (typically live, not on-Demand) video conferencing (e.g., USIT’s electronic classroom) games (e.g., Quake, …)



Requirements Application

QoS – time sensitivity resource capabilities –

support interactive streaming of multimedia content

Business scalability reliability

Architectural topology cost vs. performance



Technical Challenges User end system

real-time processing of streams (1000 MIPS for an MPEG-II decoder)

request/response delay (< 150 ms for videophones) high data rates, e.g., MPEG-II DVD quality:

average video rate of 3.5 Mbps average transport stream of 4 – 8 Mbps (video, audio, headers, error

protection) max. total video data rate of 10.08 Mbps max. user rate of 11.08 Mbps (all included like control signals)

Network real-time transport of contiguous media data TCP fairness mobility …



Technical Challenges Servers (and proxy caches)

storage real-time retrieval of contiguous media streams, e.g.:

o 4000 movies * 90 minutes * 15 Mbps (HDTV) = 40.5 TBo 2000 CDs * 74 minutes * 1.4 Mbps = 1.4 TB

I/O many concurrent clients expecting continuous playout of

DVD-quality video (~4Mbps)o disk: Seagate X15 - ~400Mbpso network: Gb Ethernet (1Gbps) o bus(ses): PCI 64-bit, 133Mhz (8Gbps)

computing in real-time encryption adaptation transcoding …



Media-on-Demand (MoD) Systems Classification parameters

interaction structure of movies presentation form

Common directions analog digital media distribution interaction media broadcasting media multicasting personalized media linear movies branched movies variable movies

Evolution broadcast - traditional, no user control near MoD - same media distributed in intervals (pay-per-view) true MoD - full user control, VCR capabilities,

bidirectional



Television (Broadcast)

sender

ch

an

nels

time

• analog or digital• traditionally, one program per channel

analog use frequency division multiplexing only digital may additionally use time division multiplexing inside one frequency (several programs per channel)

• linear movies

receiver(s)



Near Video-on-Demand (NVoD)

sender

ch

an

nels

time

• analog or digital broadcasting• one program over multiple channels• time-slotted emission of the program• linear movies

receiver(s)



(True) Video-on-Demand (VoD)

sender

movie

s

time

• digital uni- or multicasting• control channels• linear movies

receiver(s)



Comparison: NVoD vs. TVoD

NVoD TVoD

Response delayed immediate

Servicesprovide some flexibility compared to traditional

TV

improves video rentals:any video, any time

Costscheap:

more clients at lower price compared to TVoD

expensive:existing infrastructure often

not cost-competitive for households

(need to be equal to rentals)

Suiteddigital broadcast(ViaSat, TV1000)

specific (smaller) environments

(hotels, airplanes, …, cities,…)



“Interactive Vision”

sender

movie

time

• digital uni- or multicasting• control channels• fixed non-linear movies

receiver(s)



“Cyber Vision”

sender

time

• digital uni- or multicasting• control channels• variable non-linear “movies”, e.g.,

- games, virtual reality, …

receiver(s)



Application Classification Overview

movie

str

uctu

re

linear

branched

variable

interaction

unidirectional

bidirectional

pre

senta

tion

form

analog

digitalVoD

TVNVoD

Interactive Vision

Cyber Vision

HDTV

VIDEO

Media (Video) on Demand



Challenges VoD in LANs is solved: OVERPROVISIONING works

established in studio business established in hotel/hospital/plane/… business

VoD in WANs goals:

network-based distribution of media content to consumers bring control to users

assumptions: overprovisioning of resources will NOT work no central control of delivery system

programs: need for interoperability – not from a single source need for co-operative distribution systems

amount of data: estimated 65000 movies made in 1995 260 TB MPEG-2 data additionally, data from TV-series, sport clips, news, …

historically: much attention as “interactive TV (ITV)” some years ago many (not successful) field trials now: interest turned to Internet-based systems



ITV Network Architecture Approaches

WAN backbones SONET ATM

Local distribution network ADSL (asymmetric digital subscriber

line) FTTC (fiber to the curb) FTTH (fiber to the home) HFC (hybrid fiber coax) (=cable

modem) …

Internet WAN Diffserv over MPLS

(multi-protocol layer switching) point-to-point Gbps ethernet

Internet local IP over the old distribution networks

ATM / SONET

backbonenetwork

wireless

cable

telephone

ADSLInternet based systems



Concerns: Internet-Based VoD Systems

Can technical problems be mastered? broadband communication to every home user-friendly end systems server technology

Market success? what prices will consumers accept?

high equipment (HW & SW) costs data costs

will it be competitive? to existing TV programs to video rentals

what is the consequence of ITV field trials in USA and Europe? no big success only a few private consumers willing to participate trials were cancelled or switched from TV-based to PC-based

platforms



Driving Forces Hardware/software (IT) companies

computer (e.g., IBM, HP, Sun, Microsoft, …) consumer electronics (e.g., Sony, Philips, …)

Network companies telephone (e.g., Telenor, Telia, BT, AT&T, …) cable TV (e.g., UPC, Time-Warner Cable, …)

Content companies media

movies (e.g., Time-Warner, Disney, …) TV programs (e.g., NRK, TV2, TV3, …) hyper media information bases (e.g., Springer, Bertelsmann, …)

home-shopping (e.g., Amazon, Ebay, …) video games (e.g., Nintendo, Sega, Microsoft, …)



VoD Deployment Status – I Digital Video Broadcast (DVB)

no VoD cable, antenna, or satellite broadcast some NVoD scheduling approaches (e.g., TV 1000)

Digital Audio-Visual Council (DAVIC) defines interfaces only no standardization of algorithms for interoperation closed project

Internet Engineering Task Force (IETF) currently no large-scale, wide-area video

distribution sporadic use of cooperative web caching starting AV caching considerations defines protocols and inspires interoperability

testing

Broadcast world suited for Near VoD

Broadcast world suited for True VoD

Internet world suited for True VoD

MPEG 4 suited for

Interactive Vision



VoD Deployment Status – IIDATA CONTROLIE

TF

RTP: Real-Time Protocol package & timing information for

transfer profiles for each encoding format

RTCP: RTP Control Protocol report exchange, allows tuning at the sender

RTSP: Real-Time Streaming Protocol controls: setup, teardown, start, stop

SDP: Session Description Protocol carried by RTSP encoding information, timing, meta info

DA

VIC

MPEG-2 Transport: Moving Pictures Expert Group defines encoding format, packaging, timing, scaling, error correction requires other means of addressing

DSM-CC: Distributed Storage Media- Command & Control MPEG-2 substandard: addressing, setup, teardown, start, stop, … independent standard: complete management standard (usually ignored by industry)



VoD System Architecture

backbonenetwork

local distribution

network

local distribution

network

local distribution

network



VoD Storage Hierarchy Use hierarchies

Popularity of movies:not all are equally popular – most request directed to only a few (Zipf distribution)

Straight forward hierarchy: popular videos replicated

and kept close to clients locality vs.

communication vs. server costs

end-systems

local servers

master servers

regionalservers

completeness of available content



VoD Components Servers

Networks backbone local networks

Intermediate nodes routers proxy cache servers

End-systems

PCs TV sets with set-top boxes

Traditional Server Machine

Internals



General Operating System Structure and Retrieval Data Path

file systemcommunication

system

application

user space

kernel space



Example: Intel Hub Architecture (850 Chipset) – I

CPU socket

RDRAM connectors

PCI connectors

I/O Controller Hub

Memory Controller Hub

Intel D850MD Motherboard:

system bus

RDRAM interface

hub interface

PCIbus



Pentium 4Processor

registers

cache(s)

Example: Intel Hub Architecture (850 Chipset) – II

I/Ocontroller

hub

memorycontroller

hub

RDRAM

RDRAM

RDRAM

RDRAM

PCI slots

PCI slots

PCI slots

system bus(64-bit, 400/533 MHz

~24-32 Gbps)

hub interface(four 8-bit, 66 MHz

2 Gbps)

PCI bus(32-bit, 33 MHz 1 Gbps)

RAM interface(two 64-bit, 200 MHz ~24 Gbps)

network card

disk

file system

communication system

application


system

application

disk network card

Note:these transfers only show data movement between sub-systems and not the commands themselves. Additionally, data touching operations within a sub-system will require that data is moved from memory and to the CPU, e.g.: - checksum calculation - encryption - data encoding - forward error correction



POWER 4 chip

Example: IBM POWER 4

PCI slots

PCI slots

network card

disk

file system

communication system

application


system

application

disk network card

CPU

L1

CPU

L1

core interface switch

L2

fabric controller

GXcontroller L3 controller

remote I/O(RIO)

bridge

PCI host bridge

PCI host bridge

PCI-PCI bridge

PCI-PCI bridge

L3

RAM

RAM

RAM

memorycontroller

GX bus(two 32-bit, 600 MHz ~35 Gbps)

PCI busses(32/64-bit, 33/66 MHz 1-4 Gbps)

(four 6

4-bit, 400 M

Hz ~95 Gbps)

RIO bus(two 8-bit, 500 MHz ~7 Gbps)

(four 6

4-bit, 400 M

Hz ~95 Gbps)

(eight 32-bit,

400 MHz

~95 Gbps)

Note:Again, data touching operations add movement operations



Server Internals Data retrieval from disk and push to network

buffer requirements bus transfers CPU usage

concurrent streams (can be merged??) storage (disk) system:

scheduling – ensure that data is available in time block placement – contiguous, interleaving, striping

…

Stable operations: redundant HW multiple nodes

Much more, e.g., caching/prefetching, admission control, …

Video Server Structure



IBM TigerShark

HP, DEC, Novell, …switch

Video Server:Server Components & Switches

Internal content directory: External content directory:

storage device

network attachment

content directory/memory management

file system

storage management

controller

switch

switch

switch

switch

switch

[Tetzlaff & Flynn 94]

switch

switch

switch

switch

switch

IP, …

RPC in application, …

NFS, …

AFS, CODA, …

distributed OS, …

Disk arrays (RAID), …



Video Server:Simple General Server Architecture

storagesubsystem

networksubsystem

processor subsystem

dataserver

applicationserver

controlserver

clie

nts

Storage subsystem:• stores data• different devices

Network subsystem:• transmit MM data

Processor subsystem:• executing part• management and operations

Application server: • user interface• billing• content database• user database• service gateways

Control server: • administrator• admission control• optimization

Data server: • data delivery• “specialized file system”• buffer manager• data importer/exporter

datacontrol

[Sitaram & Dan 00]

A similar architecture is used in SGI/Kassena Media Base & IBM VideoCharger



Video Server:Directory Access & Data Retrieval

Two-step retrieval: “problem”:

resource management

Request redirection: “problem”:

client gets data from another machine

Network

data

content

Network

data

content



Pull model: client sends several requests deliver only small part of data fine-grained client control favors high interactivity suited for editing, searching,

etc.

Push model client sends one request streaming delivery favors capacity planning suited for retrieval,

download, playback, etc.

server client

server client

Video Server:Directory Access & Data Retrieval



Video Server:Server Topology – I

Single server easy to implement scales poorly

Partitioned server users divided into groups content : assumes equal groups location : store all data on all

servers load imbalance

Network

Network

Network



Video Server:Server Topology – II

Externally switched servers use network to make server pool manages load imbalance

(control server directs requests) still data replication problems (control server doesn’t need to

be a physical box - distributed process)

Fully switched server server pool storage device pool additional hardware costs e.g., Oracle, Intel, IBM

Network

data

data

data

control

Network

data

data

control

I/Oswitch



Video Distribution Server:Typical In the Internet Today

Push systems(pull in video editing/database systems)

Traditional (specialized) file systems – not databases – for data storage

No in-band control (control and data information in separate streams)

External directory services for data location(RTSP/control server + data pump)

Request redirection for access control

Single stand-alone servers (fully) switched servers

Server Examples



Video Server Components Researchincoming

resolve request

control/application

server

data server

content directory

delivered resolution

storage device

network attachment

memory management

file system

storage management

controller

delivered data

network attachment

incomingdata request

IBM WatsonTetzlaff, Kienzle, …



Video Server Components Research

data server

storage device

network attachment

memory management

file system

storage management

controller

delivered data

incomingdata request

Fall, Druschel, Pai, Silberschatz, Özden, Buddhikot, Miller, …

Shenoy, Nirajan, Silberschatz, Özden, Martin, Haskin, …

Goyal, Vin, Peterson, Tetzlaff, Kienzle, …

Härtig,Sitaram,Dan,Nahrstedt,Steinmetz,Klas,Shulzrinne,Coulson,Seltzer,Rangan,Zhang,Hutchinson,…

Ghandeharidzadeh, Zimmermann, Haskin, …

Jones, Nieh, Chen, Berson, Reddy, Ghandeharidzadeh, Zimmermann, Haskin, …



Video Server “Product Status”1) Real server, VXtreme, Starlight, VDO, Netscape Media Server,

MS Media Server, Apple Darwin1) Real server, VXtreme,VXtreme, Starlight, VDO, Netscape Media Server,VDO, Netscape Media Server,

MS Media Server, Apple Darwinuser level server

standardOS

all standard HW

RTPRTSP

2) IBM Mediastreamer, Oracle Video Cartridge, N-Cube

2) IBM Mediastreamer, IBM Mediastreamer, Oracle Video Cartridge, N-Cubeuser level layer

scalable, RT-aware OS,RT OS, or

OS derivation

custom/special HW

ATM, analogDSM CC, private

3) SGI/Kassena Media Base, SUN Media Center, IBM Video Charger

user level server

RTextensions

selected standard HW

RTPRTSP

standardOS

MMFS

3) SGI/SGI/Kassena Media Base, SUN Media Center,SUN Media Center, IBM Video Charger



userkernel

server

Real Server User space implementation

one control server several protocols several versions of data

in same file adapts to resources

Several formats, e.g., Real’s own MPEG-2 version with

“stream thinning”(dropped with REAL )

Does not support Quality-of-Service load leveling

track

1

track

2

index

request

IP

UDP

RTP/RTCP

Real’sprotoc

ol

TCP

1 23

back

pre

ssure

feedback



VSDwith EDF

IBM Video Charger May consist of one

machine only, or … … several IBM’s

Advanced Interactive eXecutive (AIX) machines

Servers control data

Lightly modified existing components

OS AIX4/5L virtual shared disks

(VSD)(guaranteed disk I/Os)

Special components TigerShark MMFS

(buffers, data rate, prefetching, codec, ...)

stream filters, control server, APIs, ...

control

AIX

SP2

cro

ssbar

swit

ch

specificcontrol server

RTSP

RTPencryptfilter

TigerSharkMMFS

VSD

UDP

IP

distributed computing environment RPC

video stream API

mlib API

DESCRIBE

SETUP

PLAY

TEARDOWN



IBM Mediastreamer Version of Video Charger

failed project to guarantee MPEG-2 over ATM and Cable

similar machine setup as Video Charger (machine cluster)

special HW: SCSI controller MPEG-2 ATM or analog cable out moved to Video Charger

Unlike Video Charger, Mediastreamer runs on old IBM machines due to special HW

Special components TigerShark control server APIs special board

control

AIX

SP2

cro

ssbar

swit

ch

specificcontrol server

RTSP

distributed computing environment RPC

video stream API

mlib API



n4x media hubs:• Intel 860 Chip Set • Intel 1.5 GHz Xeon CPU• Up to 2 GB Rambus Memory• Five 64 bit 66Mhz PCI slots• “Special” PCI slot (HIB board)• nHIO hypercube I/O

request

nCUBE Original research from Cal Tech (‘83)

One server scales from 1 to 256 machines, 2n, n [0, 8], using a hypercube architecture

Why a hypercube? video streaming is a switching problem hypercube is a high performance scalable switch no content replication and true linear scalability integrated adaptive routing provides resiliency

Highlights one copy of a data element scales from 5,000 to 500,000 clients exceeds 60,000 simultaneous streams 6,600 simultaneous streams at 2 - 4 Mbps each

(26 streams per machine if n = 8)

Special components boards with integrated components TRANSIT operating system n4 HAVOC (1999)

Hypercube And Vector Operations Controller ASIC-based hypercube technology

n4x nHIO (2002) nCUBE Hypercube I/O controller (8X price/performance)

memory PCI bus

configurableinterface

8 hypercube connectors

vector processorSCSI ports



nCUBE: Naturally Load-balanced

Content striped acrossall disks in the n4x server

Video Video StreamStream

Disks connected to All MediaHubs Each title striped across all MediaHUBs Streaming Hub reads content

from all disks in the video server

Automatic load balancing Immune to content usage pattern Same load if same or different title Each stream’s load spread over all

nodes

RAID Sets distributed across MediaHubs Immune to a MediaHUB failure Increasing reliability

Only 1 copy of each title ever needed Lots of room for expanded content,

network-based PVR, or HDTV content



nCUBE: Sample deployments & Cost

TWC Los Angeles – South Bay HE

Monterey Park CA 2040 streams from 18 n4

MediaHUBs

West Covina CA 840 streams from 8 n4

MediaHUBs

Pasadena CA 840 streams from 8 n4

MediaHUBs

Glendale CA 1320 streams from 12 n4

MediaHUBs

Long Beach CA 1320 streams from 12 n4

MediaHUBs



Small Comparison

Real Video Charger nCUBE

standard HW selected HW special HW

each machine its own storage, or

NFS

shared disk access,

no replication

shared disk access,

no replication

single OS image cluster machines using switch

cluster machines using wired cube

user space server user space server and loadable

kernel modules

server in both kernel and user

space



Client-Server

backbonenetwork

local distribution

network

local distribution

network

local distribution

network

Traditional distributed computing Successful architecture, and will

continue to be so (adding proxy servers) Tremendous engineering necessary to

make server farms scalable and robust



Peer-to-Peer (P2P)

backbonenetwork

local distribution

network

local distribution

network

local distribution

network

Really an old idea - a distributed system architecture No centralized control Nodes are symmetric in function

Typically, many nodes, but unreliable and heterogeneous



Promise[Hafeeda et. al. 03]

Receiver

active sender

active sender

standby senderactive sender

Thus, Promise is a multiple sender to one receiver P2P media streaming system which 1) accounts for different capabilities, 2) matches senders to achieve best quality, and 3) dynamically adapts to network fluctuations and peer failure

Each active sender: • receives a control packet specifying which data segments, data rate, etc.,• pushes data to receiver as long as no new control packet is received

standby sender

The receiver: • sends a lookup request using DHT• selects some active senders, control packet• receives data as long as no errors/changes occur • if a change/error is detected, new active senders may be selected



SplitStream

Source: full quality movie

Stripe 2

Stripe 1

[Castro et. al. 03]Each node: • joins as many multicast trees as there are stripes (K)• may specify the number of stripes they are willing to act as router for, i.e., according to the amount of resources available

Each movie is split into K stripes and each stripe is multicasted using a separate three

Thus, SplitStream is a multiple sender to multiple receiver P2P system which distributes the forwarding load while respecting each node’s resource limitations, but some effort is required to build the forest of multicast threes



Summary Multimedia applications and challenges

Media (Video) on Demand

Machine internals

Video server structures

Video server examples

Introduction and Architectures 23/8 - 2004 INF5070 – Media Storage and Distribution Systems:

Documents

Transcript of Introduction and Architectures 23/8 - 2004 INF5070 – Media Storage and Distribution Systems: