Multimedia Systems

As Presented by:Craig Tomastik

What is Multimedia?

• Multimedia consists of continuous-media, usually audio and video files.

• Examples of multimedia include: – MP3 audio files– DVD movies– Video clips, webcasts, and webcams sent over the

internet• Today, multimedia is not only sent to desktop

computers, but also PDA’s and cell phones.

Delivery of the media• To view the media correctly, the files must be

accessed at a specific rate.• For example, frames are a series of images

displayed rapidly used to represent video.• The faster the frames are displayed, the

smoother the video appears.• Video files must be accessed at a rate that

matches the rate at which it is being displayed.• This type of data is known as continuous-media

data.

Delivery (Continued)

• Clients receive the multimedia data one of two ways:– Local file systems (Local Playback)– Remote servers (Streaming)

• Local playback includes watching DVD movies on the computer or listening to an MP3 player.

• These multimedia files are stored as a file on the local system and are played back from the system.

Delivery (Continued)

• Multimedia files stored on servers are delivered to clients via streaming.

• Two streaming techniques exist:– Progressive download– Real-time streaming (Which has two variations):• Live Streaming• On-demand streaming

• Random access is not always achievable with real-time streaming.

Characteristics of Multimedia Systems

1. Multimedia files can be quite large.2. Continuous media may require high data

rates.3. Multimedia applications are sensitive to

timing delays during playback.

Operating-System Issues

• Systems must guarantee specific rate and timing requirements, also known as Quality of Service (QoS) Requirements.

• These guarantees affect many components in the computer system and influence operating-system issues.

Operating System Issues (Continued)

1. Compression and decoding may require significant CPU processing.

2. Multimedia tasks must be scheduled with certain priorities to ensure meeting the deadline requirements of continuous media.

3. File systems must be efficient to meet the rate requirements of continuous media.

4. Network protocols must support bandwidth requirements while minimizing delay and jitter.

Compression

• Multimedia files are usually compressed from their original form to a smaller form.

• Because of the smaller size, less space is required for storage and the file can be delivered to the client more quickly.

• Compression Ratio- ratio of the original file size to the compressed file size.

Compression (Continued)

• Compressed (encoded) files must be decompressed (decoded) before they can be accessed.

• There are two types of compression algorithms:– Lossless- Compressed files can be restored to its

original form.– Lossy- some of the original data is lost when the

file is decoded.

Motion Picture Experts Group (MPEG)

• A set of file formats and compression standards for digital video.

• Each standard is divided into three layers– Layers 3 and 2 apply to the audio and video portions of the

media file, respectively.– Layer 1 (Systems Layer) contains timing information that

allows the media player to multiplex the audio and video portions for synchronized playback.

• Three major MPEG Standards:– MPEG-1– MPEG-2– MPEG-4

MPEG-1

• Resolution quality (352x240, 30 frames per second, bit rate of 1.5 1.5 Mbps) slightly lower than a VCR.

• MP3 files use Layer 3 of MPEG-1.• Can achieve a compression ratio of 200:1 with

video files.• Typically used to download short video clips

from the internet.

MPEG-2

• Better quality than MPEG-1.• Used to compress DVD movies & digital

television, including high-definition television.• Idenifies levels (resolution of the video) &

profiles (video quality). The higher level of resolution & better the quality, the higher the required data rate.

• Typical bit rates range from 1.5 to 15 Mbps.• Generally used for local playback.

MPEG-4

• Most recent standard.• Used to transmit audio, video, & graphics

(including 2-D & 3-D animation layers).• Provides scalable level of quality.• MPEG-4 audio & video files can be delivered

to wireless devices (including handheld computers, PDAs, and cell phones).

Requirements of Multimedia Kernels

• Periodic Processes- tasks that request data at constant intervals (periods).

• If a part of the data file is not delivered within a specified deadline, this part of data is omitted from the stream.

• (Possible example).

Quality of Service Levels

1. Best-effort service- system makes best effort to meet requirements, but no guarantees are made.

2. Soft QoS- Traffic streams are treated differently, with some streams given higher priority, but no guarantees are made.

3. Hard QoS- Quality-of-service requirements are guaranteed.

Parameters of QoS

• Throughput- total amount of work done during a certain interval (for multimedia applications, throughput is the required data rate).

• Delay- elapsed time from when a request is first submitted to when the desired result is produced.

• Jitter- Delays that occur during playback of the stream.

• Reliability- refers to error handling during transmission & processing of continuous media.

QoS (Continued)

• QoS may be negotiated between client and server.

• Admission control admits a request for service only if there are enough resources to satisfy the request.

• Simple admission control accomplished by implementing Semaphores and Resource reservations.

• Each resource, when implementing resource reservations, is assigned a Resource manager.

CPU Scheduling

• Continuous media requires that data be delivered to a client by a certain deadline. Data that doesn’t arrive by the deadline is unusable.

• Hard real-time scheduling guarantees a critical task will be serviced in a guaranteed period of time.

• Static priority- priority remains unchanged.• Dynamic priority- priorities can change over

time.

Disk Scheduling

• Disk scheduling must be optimized to satisfy the two main constraints of continuous-media files:1. Timing deadlines2. Rate requirements

Earliest-Deadline-First Scheduling

• Orders requests in a queue based on their deadlines.

• Requests with the closest deadlines are serviced first.

• A consequence of this approach is that higher seek times are incurred.

SCAN-EDF Scheduling

• A combination of SCAN & EDF scheduling.• Uses EDF ordering, but services requests using

SCAN order.• Batching may allow requests with relatively

close deadlines to be serviced together.• Reordering requests in a batch must not

prevent the request from being serviced by its deadline.

Network Management• Currently, computer networks do not provide

network protocols that ensure data delivery with timing requirements.

• Network traffic can affect the transmission of data across the network. Any timing issues must be synchronized between the server and client.

• Real-time transport protocol (RTP)- an Internet standard for delivering audio & video.

• RTP does not provide QoS guarantees, but provides features that remove jitter caused by delays and network traffic.

Deliveries from a Server to the Client

• Unicasting- Server delivers content to a single client. If multiple clients are present, a separate unicast is established for each client.

• Broadcasting- Servers delivers content to all clients. Only used in Local Area Networks and is not possible over the public Internet.

• Multicasting- Server delivers content to a group of clients who indicate they wish to receive the content.

Real-time Streaming Protocol

• Two approaches to delivering media from a server:1. HTTP Protocol (Stateless)• The web browser requests a metafile which is then

opened in a media player. The media player will then request the media stream from the server. HTTP Protocol delivers the stream back to the media player.

2. RTSP Protocol• Same as above, but is not stateless, which allows the

client to pause and seek to random positions in the stream during playback.

RTSP Protocol Commands• Setup- Server allocates resources for a client

session.• Play- Server delivers a stream to a client session

established from a SETUP command.• Pause- Server suspends delivery of a stream but

maintains the resources for the session.• Teardown- Server breaks down the connection &

frees up resources allocated for the session.• The server can switch between the Init, Ready, &

Playing states when these commands are sent by the client.