Internet Video

By Mo Li

Video over the Internet

Introduction Video & Internet: the problems Solutions & Technologies in use Discussion

 

Video & Internet: the problems

Video Bandwidth needs Internet Delays

Packet loss Delay jitter Congestion control

Internet Delays Heavy traffic load and internal transmission problems can cause delays.using RTP to handle the time-related media delivery issues.video has high tolerance to packet losses.

Packet loss

Pack losing is inevitable ALF --A well-established principle for loss-resilient transmission of video over the Internet.

Scalable or layered coding base layer enhancement layers

Delay Jitter

The queuing delay may vary significantly during a session creating delay jitter. Play out buffers

initial minimum buffer size. change the buffer size in real time

Congestion Control

A vital parameter is the bottleneck bandwidth Rate control: control encoding rate in a predictable way in adapting its output network condition.No globally accepted solution.

Solutions & Technologies in use

Video stream properties for network useVideo compression

Video stream properties for network

use Natural breakpoints for packetization Adjustable packet sizes No bit level shifts during packetization Well defined high-priority information Flexible rate control Ease of transcoding Layered coding Resilience to error propagation

Video compression Introduction Video compression techniques–

RTP Payload formats -- H.261 -- Motion-JPEG -- MPEG-1 & MPEG-2 -- H.263+

H.261Coding is organized as a hierarchy of groupings:

Frame (picture) level : specifies the delay from the previous frame, the image format, and various indicators.GOB level: specifies the GOB number and the default quantifier that will be used for the MBs.MB level: specifies which blocks are present and which did not change, and optionally a quantifier and motion vectors.

MB is taken as the unit of fragmentation

H.261 data follow the RTP header 0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . RTP header . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | H.261 header | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | H.261 stream ... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

H.261 payload specific header 

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |SBIT |EBIT |I|V| GOBN | MBAP | QUANT | HMVD | VMVD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

♪ Start bit position (SBIT): 3 bits ♪ End bit position (EBIT): 3 bits ♪ INTRA-frame encoded data (I): 1 bit ♪ Motion Vector flag (V): 1 bit

♪ GOB number (GOBN): 4 bits ♪ Macro block address predictor (MBAP): 5 bits♪ Quantizer (QUANT): 5 bits ♪ Horizontal motion vector data (HMVD): 5 bits ♪ Vertical motion vector data (VMVD): 5 bits

Enhancing the packet loss resilience

3 ways to mitigate packet loss (1) Use only INTRA-frame encoding and MB level conditional replenishment. (2) Adjust the INTRA-frame encoding refreshment rate according to the packet loss observed by the receivers. (3) The fastest way is to request an INTRA-frame coded image refreshment after a packet loss is detected.

Use of optional H.261-specific control

packets

Two H.261-specific RTCP control packets (RTP’s companion control protocol).

Full INTRA-frame Request Negative Acknowledgement

The purpose: speed up refreshment of the video.Caution of usage: Those control packets are sent directly via unicast from the decoder to the coder.effective only when the number of receivers is small.

H.261 One of the earliest RTP payload formats for packet video transport over the internet. Demonstrates a successful approach to packetizing a video payload that was not originally designed for packet networks. It is a constant-bit-rate encoding and effective only when the number of receivers is small.

Motion JPEG

What is Motion-JPEG Review of JPEG Payload format

Overview of JPEG

The image =>one or more frames=> one or more scans=> one to four components. (3 color components). The scan contains an ordered sequence of MCUs. Each MCU defines the image data for a small rectangular block of the output image.Each frame and scan is preceded with a header containing optional definitions for compression parameters .

Three Compressed data formats Interchange format:

Contains definitions for all the tables used by the entropy-coded segments.

JPEG File Interchange Format (JFIF) is a standard that provides extra information.

Abbreviated format: Might omit some assuming they were

defined out-of-band or by a "previous" image.

Table-specification format

Restart markers in the JPEG

Denote a point where the decoder should reset its state. restart interval: a block of data containing a restart marker followed by some fixed number of MCUs.

JPEG over RTP

The RTP payload format for MJPEG: Optimized for—JPEG codec parameters change rarely from frame to frame. Assumes JPEG’s sequential DCT operating mode and single-scan, interleaved images. RTP/JPEG data format: abbreviated format

RTP/JPEG Packet Format

Each packet contains RTP header Special JPEG header

Main JPEG header A Restart Marker header and/or Quantization Table header

Main JPEG header  0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- | Type-specific | Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Q | Width | Height | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+

Main JPEG header Type-specific: 8 bits  Interpretation depends on the value of the type field. Fragment Offset: 24 bitsThe offset in bytes of the current packet in the JPEG frame data. Type field: 8 bitsDefines the abbreviated table-specification and additional JFIF-style parameters not defined by JPEG data stream.Determines whether restart markers are present. Q field: 8 bitsDefines the quantization tables. Determines whether a Quantization Table header appears. Width & Height : 8 bits/ eachEncodes the width and height of the image in 8-pixel multiples.

Restart Marker header 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Restart Interval |F|L| Restart Count | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Restart Interval field: 16 bitsSpecifies the number of MCUs that appear between restart markers.F (first) and L (last) fields: 1 bit/ eachRestart Count bit : 14 bitsThe value of F, L and Restart Count bit can be used to indicate that a receiver MUST reassemble the entire frame before decoding it.

Discussions

Codecs designed for the Internet :greater scalabilitylower computational complexitygreater resiliency to network losseslower encode/decode latency

The network properties must be considered jointly.