Rate-Distortion Optimized Layered Coding with Unequal Error Protection for Robust Internet Video

Michael Gallant, Member, IEEE, and Faouzi Kossentini, Senior Me

mber, IEEE

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, MARCH 2001

Outline Introduction Background

Layered coding Packetization Scheme Error-Concealment Method Prioritization Approach

Proposed Method Statistical Distortion Measure Rate-Distortion Optimized Mode-Selection algorithm

Experimental Results Conclusion

Introduction(1/2) How to facilitate video communications for differe

nt networks Layered coding

Network congestion and buffer overflow lead to packets being delayed and discarded

Retransmission protocols Forward-error correction(FEC) techniques

Performing error concealment through post-processing

Introduction(2/2) This paper present an effective framework fo

r video communication

based on the layered coding, packetization, error concealment, and packet prioritization

Providing a rate-distortion optimized mode-selection algorithm Want to select the coding mode for each block

BackgroundLayered Coding Layered coding, as supported in H.263+ allo

ws the source for prediction to be selected at the macroblock level The corresponding reconstructed base-layer mac

roblock A motion compensated macroblock from the pre

vious enhancement-layer reconstruction The linear interpolation of the two

This paper employ a fully standard-compliant H.263+ layered video-encoding algorithm

BackgroundPacketization Scheme(1/3)

A typical packet consists of header information for IP, UDP, RTP, and RTP payload

Header requires about 40 bytes/packet

Large packet sizes to reduce packetization overhead

The maximum size the packet on the internet should be 1500 bytes

BackgroundPacketization Scheme(2/3) Pay-load data are from one row of ma

croblocks(GOB) per packet to one entire coded frame per packet Good decoder error-concealment Reduce over head

We can interleaving even and odd GOBs into separate packets

BackgroundPacketization Scheme(3/3)

BackgroundError-Concealment Method(1/4) Packet loss must be de detected

Using the sequence number

Resynchronization markers can provide spatial error-resilience. For example ： H.263+ GOB header

Error concealment in video communications—spatial and temporal domain This paper uses the median estimate for motion

compensation

BackgroundError-Concealment Method(2/4)

This paper generating two packets per coded frame for QCIF resolution and four packets per coded frame for CIF resolution

BackgroundError-Concealment Method(3/4) For layered scenarios, we can use base-layer infor

mation to estimate enhancement layer information

Previous enhancement layer can also used for error concealment

Base layer is inter-coded -> employ median estimator from enhancement layer

Base layer is intra-coded -> using the available base-layer reconstruction

BackgroundError-Concealment Method(4/4)

BackgroundPrioritization Approach(1/2)

For layered coding, base layer has higher priority than enhancement layer

Using unequal protection to achieve transport priortization (n,k) code, k data packet, n-k parity packet

BackgroundPrioritization Approach(2/2)

Proposed method Want to select the coding mode for ea

ch block Optimal allocation of bit rate must co

nsider Source coding elements Channel coding elements Also introduce error-resilience ability

Proposed methodStatistical Distortion Measure Distortion occur from Packet loss and error propag

ated via motion compensation

n

kni

iniloss pp

in

n

n

iP

1)1()(

1))(1(1),( kNpredlosspredcorrupt lPktlP

)mode,,,()mode,,,(),()mode,,,(1

9

1ktlrDktlrwktlPtlbD predc

N

k rpredpredcorruptcurrc

Residual packet loss probabilities

(n,k)

A macroblock lost probability

Prediction layer

Statistical distortion

Macro block

Current layer

Given frame

Relative weights

Proposed methodRate-Distortion Optimized Mode-Selection algorithm

For base layer, four coding modes ： skipped, inter, intra and inter4v

For enhancement layer, five coding modes ： skipped, inter-forward, inter-upward, inter-bidirectional, and intra mode

)),,()mode,,,()(()mode,,,(

)mode,,,())1,(1(),,(

2

1mode

sccurrscurrpred

currpredcorruptcurr

RknRtlbRltlbD

tlbDtlPtlbJ

Quantization distortion

Distortion for prediction from a corrupted

Coding mode

Quantization level used to control the bit rate

Source coding rate

Channel coding rate

Experimental results(1/4)

Experimental results(2/4)

Experimental results(3/4)

Experimental results(4/4)

Conclusion Providing an effective framework for r

obust internet video communications based on the principle of layered coding with transport prioritization

A rate-distortion optimized mode-selection algorithm