MPEG-4: Multimedia Coding Standard Supporting Mobile Multimedia System

-MPEG-4 Natural Video Coding

April, 2001

MPEG-4 Natural Video Coding

MPEG-4 Video Coding BasicsVideo Coding DetailsBinary Shape CoderMotion CoderVideo Texture CoderScalable Video Coding

MPEG-4 Video Coding Basics

Semantic segmentationClass hierarchyPrediction structureLogical structureDetail structure

Semantic Segmentation of a picture into VOPs

Video ObjectVO’s instances – Video Object PlanesVOP described by texture variations and shape representation

Class hierarchy for structuring coded video Data

Video SessionVideo ObjectVideo Object LayerGroup of Video Object PlanesVideo Object Plane

An prediction structure using I-,P-,B-VOPs

I - Intra Picture (I-VOP)P - Predictive Picture (P-VOP)B – Bidirectionally Predictive Picture (B-VOP)

Logical structure of VO based codec of MPEG-4 Video

Detailed structure of video objects encoder

Binary Shape Coder

Generating binary alpha planesCompletely transparentCompletely opaquePartially transparentCoding binary alpha planesLossy or losslessly depends on thresholdselecting a maximum subsampling factor on 16*16 binary alpha that results in just acceptable distortion

Binary Shape Coder (Cont’)

Assigning mode to binary alpha planes1. Zero differential motion vector and on inter shape update2. Nonzero differential motion vector and no inter shape update3. Transparent4. Opaque5. Intra shape6. Zero differential motion vector and inter shape update7. Nonzero differential motion vector and inter shape updateDepending on the coding mode and whether it is an I-, P- or B-VOP, a variable length codeword is assigned identifying the coding type of the binary alpha block

Motion Coder

Motion Coder consists of:-Motion Estimator For P-VOPs, compute motion vectors using the current VOP and temporally previous reconstructed VOP available from the previous reconstructed VOPs store For B-VOPs, compute motion vectors using the current VOP and temporally previous reconstructed VOP from the previous reconstructed VOP store, as well as, the current VOP and temporally next VOP from the next reconstructed VOP store

Motion Coder (cont’)

-Motion Compensator Use the motion vectors to compute motion compensated prediction signal using the temporally previous reconstructed version of the same VOP

-Previous/Next VOPs Store-Motion Vector (MV) Predictor and Coder

Generate prediction for the MV to be coded

Video Texture Coder

The Texture Coder codes the luminance and chrominance variations of blocks forming macroblocks within a VOP.The blocks that lie inside the VOP are coded using DCT codingThe blocks that lie on the VOP boundary are first padded and then coded using DCT codingThe remaining blocks are not coded at all

Scalable Video Coding

Scalability of video is the property that allows a video decoder to decode portions of the coded bistreams to generate decoded video of quality commensurate with the amount of data decoded.Temporal ScalabilitySpatial Scalability

Temporal Scalability

The base layer is shown to have one-half of the total temporal resolution to be codedThe base layer is coded independently as in normal video codingThe enhancement layer uses B-VOPs that use both, an immediate temporally previous decoded base layer VOP as well as an immediate temporally following decoded base layer VOP for prediction

Spatial Scalability

The base layer is shown to have one-quarter resolution of the enhancement layerBase layer is coded independently as in normal video codingThe enhancement layer mainly uses B-VOPs that use both, an immediate previous decoded enhancement layer VOP as well as a coincident decoded base layer VOP for prediction