Audio & Video Compression and its Application in Consumer ... › acous ›...

Alain Bouffioux - Philips Hasselt 1 A/V compression applied to Consumer Products - ULGDigital Video System - Products & services, Product Innovation AHR-AB00069 - 13/12/00

Audio & Video Compressionand its Application inConsumer Products


Agendau Introduction - The evolution of Audio/Video consumer

products and the role of compression techniques.

u Audio & Video compression principles

u Audio compression

u Video compression

u Audio/Video synchronisation

u The MPEG model and its situation in a communication context

u Application to DVD (Digital Versatile Disc)

u Application to DVB (Digital Video Broadcasting)

u Conclusion


u Introduction - The evolution of Audio/Video consumerproducts and the role of compression techniques.


u Audio compression

u Video compression





u Conclusion

Agenda


The evolution of our products (1/5)

ConsumerConsumerComputerComputer

CommunicationCommunication

CDCDDCCDCC

CD-iCD-i

DVDDVD

STBSTB


The evolution of our products (2/5)u The STB (Set-Top-Box) as the link between the home and the

world-wide information infrastructure.

STB Home NetworkWorld-wide

communicationinfrastructure


The evolution of our products (3/5)u The STB (in home) as the gateway to various services.

Local Server provides 2 kind of services:

∗ BroadcastAnalogue & digital TV, NVOD, PPV

∗ Point-to-point (Home to local server)Home shopping, VOD, e-mail, Web browsing, PC connection...

Network

Local server

Internet

Local server

Up to 800 homes


The evolution of our products (4/5)u The STB as a key element of the home network

Home NetworkHome Network

Residential Residential GatewayGateway

To telephone NetworkTo telephone Network

To satelliteTo satelliteNetworkNetwork

To cable NetworkTo cable Network

ComputerComputer

DVD DVD JukeboxJukebox

TelevisionTelevision

DiskDiskRecorderRecorder


The evolution of our products (5/5)u 3C Convergence - Progressive

u New products combine all 3 functions

u Products always more and more complex

u Working in group becomes more and more important

u Philips international co-operation(avoiding to reinvent the wheel, concentration on corecompetencies)

u Products have always new features

u Lifetime of products is always shorter

u The DVD will also be provided with the communicationfeature (IEEE1394 bus) for home bus integration.


Compression is one among the various factors that enablemultimedia:

Software (methodology,user interface ...)

International cooperation(interoperability &economy of scale)

Disc capacity (DVD), communicationgoing digital (Modems, xDSL, ATM ...)

Multimedia

Electronics(Memory capacity,clock frequency,µP architecture, ...--> decoding at low cost

Audio/VideoCompression(e.g. CCIR601 vs MPEG)

Factors enabling such evolution





u Audio compression

u Video compression





u Conclusion


Compression in first A/V Products (1)u First Audio/Video products make compression without

knowing it was compression.

u How ?By removal of irrelevancies

u Audio and Video characteristics

Audio VideoSpectral

SensitivityGood Bad

SpatialSensitivity

Bad Good


Compression in first A/V Products (2)u Audio products

From 2 to 7.1 channels are enough to provide the spatialresolution.

u Video productsThree colours (RGB) are enough to provide the spectralresolution.


u Audio: Compression needed in spectral domain

u Bitrate of a stereo audio source (CD-DA encoding)

Sampling frequency : 44.1 kHzStereo16-bit per sampleBitrate = 44100 * 2 * 16 = 1.41 Mbit/sec

Audio waveform (time)

time

The need for more compression (1/5)


u Video: Compression needed in spatial domain

u Bitrate of a video source (CCIR 601 - 50 Hz countries)

25 images per secondYUV coding (Y: luminance - U,V : Chrominance)Y: 8 bit per pixel - U,V: 1 pixel on 2 coded, 8 bit per pixelBitrate = (576*720)*25*16 = 166 Mbit/sec

The need for more compression (2/5)

720 samples

576lines

Video image


The need for more compression (3/5)u Channels availables for AV transmission

∗ Analog television channel (compatibility)Cable (bandwidth = 8 MHz)Satellite (Bandwidth = 30-40 MHz)

⇒ Capacity around 40 Mbit/sec

∗ Compact disc (CD)For 74 min. play time : 1.41 Mbit/sec


The need for more compression (4/5)u MPEG-1 target

(Video-CD : 74 min. constraints)

But quality was judged too poor (about VHS quality)

Compression

Video : 166 Mbit/sec

Audio : 1.4 Mbit/sec

1.4 Mbit/sec


The need for more compression (5/5)u MPEG-2 target

∗ Program stream (DVD)

∗ Transport stream (DVB)

Compression

1 program(video, multichannelaudio, ....)

= motivation for the capacityincrease of the CD (--> DVD)

3-9 Mbit/sec (variable bitrate)(but higher quality than MPEG-1)

Compression

n programs(video, multichannelaudio, ....)

about 40 Mbit/sec (constant bitrate)(DVB-Satellite & DVB-Cable)


Principles of compression (1/2)u Compression (or source coding) is achieved by suppressing

information :

∗ redundant information∗ irrelevant information

u Suppression of redundant information ⇒ lossless compressionexample: PCM to DPCM,DCT

The original signal and the one obtained after encodingand decoding are identical

DecompressionCompressionFc(x,y,t)

Rc kbps Ri < Rc

Fp(x,y,t) = Fc(x,y,t)

Rp = Rc


Principles of compression (2/2)u Suppression of irrelevant information

⇒ lossy compressionExample: bandwidth limitation, masking in audio

The original signal and the one obtained after encodingand decoding are different but are perceived as identical

DecompressionCompressionFc(x,y,t)

Rc kbps Ri < Rc Rp = Rc

Fp(x,y,t) Fc(x,y,t)


Audio DemonstrationFrom “Borderline” Madonna - Stereo - 16 bit/channel

Compression used AAC

Compression

Decompression

Original

-

32kbps

128kbps

64kbps

16kbps

705 kbps


MOS scale (1/2)u Signal distorsion is not a good measure of the performance of a

loosy compression method⇒ an other method is necessary: MOS scale (Mean OpinionScore)

u The five-grade CCIR impairment scale (Rec.562)1(Very annoying), 2(Annoying), 3(Slightly annoying),4(Perceptible but not annoying), 5(Imperceptible)

u Example:Double blind test


MOS scale (2/2)

Compressed

3

1

4

Impairmentscale5

Min value

Sequence2 3

Original

Mean value

Max value

Original signal

Listener answers to :1. Which signal is the original ? 2 or 3 ?2. Grade the other one ?

Exchange box(Random for listener)

Selector(controlled by listener)

Compressed signal


Compression to VBR or CBR

u CBR (Constant Bit Rate) vs VBR (Variable Bit Rate)

u Scene more complex ⇒ Higher bit rate for same qualityu CBR ⇒ variable quality (example : Video CD artefact)u Constant quality ⇒ VBR necessary (e.g.: DVD-Video)

Constantbit rate

Bitrate

Complex

Distorsion

Simple

Constant quality


Video demonstration


The compression trade-offu Compression techniques are still making progress

u Trade-off Complexity/Quality/Bit Rate

u New technique may result in new trade-off

QualityQuality

BitrateBitrate

ComplexityComplexity

MPEG Layer 1MPEG Layer 1



MPEG AACMPEG AACOther TechniqueOther TechniqueSpeech codingSpeech coding





u Audio compression

u Video compression





u Conclusion


Audio compression in MPEG (1/5)u Based on psycho-acoustics

u Compress the bit rate without affecting the quality perceivedby the human ears (based on the imperfection of human ears)

u Removal of irrelevancies

u 4 main principles :∗ Threshold of audibility∗ Frequency masking∗ Critical bands∗ Temporal masking


Audio compression in MPEG (2/5)u Principle 1: Threshold of audibility

⇒ Not all frequency components need to be encoded with thesame resolution. Nr_bit(f) = (signal/threshold)db/6

0.2

Frequency (kHz)

20

0

0.02 0.10.05

100

80

60

40

Sound level(db)

50.5 1 2 2010


Audio compression in MPEG (3/5)u Principle 2: Frequency masking

⇒ Analysis of the incoming signal

0.2

Frequency (kHz)

0

20

0.050.02 0.1

60

40

80

Sound level(db) 100

Masked signal

0.5 1 2 5 10 20

New threshold of audibilitydue to masker signal

Masker signal


Audio compression in MPEG (4/5)u Principle 3: Critical bands

∗ Human ear may be modelled as a collection of narrow band filters∗ Bandwidth of these filters = critical band∗ critical band

(


Audio compression in MPEG (5/5)u Principle 4: temporal masking

⇒ selection of the frame duration for frequency analysis andencoding.

0~(-10msec) 0

Envelope of masker

Pre masking

Level(db)

Post masking

Simultaneous masking

Time~100msec


An enabling tool : the filter bank (1/2)

Digital audio signal, Fs

ENCODING / DECODING

Reconstructed signal, Fs

Synthesis

INTERPOLATION

Analysis

DECIMATION

n subband signals, Fs

Filter bank

n subband signals, Fs

n subband signals, Fs/n

n subband signals, Fs/n

Filter bank


An enabling tool : the filter bank (2/2)u After decimation, same bit rate as original signal, but signal

decomposed in various frequency ranges ⇒ possibility of frequency based compression

u Filter-bank:Aliasing occurs due to decimation

u It exists a class of filter-bank such that aliasing is compensatedin synthesis filter : QMF (Quadrature Mirror Filter) but highcomplexity

u Pseudo-QMF (Polyphase filter bank) is used. Has goodcompromise between computation cost and performances

u Remark : Aliasing may occur if signal in a adjacent band is notreconstructed with an adequate resolution.


The MPEG encoder

Signal to mask level for each subband(quantisation information)

Psychoacousticmodel.

Digital audioinput

Subbandsamples

FilterBank

Ancillary data

Quantisedsamples

Bitallocation

Bitstreamformatting

Encodedbitstream


The MPEG filter banku In MPEG, 32 equal-width subbands are used

u For each subband, necessity to define the maximum signallevel and the minimum mask level.

u BUT, at low frequencies : bandwidth of subbands > critical bands

u ⇒ Necessity to rely on an FFT in order to compensate thelack of frequency selectivity of filterbank at low frequencies


Psychoacoustic model & Bit allocation(1/2)

u An FFT compensates the lack of frequency selectivity offilterbank at low frequencies

u FFT : 512 samples (layer 1) & 1024 samples (layer 2)resolution for layer 1 : Fs/512 < 100 Hz

u A psychoacoustic model based on the FFT computes the signalto mask ratio for each subband (1 bit = 6db)

u Ideally, after allocation, quantisation noise < masking level

u The scale factors are computed for each subband from thefilterbank output (floating point representation of samples)

u The bit allocator adjust the bit allocation in order to meet thebitrate requirement.

u The bitstream syntax is dependent of the MPEG layer (See later)


Psychoacoustic model & Bit allocation(2/2)

Frequency

Signal/MaskLevel(db)

Level(db)

Signal level

Bit allocation= 1 bit

Frequency

Mask level


The MPEG decoderu Decoder is simple (Complexity is at encoder side)

u Remark 1: DCC is MPEG-1 but DCC encoder has no FFT,relies only on power in the 32 subbands ⇒ Higher bit rate (320 kbps) to reach transparent quality

u Remark 2: MPEG specifies bitstream syntax only. Encoderare given for information. Possibility of improvement.

SubbandSamplesrecons- truction

Bitstreamunpacking

Encodedbitstream

Ancillarydata

Subbandsamples

Filterbank

Decoded audio signal


Audio features in MPEGu MPEG1 :

∗ Mono/stereo/dual/joint stereo (Possibility Dolby surround)∗ Sampling frequencies : 32, 44.1 & 48 kHz∗ 3 layers : trade-off complexity/delay versus coding

efficiency of compression

∗ Various bit rate : trade-off quality versus bitrate

u MPEG2 :∗ 5.1 channels∗ Sampling frequencies extended to 16, 22.05 & 24 kHz


Dolby surround principles (1/5)u 4 channels carried by stereo pair ⇒ same tools as for stereou Compatible with stereo installation

+90°

-90°

Phaseshifter

Lt

RtSurround Surround

Left

Configuration

Center Right

Center(C)

- 3 dB- 3 dB

Right (R)

Surround(S) 100 Hz -

7000 Hz

BP

- 3 dB- 3 dB

From 4 channels to 2 channels

Left (L)


Dolby surround principles (2/5)

Encoding equations Simple decoder

Lt L .1

2

C ..j1

2

S L Lt ⇒ Ld L .1

2

C ..j1

2

S

Rt R .1

2

C ..j1

2

S R Rt ⇒ Rd R .1

2

C ..j1

2

S

CRt Lt

2⇒ Cd .

1

2

L .1

2

R C

SLt Rt

.2 j⇒ Sd ..j 1

2

L ..j1

2

R S


Dolby surround principles (3/5)

3 dBS

3 dB

3 dB

L C

3 dB

R

u Simple decoder provides only 3 dB channel separation(See previous equations) ⇒ Need for improvement ⇒ Dolby Surround pro-logic decoder (next slide)


Dolby surround principles (4/5)Dolby surround pro-logic decoder

BP

C'

S'

R'

L'

Rt

-3 dB

Lt

-3 dB Direc-tionCom-pen-sation-90°

Rd

Sd

VCA

VCA

Cd

Ld

VCA

VCA


Dolby surround principles (5/5)Performance of Dolby pro-logic decoder

Channel separation larger than 35 dB

> 35 dB

S

> 35 dB

> 35 dB

L

> 40 dB

C

> 35 dB

> 40 dB

R


5.1 surround soundMPEG-2 surround configurations (front/back)

u 3/2

u 3/0 + 2/0

u 3/1

u 2/2

u 2/0 + 2/0

u 3/0

u 2/1

u 2/0

u 1/0

+ LFE (opt.)(Fs/96)15-120Hz

SurroundLeft

SurroundRight

(+LFE)

CenterLeft Right


Virtualisationu Virtualisation has no direct relation with the MPEG standard.

It is considered here only because it may be implemented insome of the future audio products (DVD, STB ...)

u Virtualisation is a product feature.

u It allows reproduction of surround information (5.1, 3/1) on astereo installation.


Virtualisation principle

u Virtualisation = processing of the signal in such a way thesource of the signal is perceived at a selected position outsidethe loudspeaker axis (virtual loudspeaker).

u Drawback : very sensitive to listener position (stability)

u Remark : a mono signal coded in normal stereo is perceivedbetween the two loudspeakers

Listener

Real loudspeakerProcessing

Mono signal

Virtualloudspeaker


Stereo widening

u Also called Q-sound , incredible sound, azimuth positionning...

u The stereo sources are positionned at virtual locations forimproving the stereo effect (cheap analog solution exists)

u Real sound comes from real loudspeakers. Perceived sound isas if stereo signals were coming from virtual loudspeakers

Listener

Real loudspeakerProcessing

Stereo signal

Virtualloudspeaker


Virtual surroundu Virtual surround gives on a stereo installation the subjective

effect of a multichannel configuration.

u Each channels is virtually positionned at a location around thelistener.The stereo installation performs the addition of the processedsignals for each audio channel.

u Real sound comes from a stereo installation. Perceived soundis as if the various surround signals were coming from somevirtually located loudspeakers.


Summary of surround aspects

Remarks about Dolby surround pro-logic :

Only carrier is stereo, source & presentation are multichannel

Compatible with stereo installation (no surround effect except in the case ofsurround virtualisation)

Stereo receiver Multichannel receiver

Stereo source Stereo widening(incredible sound,azimuth positioning)

3-D special effects(hall, theatre, stadium)

multichannel source Virtual surround Multichannel output





u Audio compression

u Video compression





u Conclusion


Video compression in MPEG (1/6)u Principles

∗ removal of intrapicture redundancy :Image is decomposed in 8*8 pixels subimages.Each subimage contains redundant informationDCT transfomation (in frequency domain) decorrelates theinput signal.( most energy in low spatial frequencies)

∗ removal of interpicture redundancy :coding of difference with an interpolated picture (movingvectors)

∗ high frequent spatial frequencies quantized with lowerresolution than low ones(remove irrelevancy)

∗ zig-zag scan and VLC (remove redundancy)


Video compression in MPEG (2/6)u Result

∗ 4:2:2 CCIR 601 resolution : 166 Mbps(=25images/sec *576lines* 720pixels* 2(lum & chrom)*8bits)⇒ ± 3-4 Mbps (mean) in MPEG2

∗ 4:2:0 SIF resolution : 30 Mbps(=25 images/sec *288 lines *352pixels* 1.5(lum & chrom)*8bits)⇒ ±1.2 Mbps (CBR) in video CD (MPEG1)


Video compression in MPEG (3/6)u Spatial redundancy reduction (DCT example)

158 0 -1 0 0 0 0 0 -1 -1 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

139 144 149 153 155 155 155 155144 151 153 156 159 156 156 156150 155 160 163 158 156 156 156159 161 162 160 160 159 159 159159 160 161 162 162 155 155 155161 161 161 161 160 157 157 157162 162 161 163 162 157 157 157162 162 161 161 163 158 158 158

158 0 -1 -1 -1 -1 EOBzig-zag scan

1260 -1 -12 -5 2 -2 -3 1 -23 -17 -6 -3 -3 0 0 -1 -11 -9 -2 2 0 -1 -1 0 -7 -2 0 1 1 0 0 0 -1 -1 1 2 0 -1 1 1 2 0 2 0 -1 1 1 -1 -1 0 0 -1 0 2 1 -1 -3 2 -4 -2 2 1 -1 0

DCT

Quantisation


Video compression in MPEG (4/6)u Temporal redundancy reduction

B

5

Bi-directional prediction

I : Intra-coded pictureP: Predicted pictureB: Bi-directionally interpolated picture

4

B

Order ofpresentation

Order oftransmission

BI P

0 3

B P

1 2 6

B

Prediction

I B P B

Increase of compressionrate

0 1 2 3 4

7

B P B

5 9

B I P

8

P B B P I B

86 7 9


Video compression in MPEG (5/6)u Model of a possible encoder

Buffer

Data

Regulator

Motion estimation

Motionvectorsand modes

Format conversion and picture reorder

Inputpictures

Picturestore andprediction

1/DCT

DCT VLC

1/Q

Q Multi-plex


Video compression in MPEG (6/6)u MPEG1 en MPEG2 video features

∗ MPEG1F sequential picture

F resolution : SIF format 288(240)*356*24,25 or 30 Hz

∗ MPEG2F sequential or interlaced

F various levels : low level (SIF: 288*356), main level (CCIR601: 576* 720), high 1440 level (HDTV: 1152*1440), high level (EQTV:1152*1920)

F various profiles (toolboxes) : simple profile (No B picture), mainprofile (=MPEG1+interlaced), SNR scalable profile (allows gracefuldegradation (noise improvement at same resolution), spatial scalableprofile (hierarchical coding : improvement at higher resolution), highprofile.





u Audio compression

u Video compression





u Conclusion


Synchronisation

Synchronisation in the multimedia context

refers to the mechanism that ensures a temporal

consistent presentation of the audio-visual

information to the user


Intramedia synchronisation

∆T between capture & presentation = Constant ≡ Same clock frequency & Data on time ⇒ Need for corresponding tools

Network

∆

∆ T1T2 = ∆

T1

Capture time

Audio signal Encoder

∆T = Constant

Decoder

Presentation time

Audio signal


Intermedia synchronisation

∆T_Audio = ∆T_Video ≡ (Sampled at the same time ⇒ Presented at the same time) ⇒ Possible tools : common time base and presentation control (media

synchronisation with the common time base)

Ex.: Lip_sync (requirement: |delay_difference| < 80msec)

Network

T_audio = Constant

T_video = Constant = Capture time

∆

Video signal

Audio signal

Encoder

Capture time∆

T_Audio ∆Presentation time

Decoder

Video signal

Audio signal

Presentation time


Recovery of clock in CBRu CBR = Constant Bit Rate

u if the clock to recover is synchronous with transport clock ⇒Recovery of clock but not of common time base

u Remark : possibility to slave DSM to local clock

Filter

CBR stream

Phaseerror

Filling level

50%

Time Information carriedby each sample

VCO

Processing

Recoveredclock

time


Recovery of clock and time base in VBRu VBR = Variable Bit Rate

u Need for insertion of time stamps (OUTPUT TIME)Output time stamp says for example : “It is now 16h25”Receiver adjusts its own horloge to the received time stamp

u Recovery of clock & of common time baseTime information carriedonly by time stamps

Time stamp extraction

Time counter

Recovery of clock & time

First time stamp

Others

Clock

Data stream

Data stream

Time stamps

Counter

Recovered clock

Error Filter VCO

VBR Stream

Counter sample(=Time stamps)

Channel

Time stamps


Synchronisation with common time baseu Insertion of time stamp (=INPUT TIME)

Input time stamp says : “Sample has been sampled at 16h29”.Receiver presents the sample at (its input time stamp + maximum encodingand decoding delay).Alternative: transmission of presentation time stamp (input time+delay)

Buffering

Mediaoutput

Media input

Time clock(Recovered)

Comparison of time clockwith sampled time clock

Sample "Time clock"Assemble frame

Time clock

Timestamp

Processing

Processing

Channel


Getting data on timeu “On time” ⇒ Not too late, not too early

No buffer over- or underflow

u Flow control : not applicable in broadcasting

u Common time base andDefinition of a standard target decoder that describes the dataconsumption pattern of the receiver.

u Remark: Direct MPEG (microsoft) does not use timeinformation for clock recovery but relies on flow control


Streamsu Idea of continuity (pipelining)

u Carry time information for clock recovery

u No flow control (allows broadcasting)The emitter must have a precise knowledge of the receiver dataconsumption pattern (explicit in MPEG STD)

u Just-in-timeShorter delay and smaller buffer size than with flow control

u Two aspects in synchronisation :Clock recovery & timing control (model & buffering)


Requirement on the channel for streamtransport

Data information ⇒ BER (Bit Error Rate) requirementNo repetition of frame possible ⇒ FEC (Forward ErrorCorrection)

Time information ⇒ No jitter





u Audio compression

u Video compression


u The MPEG model and its situation in a communicationcontext



u Conclusion


What is MPEG ? (1/2)

u Moving Picture Expert Group

u International standard (ISO/IEC)⇒ Interoperability & economy of scale

u Compression of audio and video and multiplexing in a singlestream

u Definition of the interface not of the codecs⇒ room for improvement

u MPEG-1 : until 1.5 Mbps, for DSMProgressive picture, stereo (Dolby surround)


What is MPEG ? (2/2)u MPEG-2 : Various bit rates (CBR & VBR)

Program stream for DSM, transport stream for networkInterlaced picture, 5.1 audio channelsDefinition of various video levels (e.g. CCIR601 resolution: 4-9 Mbps, HDTV:15-25 Mbps) and profiles

u MPEG-3 : Cancelled, integrated in MPEG-2(Initially : for HDTV)

u MPEG-4 : standard for audio, video and graphics in interactive2D and 3D multimedia communication.(Initially : low bit rate for real-time personal communication)

u MPEG-7 : Multimedia contents description interface

u MPEG-21 : Focus on multimedia distribution and on DRMaspects.


The MPEG model (1/2)

Audiodecoder

Audio signal

Videosignal

Presented signals

Multiplexer

Videodecoder

Captured signals

Audioencoder

Videoencoder

Audio signal

Videosignal Digital storage medium

orNetwork

Transmission channel

Demulti-plexer


The MPEG model (2/2)u Compression of audio & video and multiplexing in a single

stream

u Guarantees intramedia and intermedia synchronisation.

u MPEG defines an interface∗ bitstream syntax∗ timing of the bitstream ⇒ STD specifying timing requirement (ideal

model)

u Consequences:∗ Decoder should compensate deviations from STD∗ Network should correct jitter introduced by the channel (RTD-LJ)

u MPEG stream must be adapted to transmission channelformatting, error correction, channel coding (b.v.video-CD)


Components of the MPEG standardu The MPEG standard is composed of 3 main parts :

∗ Audio : Specifies the compression of audio signals∗ Video : Specifies the compression of video signals∗ System : specifies how the compressed audio and video signals are

combined in the multiplexed stream (program stream or transportstream).

u Each part specifies :∗ The bitstream syntax∗ The timing requirement and the related information (bit rate, buffer

needs)


Synchronisation Mechanism (1/2)

System decoderanddemultiplexer

Multiplexer and systemencoder

Audiodecoder

Videodecoder

ComparisonPTS and STCand presentation

Videooutput

ComparisonPTS and STCand presentation

Audiooutput

Extractionof PCR (SCR)

STC

Audioencoder

Videoencoder

Assemble pictures,Sample STC for PTS

Videoinput

Assembleaudio frames,Sample STCfor PTS

Audioinput

Sample STCfor PCR(SCR)

STC

Transmission channel


Synchronisation Mechanism (2/2)u PCR for TS & SCR for PS (but same concept)

u Clock & time base recovery: Time-stamping at OUTPUT(PCR included in TS multiplex, SCR in pack header)

u Audio & video clock locked to STC ⇒ easy recovery (see nextslide)

u Synchronisation of audio & video to common time base (Timestamping at Input)

u STD is defined (because of the absence of flow control)streams are such that STD buffers never over- or underflow

u In TS, many program in a single stream but unique clock perprogram.

u Time information ⇒ No Jitter for transport


Clock recovery in receiver

VCO Audio clockdivider

Video clockdivider Video

clock

Audioclock

Audiooutput

STC

ComparisonPTS and STCandPresentationDecoded

audio

PTS

STC(Counter)

PCR

STC

Error Low Pass Filter(Integrator)

Load first PCR


MPEG program & transport streamsu Program streams:

∗ Relatively error free environment∗ program stream packet may have variable and great length∗ Single time base

u Transport streams:∗ environment where errors are likely∗ many programs (independent time base)∗ Transport stream packet : fixed, 188 bytes∗ Contains tables


MPEG in a communication context (1)“Typical” communication system

Channelencode

Channeldecode

Bit-stream

Sourcedecode

Informationsink

Digitaloutput

Format Decrypt

SourceencodeFormat

Informationsource

Digitalinput

Encrypt

Synchro-nisation

Digitalwaveform

Otherdestination

Demodu-late

Demulti-plex

Multipleaccess

Modu-late

Multi-plex

Channelbits

Othersources

Multipleaccess

Channel


MPEG in a communication context (2)u MPEG : Source coding only (bit rate reduction) + multiplexing

u The MPEG stream must be adapted to the channel in whatconcern its physical characteristics and in order to get therequired QoS (Quality of Service) & Security∗ Encryption∗ Channel coding (forward error correction, interleaving, modulation

codes)

∗ multiplexing & formatting∗ modulation (frequency allocation)∗ multiple access method

u Some channels : CD/DVD - satellite - cable - ATM - 1394


MPEG in a communication context (3)u A simple view of MPEG in the communication context

VideoEncoder

MPEG2 compression layer

Audioencoder

Audio,videosources

ES(ElementaryStream)

Adap-tationto thechannel

PS(1 pro-gram)

MPEG2 system layer

PSMulti-plexing

Adap-tationto thechannel

DVB, DVD ...

Disc

Satellite

TSMulti-plexing

TS(n pro-grams)

Adap-tationto thechannel Cable

TS (Transport Stream)orPS (Program Stream)


u Introduction - The evolution of Audio/Video consumerproducts and the role of compression techniques.


u Audio compression

u Video compression





u Conclusion

Agenda

CD : Some conceptsu Hard disk vs compact disc : more differences than just storage

technique.HD developed for data storage and recording, CD developed forstream storage (CD-DA) ⇒ their basic differences

u Questions∗ track form?∗ read direction? Why?∗ CAV or CLV? Why?∗ Access time : CD-ROM vs HD?∗ Data storage: on which face?∗ Production method?∗ Capacity?∗ Sensitivity to error? Diameter of a possible hole?∗ Standard = Interface definition : CD vs HD ?


CD-DA: Encoder model (1/3)

Modulation

6 samples = 24 bytes= 1 frame

EFM + 3 merging bits

Synchronisation pattern27 bit/frame

561 bits / frame

CD-ROM1 sector = 98 frames75 sectors/sec.

588 channel bit/frame

Right

LeftA/D conversion

PCM 44.1 kHz16 bit/sample/channel

Subcode(1 byte / frame)

Physical layer

Error correctionencoding

32 bytes/frame


CD-DA: Encoder model (2/3)u The CD-DA physical layer adapts the input stream (audio) to

the requirements of the channel

∗ Modulation : EFM (Eight to fourteen modulation + 3merging bits)Pit & land length (number of successive 0 or 1 as written todisc): between 3 and 11 channel bitsDC free codefor adaptation to the channel bandwidth & for clockrecovery considerations.

∗ Error correction (Cross-interleaved Reed-Solomon code)Interleave placed between C1 & C2 ECC.Next slide presents only principles and not real CDimplementattion.


CD-DA: Encoder model (3/3)∗ Error correction : addition of redundancy in order to be able

to correct errors (e.g. RS(28,24,5)*RS(32,28,5))Principle :

∗ Interleaving : time diversity in order to deal with errorburst.Successive errored channel bits (burst error) do not damagethe same Reed-Solomon table.

4 bytesC2 codewords

24 bytes

28 bytes

Data

4 bytes

C1code-words


CD-ROM encoder model

2336 bytes

Mode 1

Mode 2

Mode 1

2336 bytes

Additionalerror detection &error correctionencoding

Optional EDC 4 bytes

User data 2324 bytes

Subheader 8 bytes

Subheader 8 bytes


Sync Pattern 12 bytes

Additionalerror detection &error correctionencoding


Zeroes 8 bytes

Header 4 bytes

Header 4 bytes

2340 bytes

For EDC only

Video-CD uses CD-ROM mode 2 sectors

OR

12 bytes

2340 bytesScramblingOR1 CD-ROM sector= 2352 bytes


The CD standards

CD-BGMVideo-CD93

CD-RW96

CD-ROM1984

Photo-CD

CD-ROM-XA

CD-bridge

CD-i1987

CD-DA1982

CD-WO90

CD-i Ready

CD-MIDI

CD-G


From CD to DVD : the motivationu Motivation = increase the capacity

u Why ? - Requirement of the motion picture industry

∗ Playback time : more than 135 min. (duration of 90% offilms)

∗ Picture quality : superior to laser disc∗ Audio quality : 5.1 channels surround∗ Language/subtitles : 3 languages minimum.⇒ capacity needs : more than 4.7 Gbytes

u Where ? - In physical layer

u DVD : developed specifically for audio/video( ≠ video CD).


The DVD physical layer (1/2)

Recorded sectors

2366 bytes(13 *182 bytes)

8/16modu-lation

37856 channel bits

Synchronisation(2*13)*32 channel bits

38688 channel bits(eqv. to 2418 bytes)

EDC : Error Detection CodeECC : Error Correction CodeCPR-MAI : CoPyRight MAnagement Information

CPR-MAI - 6 bytes

ID (incl. sector#)4 bytes

Data2048 bytes

+EDC

Scrambling

EDC4 bytes

6 bytes

ECC(per group of 16 sectors)

Data sector

2064 bytes(12 * 172 bytes)

RowInter-lea-ving


The DVD physical layer (2/2)u Objective was the storage of 2K sectors

u Error Correction Code (Reed-Solomon) - add redundancy

u Modulation - time diversity(Number of consecutive 0 : between 2 and 10)Pit and land length : between 3 and 11 (Idem CD)

u Synchronisation : for sector reconstruction.

16 bytesPO (Outer Parity)

172 bytes

192 bytes

16 data sectors

(12 * 172 bytes)*16

10 bytes

PI(InnerParity)


DVD: the capacity improvement (1/4)u Increase of channel bit density ( gain = 4.50)

Min pit length : (0.83µ ⇒ 0.4µ)Track pitch : (1.6µ ⇒ 0.74µ)Diameter of laser spot (÷ wavelength/NA)Wavelength (780nm ⇒ 640 nm) ⇒ gain = 1.5NA (0.45⇒ 0.60) ⇒ gain = 1.78reduced margin ⇒ gain = 1.68

u Modulation:EFM (8 to 17 bit) ⇒ 8 to 16 ⇒ gain = 1.06

u Error correctionRS(32,28,5)*RS(28,24,5) ⇒ RS(182,172,11)*RS(208,192,17)

⇒gain = 1,16


DVD: the capacity improvement (2/4)u No subcode ⇒gain = 1.03u Sync pattern ⇒gain = 1.03u Better sector formatting

sector length (2352 bytes ⇒ 2064) ⇒gain = 1.14u Other (e.g. recorded area) ⇒gain = 1.07

Total gain : 7.2

Capacity per side : 650 MBytes (mode 1) ⇒ 4.7 Gbytes


DVD: the capacity improvement (3/4)

0.6 mm

0.6 mm

0.6 mm

0.6 mm

For layers 0 and 1

A side

Single-layer double-sided disc

Single-layer single-sided disc

0.6 mm

0.6 mm

B side

Dual-layer double-sided disc

0.6 mm

0.6 mm

Dual-layer single-sided disc


DVD: the capacity improvement (4/4)u Capacity of the various types

Single-layer single-side 4.7 GbytesDual-layer single-side 8.5 GbytesSingle-layer double-side 9.4 GbytesDual-layer double-side 17 Gbytes


The 3 components of the DVD-V standard

DVD = DVD (Not “Digital Video Disc”, Not “Digital VersatileDisc”)

DVD-V = DVD - Video

Part 1 : Physical specification

Part 2 : File system specification

Part 3 : Video specification

How blocks may be retrieved. Definition of the file andvolume structure.

How blocks of 2048 bytesare stored on the disc

Contents of the data block.How audio and video are mapped to the block, file and volume structure


Some DVD-V features (1/2)Presentation data = MPEG program stream, VBR, max peak bitrate = 10.08 Mbps)

Video data 1 stream Mpeg1Mpeg2 (ML@MP)

Audio data max 8 streams Mpeg2 + 7.1 ext.DolbyAC-3Linear PCM (incl. 96 kHz- 24 bits)

Sub picture data max 32 streams Run length encoded(subtitles) Bit map


Some DVD-V features (2/2)u Seamless playback

Languageparental lockMulti-angle camera

u System menuAudio stream selectionSubtitle selectionAngle

u EncryptionDecryption key hidden on the disc.


UDF & ISO9660

Part 3:Application Videospecification

Part 2:File system

Part 1:Physical

UDF

Read only

DVD-ROM DVD-Video

Audiospecification

Write-once

UDF | (UDF & ISO9660)

Rewritable

DVD-Audio DVD-R DVD-RAM

The DVD family of products


Recording on disk - principleu Products: CD-R, CD-RW, DVD-R, DVD-RAM

u CD principle: reflectivity of pits & lands are different.Pits and lands are used to store 0 and 1.

u CD-RW principle: reflectivity of the two phases of therecording material (amorphous, crystalline) are different.Controlling the phase allows storage of 0 or 1.

u To Amorphous state (low reflectivity):T above melting point (600°C) & fast cooling

u To Crystalline state (high reflectivity):T above 200°C for a sufficient time

u Recording: by the laser heating the recording layer

u Reading: by laser as for CD (-> compatibility)





u Audio compression

u Video compression





u Conclusion


Adaptation to the DVB channel

u Channel coding : transforms the TS in an other sequence ofbits containing the same information than the input stream butmore robust against the imperfections of the transmission onthe physical channelcost : a higher bit rate

u Modulation : transforms an input sequence to an analogwaveform for transmission over the physical channel

Sequence of bits

(Encrypted) TS Channelcoding

Analog waveform

Modulation Physicalchannel


Channel coding (1/3)u Unlike source coding that removes redundancy,

channel coding adds redundancy in a structured way sothat the decoder be able to detect and/or correct the errorsintroduced by the physical channel.

Channeldecoding

Source decoding

Sink

Source coding

Source Channelcoding

Quasi-error-freechannel (e.g.BER


Channel coding (2/3)u Channel coding may include :

∗ Spectral modification of the signalfor adaptation to the channel (e.g. remove DC, spectrum shaping likeuniform distribution in the frequency space ...)

∗ FEC : Forward Error CorrectionAddition of redundancy in order to allow error detection and/orcorrection (example : The total of bought articles is similar to a paritybyte)

188 bytes

188 bytes

Allows correction of 8 erroneous bytes

After error correctionencoding

Original sequence

Message Parity bytes

16 bytes


Channel coding (3/3)∗ Interleaving

Time diversity in order to deal with error bursts.The successive bytes of information are dispersed in time on thetransmission channel in such a way that an error burst does not affectneighbouring bytes.Interleaving is often combined with FEC so that error bursts could becorrected by the FEC.

Example :

AEIMBFJNCGKODHLPQ.....

AEIMBFJ&&&KODHLPQ.....

----> A burst of errors affects bytes belonging to different error correction blocks

Channel

Deinterleaving

AB&DEF&HIJKLM&OPQ.......|---|---|---|---|

Interleaving

ABCDEFGHIJKLMNOPQ.......

& : Erroneous byte| : Beginning of an error correcting block- : Element of an error correcting block


Modulation in DVB (1/3)u Different modulation techniques :

∗ Cable : QAM∗ Satellite : QPSK∗ Terrestrial : OFDM

u Why ?Modulation technique depends on :

∗ Physical characteristics of the channel∗ Compatibility constraints with actual analog transmission


Modulation in DVB (2/3)

u Example : influence of SNR on modulation technique selected⇒ QPSK for satellite and QAM for cable

14

1E-7

4

1E-5

1E-6

6 8 10 12

QPSK

BER

0.001

1E-4

0.01

1.0

0.1

16 18

16QAM

20 22

32-QAM 64-QAM

24 26 28 SNR


Modulation in DVB (3/3)u Satellite

Bandwidth : generally 27-36 MHzSNR low : about 10 db (power transmitted by satellite)direct path

u CableBandwidth : 8 MHz (50Hz countries) - 6 MHz (60Hzcountries)SNR strong (about 25 db)Echoes from impedance mismatch in the network

u TerrestrialBandwidth : idem as cableMultipath interference, signal level variation, ...


From TS to the DVB channel

u Some blocks are identical for all standards (Cable, Satellite & Terrestrial)

u Inner & outer : terminology is derived from the view of the quasi-error-freechannel composed of a transmitter and a receiver.

u Satellite & Terrestrial : More sensitive to error ⇒ inner coder is added

QAMmodulation

QPSKmodulation

OFDMmodulation

DVB-C

DVB-S

DVB-T

Spectrumshaping

TS Outercoding

Inner inter-leaver

Innercoding

Inter-leaving





u Audio compression

u Video compression





u Conclusion


Questions

?

Audio & Video Compression and its Application in Consumer ... › acous ›...

Documents

Transcript of Audio & Video Compression and its Application in Consumer ... › acous ›...