predictive coding 07 - HHIiphome.hhi.de/wiegand/assets/pdfs/DIC_predictive_coding_07.pdf · Thomas...

Thomas Wiegand: Digital Image Communication Predictive Coding - 1

Predictive CodingPredictive Coding

Prediction

Prediction in Images

Principle of Differential Pulse Code Modulation

(DPCM)

DPCM and entropy-constrained scalar

quantization

DPCM and transmission errors

Adaptive intra-interframe DPCM

Conditional Replenishment


PredictionPrediction

Prediction is difficult – especially for the future. Mark Twain

Prediction: Statistical estimation procedure where

future random variables are estimated/predicted from

past and present observable random variables.

Prediction from previous samples:

Optimization criterion

Optimum predictor:

ˆ S 0 = f (S1,S2,...,SN ) = f (S)

min})],...,,({[})ˆ{( 2

210

2

00 == NSSSfSESSE

)},...,(|{ˆ2100 N

SSSSES =


The optimum predictor

can be stored in a table (Pixels: 8 bit size 28N)

Optimal linear prediction (zero mean, Gaussian RVs)

Optimization criterion

Optimum linear predictor is solution of

In case RS=E{SSt} is invertible

StructureStructure

)},...,(|{ˆ2100 N

SSSSES =

Sat

NNSaSaSaS =+++= ...ˆ

22110

}){(})ˆ{( 2

0

2

00 Sat

SESSE =

}{ 0

t

S

tSE SRa =

}{ 0

1SRa SE

S=


Prediction in Images: Intra-frame PredictionPrediction in Images: Intra-frame Prediction

Past and present observable random variables are

prior scanned pixels within that image

When scanning from upper left corner to lower right

corner:

1-D Horizontal prediction: A only

1-D Vertical prediction: C only

Improvements for 2-D approaches (requires line store)

B

A

DC

X

),(),(),(ˆ

)0,0(),(

0

2

1

qypxsqpayxs

qp

P

Pp

Q

q

== =

43421


Prediction Example: Test PatternPrediction Example: Test Pattern

s[x,y]

uV[x,y]=

s[x,y]-0.95 s[x,y-1]

uH[x,y]=s[x,y]-0.95 s[x-1,y]

uD[x,y]=s[x,y]-0.5(s[x,y-1]+ s[x-1,y])


Prediction Example: CameramanPrediction Example: Cameraman

s[x,y] uH[x,y]=s[x,y]-0.95 s[x-1,y]

uV[x,y]=

s[x,y]-0.95 s[x,y-1]

uD[x,y]=s[x,y]-0.5(s[x,y-1]+ s[x-1,y])


Change of Histograms: CameramanChange of Histograms: Cameraman

0 50 100 150 200 2500

500

1000

1500

2000

2500

3000

Image signal Prediction error

signal (diag. pred.)

Can we use prediction for compression ?

Yes, if we reproduce the prediction signal at the decoder

-50 0 500

0.5

1

1.5

2x 104


Differential Pulse Code ModulationDifferential Pulse Code Modulation

Reconstruction error =

Prediction error Reconstruction quantization error

Coder

Decoder

quantizerentropy

coder

predictor

input

predictor

entropy

decoder output

channel

+

+

-s

's

e

s 's

s

ses ˆ'' += qeess == ''

'e

'e

sse ˆ=


DPCM and QuantizationDPCM and Quantization

Prediction is based on quantized samples

Stability problems for large quantization errors

Prediction shapes error signal (typical pdfs: Laplacian,

generalized Gaussian)

Simple and efficient: combine with entropy-

constrained scalar quantization

Higher gains: Combine with block entropy coding

Use a switched predictor

• Forward adaptation (side information)

• Backward adaptation (error resilience, accuracy)

DPCM can also be conducted for vectors

• Predict vectors (with side information)

• Quantize prediction error vectors


0 1 2 3 4 5 6 70

5

10

15

20

25

30

35

Panter & Dite App

Entropy-Constrained Opt.

R(D*), =0.9 DPCM & ECSQ

R [bits]

Comparison for Gauss-Markov Source: Comparison for Gauss-Markov Source: =0.9=0.9

Linear predictor orderN=1, a=0.9Entropy-Constrained

Scalar Quantizer with

Huffman VLC

Iterative design

algorithm applied

SNR [dB]

=10 log10

2

D


K=511, H=4.79 b/p K=15, H=1.98 b/p K=3, H=0.88 b/pK...number of reconstruction levels, H...entropy

DPCM with Entropy-Constrained Scalar QuantizationDPCM with Entropy-Constrained Scalar Quantization

from: Ohm

Example: Lena, 8 b/p


Transmission Errors in a DPCM SystemTransmission Errors in a DPCM System

For a linear DPCM decoder, the transmission error

response is superimposed to the reconstructed signalS'

For a stable DPCM decoder, the transmission error

response decays

Finite word-length effects in the decoder can lead to

residual errors that do not decay (e.g., limit cycles)

from: Girod


Error rate p=10-3.1D pred., hor. aH=0.95 1D pred., ver. aV=0.95 2D pred.*,aH=aV=0. 5

from: Ohm

Transmission Errors in a DPCM System IITransmission Errors in a DPCM System II

Example: Lena, 3 b/p (fixed code word length)


Inter-frame Coding of Video SignalsInter-frame Coding of Video Signals

Inter-frame coding exploits:

• Similarity of temporally successive pictures

• Temporal properties of human vision

Important inter-frame coding methods:

• Adaptive intra/inter-frame coding

• Conditional replenishment

• Motion-compensating prediction (in Hybrid Video Coding)

• Motion-compensating interpolation

from: Girod


Principle of Adaptive Intra/Inter-Principle of Adaptive Intra/Inter-FFrame DPCMrame DPCM

Predictor is switched between two states:

for moving or changed areas.

Intra-frame prediction Inter-frame prediction (previous frame

for moving or changed areas. prediction) for still areas of the picture.

S 22 S23 S 24

S 21 S20 S 25

FRAME N - 1

S 2 S 3 S4

S 1 S 0

FRAME N

S 22 S 23 S24

S 21 S 20 S25

FRAME N - 1

S 2 S3 S 4

S 1 S0

FRAME N

40 ms

''''ˆ44332211

SaSaSaSaS +++=20'ˆ SS

inter= from: Girod


Intra/Inter-Intra/Inter-FFrame DPCM: Adaptation Strategies, Irame DPCM: Adaptation Strategies, I

Feedforward Adaptation

Interframe

predictor

Intraframe

predictor

VLC VLCQuantizer

Interframe

predictor

Intraframe

predictor

Predictor

adaptation

+

-

Intra/inter-frame

Switching

information

Coder Decoder

s e

s

inters

intras

's

s

's'e 'e

inters

intras

from: Girod


Intra/Inter-Intra/Inter-FFrame DPCM: Adaptation Strategies, IIrame DPCM: Adaptation Strategies, II

Feedback Adaptation

Inter-frame

predictor

Intra-frame

predictor

VLCVLCQuantizer+

-

Coder Decoder

Predictor

adaptation

Inter-frame

predictor

Intra-frame

predictor

Predictor

adaptation

from: Girod

s e

s

inters

intras

's

'e 'e

s

inters

intras

's


Principle of a Conditional Replenishment CoderPrinciple of a Conditional Replenishment Coder

Still areas: repeat from frame store

Moving areas: transmit address and waveform

SIGNAL

INPUT

SEGMENTER

(MOVEMENT

DETECTOR)

CODING,

ADDRESSING,

BUFFERING

BUFFERING,

DECODING,

ADDRESSING

FRAME DELAY

(1 PICTURE MEMORY)FRAME DELAY

(1 PICTURE MEMORY)

TRANSMISSION

CHANNEL

SIGNAL

OUTPUT

Coder Decoder

from: Girod


Change DetectionChange Detection

Example of a pel-oriented change detector

Example of a block-oriented change detector

ABS

Average

of 3x3

window

Eliminate isolated

points or pairs of

points

+

-

Current

frame

Previous

frame

Decision

changed/

unchanged

Threshold

ABSAccumulate

over NxN

blocks

+

-

Current

frame

Previous

frame

Decision

changed/

unchanged

Threshold

from: Girod


The "Dirty Window" EffectThe "Dirty Window" Effect

Conditional replenishment scheme with change detection threshold

set too high leads to the subjective impression of looking through a

dirty window.

BackgroundMoving

Area

picked up

by change

detector

Moving areas

missed by

change detector

from: Girod


SummarySummary

Prediction: Estimation of random variable from past or

present observable random variables

Optimal prediction

Optimal linear prediction

Prediction in images: 1-D vs. 2-D prediction

DPCM: Prediction from previously coded/transmitted

samples (known at coder and decoder)

DPCM and quantization

DPCM and transmission errors

Adaptive Intra/Inter-frame DPCM: forward adaptation

vs. backward adaptation

Conditional Replenishment: Only changed areas of

image are transmitted

predictive coding 07 - HHIiphome.hhi.de/wiegand/assets/pdfs/DIC_predictive_coding_07.pdf · Thomas...

Documents

Transcript of predictive coding 07 - HHIiphome.hhi.de/wiegand/assets/pdfs/DIC_predictive_coding_07.pdf · Thomas...