Ch_13_1 Transform Coding - Intro & Bit Allocation Optimization (PPT)

8/2/2019 Ch_13_1 Transform Coding - Intro & Bit Allocation Optimization (PPT)

1/17

1

Ch. 13 Transform Coding

My Coverage is Different from the Book


2/17

2

Overview Transform the signal (e.g., via the DFT, etc.) into anew domain where compression can be done: (i)

better and/or (ii) easier

Optional

BlockerQuantizers

Scalar

VectorDifferential

Entropy Coders

Huffmann

Arithmethic

Etc.

Block Diagram of Transform Coding

Fig. A

Often (but not always!) done on a block-by-block basis:

Non-Overlapped Blocks (most common)

Overlapped Blocks


3/17

3

Transform as Linear Operator

Well view transforms as linear operators on a vector space (finite dimensional):

1 1

2 2

1 1N N

x y

x y

x y

= =

x y

=A

x y y Ax

A = Operator anNNMatrix

Because at the decoder we need to undo the effect of this operator we need

matrix A to be invertible:

x y

Binary

Streamy x


4/17

4

1. Information Theory Advantages Try to make y have uncorrelated elements

Try to concentrate energy into just a few elements ofy

2. Perceptual Distortion Advantages

Transform domain is often better-suited for exploiting aspects of human

perception: psychology of hearing and vision

3. Efficient Implementation

Transform Coding framework provides simple way to achieve #1 & #2 Extra cost of transform is usually not prohibitively large

Usefulness of Transform Coding


5/17

5

Need for ON Transforms

Using theory of quantization it is easy to assess transform-domain distortion:

( )1

2

0

1 ( , )

N

n n

n

d y yN

=

= y y

But what is the resulting signal-domain distortion?? ( , ) ???d =x x

Fact: If transform A is ON then ( , ) ( , )d d=x x y y

Simplifies understanding of impact of

quantization choices in the transform domain

Recall: The matrix A for an ON transform has:

Columns that are ON vectors:

Inverse is the transpose:

1,

0,

T

i j

i j

i j

= =

a a

1 T =A A

T

T

=

=

x A y

x A y


6/17

6

So if the transform is ON then the signal distortion is:

( ) ( ){ } TD E= x x x x

( ) ( ){ } T TE= y y AA y y= I

( ) ( ){ } TE= y y y y

( )1 2

0

N

n n

n

E y y

=

=

1

0

N

nn

D

=

=

Distortion ofnth

Transform Coefficient

Big Picture Result: If ON transform, then Transform-Domaindistortions add to give total distortion in signal domain


7/17

7

Bit Allocation to TC Quantizers

In Fig. A we haveNquantizers operating on the transform coefficients

Q: How do we decide how many bits each of these should use?

This is the so-called Bit Allocation Problem we have a constrained total #

of bits how do we allocate them across theNquantizers?

Q: Why not just allocate them evenly???


8/17

8

Bit Allocation Problem

RB = Total Rate Budget (Bit Budget)

Ri = # of bits allocated to the ith quantizer Total Bits Used:

Di(Ri) = Distortion ofith quantizer when usingRi bits

Assume distortions are additive (true for ON transform):

1

0

N

i

i

R R

=

=

Each quantizer has its

own R-D curve

depends on quantizer type

and char. of the ith

transform coefficient

1

0( )

N

i i

i D D R

==

Goal: Allocate bits { }1

0

N

ii

R

=

to minimize

1

0

( )N

i i

i

D D R

=

= constrained by1

0

N

i B

i

R R R

=

=

(Alternate Goal: MinimizeR subject toD DB)


9/17

9

Aspects of Bit Allocation

1. Theoretical View2. Algorithms

Average R-D Approach

Operational R-D Approach

Bit Allocation Theory

Theory Drives Algorithms

Given known functionsDi(Ri)

(Based on some appropriate signal & quantizer models)

Solve the constrained optimization problem for the optimal

allocation vector r = [R0 R1 RN-1]

Models used here determine ifwe strive for

Average R-D Solution

Operational R-D Solution

Interpret the result to understand general characteristics

Constrained Opt Lagrange Mult.


10/17

10

x1

x2 f(x1,x2) contours Equality Constraint: g(x1,x2) = Cg(x1,x2) C = h(x1,x2) = 0

0),(),(

),(),(

2121

2121

=+

=

xxhxxf

xxhxxf

Constrained Max occurs when:

( )1 2, , 1 2 1 2

( , ) ( , ) 0x x

f x x g x x C

+ =

=

=b

a

x

xxh

x

xxh

xxh

2

21

1

21

21),(

),(

),(

Ex. The grad vector has

slope ofb/a orthogonal to constraint line

Ex. ax1 + bx2 c = 0

x2 = (a/b)x1 + c/bA Linear Constraint

Constrained Optimization: Lagrange Multiplier

Unconstrained

Minimum


11/17

11

Lagrange Multiplier Approach to Bit Allocation

See paper: Shoham & Gersho, Efficient Bit Allocation for an Arbitrary Set ofQuantizers, IEEE Transactions on Acoustics, Speech and Signal Processing,

Sept. 1988, pp. 1445 1453. (See especially Sect. III)

( )*

*

min ( ) subj. to ( ) ( )

( )

B SH B R B R B

R

=

Theorem: For any 0, the solutionB*() to the unconstrained problem

{ }min ( ) ( )B S

H B R B

+

is also the solution to the constrained problem with constraint

So for each 0 we find the -dependent solution to a -dependentunconstrained problem this solution solves a particular version of the

constrained problem, where the constraint is -dependent

Constrained Minimization: min ( ) subject to ( ) cB S

H B R B R


12/17

12

Proof: SinceB*() is a solution to the unconstrained problem

( ) ( )* *

( ) ( ) ( ) ( )H B R B H B R B

B S

+ +

Re-arranging this gives: ( ) ( )* *( ) ( ) ( ) ( )H B H B R B R B

Since this is true forBS it is true forBS* S such thatR(B) R(B*())

{ }* *| ( ) ( )S B R B R B=

Set of all allocationsB that satisfy-dependent constraintR(B*())

Note:R(B) R(B*()) is negative for allBS*.

Since is positive we have that ( )*

( ) ( ) 0 H B H B ( )*

*( ) ( ) H B H B

B S

H(B*()) is minimum over allB s.t.R(B) R(B*())

B*() solves the constrained problem with constraintR(B*())


13/17

13

What does this theorem say?

To each 0

there is a constrained problem with: constraintR*()

& solution B*()

the unconstrained problem min{H(B)+R(B)} also has the same solution

Soifwe can find closed-forms forB*() &R*() as functions of

then we can adjust = c so thatR*(c) =Rc

So we get that B*(c) solves the constrained problemw/ our desired constraint

Both

depend

on

Our Actual

Constraint


14/17

14

Applying the Theorem to TC Bit Allocation

The theorem says minimize:

We want to minimize

1

0

( ) ( )

N

i i

i

D R D R

== constrained by

1

0

N

i B

i R R R

==

( ) J D R R

= + for arbitrary fixed (get results in terms of)

1 1

0 0

( )N N

i i i

i i

J D R R

= =

= +

To minimize we need to set: 0j

J jR

=

( )

set 0

j j

j j

D RJR R

= +

=

( )j j

j

D R

R

=

Aha!! Insight!!Allthe quantizersmust operate at an R-D

point that has thesame slope

Equal Slopes Requirement


15/17

15

Intuitive View of Equal Slopes ConsiderN= 2 case

R1

D1

*

1R

*

1D

R2

D2

*

2R

*

2D

Intuitive Proof by Contradiction

1. Assume an optimal operating pt. (R1*,D1

*) & (R2*,D2

*) w/ non-equal slopes

2. Because assumed optimal:

3. Now imagine small increase inR1 for quantizer #1:

4. To keep the bit budget, must decrease rateR2 by same small amount:

*

*

ii

i R Ri i

DSR

=

=

with* *

1 2S S * *1 2: / 0WLOG S S w= >

* *

1 2 B R R R+ = i.e., meets budget

* *

1 1 , 0R R + >

* *

2 2 , 0 (same )R R >


16/17

16

5. Find new distortions due to these rate changes (Use Taylor series

approximations valid because rate changes were small)

6. Find new total distortion:

( )* * * * *1 1 1 1 2* * *

2 2 2

decreases to

increases to

D D S D S

D D S

+ = +

( )

* * * *

1 2 2 2

* *

1 2

0

New

Old

D D S D S

D D

D

= + +

= + >

New Old

D D

Ch_13_1 Transform Coding - Intro & Bit Allocation Optimization (PPT)

Documents

Transcript of Ch_13_1 Transform Coding - Intro & Bit Allocation Optimization (PPT)