Deblocking Filter for Low Bit Rate MPEG-4 Video

Image Communication & Signal Process-ing Lab.

Ilseung Kim (master’s student)

ContentReferencesIntroductionConventional Algorithm

◦(Deblocking Filter in MPEG-4 VM 8.0 coder)

Proposed AlgorithmExperimental ResultsPros & ConsConclusion & Future WorkQ & A

References [1] Shen-Chuan Tai, Yen-Yu Chen, and Shin-Feng

Sheu, “Deblocking Filter for Low Bit Rate MPEF-4 Video,” IEEE Trans. Circuits and Systems for Video Tech., vol. 15, no. 6, pp. 733-741, June. 2005.

[2] Sung Deuk Kim, Jaeyoun Yi, Hyun Mun Kim, and Jong Beom Ra, “A Deblocking Filter with Two Separate Modes in Block-Based Video Coding,” IEEE Trans. Cir-cuits and Systems for Video Tech., vol. 9, no. 1, pp.156 – 160, Feb. 1999.

Introduction Block-based discrete cosine transform (BDCT) has

been employed in several video coding standards. For high compression, however, BDCT suffers from

noticeable artifacts, including the blocking. Methods for reducing the number of blocking artifacts

◦ low-pass filtering

◦ statistical estimation

◦ set-theoretic reconstruction Projection onto convex sets.

◦ post-processing methods. the most practical solution

it does not require existing standards to be changed.

Iterative schemes : high complexity & costly

Over-smoothing of im-age

Conventional Algorithm (Deblocking Filter in MPEG-4 VM 8.0 coder)

Observation

1. The HVS(human visual system)• More sensitive in flat regions.

2. Undesirable blur • Adaptive smoothing is desirable.

3. Blocking artifacts are propagated• Smoothing flat regions must cover the inside of a

block

(a) Original first frame. (b) First decoded frame of

Foreman at QP = 13. (c) Original first frame.(d) First decoded frame of Coast guard at QP = 29.

8x8 Block Boundaries

V0 V1 V2 V3 V4 V5 V6 V7 V8 V9

V0

V1

V2

V3

V4

V5

V6

V7

V8

V9

Block bound-ary

Pixels for fil-tering on a vertical edge

Pixels for filter-ing on a hori-zontal edge

S0 S2

S1

Structure of the Deblocking Scheme

4

4

0 1 00

1

9 9 89

8

0

9

1' , 1 8,

16

,

,

,

,

,

, 1

, 1 8

, 8

{ : - 4 4} {1,1,2,2,4,2,2,1,1}

k n kk

m m

k

V b p n

where

v v v QPp

v otherwise

v v v QPp

v otherwise

p m

p v m

p m

and

b k

4 4 5 5

4 51 3,1 3,1

3,0 3,1 3,2

3,1 3,1

3,13,1

3,0

' - , '

,

- ( ( ' ), 0, )

2

( , , ), 0

'

0, 0

and CLIP(x,p,q) clips x to

v v d v v d

where

v vd CLIP k a a

MIN a a aa a

aa

a

a value between p and q.

2.Smooth region mode

3.Default mode

YES

NO

max - min 2 QP 3,1 a QPNONO

YES

YES

' V V

1. Mode deci-sion

2( ) (= 6)F v T

1 2 8

8

10

1

[ ]

max ( ) , min ( )

( ) ( - ),

1, ( 2), ( )

0,

i ii

V v v v

MAX V MIN V

F v v v

Twhere

otherwise

Mode Decision

F(v) : Flatness of the region T1 (= 2) : set to a small value If , Smooth region mode is set. Else, Default mode is set.

YES

NO

1. Mode deci-sion

2( ) ( 6)F v T

1 2 8

8

10

1

[ ]

max ( ) , min ( )

( ) ( - ),

1, ( 2), ( )

0,

i ii

V v v v

MAX V MIN V

F v v v

Twhere

otherwise

2.Smooth region mode

3.Default mode

2( ) ( 6)F v T

Smooth Region Mode (Nine-tap filter)

A nine-tab smoothing filter is applied inside the block as well as on the block bound-aries.

|Maxv - Minv |<2QP

◦ Filtering is not performed

4

4

0 1 00

1

9 9 89

8

0

9

1' , 1 8,

16

,

,

,

,

,

, 1

, 1 8

, 8

{ : - 4 4} {1,1,2, 2,4, 2,2,1,1}

k n kk

m m

k

V b p n

where

v v v QPp

v otherwise

v v v QPp

v otherwise

p m

p v m

p m

and

b k

2.Smooth region mode

max - min 2 QP NO

YES

' V V

A nine-tab smoothing filter

Ex) 0 0 0 0 0 128 128 128 128 128

After filtering

0 8 16 32 48 80 96 112 120 128

Default Mode (1)0,1 0 0 0 0 3

1,1 1 3 3 1 41

52,1 2 2 2 2

3 1 1 3 63,1

0 1 2 33

0 3 2 14

5 0 3 2 1

6 0 1 2

- - DCT of S

- -

- -

- -

- -

- -

a k k k k v

a k k k k v

va k k k k

k k k k va

k k k kv

k k k kv

v k k k k

v k k k

0,1

1,1

12,1

3 3,1

0

1

2

3

IDCT of S

0.5

1 cos 0.6533

821

cos 0.542

1 3 cos 0.2706

82

a

a

a

k a

where

k

k

k

k

Pixels for filter-ing on a verti-cal edge

S0 S2

S1

v0

v1

v2

v3

v4

v5

v6

v7

v8

v9

A major factor affecting the blocking artifact

Default Mode (2)Default mode Complex re-

gions

4 4 5 5

4 51 3,1 3,1

3,0 3,1 3,2

3,1 3,1

3,13,1

3,0

' - , '

,

- ( ( ' ), 0, )

2

( , , ), 0

'

0, 0

and CLIP(x,p,q) clips x to

v v d v v d

where

v vd CLIP k a a

MIN a a aa a

aa

a

a value between p and q.

3.Default mode

3,1 < QPaNO

YES

' V V

4 4 1

5 5 1

3,1 3,1

' -

'

, ' -

v v k

v v k

where a a

3,1 3,1

4 5 4 5

'

, ' , '

a a

v v v v

< Key Idea>

Proposed Algorithm(Deblocking Filter for Low Bit Rate MPEG-4 Video)

Structure of the Deblocking Scheme

Simpli-fied

One mode is added

Mode Decision

5

1 0

( ) ( )

0, ( 2), ( )

1, .

i ii

A v v v

Swhere

otherwise

1

2

( ) ( 2) : Deblocking for Smooth Region

( ) ( 3) : Deblocking for Complex Region

otherwise : Deblocking for the Intermediate Region

A v T

A v T

The number of pixels for measuring activity is reduced. Intermediate Region Mode is added to improve in terms

PSNR and visual quality.

Deblocking for Smooth Region

Simple offset-based filtering method

Steps:◦ 1) Offset: the difference between

two pixels across a block boundary: Offset = D – C

◦ 2) Update A, B, and C using the offset a = A + offset / 8 b = B + offset / 4 c = C + offset / 2

◦ 3) Update D, E, and F by means of the offset d = D – offset / 2 e = E – offset / 4 f = F – offset / 8

◦ 4) Adjust these updated pixels value within 0 to 255clip(0, 255, a) clip(0, 255, b) clip(0, 255, c)clip(0, 255, d) clip(0, 255, e) clip(0, 255, f)where clip (x, y, v) is a function that clips v

into the range from x to y.

Deblocking for Complex Region

The total length of the line of fil-tered pixels is limited to four.

Steps :◦ 1) The difference between two pix-

els across a block boundary, are calculated as follows: Offset = D – C

◦ 2) Update A, B, and C using the offset b = B + offset / 8 c = C + offset / 2

◦ 3) Update D, E, and F by means of the offset d = D – offset / 2 e = E – offset / 8

◦ 4) Adjust these updated pixels value within 0 to 255clip(0, 255, b) clip(0, 255, c)clip(0, 255, d) clip(0, 255, e)where clip (x, y, v) is a function that clips

v into the range from x to y.

Deblocking for the Intermediate Region

A 3 x 3 low pass filter is presented as an intermediate mode filtering.

Applying this LPF to the pixels on either side of the block boundary, C and D, significantly reduces the blocking effect with minimal loss of image content.

9

1 1 5

5

9

1 1 5

5 15

2

5

1, -

0,

( )'

( )

,

if belongs to an intracoded block,

is set.

i iii

ii

iii

S p

p Thp

otherwise

S

p Sp

S

where

p

Th QP

5 if belongs to an intercoded block,

/ 2 is set.

:controls the extent of smoothing

and typically lies between 8 - 16.

p

Th QP

Test ConditionSoftware : Off codec.Sequences : CIF(Hall monitor,

Coast guard, Foreman) 100 frames.

QP : 29Test algorithm

◦Coventional algorithm (Deblocking Filter in MPEG-4 VM 8.0

coder )

◦proposed algorithm.

Experimental ResultsVideo format Frame rates

Sequence QPBitrate(kbps)

No proc-seeing

Ref Proposed

Avg. PSNR

QCIF(10Hz)

Mother & daughter

8 23 35.33 35.43 35.49

Hall monitor 9 24 34.04 34.08 34.25

Foreman 13 50 31.03 31.13 31.19

Coast guard 14 47 29.04 29.01 29.12

CIF(15Hz)

Hall monitor 29 30 29.55 29.64 29.70

Coast guard 29 132 26.79 26.84 26.87

Foreman 29 138 29.26 29.39 29.45

Table I. PSNR Comparison of Test Se-quences

Instruction

QCIF, 100 framesTotal number of in-

structions(computation/pixel)

Ref Proposed Ref Proposed

Addition 8.25 3.17 1970593 758410

Multiplication 0 0 0 0

Division 0 0 0 0

Shift operator 2.82 1.1 565207 222334

Comparison 9.35 3.65 1896809 741120

Table II. Comparing the Computational Complexity

Pros & ConsPros

◦Simple offset-based filtering method Operation only in spatial domain Computational complexity reduction

◦Intermediate Region Avoid to be too coarse

Excessive blurring or inadequate mode

Cons◦Post-filters are not accepted in H.264

In-loop filters are accepted.

Conclusion & Future WorkAs compare with MPEG4 VM8 filter,

the computational complexity of the proposed scheme is reduced without sacrificing the PSNR.

I’m going to study in-loop deblocking filter and H.264 code with JM software.

Q & A

Thank you~