Bernd Girod: EE398A Image Communication I Image Compression Overview no. 1
Image Compression Overview
Goal of this chapter: provide a first introduction of some key terms and ideas of image compression, without rigorous treatmentLossless vs lossy compressionMeasuring distortion and compressionStatistical redundancy and entropyCompression as a vector quantization problemQuantizationTransformsCompression with predictive feedback
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 2
Digital images
A digital image is a two-dimensional sequence of samples
[ ]1 2 1 1 2 2, , 0 , 0 x n n n N n N≤ < ≤ <
2 1N −
1 1N −
01
0 1 2 2n
2
[ ]1 2,x n n1n
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 3
Discrete image intensities
Unsigned B-bit imagery
Signed B-bit imageryMost common: B=8, but larger B are used in medical, military, or scientific applications.Useful interpretation
[ ] { }1 2, 0,1, , 2 1Bx n n ∈ −…
[ ] { }1 1 11 2, 2 , 2 1, , 2 1B B Bx n n − − −∈ − − + −…
[ ] [ ]1 2 1 2, 2 ,Bx n n x n n′=
Real-valued intensities, range 0…1
or –1/2…+1/2
Rounding to nearest
integer
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 4
Multiple image components
Color images typically represented by three values per sample location, for red, green and blue primary components
General multi-component image
Examples:Color printing: cyan, magenta, yellow, black dyes, sometimes moreHyperspectral satellite imaging: 100s of channels
[ ] [ ] [ ]1 2 1 2 1 2, , , , , R G Bx n n x n n x n n
[ ]1 2, , 1, 2, , Cx n n c C= …
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 5
Some classes of imagery
“Natural” image
Text image
Graphics
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 6
Lossless compression
Minimize number of bits required to represent original digital image samples w/o any loss of information.All B bits of each sample must be reconstructed perfectly.Achievable compression usually rather limited.Applications
Binary images (facsimile)Medical imagesMaster copy before editingPalettized color images
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 7
Lossy compression
Some deviation of decompressed image from original(“distortion”) is often acceptable:
Human visual system might not perceive loss, or tolerate it.Digital input to compression algorithm is imperfect representation of real-world scene
Much higher compression than with lossless.Lossy compression used widely for natural images (e.g. JPEG) and motion video (e.g. MPEG).
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 8
Lossy compression: measuring distortion
Most commonly employed: Mean Squared Error
. . . or, equivalently, Peak Signal to Noise Ratio
AdvantagesEasy calculationMathematical tractability in optimization problems
DisadvantageNeglects properties of human vision
[ ] [ ]( )1 2
1 2
1 12
1 2 1 20 01 2
1 ˆMSE= , ,N N
n nx n n x n n
N N
− −
= =
−∑ ∑
( )2
10
2 1PSNR=10log dB
MSE
B −
Gamma correction
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 9
Display devices (e.g. CRTs) highly nonlinear
Cameras compensate by γ-predistortion circuitryIEC 61966-2-1 standard for γ-predistorted color space sRGB (“standard RGB”), each normalized linear RGB component mapped by
~ , 1.8 2.8L xγ γ = …Screen
luminance
( ) 1/
if 01 if 1
lin lin
lin lin
gx xx
x xγ
εβ β ε
≤ ≤⎧′ = ⎨ + − ≤ ≤⎩
( ) ( )2.4, 0.055, ,
111 1g
γ
β βγ β εε γ
βγ
⎛ ⎞⎜ ⎟⎜ ⎟= = = =⎜ ⎟ −⎛ ⎞
− −⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠⎝ ⎠
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 10
sRGB γ-characteristic
linx
x′
Linearpiece
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 11
Effect of a small error
linx
x′
dx′
lindx
( )( )
1
1 1/
lin
lin
dx x dx
x dx
γ
γ
γ
γ
−
−
′ ′=
′=
Roughly matchesWeber’s Law.
PSNR distortionmeasure only usefulin the γ-correcteddomain.
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 12
Measures of compression
Image represented by “bit-stream” c of length ||c||.Compare no. of bits w/ and w/o compression
Alternatively
For lossy compression, bit-rate more meaningful than compression ratio, as B is somewhat arbitrary.
1 2compression ratio = N N Bc
1 2
bit-rate = bits/pixelN N
c
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 13
Typical bit-rates after compression
Substantially dependent on image content: consider typical natural imagesLossless compression: (B-3) bpp (bits per pixel)Assume viewing on computer monitor, 90 pixels/inch.Lossy compression,
high quality: 1 bppmoderate quality: 0.5 bppusable quality: 0.25 bpp
Perceived distortion depends on sampling density and contrast
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 14
How does compression work?
Exploit statistical redundancy.Take advantage of patterns in the signal.Describe frequently occuring events efficiently.Lossless coding: only statistical redundancy
Introduce acceptable deviations.Omit “irrelevant” detail that humans cannot perceive.Match the signal resolution (in space, time, amplitude) to the applicationLossy coding: exploit statistical and visual redundancy
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 15
Statistical redundancy
Trivial example: Given two B-bit integers (e.g., representing two adjacent pixels)
Assume that only takes on values {0,1}Compression to 1 bpp
Further assume, that Compression to 0.5 bpp
Hope: bit-rate increases only slightly, as long as the above assumptions hold with high probability
{ }1 2, 0,1, , 2 1Bx x ∈ −…
1 2,x x
1 2x x=
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 16
Joint histogram of two horizontally adjacent pixels
relative frequency
of occurence
amplitude of adjacent pixel
amplitude of current pixel
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 17
Entropy
Statistical redundancy is formalized by the concept of entropy of Shannon’s information theoryMinimum bit-rate for encoding or separately
Minimum bit-rate for encoding and jointly
Entropies are determined entirely by the statistical distribution of values and
1x 2x
( ) ( )1 2 H x H x
1x 2x
( ) ( ) ( )1 2 1 2, H x x H x H x≤ +
( ) ( ) ( )1 2 1 2, , , H x H x H x x1x 2x
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 18
Visual Irrelevance
For images to be viewed by humans, no need to represent more than the visible resolution in
spacetimebrightnesscolor
Required resolution might depend on image content (“masking”)For some applications, only a specific region of the image might be relevant, e.g., in
medical imagingmilitary imaging
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 19
Irrelevance in Color Imagery
Human visual system has much lower acuity for color hue and saturation than for brightnessUse color transform to facilitate exploiting that property
Note
Cb and Cr often sub-sampled 2:1 relative to Y.
0.299 0.587 0.1140.169 0.331 0.50.5 0.419 0.0813
Y R
Cb G
Cr B
x xx xx x
⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟ ⎜ ⎟= − − ⋅⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟ ⎜ ⎟− −⎝ ⎠ ⎝ ⎠ ⎝ ⎠
RGB components (γ-predistorted)
Chrominancecomponents
( ) ( )0.564 0.713Cb B Y Cr R Yx x x x x x= − = −
Luminancecomponent
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 20
Compression as a global mapping
( )compressor
M=c x ( )1
decompressor
ˆ M −=x c
image x bit-stream c
reconstructed ˆimage x
1 2
lookup table2 entriesN N B
lookup table
2 entriescfixed length cLookup table
interpretation
Lossless compressionLossy compression
Leads to idempotent compression system
1 1M M− −=
( ) ( )( )1arg min ,M D M −
′′= =
cc x x c
( )( )1M M − =c c
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 21
Compression as a global mapping (cont.)
Variable length encoding: need formulation of trade-off between rate and distortionLookup table formulation corresponds to approach taken by fixed-length vector quantizationSize of lookup tables for entire image is impractical (both for encoder and decoder)Impose structure on mappings and
Avoid exponential increase of complexity with image size and size of bit-streamMaintain achievable performance close to unconstrained system
1M −M
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 22
Typical structured compression system
( )transform
T=y x ( )quantizer
Q=q y ( )encoder
C=c qsamples yimage x indices q
( )1
inversetransform
ˆ ˆT −=x y ( )1
dequantizer
ˆ Q−=y q ( )1
decoderC−=q c
indices qˆsamples yreconstructed
ˆimage x
bit-stream c
Transform T(x) usually invertibleQuantization not invertible, introduces distortionCombination of encoder and decoder lossless
( )Q y( )C q ( )1C− c
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 23
Variable length coding
Goal: exploit non-uniformity of probability distribution of quantization indicesSimple example: Probabilities Code words
Average code word length
Often much bigger gains for image compression
1/ 2 if 0 11/ 4 if 1 01
( ) 1/ 8 if 2 0011/ 8 if 3 000
q
P qqq
=⎧⎪ =⎪= ⎨ =⎪⎪ =⎩
1 1 11 bits 2bits 2 3bits 1.75 bits2 4 8⋅ + ⋅ + ⋅ ⋅ =
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 24
Quantization
Goal: reduce the number of possible amplitude values for codingSimple scalar quantizer with four output indices
Dequantization
Interval 1 Interval 2 Interval 3 Interval 4
y
q=3
Interval 1 Interval 2 Interval 3 Interval 4
2y 3y4y1y
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 25
Transforms
Goal: represent the image samples in a different form, such that statistical dependencies are greatly reducedSimple example:
-- -
imagesamples
x[m,n]
transformsamplesy[m,n]
[ ][ ]
}{
[ ]
, 0,1
1 , if , both even4
,, 2 ,2 otherwise
2 2
i j
x m i n j m n
y m nm nx m n y
∈
⎧ + +⎪⎪= ⎨
⎡ ⎤⎢ ⎥ ⎢ ⎥⎪ − ⎢ ⎥⎢ ⎥ ⎢ ⎥⎪ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎩
∑[ ]
[ ] [ ]}{
[ ]
, 0,1
ˆ ˆ2 , , if , both even
ˆ ,ˆ ˆ, 2 ,2 otherwise
2 2
i j
y m n y m i n j m n
x m nm ny m n y
∈
⎧ − + +⎪⎪=⎨ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎪ − ⎢ ⎥⎢ ⎥ ⎢ ⎥⎪ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎩
∑
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 26
[ ]indices q k
Compression with predictive feedbackin the transform domain
( )transform
T=y x [ ] [ ]( )quantizer
Q e=q k k ( )encoder
C=c q[ ]
samples y kimage x
[ ]indices q k
( )1
inversetransform
ˆ ˆT −=x y ( )1
decoderC−=q c
reconstructedˆimage x
bit-stream c
Σ [ ] [ ]( )1
dequantizer
e Q q−=k k
Σ
[ ] [ ]( )1 2
causal predictorˆ , 1 ,py P y k k= −k …
[ ] [ ]( )1
dequantizer
e Q q−=k kΣ
[ ] [ ]( )1 2
causal predictorˆ , 1 ,py P y k k= −k …
- [ ]residual e k
[ ]e k[ ]y k
[ ]e k[ ]y k
[ ]p
prediction y k
[ ]p
prediction y k
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 27
Compression with predictive feedbackin the image domain
residual
e ( )
transformT=y e ( )
quantizerQ=q y ( )
encoderC=c q
image x indices q
( )1
decoderC−=q c
reconstructedˆimage x
bit-stream c
Σ( )1
dequantizer
ˆ Q−=y q
Σ
( )causal predictor
ˆP=px x
Σ
-
exprediction
px
( )1
inversetransform
ˆ ˆT−=e yy
prediction
px( )
causal predictorˆP=px x
( )1
dequantizer
ˆ Q−=y qe
( )1
inversetransform
ˆ ˆT−=e yy indices q
Bernd Girod: EE398A Image Communication I Image Compression Overview no. 28
Reading Assignment
Taubman+Marcellin, Chapters 2.1 and 2.2
Top Related