lecture2 color quant - NYU Tandon School of...

Lecture 2Color Representation

and Quantizationand QuantizationYao Wangg

Polytechnic Institute of NYU, Brooklyn, NY 11201

With contributions from Zhu Liu, AT&T LabsMost figures from Gonzalez/Woods: Digital Image Processing

Lecture Outline• Color perception and representation

– Human perception of colorp p– Trichromatic color mixing theory– Different color representations

C l i di l• Color image display– True color image– Indexed color imagesIndexed color images

• Pseudo color images• Quantization FundamentalsQuantization Fundamentals

– Uniform, non-uniform, dithering• Color quantization

Yao Wang, NYU-Poly EL5123: color and quantization 2

Light is part of the EM wave


Illuminating and Reflecting Light• Illuminating sources (primary light):

– emit light (e.g. the sun, light bulb, TV monitors)emit light (e.g. the sun, light bulb, TV monitors)– perceived color depends on the emitted freq. – follow additive rule

• R+G+B=White

• Reflecting sources (secondary light): – reflect an incoming light (e.g. the color dye, matte

surface, cloth)– perceived color depends on reflected freq (=emitted– perceived color depends on reflected freq (=emitted

freq-absorbed freq.)– follow subtractive rule


• R+G+B=Black

Eye vs. Camera

Camera components Eye componentsCamera components Eye componentsLens Lens, corneaShutter Iris, pupilFil R ti


Film RetinaCable to transfer images Optic nerve send the info to

the brain

Human Perception of Color

• Retina contains receptorsCones– Cones

• Day vision, can perceive color tone• Red, green, and blue cones

– Rods• Night vision, perceive brightness only

Color sensation• Color sensation– Luminance (brightness)

Chrominance From – Chrominance• Hue (color tone)• Saturation (color purity)

http://www.macula.org/anatomy/retinaframe.html


Frequency Responses of Cones and the Luminous Efficiency FunctionLuminous Efficiency Function


ybgridaCC ii ,,,,)()(

Tri-chromatic Color Mixing• Tri-chromatic color mixing theory

– Any color can be obtained by mixing three primary colors with a right proportionright proportion

• Primary colors for illuminating sources:– Red, Green, Blue (RGB)– Color monitor works by exciting red, green, blue phosphors

using separate electronic guns– follow additive rule: R+G+B=White

• Primary colors for reflecting sources (also known as secondary colors):– Cyan, Magenta, Yellow (CMY)– Color printer works by using cyan, magenta, yellow and black

(CMYK) dyes– follow subtractive rule: R+G+B=Black


RGB vs CMY

Magenta = Red + BlueCyan = Blue + Green

Magenta = White - GreenCyan = White - Red


Cyan Blue GreenYellow = Green + Red

Cyan White RedYellow = White - Blue

Demo with Photoshop

• Show the mixing using RGB, CMY colorsUsing the “window>color” tool (in “show– Using the window>color tool (in show color”, click on right arrow button to choose different color sliders) (can also click F6)different color sliders) (can also click F6)

– Sample image: RGBadd, CMYsub• Use the dropper tool, click on different part of theUse the dropper tool, click on different part of the

image, see the color information in “show color”• Select different color coordinates


A Color Image

Red

Green Blue


Tristimuls Values• Tristimulus value

– The amounts of red green and blue neededThe amounts of red, green, and blue needed to form any particular color are called the tristimulus values, denoted by X, Y, and Z.

– Trichromatic coefficients

.,,ZYX

ZzZYX

YyZYX

Xx

– Only two chromaticity coefficients are necessary to specify the chrominance of a

,,ZYXZYX

yZYX

necessary to specify the chrominance of a light. 1 zyx


CIE Chromaticity DiagramCIE (Commission Internationale de L’Eclairage, International Commission onCommission on Illumination ) system of color specification

x axis: redy axis: greene.g. GREEN:gx: 25%, y: 62%, z: 13%.

Colors on the boundary: spectrum colors highest


spectrum colors, highest saturation

Color Gamut

A line segment indicates all colors that can be produced by mixing two colors corresponding to the end points of the line.E h l d l h diff t l ( t) RGB d l h l t th CMY


Each color model has different color range (or gamut). RGB model has a larger gamut than CMY. Therefore, some color that appears on a screen may not be printable and is replaced by the closest color in the CMY gamut.

Demo with Photoshop• Using photoshop to show how to replace a out of

gamut color by its closest in-gamut color.gamut color by its closest in gamut color.• Choose “window->show color”, choose the

“CMYK slider”• While having “RGBadd” in display, use

Eyedropper tool to click on “red”, “green” or “blue”, you will see a warning about out of gamut color, and the suggested replacement color

• Also open “gamut” image, click on out of gamut region


Three Attributes of Color• Luminance (brightness)• Chrominance

– Hue (color tone) and Saturation (color purity)• Represented by a “color cone” or “color solid”

θ


Color Models• Specify three primary or secondary colors

– Red, Green, Blue.– Cyan, Magenta, Yellow.

• Specify the luminance and chrominance– HSB or HSI (Hue, saturation, and brightness or

intensity)– YIQ (used in NTSC color TV)Q ( )– YCbCr (used in digital color TV)

• Amplitude specification: – 8 bits per color component, or 24 bits per pixel– Total of 16 million colors

A 1kx1k true RGB color requires 3 MB memory


– A 1kx1k true RGB color requires 3 MB memory

RGB Color Model

RGB 24-bit color cube


CMY and CMYK Color Models• Conversion between RGB and CMY (assuming

maximum value is 1)

11

11

MC

GR

GR

MC

.11,

11

Y

MBG

BG

YM

• Equal amounts of Cyan, Magenta, and Yellow produce black. In practice, this produce muddy-looking black. To p p y gproduce true black, a fourth color, black is added, which is CMYK color model.


HSI Color Model

• Hue represents dominant color as perceived by an observer It is an attributeperceived by an observer. It is an attribute associated with the dominant wavelength.Saturation refers to the relative purity or• Saturation refers to the relative purity or the amount of white light mixed with a hue. The pure spectrum colors are fullyThe pure spectrum colors are fully saturated. Pink and lavender are less saturatedsaturated.

• Intensity reflects the brightness.


Conversion Between RGB and HSI

• Converting color from RGB to HSI1

3

))(()(

)()(21

cos360

21

2

1

BGBRGR

BRGRwith

GBifGBif

H

][31

)],,[min()(

31

BGRI

BGRBGR

S

• Converting color from HSI to RGBRG sector (0≤H<120) GB sector (120≤H<240) BR sector (240≤H<360)

)60cos(cos1

)1(

HHSIR

SIB

( )

))120(60cos()120cos(1

)1(

HHSIG

SIR

))240(60cos()240cos(1

)1(

HHSIB

SIG


)(1)(

BRG

)(1 GRB )(1 BGR

YIQ Color Coordinate System

• YIQ is defined by the National Television System Committee (NTSC)System Committee (NTSC)– Y describes the luminance, I and Q describes

the chrominancethe chrominance.– A more compact representation of the color.

YUV plays similar role in PAL and SECAM– YUV plays similar role in PAL and SECAM.• Conversion between RGB and YIQ

QIY

BGR

BGR

QIY

703.1106.10.1649.0272.00.1

621.0956.00.1,

311.0523.0211.0322.0274.0596.0

114.0587.0299.0


QQ 703.06.0.3.05 3.0.0

YUV/YCbCr Coordinate• YUV is the color coordinate used in color TV in PAL

system, somewhat different from YIQ. • YCbCr is the digital equivalent of YUV, used for digital

TV, with 8 bit for each component, in range of 0-255


Demo with Photoshop

• Show the RGB, CMY, HSI modelsUsing the “window>show color” tool (in “show– Using the window>show color tool (in show color”, click on right arrow button to choose different color sliders)different color sliders)

– Sample image: RGBadd, CMYsub, fruit• Use the dropper tool, click on different part of theUse the dropper tool, click on different part of the

image, see the color information in “show color”• Select different color coordinates


Criteria for Choosing the Color CoordinatesCoordinates

• The type of representation depends on the applications at handapplications at hand.– For display or printing, choose primary colors

so that more colors can be produced E gso that more colors can be produced. E.g. RGB for displaying and CMY for printing.

– For analytical analysis of color differencesFor analytical analysis of color differences, HSI is more suitable.

– For transmission or storage, choose a lessFor transmission or storage, choose a less redundant representation, eg. YIQ or YUV or YCbCr


Comparison of Different Color SpacesSpaces


Pseudo Color Display• Intensity slicing: Display different gray levels

as different colorsas different colors– Can be useful to visualize medical / scientific /

vegetation imagery• E.g. if one is interested in features with a certain intensity

range or several intensity ranges

• Frequency slicing: Decomposing an image intoFrequency slicing: Decomposing an image into different frequency components and represent them using different colors.


Intensity Slicing

Color

C4

C3

C2

C1

Pixels with gray scale (intensity) value in the range of (f f ) are rendered with color C

f0 =0 f2f1 f3 f4 Gray level


Pixels with gray-scale (intensity) value in the range of (f i-1 , fi) are rendered with color Ci

Example


Another Example


Pseudo Color Display of Multiple Images

• Display multi-sensor images as a single color imageimage– Multi-sensor images: e.g. multi-spectral images by

satellite


An Example


Example


Three Processes in A/D Conversion

Quanti-zation

Sampling Binary Encoding

xc(t) x[n] = xc(nT) [ ]x n c[n] Sampling

P i dQuantization

I t lBinary

• Sampling: take samples at time nT

Period T

IntervalQ

codebook

– T: sampling period; – fs = 1/T: sampling frequency

• Quantization: map amplitude values into a set of discrete values kQ– Q: quantization interval or stepsize

• Binary Encoding


y g– Convert each quantized value into a binary codeword

Analog to Digital Conversion

1T=0.1Q=0.25

0

0.5

-0.5

A2D plot m

0 0 2 0 4 0 6 0 8 1

-1

A2D_plot.m


0 0.2 0.4 0.6 0.8 1

How to determine T and Q?• T (or fs) depends on the signal frequency range

– A fast varying signal should be sampled more frequently!– Theoretically governed by the Nyquist sampling theorem

• fs > 2 fm (fm is the maximum signal frequency)• For speech: fs >= 8 KHz; For music: fs >= 44 KHz;

• Q depends on the dynamic range of the signal amplitude and perceptual sensitivity– Q and the signal range D determine bits/sample Rg g p

• 2R=D/Q• For speech and image: R = 8 bits; For music: R =16 bits;

• One can trade off T (or fs) and Q (or R)( s) Q ( )– lower R -> higher fs; higher R -> lower fs

• We discuss quantization in this lecture. We will discuss sampling later


discuss sampling later.

A Quantizer

Quantizerf Q(f)Q(f)

r7

r5

Decision Levels {tk, k = 0, …, L}

Reconstruction Levels {rk, k = 0, …, L-1}

)[f

r6

r7

ft1 t2 t5

If ),[ 1 kk ttfThen Q(f) = rk t6L levels need bits LR log

t3 t7r3

t4

r4

r1

Quantizererror

L levels need bits LR 2logr2

r3

x returns the smallest integerthat is bigger than or equal to x

r0

f

r1 r2 r3 r4 r5 r6 r7r0


t1 t2 t3 t4 t5 t6 t7t0=-∞ t8=∞

Uniform Quantization• Equal distances between adjacent decision levels and

between adjacent reconstruction levelst t– tl – tl-1 = rl – rl-1 = q

• Parameters of Uniform Quantization– L: levels (L = 2R)– B: dynamic range B = fmax – fmin– q: quantization interval (step size)– q = B/L = B2-Rq

• Quantization function

N t min

min

2)( fqq

qfffQ

Note:

returns the biggest integer that is smaller

x


gthan or equal to x

Uniform Quantization: Functional Representation

Q(f)

r6

r7=fmax-q/2

r

r4

r5

stepsize q=(fmax-fmin)/8

r

r2

r3

ft t1 t2 t3 t4 t5 t6 t7 t8

r0=fmin+q/2

r1


t0 t1 t2 t3 t4 t5 t6 t7 t8fmin fmax

Truncated Uniform Quantization• Signal with infinite dynamic range• Truncate the lower and higher values to• Truncate the lower and higher values to

fmin and fmax• Parameters• Parameters

– fmin, fmax– L (R)– L (R)

• Quantization function i qff

minmin

maxminminmin

/2/2

)(fffffqf

ffffqqqff

fQ


maxmax 2/ ffqf

Truncated Uniform Quantization:Functional Representation

Q(f)

r =f -q/2

overloadregion

r5

r6

r7=fmax-q/2

r3

r4

r5

r1

r2

ft0 =-∞ t1 t2 t3 t4 t5 t6 t7fmin fmax

r0=fmin+q/2

overloadt8 =∞


overloadregion

Example 1 of Uniform Quantizer

• Input signal is continuousThe output of a Charge Coupled Device– The output of a Charge-Coupled Device(CCD) camera is in the range of 0.0 to 5.0 volt.volt.

– L = 256– q = 5 0 / 256q 5.0 / 256– The output value in the interval (l * q, (l + 1) *

q] is represented by index l, l = 0, …, 255.q] is represented by index l, l 0, …, 255.– The reconstruction level rl = l * q + q/2, l =

0,…, 255.


, ,

Example 2 of Uniform Quantizer

• Input signal is discreteDigital Image of 256 gray levels– Digital Image of 256 gray levels

– Quantize it into 4 levelsf 0– fmin= 0,

– fmax = 256,256 / 4 64– q = 256 / 4 = 64,

– q/2 = 32, – 3264*64/)( ffQ


Uniform Quantization on Images=2

56

32

Orig

inal

, L=

q=8,

L=

O16

, L=1

6

4, L

=4

q=1

q=64


Quantizer Performance• Ideally we want to measure the performance by how

close is the quantized image to the original image --Perceptual Difference

• But it is very hard to come up with an objective measure that correlates very well with the perceptual qualitythat correlates very well with the perceptual quality

• Frequently used objective measure – mean square error (MSE) between original and quantized samples or signal to noise ratio (SNR) or Peak SNR (PSNR)


MSE• Mean square error (MSE) of a quantizer for a continuous

valued signal

t L t

q ffQEMSE1

22 }))({(

Wh (f) i th b bilit d it f ti f f

L l

l

t

tl

t

tl dffprfdffpfQf

0

1

0

22 )()()())((

– Where p(f) is the probability density function of f

• MSE for a specific image 1 1N M

1

0

1

0

22 ))),((),((1 N

i

M

jq jifQjif

MNMSE


0 0i j

SNR and PSNR• Signal to Noise Ratio (SNR)

2f

1 1

2

1

log10)(

N M

q

fdBSNR

P k SNR

0 0

22 )),((1 Energy SignalVariance Signali j

f meanjifMN

• Peak SNR– For the error measure to be

independent of the signal energy, use the dynamic gy, yrange square of the image as the signal energy

– For 8 bit image, peak=255 2

2255log10)(q

dBPSNR


MSE of a Uniform Quantizer for A Uniform SourceUniform Source

otherwise

fffBfffp

,0),(,/1)/(1

)( maxminminmax

,

12

1 222 max

min

BdfB

ff

ff

q/2) (-q/2,in ddistributeuniformly is and same, theisbin each in Error //:levels L intoon quantizati Uniform LBLBq

2 R

fRq

qq

BLBqde

qe 222

22

22/

2/

22 22121212

1

dbhibiddi i l

)(62log202log10log10 102

102

2

10 dBRRSNR R

q

f


6dB!by SNRtheincreasesbit additionalEvery

Example: Nonuniform Source• The pdf of a signal is shown below, we want to quantize

it to 2 levels. Determine the partition and reconstruction levels that minimizes the quantization error (in terms of MSE). Also compute the MSE and SNR.

• Go through in class• Go through in class.

p(f)1/2

fExp(-f)Exp(f)


MMSE Quantizer

• For any pdf, the quantizer that minimizes MSE is known as Minimal MSE (MMSE)MSE is known as Minimal MSE (MMSE) quantizer. For uniform source what is the MMSE• For uniform source, what is the MMSE quantizer?


Implementation of Quantization• Two options• Option 1:Option 1:

– At each pixel, calculating quantized value for the given pixel valueF if ti ti i l di di i i d– For uniform quantization, including one division and rounding, then multiplication

– Must repeat for all pixels• Option 2:

– Precompute the quantized values for all possible input values and save in a look up tableinput values and save in a look-up table

– Change each pixel based on the look-up table• Which one is more efficient?


Which one is more efficient?

Sample Matlab Code for Option 2• Setup the quantization function Q(f) for all possible input

level first.i d('l 256 8 b ')

• Matlab codex = imread('lena256_8.bmp');[height, width] = size(x);x=int(x);B = 256;% Q i th ti ti t bl% Q is the quantization tableQ = zeros(256, 1);% Quantized to 64 levelsL = 16;q B / Lq = B / L;for i = 0:255,

Q(i+1, 1) = floor(i / q) * q + q /2; endy = zeros(size(x));

L = 16 MSE = 21 6

y = zeros(size(x));for i = 1:height,

for j = 1:width,y(i, j) = Q(x(i,j) + 1);

end


L = 16, MSE = 21.6 endEndMSE = mean(mean((x-y).^2));Is there a more efficient implementation?

Quantization Effect – False Contour

• 1-D Signal f(t) )(ˆ tf

• 2-D Image t t

(256 256)x=zeros(256,256);for I =1:256,

for j = 1:256,x(i,j)=max(i,j)-1;dend

endFigure;imshow(uint(x));


f(i,j)=max(I,j)

Perception Optimized Quantization

• Reduce the artifact of false contourBetter is al perception (H man e es are• Better visual perception (Human eyes are very sensitive to edges)

• MSE might be worse


Pseudo Random Noise Quantizer

+Uniform Quantizer

Q()Step size: q

++

+

+

-

f g g f

h

Random NoiseGenerator

Why do we first subtract and then add back the random

i ? Wh t if ki fUniform in (-q/2, q/2)

noise? What if we skip one of them?


Example

• Comparison

8 Level Uniform Quantizer8 Level Pseudo Random Noise Quantizer


MSE=83.7o se Qua t e

MSE=173.45

Which one is better to your eye?

Halftoning• Halftoning – Representing gray scale image using 2

levels (black and white)• Used for printing gray scale photographs in B/W

newspaper/faxCan be extended to more than two levels• Can be extended to more than two levels

+

Two Level Quantizer+f g ffw+

+ fb

Dither Signald

Why don’t we subtract the added dithered signal?


Color Quantization• In low end monitors, the monitor cannot display all

possible colors.• Select a set of colors, save them in a look-up table (also

known as color map or color palette)Any color is quantized to one of the indexed colors• Any color is quantized to one of the indexed colors

• Only needs to save the index as the image pixel value and in the display bufferp y

• Typically: k=8, m=8 (selecting 256 out of 16 million)

Input index (k bits) Red color (m bits) Green color (m bits) Blue color (m bits)

Index 1 … … …


……. … … …

Index 2^k … … …

Web Colors: 216 Safe RGB Colors

These colors are those that can be rendered consistently by different computer systems. They are obtained by quantizing the R,G,B component independently using uniform quanitization. Each component is

i d 6 ibl l 0(0 00) 51(0 33) 102(0 66) 153(0 99) 204(0 CC) 255(0 FF)


quantized to 6 possible values: 0(0x00), 51(0x33), 102(0x66), 153(0x99), 204(0xCC), 255(0xFF).

Uniform vs. Adaptive Quantization• Uniform (scalar quantization)

– Quantize each color component uniformly• E.g. 24 bit-> 8 bit can be realized by using 3 bits (8 levels) for

red, 3 bits (8 levels) for green, 2 bits (4 levels) for blue• Do not produce good resultp g

• Adaptive (vector quantization) – Treat each color (a 3-D vector) as one entity. Find the ( ) y

N colors (vectors) that appear most often in a given image, save them in the color palette (codebook). Replace the color at each pixel by the closestReplace the color at each pixel by the closest color in the codebook

– The codebook (I.e. color palette) varies from image to


The codebook (I.e. color palette) varies from image to image -> adaptive

Illustration of Vector Quantization

y yCodebook size: 25

x x


Uniform Quantization Vector Quantization

Example of Color Quantization24 bits -> 8 bits

Adaptive (non-uniform) quantization(vector quantization)

Uniform quantization(3 bits for R,G, 2 bits for B)


(vector quantization)

Color Dithering• Color quantization may cause contour effect when the

number of colors is not sufficient• Dithering: randomly perturb the color values slightly to

break up the contour effect, by applying the dithering technique for gray scale images to each colortechnique for gray scale images to each color component– fixed pattern or random noise dithering– diffusion dithering (the perturbed value of the next pixel depends

on the previous one)


Example of Color Dithering

8 bit uniformwithout dithering

8 bit uniformwith diffusion dithering


without dithering

Demo Using Photoshop• Show quantization results with different methods

using “image->mode->index color”using image mode index color


Homework1. Describe how the cones in the retina behind the human eye enable us to perceive color. What

are the different types of cones we have? what colors are they each sensitive to?2. What color do you see when you project equal amounts of red, green and blue lights on a screen?

What about red and green, and green and blue?f ?3. What color do you see when you mix equal amounts of cyan, magenta, and yellow? What about

cyan and magenta, magenta and yellow? what about mixing red, green, and blue in equal proportions?

4. Consider the conversion between RGB and YCbCr coordinate. a) Determine the YCbCr coordinate for a color that is specified in (R,G,B) coordinate with R=100,G=200, B=50. b) Determine the (R G B) coordinate for a color specified in the YCbCr coordinate withDetermine the (R,G,B) coordinate for a color specified in the YCbCr coordinate with Y=100,Cb=50,Cr=30. Note that if you get R,G,B values that are outside the range of (0,255) you should clip the values, so that values below 0 become 0, and values above 255 become 255.

5. Suppose you are given a 24 bit color image, with color components red, green, blue each represented by 8 bits per pixel. Now, we would like to reduce it to an 8 bit image by uniformly quantizing the red and green components to 3 bits each, and the blue component to 2 bits. g gIllustrate the quantizer that you would use for different components. For designing the quantizer, you should assume the image values are uniformly distributed between 0 and 256 (instead of 255). Suppose a pixel has a color value of (R,G,B)=(200, 150, 40). What is the color after quantization?

6. Consider a source with a pdf as shown below. If we want to quantize it into L=3 levels, what should be the decision levels and reconstruction levels? Illustrate the results on the pdf andshould be the decision levels and reconstruction levels? Illustrate the results on the pdf, and compute the quantizer MSE and SNR.

p(f) 1


f-1 1

1

Computer Assignment1. Using Matlab or C, to implement uniform quantization of a grayscale image. Your

program should do the following: i) read a grayscale image into an array; ii) quantize and save the quantized image in a different array; iii) compute the MSE, SNR, and PSNR between the original and quantized images; and iv) display and print the g q g ) p y pquantized image. Your program should assume the input values are in the range of (0,256), but allow you to vary the reconstruction level (L). Record the MSE obtained with L=64, 32, 16, 8, and print the quantized images with corresponding L values. Include the program, the image printout and MSE/SNR/PSNR values in your submission. Comment on the image quality as you vary L.g q y y y

2. Choose a 24 bit color RGB image, quantize the R, G, and B components to 3, 3, and 2 bits, respectively, using a uniform quantizer. Display the original and quantized color image. Comment on the difference in color accuracy. Make sure you use a computer that has a 24 bit color display, and the test image has good color contrast. Note you can apply the uniform quantization function developed for Prob. 1 to eachNote you can apply the uniform quantization function developed for Prob. 1 to each color component. For this assignment, please include color printout of original and quantized images. If you don’t have access to color printer, include a disk containing your image files, or email the files to the grader.

3. Choose a 24 bit RGB color image, convert it to HSI format, and display the HSI components as separate gray scale images Observe these images to comment oncomponents as separate gray scale images. Observe these images to comment on what does each of the H, S, I components represent. Include printout of the images and your program and comments in your report. For this assignment, you should not use the MATLAB built-in function for RGB to HSV conversion. You should write your own function. The HSV images should be saved in double precision. You can use imagesc( ) to display these images directly which automatically scale each image to


imagesc( ) to display these images directly, which automatically scale each image to a gray scale between 0-255.

Reading

• Gonzalez, “Digital Image Processing”, Chapter 2 (Section 2 1 2 2 2 3 2 4 1 –Chapter 2 (Section 2.1, 2.2, 2.3, 2.4.1 –2.4.3), Chap. 6. (Sec. 6.1-6.3), Sec. 8.2 (p602-603 MMSE quantizer)(p602-603, MMSE quantizer)

• Jain, “Fundamentals of Digital Image Processing” Chapter 4 (Section 4 5Processing”, Chapter 4 (Section 4.5 –4.11)


lecture2 color quant - NYU Tandon School of...

Documents

Transcript of lecture2 color quant - NYU Tandon School of...