ISAN-DSP GROUP 168 442 Introduction to Image Processing The First Semester of Class 2546 Dr. Nawapak...

ISAN-DSP GROUP

168 442 Introduction to Image Processing

The First Semester of Class 2546

Dr. Nawapak Eua-Anant

Department of Computer Engineering

Khon Kaen University

ISAN-DSP GROUP

Course Syllabus

Date and Time : MW 11.00-12.00 EN 4510, LAB1 TU17-20, LAB2 TH17-20Assessments:

Attendance & Homework 5%Lab and Homework 35%Midterm 30%

Final 30%Grading:

85-100% A, 75-85% B+, 70-75% B, 65-70% C+, 60-65% C, 55-60% D+, 50-55% D, 0-50% F

References:1. Rafael C. Gonzalez and Richard E. Woods, “Digital Image Processing”,

Addison Wesley, 1992.2. Anil K. Jain, “Fundamentals of Digital Image Processing”,

Prentice-Hall, Inc., 1989.3. William K. Pratt, “Digital Image Processing”, 2nd Edition,

Wiley & Sons, Inc., 1991.

ISAN-DSP GROUP

Course Outline

1. Introduction

2. Digital Image Fundamentals

3. Image Transforms

4. Image Enhancement

5. Image Segmentation

6. Image Compression

7. Image Morphology

ISAN-DSP GROUP

Chapter 1

Introduction to Image Processing

ISAN-DSP GROUP

What is Digital Image Processing ?

Processing of a multidimensional pictures by a digital computer

การประมวลผลสั�ญญาณร�ปภาพโดยใช้�ด�จิ�ตอลคอมพ�วเตอร�

Why we need Digital Image Processing ?

1 .เพ��อบั�นทึ#กและจิ�ดเก%บัภาพ2. เพ��อปร�บัปร&งภาพให้�ด)ขึ้#+นโดยใช้�กระบัวนการ

ทึางคณ�ตศาสัตร�3. เพ��อช้-วยในการว�เคราะห้�ร�ปภาพ4. เพ��อสั�งเคราะห้�ภาพ5. เพ��อสัร�างระบับัการมองเห้%นให้�ก�บั

คอมพ�วเตอร�

ISAN-DSP GROUP

Digital Image

Digital image = a multidimensionalarray of numbers (such as intensity image) or vectors (such as color image)

Each component in the imagecalled pixel associates withthe pixel value (a single number in the case of intensity images or a vector in the case of color images).

39871532

22132515

372669

28161010

39656554

42475421

67965432

43567065

99876532

92438585

67969060

78567099

ISAN-DSP GROUP

Visual Perception: Human Eye

(Picture from Microsoft Encarta 2000)

ISAN-DSP GROUP

Chapter 2: Digital Image FundamentalsChapter 2: Digital Image Fundamentals

ISAN-DSP GROUP

Visual Perception: Human Eye (cont.)

1. The lens contains 60-70% water, 6% of fat.

2. The iris diaphragm controls amount of light that enters the eye.

3. Light receptors in the retina- About 6-7 millions cones for bright light vision called photopic

- Density of cones is about 150,000 elements/mm2.- Cones involve in color vision.- Cones are concentrated in fovea about 1.5x1.5 mm2.

- About 75-150 millions rods for dim light vision called scotopic- Rods are sensitive to low level of light and are not involved color vision.

4. Blind spot is the region of emergence of the optic nerve from the eye.

ISAN-DSP GROUP


ISAN-DSP GROUP

Image Formation in Human Eye

ISAN-DSP GROUP


ISAN-DSP GROUP

Brightness Adaptation of Human Eye

Position

Inte

nsi

ty

Intensities of surrounding points effect perceived brightness at each point.In this image, edges between bars appear brighter on the right side and darkerOn the left side.

ISAN-DSP GROUP

In area A, brightness perceived is darker while in area B isbrighter. This phenomenon is called Mach Band Effect.

Brightness Adaptation of Human Eye (cont.)

Position

Inte

nsity

AB

ISAN-DSP GROUP

Brightness Adaptation of Human Eye (cont.)

Simultaneous contrast. All small squares have exactly the same intensitybut they appear progressively darker as background becomes lighter.

ISAN-DSP GROUP

Imaging Geometry: Perspective Transformation

Lens center

x,Xy,Y

(x,y)

z,Z

(X,Y,Z) = world coordinate

(x,y,z) = Camera coordinate system

(X,Y,Z)

= focal length

Image plane

Z

YyZ

Xx

Eq. 1.1

ISAN-DSP GROUP

Z

ZZ

YZ

X

z

y

x

c

Imaging Geometry: Perspective Transformation (cont.)

Question: How can we project the real world object at (X,Y,Z) ontothe image plane (such as photographic film)?

Answer: Relation between camera coordinate (x,y,z) and world coordinate (X,Y,Z) are given by

Since on the image plane z is always zero, z=0, we consider only (x,y)while z is neglected.

Eq. 1.2

ISAN-DSP GROUP


Equation 1.2 is not linear because of Z in the dividers so weintroduce the homogeneous coordinate to solve this problem.

Z

Y

X

wCartesian coordinate

k

kZ

kY

kX

whHomogeneous

coordinate

k = nonzero constant

To convert from the homogeneous coordinate wh to the Cartesian coordinate w, we divide the first 3 components of wh by the fourth component.

ISAN-DSP GROUP

)(100

0100

0010

0001

1ZkkZ

kY

kX

k

kZ

kY

kX

Pwc hh


100

0100

0010

0001

1

P

The perspective transformation matrix for the homogeneous coordinate:

Perspective transformation becomes:

Eq. 1.3

ISAN-DSP GROUP

)(Zk

kZ

kY

kX

c h


z

y

x

Z

ZZ

YZ

X

ZkkZ

ZkkY

ZkkX

c

)(

)(

)(

From homogeneouscoordinate

We get camera coordinate in the image plane:

ISAN-DSP GROUP

Imaging Geometry: Inverse Perspective Transformation

100

0100

0010

0001

1

1

Phh cPw 1 where Eq. 1.4

ISAN-DSP GROUP

For an image point (x0,y0), since on the image plane z=0, we have

k

ky

kx

ch 00

0

Inverse Perspective Transformation (cont.)

???

0

or 0 0

00

0

1

y

x

w

k

ky

kx

cPw hh

We get the world coordinate :

Since the perspective transformation maps 3-D coordinates to 2-D Coordinates,we cannot get the inverse transform unless we have additional information.

ISAN-DSP GROUP

To find the solution, let

k

kz

ky

kx

ch0

0


We get

z

zZ

z

yY

z

xX

w

zkkz

ky

kx

cPw hh

0

0

0

0

1 or

)(Eq. 1.5

ISAN-DSP GROUP

)(

)(

0

0

Zy

Y

Zx

X


From Eq. 1.5,

Z

Zz

We get Eq. 1.6

Substituting Eq. 1.6 into Eq.1.5, we get

Eq. 1.7

Equations 1.7 show that inverse perspective transformation requiresinformation of at least one component of the world coordinate of the point.

ISAN-DSP GROUP


)( and )( 00 Zy

YZx

X

These equations:

show that points on Line L in the world coordinatemap to a single point in the image plane.

Lens center

x,Xy,Y

z,Z

(x,y)

Image plane

(X2,Y2,Z2)(X3,Y3,Z3)

Points (X1,Y1,Z1) , (X2,Y2,Z2), and (X3,Y3,Z3) map to Point (x,y) in the image plane.

(X1,Y1,Z1)

Line L

ISAN-DSP GROUP

yx

Stereo Imaging: How we get depth information from 2 eyes

Left lens center

(x2,y2)

Image 2

yx

right lens center

B

World point (X,Y,Z)

Optical axis

Image 1

(x1,y1)

ISAN-DSP GROUP

Stereo Imaging: How we get depth information from 2 eyes (cont.)

Problem: we know camera coordinates of the object on left and right image planes (x1,y1) and (x2,y2) and want to how far from the camera the object is located.Note: when y-axis is parallel to the ground, we have y1 = y2

(x2,y2)

(x1,y1)

B w

Z

Image 2

Image 1

Plane of constant Z

X

Origin of world coordinate

system

ISAN-DSP GROUP

1. From the inverse perspective transform, we compute X1 and X2:

)( and )( 22

211

1 Zx

XZx

X

3. Since left and right lenses are separated by distance B, we haveBXX 12

2. Z1 and Z2 must be equal, we get 21 ZZ


4. From 1, 2 and 3, we get

)( and )( 21

11 Z

xBXZ

xX

Solving Z yields

12 xx

BZ

ISAN-DSP GROUP


12 xx

BZ

We can locate the object if we know positions of the objectin left and right image planes using Equation:

Question: While the equation is so simple but why it is verydifficult to built an automatic stereo vision system that canreconstruct 3-D scene from images obtained from 2 cameras?

Answers: for a computer, locating the corresponding points on left and right images is the most difficult task.

ISAN-DSP GROUP

Imaging Geometry : Affine Transformations

1. Translation

2. Scaling

3. Rotating

ISAN-DSP GROUP

Image Geometry: Translation of Object

Y

Z

X

Displace the object by vector (X0,Y0,Z0) with respect toits old position.

11000

100

010

001

10

0

0

*

*

*

Z

Y

X

Z

Y

X

Z

Y

X

0*

0*

0*

ZZZ

YYY

XXX

(X+X0,Y+Y0,Z+Z0)

(X,Y,Z)

(X0,Y 0

,Z 0)

ISAN-DSP GROUP

Image Geometry: Translation of Frame

Y

Z

X

(0,0,0)

Translate the origin point of the frame by (X0,Y0,Z0)with respect to the old frame

11000

100

010

001

10

0

0

*

*

*

Z

Y

X

Z

Y

X

Z

Y

X

0*

0*

0*

ZZZ

YYY

XXX

Y

Z

X (X0,Y

0,Z

0)

The object still stays at the same position. Only the frame is moved.

ISAN-DSP GROUP

Image Geometry: Scaling Scale by factors Sx, Sy, Sz along X, Y, and Z axes.

a b

c

de

f g

A B

C

DE

F G

11000

000

000

000

1

*

*

*

Z

Y

X

S

S

S

Z

Y

X

z

y

x

ZSZ

YSY

XSX

z

y

x

*

*

*

Note: Origin point is unchanged.

ISAN-DSP GROUP

Image Geometry: Rotating an object about X-axis Rotate an object about X-axis by x in a counterclockwisedirection.

Y

Z

X=X

x

11000

0cossin0

0sincos0

0001

1

*

*

*

Z

Y

X

Z

Y

X

xx

xx

Note : In this case the object is moved. Only y and z are changed while x stills the same.

ISAN-DSP GROUP

Image Geometry: Rotating a frame about X-axis Rotate the frame about X-axis by x in a counterclockwise direction.

Y

Z

x

11000

0cossin0

0sincos0

0001

1

*

*

*

Z

Y

X

Z

Y

X

xx

xx

Y

Z

Note : In this case the object is not moved. The frame is rotated instead.

X=X

ISAN-DSP GROUP

Image Geometry: Rotating an object about Y-axis Rotate an object about Y-axis by y in a counterclockwisedirection.

Z

X

y

11000

0cos0sin

0010

0sin0cos

1

*

*

*

Z

Y

X

Z

Y

X

yy

yy

Note : In this case the object is moved. Only x and z are changed while y stills the same.

Y= Y

ISAN-DSP GROUP

Image Geometry: Rotating a frame about Y-axis Rotate the frame about Y-axis by y in a counterclockwise direction.

Y= Y

Z

X

y


Z

X

11000

0cos0sin

0010

0sin0cos

1

*

*

*

Z

Y

X

Z

Y

X

yy

yy

ISAN-DSP GROUP

Image Geometry: Rotating an object about Z-axis Rotate an object about Z-axis by z in a counterclockwisedirection.

Y

Z

X

z

Note : In this case the object is moved. Only x and y are changed while z stills the same.

11000

0100

00cossin

00sincos

1

*

*

*

Z

Y

X

Z

Y

X

zz

zz

ISAN-DSP GROUP

Image Geometry: Rotating a frame about Z-axis Rotate the frame about Z-axis by z in a counterclockwise direction.

Y

Z= Z

X

z

Y


11000

0100

00cossin

00sincos

1

*

*

*

Z

Y

X

Z

Y

X

zz

zz

X

ISAN-DSP GROUP

X

Y

Z

World Coordinate System

Image Geometry: How to compute a point on an imageplane from the world coordinate

Problem: we know the location of the objectand want to know where it will be projectedon the film (image plane).

y

z

x

Camera Coordinate

System

Answer: 1. Transform the world coordinate to the camera coordinate 2. Perform the perspective transformation

ISAN-DSP GROUP

Image Geometry: How to compute a point on the imageplane from the world coordinate (cont.)

Lens center

x

y

(x,y)

z

(x,y,z) = Camera coordinate, (X,Y,Z) = World coordinate

Image plane (z=0)

Before using the perspective transformation, the world axes X-Y-Z must coincide with the camera axes x-y-z, (we need some transformations).

(X,Y,Z)X

Z

Y

ISAN-DSP GROUP

Image Geometry: Compute the camera coordinate fromthe world coordinate

X

Y

Z


y

z

x

Camera Coordinate

System

2

3

Gimbal center

1. Translate* by w0

Note : *perform on the frame

2. Pan the camera* (rotate about Z-axis)

4. Translate* by z = r

Steps from Gonzalez’s book

3. Tilt the camera* (rotate about X-axis)

5. Compute the perspective Tr.

1w 0

4r

ISAN-DSP GROUP

Formula from Gonzalez’s book

hh wGRCPc

Perspective tr.

Translate to the image planeCenter by z = r

Translate to thegimbal center w0

World coordinate

Camera coordinate

Image Geometry: Compute the camera coordinate fromthe world coordinate (cont.)

Rotate byPan (z)

and Tilt (x)

ISAN-DSP GROUP

X

Y

Z


y

z

x

Camera Coordinate

System

2

3

4

Gimbal center

1. Translate* by w1

2. Pan the camera*

3. Tilt the camera*

Note : *perform on the frame

General case

4. Twist the camera* (rotate about Y-axis)

5. Compute the perspective Tr.

w11


ISAN-DSP GROUP


General case

hh wTRPc

Perspective tr.

Translate to the image

plane center w1

World coordinate

Camera coordinate

Rotate byPan (z), Tilt (x),

Twist (y)

ISAN-DSP GROUP 168 442 Introduction to Image Processing The First Semester of Class 2546 Dr. Nawapak...

Documents

Transcript of ISAN-DSP GROUP 168 442 Introduction to Image Processing The First Semester of Class 2546 Dr. Nawapak...