In the name of Allah - ce.sharif.educe.sharif.edu/courses/85-86/1/ce823/resources/root/lecture...

In the name of Allah

the compassionate, the merciful

Digital Image ProcessingDigital Image Processing

S. KasaeiS. KasaeiSharif University of Technology

Room: CE 307 E-Mail: [email protected]

Home Page: http://ce.sharif.eduhttp://ipl.ce.sharif.eduhttp://sharif.edu/~skasaei

Kasaei 4

Course SyllabusCourse SyllabusLecture: Sundays & Tuesdays, 13:30-15:00, Room Kh. 7.

Website:

http://ce.sharif.edu/courses/85-86/1/ce823/Check this site often for important announcements, files needed for computer exercises, and the PDF versions of handouts & homework.

Course Description: 40-823+ provides an introduction to image processing theory and techniques.

Kasaei 5

Course SyllabusCourse SyllabusTopics include: 2-D system theory, image perception, image sampling and quantization, image transforms, image representation, image enhancement, image filtering and restoration, image analysis, and image/video compression.

Prerequisites: 40-763 (Digital Signal Processing) or40-933 (Digital Image Processing)

Text Book:Fundamentals of Digital Image Processing, by Anil K. Jain, Prentice Hall, 1989.Chapters 1, 2, 3, 4, 5, 7, & 9 will be wholly covered, the rest will be partially covered.

Kasaei 6

Course SyllabusCourse SyllabusAdditional topics will be included (e.g., video compression, wavelet transform, texture analysis, watershed, snakes…).

Written Homework Problems:Written homework problems will be assigned by the end of each chapter.

Computer Exercises:Computer exercises will also be assigned over the course. They can be provided either in Matlab or C.

Term Project:There will be a term project, which can be proposed by the student. Students are supposed to present the final result, associated with related software & technical report.

Kasaei 7

Course SyllabusCourse SyllabusExams:There will be one midterm and one final exam.

Grading Policy:Written & computer assignment: 3 pts.Project: 3 pts.Project Report: 1 pts.Project Presentation: 1 pts.Midterm exam: 3 pts. (hold at: 1385.8.30)Final exam: 9 pts. (hold at: 1385.10.20, 14:30)Submitted paper: 2 extra pts.

Project Topic Confirmation Due:1385.10.20

Kasaei 8

Course SyllabusCourse Syllabus

Instructor Office Hour:Sundays, 17:00-18:00, Room CE 307.

Teaching Assistants:Ms. M. Hassanzadeh & Mr. A.A. Darabi

Kasaei 9

Chapter 1:Chapter 1:

Introduction to Image Processing

Kasaei 10

IntroductionIntroduction

Kasaei 11


Kasaei 12


Kasaei 13


Kasaei 14

IntroductionIntroductionImage:any 2-D function that bears information.

Digital Image:an array of real or complex numbers represented by a finitenumber of bits.

Digital Image Processing:digital processing of any 2-D data.

Applications:remote sensing via satellites, image transmission & storage, medical processing, radar, sonar, robotics.

Kasaei 15

Basic SubjectsBasic SubjectsImage representation & modeling,

Image transform,

Image enhancement,

Image restoration,

Image analysis,

Image data compression.

Kasaei 16

Image Representation & ModelingImage Representation & ModelingAn image can present:

luminance of objects in a scene (picture),

absorption characteristics of body tissue (X-ray),

radar cross section of a target (radar imaging),

temperature profile of a region (infrared imaging),

gravitation field in an area (geophysical imaging).

Kasaei 17

Image Representation & ModelingImage Representation & Modeling

Infrared image. Angiography image.

Kasaei 18


X-ray images.

Kasaei 19


UV images.

Kasaei 20


CT image.

Kasaei 21


Multispectral Geostationary Operational Environment Satellite image.

Kasaei 22


Scanning Electron Microscope images.

Kasaei 23


Computer Generated Images.

Kasaei 24

Image Representation & ModelingImage Representation & ModelingFundamental requirement of digital processing: sampling & quantization.

The sampling rate (# pixels/unit area) has to be large enough to preserve useful information.

Quantization is an A/D conversion of a sampled image into a finite number of gray levels.

e.g., raster scanned common TV signal bandwidth: 4 MHzminimum sampling rate: 8 MHzframe pixels for 30 frames/sec: 8x10m/30=266,000image size for 512-line raster: 512x512.

Kasaei 25

Image Representation & ModelingImage Representation & ModelingA classical method of signal representation is by an orthogonal series expansion, such as Fourier series.

For images, analogous representation is possible via 2-D orthogonal function called basis images.

For sampled images, the basis images can be determined from unitary matrices called image transforms.

Any given image can be expressed as a weighted sum of basis images.

Statistical models describe an image as a member of an ensemble.

Kasaei 26


Kasaei 27

Image FormatsImage Formats

Kasaei 28

Switching Between FormatsSwitching Between Formats

Kasaei 29

Choosing a ThresholdChoosing a Threshold

Kasaei 30

Using MasksUsing Masks

Kasaei 31

Image EnhancementImage EnhancementImage enhancement goal is to accentuate certain image features for subsequent analysis or display.

Enhancement process itself does not increase the inherent information content in the data.

It is usually the first main step of every image processing task whose performance efficiently affects the algorithm.

It is interactive, application-dependent, image-dependent, and its performance criteria is subjective (perceptual).

Examples include: contrast & edge enhancement, pseudocoloring, noise filtering, sharpening, & magnifying.

Kasaei 32

Image EnhancementImage Enhancement

Kasaei 33

Transformed HistogramTransformed Histogram

Kasaei 34

Histogram EqualizationHistogram Equalization

Kasaei 35

Geometric TransformsGeometric Transforms

Kasaei 36

NearestNearest--Neighbor vs. BilinearNeighbor vs. Bilinear

Kasaei 37

Bilinear InterpolationBilinear Interpolation

Kasaei 38

Resizing ImagesResizing Images

Kasaei 39

Rotating ImagesRotating Images

Kasaei 40

Median FilteringMedian Filtering

Kasaei 41

Image RestorationImage RestorationRefers to removal or minimization of known degradations in images.

A typical problem is to find an estimate of image, given the point spread function, the degraded image, and statistical properties of the noise.

Examples include deblurring due to sensor limitations or its environment, noise filtering, blotch & scratch removal, correction of geometric distortion.

Wiener filter gives the best linear MSE of the object from the observations.

Kasaei 42

Image RestorationImage Restoration

Kasaei 43(a) Original Image. (b) Blotch detection. (c) Motion vectors. (d) Blotch removal. (e) Scratch removal


(a) (b) (c)

(d) (e)

Kasaei 44

(a) Original Image. (b) Image with 64x64 zeros in the SDFT domain. (c) Image with a lost block of size 64x64 pixels. (d) Restored image, PSNR = 21.26 dB.


(a) (b) (c) (d)

Kasaei 45

Image AnalysisImage AnalysisIt is concerned with making quantitative measurementsfrom an image to produce a description of it.e.g., reading a label on a grocery item,

sorting different parts on an assembly line,measuring quantitative information to make a decision.

It requires extraction of certain features that aid in identification of the object.

Segmentation techniques are used to isolate the desired object from the scene so that measurements can be made on it subsequently.

Kasaei 46

Image WatermarkingImage Watermarking

Original Image

WatermarkSignal

WatermarkedImage

Kasaei 47

ContentContent--Based Image RetrievalBased Image Retrieval

Query Image

Kasaei 48

ContentContent--Based Image RetrievalBased Image Retrieval

Two images with similar color histogram.

Kasaei 49

Proposed MethodProposed Method

Our Method. Gabor Wavelet.

Kasaei 50


Kasaei 51


After imposing Zernike.

Kasaei 52

Camera parameter estimation using hardware or software solution

Video out Removing of unwanted background (using chroma keyer)

Camera parameters

Virtual scenes generator

MixerSensors

Image

Virtual StudiosVirtual Studios

Kasaei 53Marker-based systems.


Kasaei 54Orad blue screen pattern.


Kasaei 55

Proposed Virtual StudioProposed Virtual Studio

Kasaei 56

Proposed Virtual StudioProposed Virtual Studio

Kasaei 57

Fingerprint Face Hand Iris

Retina Signature Voice Facial Thermograms

Biometric TechniquesBiometric Techniques

Kasaei 58

Optical sensor. Different silicon sensors.

Thermal silicon sensor.


Kasaei 59Examples of poor FP images.


Kasaei 60

Two different FP images of the same individual.


Kasaei 61


Commercial FP authentication systems.

Kasaei 62

Proposed Authentication TechniqueProposed Authentication Technique

OriginalImage

Block Directions

SegmentedImage

ExtractedFeatures

Kasaei 63Two photos of the same individuals.


Kasaei 64Facial recognition device and method.


Kasaei 65

Proposed Face Detection TechniqueProposed Face Detection Technique

Face detection results.

Kasaei 66

Facial recognition systems.


Kasaei 67

Security monitoring.


Kasaei 68

Measures the unique pattern of the iris-colored portion of theeye, to identify individuals.

Iris sensors.


Kasaei 69

Retinal sensor.


Kasaei 70Comparison of biometric technologies.


Kasaei 71Comparison of biometric technologies.


Kasaei 72

Image Data CompressionImage Data CompressionAmount of data associated with visual information is very large.

Transmission & storage of raw data is a enormous task.

Typical TV images generate data rates exceeding 10 million bps.

Data compression techniques aim at reduction of interpixel, psychovisual, coding, temporal, & spectral redundancies.

They fall into two main categories: lossless & lossly.

Applications include: broadcast TV, remote sensing, computer communications, teleconferencing, facsimile transmission.

Image storage is required for education & business documents, medical images, and large databases.

Kasaei 73

Image Data CompressionImage Data Compression

Kasaei 74

Image Data CompressionImage Data Compression

Compression performance.

Kasaei 75

Image/Video Compression StandardsImage/Video Compression Standards

Kasaei 76

Image/Video Compression StandardsImage/Video Compression Standards

Kasaei 77

JPEG at 0.125 bpp(192:1)

JPEG2000 at 0.125 bpp(192:1)

JPEG JPEG vs. JPEG2000vs. JPEG2000

Kasaei 78

Proposed Image CompressionProposed Image Compression

OriginalImages

ReconstructedImages

CR: 66.5:1bpp: 0.120PSNR: 38.1

CR: 56:1bpp: 0.143PSNR: 23.98

Kasaei 79

ObjectObject--Based Video CompressionBased Video Compression

Decomposition of Scene:– Object segmentation process is the most difficult task!– This stage is not standardized yet.– Each object is specified by its shape, motion, & texture.– Both shape & texture change.

Examples of video object planes (VOP)s:

Kasaei 80

ObjectObject--Based Video CompressionBased Video Compression

Video Object Plane (VOP) Extraction:– A fully adaptive, noise robust, & fast method.

• Stationary Background Assumed.• Object Extraction Techniques:

– Global Object Extraction.– Moving Object Extraction.– Still Object Extraction.

• Noise Removal:– Discrete Wavelet Transform:

• Decreasing Computational Cost.• Decreasing Noise Effect.• Spatial Scalability.

– Adaptive Spatial & Temporal Noise Cancellation.

Kasaei 81

Proposed Coding AlgorithmProposed Coding AlgorithmContent-Based Coding:– Background Coding:

• EZW Coding Scheme.– Moving VOP Coding:

• Shape Coding.• Texture & Motion Coding.

– Still VOP Coding:• Smooth Changing Objects (scripts):

– DPCM with no motion compensation.• Burst Changing Objects (slides/presentations):

– Slides & Presentations:– EZW Coding.

– Multiplexer:• Based on MPLS protocol.• High priority objects use the maximum bandwidth.

Kasaei 82

Object SegmentationObject Segmentation

( ) ( ) ( )yxBgyxFyxObj ii ,,, −=

Kasaei 83

Object SegmentationObject Segmentation

Difference Noise:– Gaussian Noise:– Due to intensity changes.– Can be reduced by a

Hypothesis test.Hypothesis test: – Noise if H0 True.– Object if H1 True.

Post-Processing:– Morphological Opening.

( )

( )( )( )( )objx

objx

dif

i

dif

i

dif

cameradif

thyxDPHthyxDPH

yxObjD

dDif

>=

≤=

=

−⋅=

=

∑

,

,

,

2exp

2

1

2

1

0

2

2

2

2

2

σ

σπσ

σσ

χ

Kasaei 84

Change Detection Masks (Change Detection Masks (CDMsCDMs))

Kasaei 85

Noise ReductionNoise Reduction

Using spatial distance.

Using temporal information.

Kasaei 86

Object ExtractionObject Extraction

Kasaei 87

Multiplexer Script ModeMultiplexer Script Mode

Multiplexer:• Is applied to assign maximum bit-rate to

higher priority bit-streams.

Script Mode:• As scripts change rarely & are DPCM coded,

their size is far less than the moving VOPsScripts have higher Priority.

• Assign maximum bandwidth to still images when ready.

• Status Change Detection.

Kasaei 88

Multiplexer Slide ModeMultiplexer Slide Mode

Slide Mode:• Slides are a burst of information.• The entire still image changes in a very few frames.• The size of compressed still images is more than

Intra-VOP.• Moving VOPs have higher priority.• Assign Maximum bandwidth to moving VOPs when

ready.• Status Change Detection.

In both modes:• the higher priority bit-streams is transmitted between

the lower priority bit-streams.

Kasaei 89

Experimental ResultsExperimental ResultsB

ackg

roun

d

10 fps, 80dB

Sta

ndar

d JP

G

10 fps, 99dB

Pro

pose

d E

ZW

56.61.28

1280.56

2560.28

3840.18

5120.14

99 dB7 KB

BandwidthKbps

Transmission TimeSec.

PSNRSize

Kasaei 90

Experimental ResultsExperimental Results

10 fps, ~48 dB

10 fps, ~37 dB

Pro

pose

d In

tra

Cod

ing

Intra

-Fra

me

MP

EG

-456.60.11

1280.05

2560.025

3840.0167

5120.0125

49.04 dB<1KB

Bandwidth(Kbps)

Transmission Time(Sec.)

PSNRSize

The EndThe End

In the name of Allah - ce.sharif.educe.sharif.edu/courses/85-86/1/ce823/resources/root/lecture...

Documents

Transcript of In the name of Allah - ce.sharif.educe.sharif.edu/courses/85-86/1/ce823/resources/root/lecture...