WeekNo.%01 IntroducMon%% (course:%Computer%Vision) · • Vision,%Visual%system% •...

Naeem A. Mahoto e-‐mail: [email protected]

Department of So9ware Engineering, Mehran UET

Jamshoro, Sind, Pakistan

Monday, July 27, 2015

Week No. 01 IntroducMon

(course: Computer Vision)

•  Vision, Visual system

•  Color vision, computer vision

•  Image processing steps

•  ApplicaMons

Naeem A. Mahoto

Outline


•  Vision –  Vision is the process of discovering,

what is present in the world and where it is.

•  PercepMon –  percepMon is the process of acquiring, interpreMng, selecMng, and organizing sensory informaMon.

Naeem A. Mahoto

Vision


Image source: Gonzales, R. C., & Woods, R. E. Digital image processing, 1993.

state of being able to see

Process of becoming aware of something through senses

•  The visual system allows to assimilate informaMon from the environment

•  The act of seeing starts when the lens of the EYE focus an image of the outside world onto a light-‐sensiMve membrane in the back of the eye, called the ReMna

•  The reMna is actually part of the brain that is isolated to serve as a transducer for the conversion of pa]erns of light into neuronal signals

Naeem A. Mahoto

Visual System


Visual = of vision System: set of connectors/parts/nodes forming as a whole ReMna: Layer at back of eyeball, it has cells & is sensiMve to light, it triggers neurons Impulses to brain

Naeem A. Mahoto

Visual System


•  The human eye achieves proper focus of the object by changing the shape of the lens -‐ changing its focal-‐length

•  The lens gets fla]ened (thinned) to focus the distant objects, and gets thickened to focus the near objects

•  The focal length of the lens varies between 14 mm and 17 mm

•  The distance between the lens and the reMna along the visual axis is around 17 mm

•  An inverted image of the object is formed on the fovea region of the reMna

Naeem A. Mahoto

Visual System


Naeem A. Mahoto

Visual System


•  A simplest imaging device which can map a 3-‐D scene onto a 2-‐D image plane.

•  The projecMon from 3-‐D to 2-‐D is a perspecMve projecMon. •  A pinhole camera forms an inverted image of the object, so

does the Human eye

Naeem A. Mahoto

Pin hole Camera


•  Color vision is the capacity of an organism or machine to disMnguish objects based on the wavelengths (or frequencies) of the light they reflect or emit.

•  The nervous system derives color by comparing the responses to light from the several types of cone photoreceptors in the eye.

•  For humans, the visible spectrum ranges approximately from 380 to 750 nm.

Naeem A. Mahoto

Color vision


•  A 'red' apple does not emit red light. Rather, it simply absorbs all the frequencies of visible light shining on it except for a group of frequencies that is perceived as red, which are reflected.

•  An apple is perceived to be red, only, because the human Eye can disMnguish between different wavelengths.

•  Three things are needed to see color –  a light source, –  a detector (e.g. the Eye) –  a sample to view.

Naeem A. Mahoto

Color vision


•  Computer vision is the science and technology of machines that see.

•  Computer Vision is the study of analysis of pictures and videos in order to achieve results similar to those as by human.

Naeem A. Mahoto

Computer vision


•  Since percepMon can be seen as the extracMon of informaMon from sensory signals,

•  computer vision can be seen as the scienMfic invesMgaMon of arMficial systems for percepMon from images or mulM-‐dimensional data

•  Computer vision can also be described as a complement of Biological Vision, as computer vision, studies and describes arMficial vision system that are implemented in so9ware and/or hardware.

Naeem A. Mahoto

Computer vision


•  Image processing •  Computer graphics •  Pa]ern recogniMon •  ArMficial intelligence •  Applied mathemaMcs •  Learning

Naeem A. Mahoto

Related Disciplines


Naeem A. Mahoto

Related Disciplines


Naeem A. Mahoto

Fields of an IMAGE


Image Processing

Computer Graphics

Pattern Recognition

Computer Vision

Output Image Input Image

Image Description

Image

Image Description Statistics

Image Action

•  Images can be categorized according to their source (e.g., visual, X-‐ray, etc)

•  Principle energy sources for images –  ElectromagneMc energy spectrum –  acousMc, ultrasonic –  Electronic (in the form of electron beams used in electron microscopy)

–  SyntheMc images, used for modeling and visualizaMon,are generated by computer

Naeem A. Mahoto

Image Sources


•  AchromaMc light –  Light that voids color is called achromatic or

monochromatic light –  The only attribute of such light is its intensity, or amount –  The term gray level generally is used to describe

monochromatic intensity because it ranges from black to grays, and finally to white

•  ChromaMc light –  Chromatic light spans the electromagnetic energy spectrum

from approximately 0.43 to 0.79 µ.m

Naeem A. Mahoto

Light Types


•  ChromaMc light –  Three basic quantities are used to describe the quality of a

chromatic light source: 1)  Radiance 2)  Luminance 3)  Brightness

– Radiance: The total amount of energy that flows from the light source, and it is usually measured in watts (W)

– Luminance: measured in lumens (lm), gives a measure of the amount of energy an observer perceives from a light source

– Brightness: Brightness is a subjective descriptor of light perception that is practically impossible to measure •  It embodies the achromatic notion of intensity and is one of the key

factors in describing color sensation

Naeem A. Mahoto

ChromaMc Light


•  Example –  light emitted from a source operating in the far infrared

region of the spectrum could have significant energy (radiance), but an observer would hardly perceive it; its luminance would be almost zero

Naeem A. Mahoto

ChromaMc Light


Naeem A. Mahoto

ElectromagneMc Spectrum


The electromagnetic spectrum arranged according to energy per photon

Energy of one photon (electron volts)

Planck's constant, or h i-‐e

ElectromagneMc Spectrum: Band of Radiations

RadiaMon: Energy that travels and spreads out as it goes

•  Visible light: Light that comes from a lamp •  Radio waves: Light that comes from a radio staMon and are types of electromagneMc radiaMon – Microwaves –  Infrared –  Ultraviolet –  X-‐rays –  Gamma-‐rays

•  Ho]er, more energeMc objects and events create higher energy radiaMon than cool objects

•  Only extremely hot objects or parMcles moving at very high velociMes can create high-‐energy radiaMon like X-‐rays and gamma-‐rays

Naeem A. Mahoto Monday, July 27, 2015


•  Gamma-‐rays –  The highest energy, shortest wavelength electromagnetic

radiations. Usually, they are thought of as any photons having energies greater than about 100 keV

–  Radioactive materials (some natural and others made by man in things like nuclear power plants) can emit gamma-rays

–  Big particle accelerators that scientists use to help them understand what matter is made of. The biggest gamma-ray generator of all is the Universe! It makes gamma radiation in all kinds of ways

–  Major uses of imaging based on gamma rays include nuclear medicine and astronomical observations

–  In nuclear medicine, the approach is to inject a patient with a radioactive isotope that emits gamma rays as it decays

–  Images are produced from the emissions collected by gamma ray detectors



•  X-‐rays –  Electromagnetic radiation of very short wavelength and very

high-energy; X-rays have shorter wavelengths than ultraviolet light but longer wavelengths than gamma rays

–  Doctors use them to look at bones, Dentist to look at teeth. –  Hot gases in the Universe also emit X-rays, X-rays are among

the oldest sources of EM radiation used for imaging –  The best known use of X-rays is medical diagnostics, but they

are also used extensively in industry and other areas, like astronomy

–  Angiography is another major application in an area called contrast-enhancement radiography. This procedure is used to obtain images (called angiograms) of blood vessels

–  Perhaps the best known of all uses of X-rays in medical imaging is computerized axial tomography (CAT)

–  X-rays, are used to examine circuit boards for flaws in manufacturing, such as missing components or broken traces



•  Ultraviolet –  Electromagnetic radiation at wavelengths shorter than the

violet end of visible light –  X-rays, are used to examine circuit boards for flaws in

manufacturing, such as missing components or broken traces –  Sun is a source of ultraviolet (UV) radiation, because it is the

UV rays that cause our skin to burn! Stars and other "hot" objects in space emit UV radiation

–  The atmosphere of earth effectively blocks the transmission of most ultraviolet light

–  Applications of ultraviolet "light” include lithography, industrial inspection, microscopy, lasers, biological imaging, and astronomical observations



Lithography :A method of planographic prinMng from a metal or stone surface

•  Infrared –  Electromagnetic radiation at wavelengths longer than the red

end of visible light and shorter than the microwaves (roughly between 1 and 100 microns)

–  Almost none of the infrared portion of electromagnetic spectrum can reach the surface of earth

–  Applications include light microscopy, astronomy, remote sensing, industry, and law enforcement



ThemaMc Bands in NASA’s LANDSAT Satellite



LANDSAT satellite obtains and transmits images of Earth from space for monitoring environmental condiMons of the planet.

•  Microwave –  Electromagnetic radiation which has a longer wavelength

(between 1 mm and 30 cm) than visible light –  Microwaves can be used to study the Universe, communicate

with satellites in Earth orbit, and cook popcorn –  The dominant application of imaging in the microwave band is

radar. The unique feature of imaging radar is its ability to collect data over virtually any region at any time, regardless of weather or ambient lighting conditions

–  Some radar waves can penetrate clouds, and under certain conditions can also see through vegetation, ice, and extremely dry sand. In many cases, radar is the only way to explore inaccessible regions of the Earth's surface

–  An imaging radar works like a flash camera in that it provides its own illumination (microwave pulses) to illuminate an area on the ground and take a snapshot image

–  Instead of a camera lens, a radar uses an antenna and digital computer processing to record its images



•  Radio waves –  Electromagnetic radiation which has the lowest frequency, the

longest wavelength, and is produced by charged particles moving back and forth

–  The atmosphere of the Earth is transparent to radio waves with wavelengths from a few millimeters to about twenty meters

–  Yes, this is the same kind of energy that radio stations emit into the air for boom box to capture and turn into favorite tunes. But radio waves are also emitted by other things ... such as stars and gases in space

–  The major applications of imaging in the radio band are in medicine and astronomy

–  In medicine radio waves are used in magnetic resonance imaging (MRI)

–  This technique places a patient in a powerful magnet and passes radio waves through his or her body in short pulses



•  Although imaging in the electromagnetic spectrum is dominant by far, there are a number of other imaging modalities that also are important

•  Other imaging modalities are: –  acoustic imaging –  electron microscopy –  Synthetic (computer-generated) imaging

•  Imaging using "sound" finds application in –  Geological exploration –  Industry –  Medicine

•  Geological applications use sound in the low end of the sound spectrum (hundreds of Hertz)

•  Imaging in other areas use ultrasound (millions of Hertz)


Imaging ModaliMes

•  Ultrasound Imaging –  ultrasound imaging is used routinely in manufacturing –  The best known applications of this technique are in medicine,

especially in obstetrics, where unborn babies are imaged to determine the health of their development


Imaging ModaliMes

Obstetrics: The branch of medicine dealing with childbirth and care of the mother

Fractal images: are striking examples of computer generated images

1. Image acquisiMon

2. Pre-‐processing

3. Feature extracMon

4. DetecMon/SegmentaMon

Naeem A. Mahoto

Image processing steps


•  A digital image is produced by one or several image sensors.

•  These sensors may include: –  Light-‐sensiMve cameras –  Range sensors –  Tomography devices –  Radar and ultra-‐sonic cameras, etc.

•  Depending on the type of sensor, the resulMng image data is an ordinary 2D image, a 3D volume, or an image sequence.

•  The pixel values typically correspond to light intensity in one or several spectral bands

Naeem A. Mahoto

1) Image acquisiMon


Naeem A. Mahoto

1) Image acquisiMon


Image source: Gonzales, R. C., & Woods, R. E. Digital image processing, 1993.

Naeem A. Mahoto

2) Pre-‐processing


•  Before a computer vision method can be applied to image data in order to extract some specific piece of informaMon.

•  It is usually necessary to process the data in order to assure that it saMsfies certain assumpMons implied by the method.

•  Examples may include: –  Re-‐sampling in order to assure that the image coordinate system is

correct –  Noise reducMon in order to assure that sensor noise does not

introduce false informaMon –  Contrast enhancement to assure that relevant informaMon can be

detected

•  Image features at various levels of complexity are extracted from the image data

•  Typical examples of such features are: –  Lines, edges and ridges –  Localized interest points such as corners, blobs or points –  More complex features may be related to texture, shape or moMon

Naeem A. Mahoto

3) Feature extracMon


Ridge: Edge formed where two sloping sides of roof meet at the top. Blob: Spot of color

•  At some point in the processing a decision is made about which image points or regions of the image are relevant for further processing

•  Examples: –  SelecMon of a specific set of interest points –  SegmentaMon of one or mulMple image regions which contain a specific object of interest

Naeem A. Mahoto

4) DetecMon/SegmentaMon


Naeem A. Mahoto

Vision and Graphics


•  Law enforcement •  Nuclear medicine and Defense •  AutomaMc character recogniMon •  Industrial applicaMons (machine vision) •  Satellite imagery for weather predicMon •  Solving problems with machine percepMon •  Enhance the contrast or code the intensity levels into color for easier interpretaMon

•  InterpretaMon of X-‐rays and other Images used in industry, medicine and biological sciences

•  Remote Sensing

Naeem A. Mahoto

ApplicaMon Areas


•  Two principal applicaMon areas are: 1.   Improvement of pictorial informaBon for human

interpretaBon 2.   Processing of image data for storage, transmission, and

representaBon for autonomous machine percepBon

•  Vision is the most advanced of human senses

•  Images play the most important role in human percepMon

•  Humans are limited to the visual band of the electromagneMc (EM) spectrum

•  Imaging machines cover almost the enMre EM spectrum, ranging from gamma to radio waves

Naeem A. Mahoto

Digital Image Processing


•  UlMmate goal of computer vision is to use computers to emulate human vision, including learning and being able to make inferences and take acMons based on visual inputs

•  This area itself is a branch of arMficial intelligence (AI) whose objecMve is to emulate human intelligence

•  The area of image analysis (also called image understanding) is in between image processing and computer vision

•  There are no clear-‐cut boundaries in the conMnuum from image processing at one end to computer vision at the other. However, one useful paradigm is to consider three types of computerized processes

Naeem A. Mahoto



•  Three computerized processes are: –  Low Level processes – Medium Level processes –  High Level processes

•  Low-‐level processes involve primiMve operaMons such as image preprocessing to reduce noise, contrast enhancement, and image sharpening. A low-‐level process is characterized by the fact that both its inputs and outputs are images

•  Mid-‐level processing on images involves tasks such as segmentaMon, descripMon of those objects to reduce them to a form suitable for computer processing, and classificaMon (recogniMon) of individual objects

Naeem A. Mahoto



•  A mid-‐level process is characterized by the fact that its inputs generally are images, but its outputs are a]ributes extracted from those images (e.g., edges, contours, and the idenMty of individual objects)

•  Finally, higher-‐level processing involves "making sense" of an ensemble of recognized objects, as in image analysis, and, at the far end of the conMnuum, performing the cogniMve funcMons normally associated with vision

Naeem A. Mahoto



•  Digital Image Processing –  Set of operaMons performed on digital image

•  Image Enhancement –  The simplest and most appealing area of digital image processing, it is subjecMve technique in a sense that is based on human subjecMve preferences regarding what consMtutes a "good" enhancement result

–  The idea behind enhancement techniques is to bring out detail that is obscured, or simply to highlight certain features of interest in an image

–  Example: when someone increases the contrast of an image because "it looks be]er.”

Naeem A. Mahoto



•  Image RestoraMon –  An area that also deals with improving the appearance of

an image –  it is objective technique in the sense that restoration

techniques tend to be based on mathematical or probabilistic models of image degradation

•  Color Image Processing –  An area that deals with the spectrum of frequencies of an

image. –  It has been gaining in importance because of the significant

increase in the use of digital images over the Internet •  Wavelets –  Wavelets are the foundation for representing images in

various degrees of resolution

Naeem A. Mahoto



•  Compression –  deals with techniques for reducing the storage required to

save an image, or the bandwidth required to transmit it

•  Morphological Processing –  deals with tools for extracting image components that are

useful in the representation and description of shape

•  SegmentaMon –  partitioning an image into its constituent parts or objects –  In general, autonomous segmentation is one of the most

difficult tasks in digital image processing

Naeem A. Mahoto



Naeem A. Mahoto

Components of General Purpose Image Processing System


Naeem A. Mahoto



•  Image Sensors –  Two elements are required to acquire digital images.

•  The first is a physical device that is sensitive to the energy radiated by the object intended to image.

•  The second, called a digitizer, is a device for converting the output of the physical sensing device into digital form.

•  Specialized Image Processing –  hardware usually consists of the digitizer and hardware that

performs other primitive operations, such as an arithmetic logic unit (ALU) performing arithmetic and logical operations in parallel on entire images.

–  This type of hardware sometimes is called a front-end subsystem

•  Image Processing So9ware –  It consists of specialized modules performing specific

tasks. A well-designed package includes the capability for the user to write code that, as a minimum, utilizes the specialized modules

Naeem A. Mahoto



•  Image Processing So9ware –  More sophisticated software packages allow the integration

of those modules and general-purpose software commands from at least one computer language

•  Mass Storage –  Mass storage capability is a mandatory in image

processing applications –  An image of size 1024 X 1024 pixels, in which the

intensity of each pixel is an 8-bit quantity, requires one megabyte of storage space if the image is not compressed

–  Digital storage for image processing applications falls into three principal categories:

1)  short term storage required during processing 2)  on-line storage for relatively fast recall 3)  archival storage characterized by infrequent access

Naeem A. Mahoto



•  Image Displays –  Image displays in use today are mainly color (preferably flat screen)

TV monitors. Monitors are driven by the outputs of image and graphics display cards that are an integral part of the computer system

•  Hardcopy devices –  Hardcopy devices for recording images include laser printers, film

cameras, heat-sensitive devices, inkjet units, and digital units, such as optical and CD-ROM disks

•  Networking –  Networking is almost a default function in any computer system in use

today. Because of the large amount of data inherent in image processing applications, the key consideration in image transmission is bandwidth. In dedicated networks, this typically is not a problem, but communications with remote sites via the Internet are not always as efficient. Fortunately, this situation is improving quickly as a result of optical fiber and other broadband technologies

UlMmate goal of Computer vision is to emulate human vision

Including: –  Learning, to make inferences and take acMons based on visual input

Naeem A. Mahoto

Conclusions


WeekNo.%01 IntroducMon%% (course:%Computer%Vision) · • Vision,%Visual%system% •...

Documents

Transcript of WeekNo.%01 IntroducMon%% (course:%Computer%Vision) · • Vision,%Visual%system% •...