Applications of Machine Vision

OOverview and Introduction

Instructor: Chao-Ching HoNational Yunlin University of Science and Technology

Robotic Embedded System LabRobotic Embedded System Lab

Overview of the Machine Overview of the Machine Vision classVision class

When and where to take class Who should take the class Why machine vision What is machine vision How to learn machine vision by steps

When and where to take When and where to take classclass Office Hours

10:00 -12 :00, Wednesday Room EM 317

Classroom EB205 Wednesday, 14:10- 17:00 (FGH)

Class web page http://sites.google.com/a/smartrobot.co.cc/robot/courses/machinevisionappl

TA EM330

http://sites.google.com/a/smartrobot.co.cc/robot/courses/machinevisionapplmailto:g9811706@yuntech.edu.tw

TextbooksTextbooks

Lecture Notes:

References: Visual Servoing Control Based Three-Dimensional Tracking: Theory, Algorithms, Practicalities

(Paperback), Chao-Ching Ho, LAP, 2009. Ramesh Jain and R. Kasturi, Machine Vision,

McGraw Hill, April 1995.

http://www.amazon.com/Visual-Servoing-Control-Three-Dimensional-Tracking/dp/3838305353/ref=sr_1_1?ie=UTF8&s=books&qid=1257845330&sr=1-1http://www.libwebpac.yuntech.edu.tw/Webpac2/store.dll/?ID=110317&T=2&S=ASC&ty=ns

Grading Policy

Course participance (10%) Students are encouraged to exercise the

assignments and practice to present in class Mid-term Report (40%)

Propose a 1or 2 page report to express the motivation of specified machine vision topic.

Final Report (50%) Based on the learned machine vision knowledge,

select one topic, study and give a presentation.

Course Topics Machine Vision Overview and Introduction ( ) ( ) ( ) ( ) ( ) ( )

( ) ( )

( ) ( ) ( )

Who should take the class

If youre interested in the fields of Image Processing Animation Computer Graphics Automatic Optical Inspection Robotic Vision 3D Scanning technology Combined vision with industrial engineering Combined vision with consumer engineering

Why machine vision

An image is worth 1000 words Many biological systems rely on vision. The world is 3D and dynamic. Cameras and computers are cheap and popular Sometimes, the sensor or the environment are

actively changed to make a task easier. This is called Active Vision

Application areas

Industrial inspection, quality control, Reverse Engineering

Video Surveillance and security, road monitoring Pattern Recognition, Face/Gesture Recognition,

Human-computer interfaces Artificial Intelligence, unmanned vehicles Robotics, Visual Servoing Virtual Reality, tele-operations Medical Image analysis (MRI, CT, X-ray)

What is machine vision

Also termed computer vision, robot vision A machine automatically processes an image and

reports what is in the image Recognize the content oft he image Locate and inspect the objects in the image

Machine vision is vision for machine Measurement of features Pattern classification based on those features

Measurement of features focuses on processing the image pixels and extract sets of measurement

Vision = Geometry + Measurement + Interpretation

(Machine Vision)(Computer Vision)

(Automated Optical Inspection)

2004 ( )

Machine Vision focuses on:

What information should be extracted?

How can it be extracted?

How should it be represented?

How can it be used to achieve the goal?

Related disciplines Image processing Pattern recognition Photogrammetry Computer graphics Artificial intelligence Projective geometry Control theory

Sensors

Intensity Images

Light coming from the world hits the sensor.Light coming from the world hits the sensor.

Digital Images

are 2D arrays (matrices) of numbers:

Applications

Robot Soccer Initiative

Basic Architecture for Robot Soccer Systems

Robots on the

playing field

Host comput

er

Host compu

ter

Vision system

Communication

SystemCommunic

ation System

Reverse Engineering

Combine color and range dataUse knowledge about most likely shapes

The Pennsylvania State University

(AOI) What is AOI

Automatic Optical Inspection

CCD

CCD

Copyright 2008 ITRI

AOI

IC PCB LCD BGA AOI

- - -

2004 ( )

AOI ( )

AOI : : :X : : ( )

: :AOI

2004 ( )

National Instruments

Average intensity of a region

Standard deviation Line profile

National Instruments

Coordinate SystemsOrigin of

coordinate system is based on a pattern match

Defined by the location and angle of a reference point (Origin) within the image.

Based on the Origin, regions of interest will shift and rotate with the unit under test.

National Instruments

Identification & Classification

1D Barcodes 2D Barcodes

DataMatrix PDF 417

Optical Character Recognition

National Instruments

IC

National Instruments

Image AcquisitionCo

mpute

r

MotionControlDataAcquisition

I/O, Monitoring, Control

National Instruments

/

/

/

/

Stage

/Flat-field correction

Gain and offset

Image alignment

:

Copyright 2008 ITRI

OpenOpen

ShortShort

Mouse-biteMouse-bite

Etc.Etc.

( )

Copyright 2008 ITRI

(3D Visual Servoing

Control)

USB

dsPIC

UART

5-axis manipulator

color camera

encorder

motor drivers

PC

(3D Visual Servoing

Control)

KERNEL MODE USER MODE

KEY

Device Control Block

USB CameraRobot DSP WDM Driver

SyncFilter

MODE Class BaseFunc Function Multiply Function

PID DSP

RS232

User InterfaceCalibration Intrinsic ParametersExtrinsic Parameters

Hand-Eye HomogeneousCAMSHIFT Algorithm HSV Transform Back-ProjectionStereo Computing 3D Position of Fish 3D Position Robot Arm

(Mobile Robot) USB

dsPIC

dsPIC UART

UART

DC motor

servo motor

color camera

encorder PC

Far Obstacle

Tracking Target

Middle Obstacle

Near Obstacle

Wheel Mobile Robot

3D Scan

3D Scan

Machine vision vs. Image Processing Image processing tries to make images look

better, the output of an image processing system is an image.

The output of the machine vision system is information about the content of the image.

+ + +Lighting

andOptics

Camera or

Sensor

Frame Grabber or

Vision SystemApplicationSoftware

National Instruments

NI Vision Acquisition Software

RGB

with

Still

Colo

r

IEEE

-139

4

Came

ra lin

k

Came

ra lin

kNational Instruments

General structure of a CV problem

ProblemProblem

ImageImageAcquisitionAcquisition Pre-processingPre-processing

Feature Feature ExtractionExtraction

SensorSensor IlluminationIllumination

NoiseNoise Img enhancementImg enhancement TransformTransform

LinesLines CornersCorners ContoursContours RegionsRegions Optical flowOptical flow

AnalysisAnalysis

Knowledge BaseKnowledge Base

InterpretationInterpretation

Keyence Machine Vision System Applications

Task: Part IdentificationDifferentiation of the buttons on a mobile phone

Task: Defect InspectionInspection of burnt marks/short-circuit of connector resin

Task: Presence/Absence detectionDetecting defective pin plating Keyence

Task: Defect InspectionInspection of trimmer switch position

Task: Part IdentificationChecking correct cable assembly

Task: Presence/Absence detectionDetecting reject marks on electronic components

Dimension MeasurementMeasuring the coplanarity of connector pins

Task: PositioningPositioning of a CCD device

Electrical and Electronic Industries Keyence

Task: CountingChecking BGA solder balls

Task: Dimension MeasurementPositioning confirmation for silicon wafers

Task: PositioningPositioning of LCD glass substrates Task: Counting

Detecting ink marks on a silicon wafer

Electrical and Electronic Industries Keyence

Automotive and Metal Industries Keyence

Task: Part IdentificationDifferentiation of cylinder blocks

Task: Defect InspectionCrankshaft porosity detection

Task: Presence/Absence detectionDetecting the presence/absence of bearing grease

Task: Part IdentificationDifferentiation of tires

Automotive and Metal Industries Keyence

Task: PositioningPosition control of a robot

Task: Dimension MeasurementMeasuring the notch position of a gear

Task: Defect InspectionInspection of flaws on a steel plate

Task: Presence/Absence detectionDetecting the groove defect in a piston head

Automotive and Metal Industries Keyence

Task: CountingCheck correct seating of parts for die protection

Task: Dimension MeasurementMeasuring deformation of metal materials

Task: CountingCounting bearing balls

Task: PositioningChecking fit of body panels

Food, Pharmaceutical and Other Industries Keyence

Task: Part IdentificationInspection of different types of medicine capsules

Task: Defect InspectionInspection of pinholes and foreign materials on a sheet

Task: Part IdentificationInspection of plastic cups and printing

Task: Presence/Absence detectionInspecting for molded products remaining in a mold

Food, Pharmaceutical and Other Industries Keyence

Task: Defect InspectionInspection of stains on the bottom of beverage cans

Task: Presence/Absence detectionDetecting the presence/absence of package inserts and missing items

Task: PositioningChecking mis-positioned labels

Task: Dimension MeasurementPitch measurement of construction boards

Food, Pharmaceutical and Other Industries Keyence

Task: CountingCounting items in a carton

Task: Dimension MeasurementMeasuring the thickness of building materials

Task: CountingCounting beverage cans

Task: PositioningDetecting liquid level in a bottle

Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40Slide 41Slide 42Slide 43Slide 44Slide 45Slide 46Slide 47Slide 48