Jack Pinches INFO410 & INFO350 S2 2015 INFORMATION SCIENCE Computer Vision I.
-
Upload
samuel-branden-maxwell -
Category
Documents
-
view
216 -
download
0
Transcript of Jack Pinches INFO410 & INFO350 S2 2015 INFORMATION SCIENCE Computer Vision I.
Jack Pinches
INFO410 & INFO350 S2 2015
INFORMATIONSCIENCE
Computer Vision I
INFO410 S2 2015 COMPUTER VISION I SLIDE 2INFORMATION SCIENCE
Topics Definition Related Fields Application Examples Application Overview
Typical Tasks Methods
Feature Detection Feature Description Feature Matching Exam Questions Resources
Defining Computer Vision
INFO410 S2 2015 COMPUTER VISION I SLIDE 4INFORMATION SCIENCE
Formal Definition Making useful decisions about real physical objects and
scenes based on images (Shapiro & Stockman, 2001). Extracting descriptions of the world from pictures of
sequences of pictures (Forsyth & Ponce, 2003). Analysing images and producing descriptions that can be
used to interact with the environment (Horn, 1986). Designing representations and algorithms for relating images
to models of the world (Ballard & Brown, 1982).
INFO410 S2 2015 COMPUTER VISION I SLIDE 5INFORMATION SCIENCE
Informal Definition
Consists of acquiring, processing, analysing images/video of the real world
To produce information or descriptions Often used as part of decision making
INFO410 S2 2015 COMPUTER VISION I SLIDE 6INFORMATION SCIENCE
Definition Extract “meaning” from
pixels
What we see What a computer sees
http://www.cs.unc.edu/~lazebnik/spring11/lec01_intro.pdf
INFO410 S2 2015 COMPUTER VISION I SLIDE 7INFORMATION SCIENCE
Why is it important? Images and videos are everywhere, and growing in
numbers. Trying to mimic human vision and understanding with
computers. Humans have good visual understanding – large
amount of the brain devoted to visual processing. Difficult to mimic.
Human level visual perception may be strong AI or “AI complete”.
INFO410 S2 2015 COMPUTER VISION I SLIDE 8INFORMATION SCIENCE
Related Fields
https://en.wikipedia.org/wiki/Computer_vision#/media/File:CVoverview2.svg
Application Examples
INFO410 S2 2015 COMPUTER VISION I SLIDE 10INFORMATION SCIENCE
ApplicationsAgriculture
Augmented Reality
Autonomous Vehicles
Biometrics
Character Recognition
Forensics
Industrial Quality Inspection
Face Recognition
Medical Image Analysis
Pollution Monitoring
Process Control
Remote Sensing
Robotics
Security and Surveillance
Transport
http://www.bmva.org/visionoverview
INFO410 S2 2015 COMPUTER VISION I SLIDE 11INFORMATION SCIENCE
Application Examples
Face Detection: “Feature” included
on many digital cameras
https://courses.cs.washington.edu/courses/cse455/09wi/Lects/lect1.pdf\
INFO410 S2 2015 COMPUTER VISION I SLIDE 12INFORMATION SCIENCE
Application Examples Automatic number plate recognition:
Used by law enforcement Automatic tolling stations Road surveillance
https://en.wikipedia.org/wiki/File:California_license_plate_ANPR.png
INFO410 S2 2015 COMPUTER VISION I SLIDE 13INFORMATION SCIENCE
Application Examples
Facial Recognition: Airport Security Check the person
matches the passport photo
Used in NZ
http://www.secureidnews.com/news-item/uks-stansted-airport-deploys-biometric-e-passport-gates/
Application Overview
INFO410 S2 2015 COMPUTER VISION I SLIDE 15INFORMATION SCIENCE
Typical Tasks Detection:
Image scanned for a specific condition. E.g. detection of humans as part of a security system, or vehicles at automatic toll system.
Recognition: Pre-learned objects can be recognized. E.g. recognition of
a specific album cover.
Description: Extracting data about specific object of interest. E.g.
colour information, 2D position or 3D position.
INFO410 S2 2015 COMPUTER VISION I SLIDE 16INFORMATION SCIENCE
Application Methods Image Acquisition:
Image is produced by one or several image sensors. The image may have some associated meta data.
Pre-processing: Process image to ensure it meets requirements. Examples:
Re-sample the image to check it’s what’s expected to be outputted.
Noise reduction so sensor noise doesn’t add false information.
Contrast alteration so important features can be detected.
INFO410 S2 2015 COMPUTER VISION I SLIDE 17INFORMATION SCIENCE
Application Methods Feature extraction:
Image features are extracted from the image data. Examples:
Lines, edges, intersections Complex examples could include texture, shape, or
motion
Detection: Decide which image regions are required for further
processing.
INFO410 S2 2015 COMPUTER VISION I SLIDE 18INFORMATION SCIENCE
Application Methods High level processing:
Usually small set of the original data. Image region which assume contains a specific object.
Examples: Estimation of object pose or size Object recognition
Final decision: Make final decision for the application. Examples:
Match/no match in recognition application Flag if human review needed (medical, military
applications)
Feature Detection
INFO410 S2 2015 COMPUTER VISION I SLIDE 20INFORMATION SCIENCE
Feature Detection Methods that try to extract
image information. Decisions made if there is an
image feature or not. Features will be a subset of the image. Different approaches available.
INFO410 S2 2015 COMPUTER VISION I SLIDE 21INFORMATION SCIENCE
Feature Detection – Image Features Edges:
Points where there is a boundary between two image regions. Can be almost any shape.
http://stackoverflow.com/questions/11319937/c-sharp-paint-image-within-edges
INFO410 S2 2015 COMPUTER VISION I SLIDE 22INFORMATION SCIENCE
Feature Detection – Image Features Corners (interest points):
Originally called corner as early algorithms performed edge detection. Then looked for edges which have rapid direction change.
Can look for high level of curvature.
http://fahmifahim.com/2010/10/22/opencv-corner-detection/
INFO410 S2 2015 COMPUTER VISION I SLIDE 23INFORMATION SCIENCE
Feature Detection – Image Features Blobs:
Aimed at detecting regions of an image. The region may differ in properties
(such as brightness or colour)compared to surrounding images.
https://wwwx.cs.unc.edu/~sjguy/CompVis/Features/
Feature Description
INFO410 S2 2015 COMPUTER VISION I SLIDE 25INFORMATION SCIENCE
Feature Description
Feature Description Methods: SIFT: Scale Invariant Feature Transform SURF: Speeded up Robust Features GLOH: Gradient Location and Orientation Histogram HOG: Histogram of Orientated Gradients
INFO410 S2 2015 COMPUTER VISION I SLIDE 26INFORMATION SCIENCE
Feature Description - SIFT Detect and describe features in images. Developed by David Lowe in 1999. Algorithm described in:
Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision 2. pp. 1150–1157. doi:10.1109/ICCV.1999.790410
C++ Implementation: http://www.robots.ox.ac.uk/~vedaldi/code/siftpp.html
https://copilosk.fbk.eu/images/f/f6/Sift.pdf
INFO410 S2 2015 COMPUTER VISION I SLIDE 27INFORMATION SCIENCE
Input an image. Output is a list of points on the image each associated to a vector
of low level descriptors. These points are called the keypoints.
Provide a local image description. The points and their descriptions are invariant by rescaling,
orientation, and some changes of illumination. Used to find visual correspondences between images for different
applications, such as image alignment or object recognition.
Feature Description - SIFT
INFO410 S2 2015 COMPUTER VISION I SLIDE 28INFORMATION SCIENCE
Feature Description – SIFT Image Description
813 Keypoints
https://copilosk.fbk.eu/images/f/f6/Sift.pdf (slide 4)
INFO410 S2 2015 COMPUTER VISION I SLIDE 29INFORMATION SCIENCE
Feature Description – SIFT Application
Align two imagesto produce a newimage.
Feature Matching
INFO410 S2 2015 COMPUTER VISION I SLIDE 31INFORMATION SCIENCE
Feature Matching – Why do it? Fusing image from different sensors or times Detect changes between images Produce a new large image by overlapping/stitching smaller images
E.g. Panorama Object recognition by comparing an image to a reference image
or model
INFO410 S2 2015 COMPUTER VISION I SLIDE 32INFORMATION SCIENCE
Feature Matching
INFO410 S2 2015 COMPUTER VISION I SLIDE 33INFORMATION SCIENCE
Feature Matching Different Techniques:
SIFT RANSAC (Random Sample Consensus) Cross Correlation Nearest Neighbour Techniques Exhaustive Comparison
INFO410 S2 2015 COMPUTER VISION I SLIDE 34INFORMATION SCIENCE
Feature Matching – What makes a good feature? Uniqueness:
No ambiguous matches between images Look for “interest points”: image regions that unique are
(or at least unusual) Defining unusual?
INFO410 S2 2015 COMPUTER VISION I SLIDE 35INFORMATION SCIENCE
Feature Matching – With SIFT Can detect interest points, create feature descriptors. Compare feature descriptors between images.
INFO410 S2 2015 COMPUTER VISION I SLIDE 36INFORMATION SCIENCE
Feature Matching – With Cross Correlation Determine the important features:
Unique colour/structure of the neighbourhood Geometry or topology of neighbourhood
Vectors show the movement of the neighbourhood.
www.cse.msu.edu/~stockman/RegTutorial/matchingPart1.ppt
INFO410 S2 2015 COMPUTER VISION I SLIDE 37INFORMATION SCIENCE
Feature Matching
www.cse.msu.edu/~stockman/RegTutorial/matchingPart1.ppt
1) Determine points2) Match Neighbourhoods
Potential Exam Questions
INFO410 S2 2015 COMPUTER VISION I SLIDE 39INFORMATION SCIENCE
Exam Questions
Define computer vision, and include the typical tasks that could be expected in a computer vision application.
Explain why it is important to use good feature descriptors for feature matching, use applications examples to support your reasoning.
Explain how you would design and implement a license plate detection and recognition application.
INFO410 S2 2015 COMPUTER VISION I SLIDE 40INFORMATION SCIENCE
Resources Computer Vision:
http://www.cs.unc.edu/~lazebnik/spring11/ http://users.eecs.northwestern.edu/~yingwu/teaching/EECS432/Notes/intro.pdf
SIFT: https://copilosk.fbk.eu/images/f/f6/Sift.pdf http://
www.quora.com/What-is-the-best-explanation-of-SIFT-that-you-have-seen-or-heard Features, Matching, and Detection:
http://www.cs.toronto.edu/~kyros/courses/2503/Handouts/features.pdf https://courses.cs.washington.edu/courses/cse455/09wi/Lects/lect6.pdf www.cse.msu.edu/~stockman/RegTutorial/matchingPart1.ppt www.cse.msu.edu/~stockman/RegTutorial/matchingPart2.ppt http://people.cs.umass.edu/~elm/Teaching/ppt/370/370_10_RANSAC.pptx.pdf