GM-Carnegie Mellon Autonomous Driving CRL Curb Detector 1.
-
Upload
brianna-simmons -
Category
Documents
-
view
221 -
download
0
Transcript of GM-Carnegie Mellon Autonomous Driving CRL Curb Detector 1.
GM-Carnegie Mellon Autonomous Driving CRL
Curb DetectorCurb Detector
1
GM-Carnegie Mellon Autonomous Driving CRL
Project Leader Name & Functional AreaWende Zhang (GM R&D / ECS Lab) David Wettergreen (CMU)
Timing Date______________ Initial May, 2014Midterm iMay, 2015Final May, 2016
Resources 2013 2014 2015Total Material Cost[US$] 100k 100k 100kTotal Headcount (GM) 0.1 0.1 0.1Total Headcount (CMU) 1 1 1
GM Confidential
Automated Image Analysis for Robust Detection of Curbs
DescriptionCurbs are important cues on identifying the boundary of a roadway. Drivers understand an appropriate parking spot as defined by the curbs when reverse or parallel parking. Detecting curbs and providing information to assist drivers is an important task for active safety. Curb location is also crucial to autonomous parking systems.Visual indications of curbs are widely various in the appearance. For example, under perspective imaging, projection of 3-dimensional curbs into 2-dimensional image plane distorts most of the curbs’ geometry properties, such as its angle, distance, and ratio of angles. Also, all curbs might be seen different because of age, wear, damage and lighting. Methods of detecting, localizing, and classifying curbs must address this diversity. This is to say, there is not a fixed template or set of templates that could be applied to reliably detect curbs through images.Nevertheless visual appearance is how human drivers successfully detect curbs. Although physical structure can be sensed with some ranging sensors it not distinctive (two offset planes) or diagnostic of the roadway edge. Therefore we choose to pursue visual appearance. This new project will develop an automated curb detection through :• Choosing appropriate features, learning those features to detect, and classifying the detected curbs• Utilizing the calibrated camera to fuse the 3D geometry information• One year development plan: detect, localize, and classify curbs using in-vehicle vision sensor with backward looking view with wide field of viewMotivation/Benefits• Identify the boundary of a road way in urban driving• Understand an appropriate parking spot as defined by the curbs when reverse or parallel parkingDeliverable / Technology Insertion into GM (What, When, Where)• Problem Definition: Survey of curbs• Data collection: Database of definite curb images and diverse curb images• Application: Detect curb features in perspective imagery• Experimental validation and performance analysis• Annual report
Detect and classify features using learning-based
method
GM-Carnegie Mellon Autonomous Driving CRL
Use Case SlidesUse Case Slides
3
GM-Carnegie Mellon Autonomous Driving CRL
AssumptionsAssumptions
• Color monocular camera
• Known camera motion
• Known intrinsic parameters
• Maximum speed dependent upon frame rate
4
GM-Carnegie Mellon Autonomous Driving CRL
Use CasesUse Cases
• Parking lots– Backward parking– Parallel parking
• Driveways
• Roadways– Single lane– Multi-lane
5
GM-Carnegie Mellon Autonomous Driving CRL
Parking lotsParking lots
• Scenario : Curbs exist behind of a vehicle; rear-view camera with wide field of view
• Success : Detect and localize curbs on images;(Optional) estimate the distance from a vehicle to curbs
6
GM
CURB
CURB
GM-Carnegie Mellon Autonomous Driving CRL
Parking lotsParking lots
• Scenario : Parking curbs exist behind of a vehicle; rear-view camera with wide field of view
• Success : Detect and localize parking curbs on images;(Optional) estimate the distance from a vehicle to parking curbs
7
GM
CURB
CURB
GM-Carnegie Mellon Autonomous Driving CRL
DrivewaysDriveways
• Scenario : Curbs exist at the side of the entrance of driveway; front-view camera with wide field of view
• Success : Detect and localize curbs on images and indicates driveways as traversable path
8
GM
CURB CURBDriveways
GM-Carnegie Mellon Autonomous Driving CRL
RoadwaysRoadways
• Scenario : Curbs exist at the side of the road; wide field of view camera
• Success : Detect and localize curbs on images and indicates curbs as the non-traversable path and the boundary of road
9
GM
CURB
CURB
GM
GM-Carnegie Mellon Autonomous Driving CRL
Flow ChartFlow Chart
10
Detection
Tracking
: Localize relevant curbs in each image
: Localize the detected curbs in remained images
Edge
Texture
Segmentation
GM-Carnegie Mellon Autonomous Driving CRL
Edge DetectionEdge Detection
11
UndistortedDistorted
Bird’s-eye view
Edge
Edge
GM-Carnegie Mellon Autonomous Driving CRL
SegmentationSegmentation
12
GM-Carnegie Mellon Autonomous Driving CRL
Texture ClassificationTexture Classification
13
GM-Carnegie Mellon Autonomous Driving CRL
Development PlanDevelopment Plan
• Develop and test simple features
• Train classifiers to detect and localize curbs
• Evaluate classifier performance
• Add complex features
• Test quantify detection and localization performance
• Train color classifiers to interpret appropriate parking spots
• Motion Stereo to exploit 3D geometry
14
GM-Carnegie Mellon Autonomous Driving CRL
SchemeScheme
15
ExtractFeatures
TrackingEdge
DetectionClassification
- Geometricconsideration
- Horizontal- Long features- Thin features- Color- Texture- Curvature
- Filters• Edges• Intensity
differences• Gradients
- Appearance based tracking
GM-Carnegie Mellon Autonomous Driving CRL
Data collectionData collection
16
• Using 180 degree field of view camera
• Install underneath the side mirror, tilt 45 degree down to the ground
Sample images
GM-Carnegie Mellon Autonomous Driving CRL
Camera CalibrationCamera Calibration
• Wider field of view, more distortion
• Camera calibration is necessary in order to find geometry constrains (e.g., edges…)
• Using OCamCalib (Omnidirectional Camera Calibration Toolbox) to calibrate camera
17
Sample undistorted images
GM-Carnegie Mellon Autonomous Driving CRL
Shape InformationShape Information
• Edge detection
• HOG feature
18
GM-Carnegie Mellon Autonomous Driving CRL
Edge DetectionEdge Detection
19
Input Image at t
Undistorted Image
Edge Detection
SequentialRANSAC
Extract Dominant
Edges
GM-Carnegie Mellon Autonomous Driving CRL
Edge DetectionEdge Detection
20
Input Image at t
Undistorted Image
Edge Detection
SequentialRANSAC
Extract Dominant
Edges
GM-Carnegie Mellon Autonomous Driving CRL
Edge DetectionEdge Detection
21
Input Image at t
Undistorted Image
Edge Detection
SequentialRANSAC
Extract Dominant
Edges
Two or Three parallel lines with small offsets are important cue for curbs
GM-Carnegie Mellon Autonomous Driving CRL
HOG featureHOG feature
22
GM-Carnegie Mellon Autonomous Driving CRL
HOG featureHOG feature
23
GM-Carnegie Mellon Autonomous Driving CRL
HOG featureHOG feature
24
Input image HOG map
Score map Output image
GM-Carnegie Mellon Autonomous Driving CRL
PrerequisitePrerequisite
• The maximum distance of ‘Curb Detection’ from a vehicle should be defined.
• Given extrinsic parameters and the maximum distance, the followings can be estimated.– Different size of HOG model– Region of interest
25
short medium long
GM-Carnegie Mellon Autonomous Driving CRL
Geometry calculationGeometry calculation
26
GM
CURB
Local Area2.7-3.6 m (9-12 feet)
GM
Maximum detect distance
GM-Carnegie Mellon Autonomous Driving CRL 27
http://www.cadillac.com/srx-luxury-crossover/features-specs/dimensions.html
Examples
http://www.cadillac.com/cts-sport-sedan/features-specs/dimensions.html
Cadillac SRX
Cadillac CTS
GM-Carnegie Mellon Autonomous Driving CRL
Geometry calculationGeometry calculation
28
CURBG
M
Maximum detect distance = 2.1 meter
GM-Carnegie Mellon Autonomous Driving CRL 29
1m 1m 1m
Image Sample with distance measure
- The center of the camera is 1.05m from the ground.
- The angle of the camera is 45 degree down from the horizontal.
- ROI will be reduced. (Red transparent rectangle)
- ROI will be changed based on the extrinsic parameter.
GM-Carnegie Mellon Autonomous Driving CRL
Lane MarkingsLane Markings
• Since lane markings have strong edges, we need to eliminate outputs from lane markings.
• Parts of images which contain lane markings can be removed by detecting white blobs.
30
GM-Carnegie Mellon Autonomous Driving CRL
Result VideoResult Video
31
GM-Carnegie Mellon Autonomous Driving CRL
Performance MeasurePerformance Measure
• Choose 300 testing images– Positive samples: images which contains full length of
curbs– Negative samples: images without curbs
• We consider curbs are detected when the horizontal length of the detected curbs are bigger than half of the horizontal length of image.– Since the size of image is 480 by 720, we consider
curbs are detected and the sum of the length of the detected curbs are bigger than 360.
32
GM-Carnegie Mellon Autonomous Driving CRL
Performance MeasurePerformance Measure
33
Groundtruth
Positive Negative
Positive 80 13
Negative 24 183
length of detected curb
total length of image> 0.5
GM-Carnegie Mellon Autonomous Driving CRL
Future WorksFuture Works
• Features of curb detection– Redundant information through multiple images
• Include tracking system to recover false negatives
– Continuity• Develop likelihood function to recover false negatives and
remove false positives
– Height
• Front-view camera– Mount 180 degree field of view camera on the front
bumper
34
GM-Carnegie Mellon Autonomous Driving CRL
Front-view Camera ConfigurationFront-view Camera Configuration
35
GM-Carnegie Mellon Autonomous Driving CRL 36