Pedestrian Detection with Radar and Computer Vision

Stefan Milch, Marc Behrens, smart microwave sensors GmbH

radar sensor

camera

Darmstadt, September 25 / 26, 2001 Darmstadt, September 25 / 26, 2001

smart microwave sensors GmbH

70%

3%

17%10%border of roadway

Source: Pedestrian injury accidents -Report by WG 7 of the EuropeanExperimental Vehicles Committee, 1982

opposite roadway

Impact zone: Active pedestrian protection systems:

• lift bonnet in case of collision

• bonnet airbag

• active bumper(active soft nose)

• lower front spring strut(off-road-vehicles)

⇒ Future aim: Driver assistance system to avoid the accident

Pedestrian accidents and protection systems

smart microwave sensors GmbH

objects

3-dB detection field

2D Space

Properties of the radar sensor

sensors

smart microwave sensors GmbH

objects

3-dB detection field

2D Space

Properties of the radar sensor

Location, velocity, RCS,(dimension)

Object properties

Clustering of reflection-points (without model)

Object detection

LowData rate

ActiveSensor principle

List of reflection-points (range, angle, velocity, RCS)

Result of measurement

Radar

smart microwave sensors GmbH

Properties of the video-sensor

camera

camera target

3D space 2D space

Model depending (derived from gray-level matrix)Object properties

Knowledge based interpretation (with model)Object detection

HighData rate

PassiveSensor principle

Gray-level matrix (brightness distribution)Result of measurement

Video

smart microwave sensors GmbH

• Purpose of sensor fusion: Merge data coming from different information sources.

• No perfect sensor: use smart combination to get favorable properties of one sensor and to suppress disadvantages

⇒ Aim: Obtain information of greater quality; definition of «greater quality» will depend upon application

Target Location

Tar

get

Typ

e Radar

Video

Sensor Fusion / Sensor Combination

smart microwave sensors GmbH

Sensing Perception

radar

camera

Vehicle:

pre-selectioncriteria: r, v, (dimension)

examinationof pedestrianhypotheses

hypothesislist

examinedhypothesis

list

trackhypotheses

results

time basedstabilization

pedestrian listApplication(s)

eigenv

target list ( )ϕ,, relvr

image

tracktargets

Topology of the pedestrian detection system,steering

smart microwave sensors GmbH

hypotheses

camera modelcamera model

projection

Image space

Generation of hypotheses

smart microwave sensors GmbH

hypotheses

camera modelcamera model

projection

Image space

Generation of hypotheses

Image plane

projection

hypotheses

Classification(Computer

Vision)

Classification(Computer

Vision)

smart microwave sensors GmbH

• What forms of representations are suitable for the perception ofdynamic visual objects such as moving persons?

• Meaningful perception requires a prioriknowledge

• Some kind of simplifying assumptions are generally required

Mask from a particular pedestrian / pose

Mask from a particular pedestrian / pose

Generic deformable3D parametric surface

representation

Generic deformable3D parametric surface

representation

Dimensionality of model-space

(Computational expense)

Model Matching

Image

Pedestrian Model: What to Look For

smart microwave sensors GmbH

• 2D outline of human is modeled

• View based shape models

• Statistical Shape Model

Side view

Front/back view

Pedestrian Model: Approach taken

Normal to ModelBoundary

Model Point (x,y)

Model BoundaryImage Object

Nearest edge on Normal (x´,y´)

smart microwave sensors GmbH

• «Sensor» that recognizes pedestrians is the assumption for anactive protection system

• Sensors and parts of signal processing can support more applications like,

• adaptive cruise control (ACC)• pre-crash detection

• Feasibility of the approach was shown,

• Performance analyzes, enhancements and optimization under examination

Outlook and conclusion