People Detection in Video Stream

Presented By:Engy Foda

Supervised By:Dr. Ahmed Darwish

Dr. Ihab TalkhanDr. Salah El Tawil

Cairo University

Faculty of Engineering

Computer Engineering Department

Contents

Problem Definition Motivation Literature Survey Art Theories Artistic People Detection System Experimental Results in Images Experimental Results in Video Future work

Problem Definition

Motivation

It is needed by many applications; multimedia applications, traffic control, humanoids and robotics, intelligent cars embedded systems, security.

Challenges

Edge detection, color detectors techniques.

It is hard to model as it is non-rigid object.

Literature survey

People detection in still images

People detection in video

Kalman filter

3D modeling

Tracking

Dynamic detection information

Detection by components

Wavelets and Haar Transform

Art Theories

Vitruvian Man by Ancient Roman architect Vitruvius

Vitruvian Man by Leonardo Da Vinci

Human Body Proportions Standards Theory

Proportions used in our system

Human Body Proportions Standards Theory

The human body is -in average- of 7 heads high.

Shoulder to shoulder width is 3 heads.

Hip to toes height is 4 heads. Top of the head to the bottom of the

chest is 2 heads high. Wrist to the end of the outstretched

fingers of the hand is 1 head in length.

Top to bottom of the buttocks is 1 head in length.

Elbow to the end of outstretched fingers is 2 heads in length.

Proportions used in the system

Artistic People Detection System

Skin Detection

Face Detection

Human Body Detection

Detection Technique Detect probable skin regions from the image. Discard skin regions of area <3% of the whole image area.

DISCARDED DISCARDED

Template resize and orientation. Perform cross correlation. Apply body proportions and mark body components.

Video Detection Technique

Break the video into successive frames .

Apply the whole image detection technique on each frame.

Assemble the detected frames in a new video file showing the detected persons.

Contributions

Human Body detection based on artistic theory.

Selecting the appropriate proportions from the standard theory.

Using the skin detection and face detection as phases for body detection.

Experimental values of cross correlation [0.5, 0.7].

Advantages

Ability to detect partial bodies.

Detect human body by components.

Does not require fixed setup.

Simple Processing.

Limitations

The following cases are not resolved by this system:

Covered faces.

Body is in up side down position.

Pygmies.

Faces with sun glasses, beards, hats. (resolved with low accuracy)

Filtering the regions by area experimentally by <3%.

Experimental Results in Images

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

per

cen

tag

e

Whole bodyw ithout

background

Whole bodyw ith

background

Partial bodyw ithout

background

Partial bodyw ith

background

Whole bodyw ith

orientationand w ithoutbackground

Whole bodyw ith

orientationand w ith

background

Moustache,glasses

MultiplePeople

types

Image detection resultsfail face andbody detection

Correct Facedetection fail inbody detection

Face notdetectedexactly correctbody detectedcorrect

Face & bodydetectedcorrectly

Experimental Results in Images

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

percentage

1

Image detection results

fail face and body detection

Correct Face detection fail inbody detection

Face not detected exactlycorrect body detected correct

Face & body detected correctly58.46%

17.09%

24.45%

585

Experimental Results in Images

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

percentage

Whole bodyw ithout

background

Whole bodyw ith

background

Partial bodyw ithout

background

Partial bodyw ith

background

Whole bodyw ith

orientationand w ithoutbackground

Whole bodyw ith

orientationand w ith

background

Moustache,glasses

MultiplePeople

Types

imagedetection results

Fail

FALSE

Correct

Experimental Results in Images

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

percentage

1

image detection results

fail

false

correct

43%

33%

24%

585

Experimental Results in video

0

5

10

15

20

25

30

no of frames

1 2 3 4 5 6 7 8 9 10 11

sample no

video sample results

w rong

not detected

missing

correct

Experimental Results in video

0

20

40

60

80

100

120

140

Total no frames

1

video sample

w rong

not detected

missing

correct

26.30%

17.29%

33.83%

22.58%

133

Whole Body without background Correct:

1. Exact 3 parts2. Whole body3. 2 parts

False Fail:

1. Background2. Not Detected3. Wrong

Samples of results in images

Results of Video Part

Future Work

Modifications on image processing part.

Modifications on video processing part.

Modifications on Image Part

Boundary or contour detection for the human body.

More body components, e.g. every arm, every leg.

Neural networks to learn the human body architecture.

Modifications on Video Part

More processing to the dynamic information of the video sequence.

Thank You

efoda@ieee.org

Exact 3 parts

Whole Body

2 Parts

False Detection

Background

Not Detected

Wrong Detection

More Detailed StatisticsType Total Correct False Fail

Exact 3 parts Whole 2 parts BK ND Wrong

Whole body without

background128 69 15 13 22 2 6 1

Whole body with background

245 37 11 31 79 52 3 32

Partial body without

background61 15 2 6 30 3 0 5

Partial body with background

63 8 4 3 30 8 2 8

Whole body with orientation and without background

8 3 0 0 4 0 1 0

Whole body with orientation and with

background

46 5 3 8 15 6 0 9

Moustache, glasses 9 4 0 0 3 0 2 0

Multiple People 25 9 0 4 9 0 1 2

Total 585 150 35 65 192 71 15 57