Driver Fatigue

29
Using digital image processing in MATLAB DROWSY DRIVER DETECTION SYSTEM Prepared by: Manali K. Shukla (07cp621) Niraj A. Chandrani (07cp622) Guided by: Ms. Hetal Gaudani

Transcript of Driver Fatigue

Page 1: Driver Fatigue

Using digital image processing in MATLAB

DROWSY DRIVER DETECTION SYSTEM

Prepared by: Manali K. Shukla (07cp621)Niraj A. Chandrani (07cp622)

Guided by: Ms. Hetal Gaudani

Page 2: Driver Fatigue

AIM

The aim of this project is to develop a prototype drowsiness detection system.

The focus will be placed on designing a system that will accurately monitor the open or closed state of the driver’s eyes in real-time.

Page 3: Driver Fatigue

INTRODUCTIONDriver fatigue is a significant factor in a large number of vehicle accidents. Recent statistics estimate that annually 1,200 deaths and 76,000 injuries can be attributed to fatigue related crashes. The development of technologies for detecting or preventing drowsiness at the wheel is a major challenge in the field of accident avoidance systems. Because of the hazard that drowsiness presents on the road, methods need to be developed for counteracting its affects.

Page 4: Driver Fatigue

Detection of fatigue can be done in one of the following ways mentioned or by combining them.

Measuring changes in physiological signals, such as brain waves, heart rate and eye blinking.

Measuring physical changes such as sagging posture, leaning of the driver’s head and the open/closed states of the eyes.

INTRODUCTION

Page 5: Driver Fatigue

INTRODUCTION

Monitoring the steering wheel movement, accelerator or brake patterns, vehicle speed, lateral acceleration, and lateral displacement.

Monitoring the response of the driver. This involves periodically requesting the driver to send a response to the system to indicate alertness.

Page 6: Driver Fatigue

INTRODUCTIONBy monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected early enough to avoid a car accident.

Hence we have used the eye open-closed detection technique combined with detection of blinks to detect drowsiness.

It involves a sequence of images of a face, and the observation of eye position in each image and analysis blink patterns.

Page 7: Driver Fatigue

OVERVIEW OF IMPLEMENTATION

Page 8: Driver Fatigue

IMAGE ACQUISITION

IMAGE PROCESSING

OUTPUT GENERATION

‘CORE’ OF IMPLEMENTATION

Implementation of our project is divided into 3 core processes: Image acquisition, Image Processing, Output Generation.

Page 9: Driver Fatigue

IMAGE ACQUISITION

This process works at getting data into MATLAB and passing it to other processes.

It works in 3 phases as shown.

INTERFACE WEBCAM AND

START LIVE VIDEO

REPLACE PROCESSED

FRAME WITH NEW ONE

EXTRACT THE FRAMES FROM

VIDEO

Page 10: Driver Fatigue

IMAGE ACQUISITION

• Interface the webcam with matlab. ‘videoinput()’ function is used.

• Make the required changes in the properties ‘set()’ function is used to set properties such as

‘framegrabinterval’ etc.

• Obtain the frames from the live feed into MATLAB memory. ‘getdata()’ function is used

Page 11: Driver Fatigue

IMAGE PROCESSING

The obtained images from the image acquisition are processed here to get the desired output.

Image processing also works in 3 phases as shown

SKIN COLOR SEGMENTATION

EYE DETECTION FACE

DETECTION

Page 12: Driver Fatigue

SKIN COLOR SEGMENTATION

This phase is used to find the skin color regions from the image.

We have used HSI model for segmenting skin color from the entire image.

The benefit of using HSI instead of RBG to segment a particular color is that it requires us to filter only the hue part as hue alone defines color

HSV filter applied. 0<h<0.11 or 0.90<h<1.00

Page 13: Driver Fatigue

Results of skin color segmentation

Page 14: Driver Fatigue

FACE DETECTION

The job of face detection is to locate a face out of the skin color segmented image.

The phase starts with binarization ie converting the obtained skin color image to binary image. This binary image is subjected to morphological operation followed by connected component analysis to obtain the face component.

Using this face component we can design a mask that would give the entire face from original image.

Page 15: Driver Fatigue

BINARIZATION

CONNECTED COMPONENT ANALYSIS

MOPHOLOGICAL OPERATION

FACE COMPONENT

MASK

Page 16: Driver Fatigue

FACE DETECTION

• Binarization of the segmented image. Applying the threshold on gray scale

image. Threshold used is 0.20.

• Morphological operation. To separate the unnecessary joints.

• Connected component analysis To segment catchment area type regions

created by eyes, lips.

Page 17: Driver Fatigue

FACE DETECTION• Face component

Fill back the watershed components to a bigger single component.

The biggest such component is face

• Mask Obtain a rectangular bounding box around

the biggest component. This is required mask.

Page 18: Driver Fatigue

Results of face detection

Page 19: Driver Fatigue

EYEDETECTION

The job of eye detection is to locate the eye portion, find the eye-brow and eye-lashes and compute the distance between them. This distance is sent to the output generation phase.

The technique that we have used is known as Horizontal Average Intensity Technique

The working flow in this module is shown in next slide

Page 20: Driver Fatigue

SEPARATE RIGHT AND LEFT EYE

SEGMENT OUT BLACK REGION

COMPUTE HORIZONATAL AVERAGES

COMPUTE THE DISTANCE BETWEEN THEM

LOCATE EYE-BROW AND UPPER EYE-LASHES

Page 21: Driver Fatigue

EYE DETECTION

• Separate right and left eye This is done to avoid getting wrong results

due to tilting of face.• Horizontal average intensities

We calculate the horizontal average intensities over each y-coordinate for both eye regions

• Segment out black regions Using the dips observed in the horizontal

intensities we segment out the black regions

Page 22: Driver Fatigue

EYE DETECTION• Locate eye-brow and upper eye-lash

Find all the valleys in horizontal average intensity vs y-coordinate graph

Select the two biggest valleys satisfying the conditions – valleys are at most 65 coordinates apart and at least 5 coordinates apart.

• Calculate the distance The distance between valleys satisfying

above condition is the required distance. Find it for both eyes

Page 23: Driver Fatigue

OPEN EYE CLOSED EYE

Comparison of graphs of horizontal avg intensity for open and closed eye

Page 24: Driver Fatigue

OUTPUT GENERATION

The job of output generation as the name suggests is to give the output.

It receives the distance between eye-brow and upper eye-lashes from eye-detection module.

It has to work on it to decide if this distance represents open eye or closed eye

Page 25: Driver Fatigue

CALCULATE REFRENCE DISTANCE

COMPUTE NUMBER OF BLINKS AND EYE-CLOSE FRAMES

COMPARE DISTANCE IN EACH FRAME WITH REFERENCE DISTANCE

DECIDE IF THE DRIVER IS DROWSY

Page 26: Driver Fatigue

OUTPUT GENERATION

• Calculate reference distance The average distance of first 5 frames

supplied to output generation is used as reference.

• Compare distance in each frame with reference distance As each frame arrives, compare its

distance with reference and accordingly send it to eye-close or eye-open module.

Page 27: Driver Fatigue

OUTPUT GENERATION

• Compute number of blinks and eye-close frames. Each time eye-close is detected, increment

a sleeping counter and return it Each time an eye open is detected after an

eye close, increment the blink count and return it.

• Decide if the driver is drowsy If the sleeping counter or blink counter is

5, the driver is detected to be drowsy.

Page 28: Driver Fatigue

FUTURE SCOPE OF IMPROVEMENT A circuit can be designed to raise an

alarm as soon the software returns that driver is drowsy.

If 3D images can be captured localizing eyes can be made more robust by capturing deepest part of a 3D image.

Processing of all frames can be ensured by allocating more memory for software to run in.

This system does not work very efficiently for dark skinned individuals. This can be improved by using an adaptive light source.

Page 29: Driver Fatigue

THANK YOU.