International Journal of Communication Engineering and Technology. ISSN 2277-3150 Volume 3, Number 1 (2013), pp. 1-11 © Research India Publications http://www.ripublication.com

Monitoring the Fatigue Conditions and Controlling the Speed of the Vehicle

A. Nivetha, R. Priya, V. Radhika and Dr. S.Mary Praveena*

Sri Ramakrishna Institute of Technology, CBE-10. E-mail: nivethaa.sritece@gmail.com, radhikachinnapan@gmail.com *Associate Professor, Department of Electronics and Communication

Engineering Sri Ramakrishna Institute of Technology, CBE-10. E-mail: praveena_infant@yahoo.co.in

Abstract

A monitoring system is designed in where it receives sensory data via wireless sensor network and further processes the data to indicate the current driving aptitude of the driver. Analyzing information related to fatigue using two distinct methods eye movement monitoring and bio signal processing. It is critical that several sensors are integrated and synchronized for a more realistic evaluation of the driver behaviour. The sensors applied include a video sensor to capture the driver image and a bio signal sensor to gather the driver photoplethysmograph signal. A warning alarm is sounded if driver fatigue is believed to reach a defined threshold. The manifold testing of the system demonstrates the practical use of multiple features, particularly with discrete methods, and their fusion enables a more authentic and ample fatigue detection.

Introduction The growth of sensor technology and network based information technology has expanded the reach of wire-less sensor networks into numerous areas such as health care, remote control, wildlife habitat monitoring, military explosive detection, intelligent home monitoring and environment observation and forecasting system. On the other hand, the recent increase in traffic accidents is possibly four distinctive driving patterns through analysis by a hidden Markov model (HMM) studied the reliability of steering behaviour to detect caused by driver distraction and low attention during driving. Intelligent transport systems are promoted by integrating the sensor technology in to the transport to measure the driver alertness level.

A. Nivetha et. al.

2

A nonintrusive prototype computer vision system has been proposed for monitoring driver’s attentiveness in real-time. Kasukabe‘s manuscript was supported in part by the Ministry of Education, Science and Technology, and by the National Research Foundation of Korea through the Human Resource Training Project for Regional Innovation. Pauwelussen [7] developed a traffic simulation model in which a vehicle is equipped with an adaptive cruise-control (ACC) and lane-departure warning (LDW) system to monitor driver behaviour in a real traffic environment. Moreover, Lee [8] proposed a system with two fixed cameras to capture images of the driver and the road respectively, and then the images are mapped to global coordinates to monitor the driver sight line. The authors found driver fatigue by multi wavelet packet energy spectrum using a support vector machine (SVM).

Lee [10] developed a video sensor based eye-tracking and blink-detection system with Haar like features and template matching for an automated drowsiness warning system. In addition, Yang demonstrated that drowsiness has a greater effect on rule-based driving tasks than on skill-based tasks using a Bayesian network (BN) paradigm through simulator based human in the loop experiments.

A webcam is placed on the dashboard in front of the driver to capture the driver image whereas a PPG sensor is installed at the steering wheel to collect the driver PPG signals through a finger underneath skin. The smart phone received the signals via connected wireless sensor.

Figure 1: Flow Chart of System Flow Design.

In figure 1, the flowchart of the entire system is given. Here the driver’s image and bio signals were taken from the webcam and from the bio sensor respectively. Then from the input image, the features such as eye blink rate detection, percentage of eye closure are extracted and from the bio sensor the bio signal like photoplethymography signal will be extracted. Once if the fatigue is believed a warning will be given.

Monitoring the Fatigue Conditions and Controlling the Speed of the Vehicle 3

System Design

Figure 2: Block Diagram of Monitoring the fatigue conditions and controlling the speed of the vehicle.

In figure 2, the software is written into the PIC microcontroller through the universal asynchronous receiver transmitter that is the MATlab coding to receive the bio signal is written. Then the signal will be an analog signal. The PIC microcontroller accepts data in digital format. So the analog signal is converted into digital by using A/D converter prior to PIC microcontroller.In the PIC microcontroller, this signal is compared with pre defined value. Until the signal voltage reaches the threshold voltage, the driver fatigue will be monitored. Once the signal voltage increases beyond the threshold voltage then the fuel flow will be reduced by using relays and solenoid valve. Once if the fuel flow reduces then the speed of the vehicle will automatically get decreases.

Sensor Methodology There are four significant modules consisting of hardware design and software implementation that build up the complete system for driver fatigue monitoring. Those modules are

1. Biomedical sensor 2. Video sensor 3. Information fusion module

Biomedical Sensor Module The driver PPG (PhotoPlethysmoGraph) signal is obtained using a pulse oximeter that measures the changes in light absorption under a finger skin. The PPG sensor contains an infrared light-emitting diode(LED) The analog signal acquired is filtered and amplified by an analog conditioning circuit which comprised of an amplifier, a low pass filter (LPF), and a differential amplifier as depicted. The analog signal is converted into digital signal and transmitted to the smart phone for analysis. The PPG data packet format is illustrated in Fig. 3. Each packet contains a source address, source node ID, sequence number, channel number, data and etc. in hexadecimal format.

A. Nivetha et. al.

4

Two types of bio medical sensors are used here. They are 1. Heart beat sensor. 2. Temperarure sensor

Heart Beat Sensor The Heart Beat Sensor provides a simple way to study the heart's function. This sensor monitors the flow of blood through ear lobe. As the heart forces blood through the blood vessels in the ear lobe, the amount of blood in the ear changes with time. The sensor shines a light lobe (small incandescent lamp) through the ear and measures the light that is transmitted. The clip can also be used on a fingertip or on the web of skin between the thumb and index finger. The signal is amplified, inverted and filtered, in the box. By graphing this signal, the heart rate can be determined. Heart rate varies between individuals. At rest, an adult man has an average pulse of 72 per minute. Athletes normally have a lower pulse rate than less active people. Children have a higher heart rate (approx. 90 beats per minute), but also show large variations. The heart rate rises during exercise and returns slowly to the rest frequency after exercise. The rate at which the pulse returns to normal can be used as an indication of fitness. Table 1: Pulse condition

S.NO Pulse rate Relay 1 Relay 2 LCD 1. 99 ON OFF Serial port 2. 100 ON ON Emergency 3. 120 ON ON Emergency

In table 1, the relay 2 will be in the OFF condition, when the pulse ranges below 100. In that period, the solenoid valve will be in open condition. When the pulse range exceeds 100, then the relay 2 becomes ON. At that time the LCD display as emergency.

Temperature Sensor The sensor which collects and displays data about a source’s temperature is known as temperature sensor. It is a wide range low power temperature sensor outputs an analog voltage that is proportional to the ambient temperature. To use connect pin1 (left) to power (between 2.7 and 5.5), pin3 (right) to ground and pin2 to analog input on your microcontroller. The voltage out is 0v at -50ºc and 1.75v at 125ºc. The LM135 series are precision easily calibrated, integrated circuit temperature sensors. Operating as a two terminal zener diode, the LM135 has a breakdown voltage directly proportional to absolute temperature at +10mv/ºk with less than 1ºc error over a 100ºc temperature range. It sensing over a -55ºc to 150ºc temperature range.

Monitoring the Fatigue Conditions and Controlling the Speed of the Vehicle 5

Table 2: Temperature conditions. S. No Temperature Relay 1 Relay 2 LCD

1. 25º C ON OFF Serial port 2. 72º C ON OFF Serial port 3. 80º C ON ON Emergency

In table 2, the relay 2 will be in the OFF condition, when the temperature ranges below 80º C. In that period, the solenoid valve will be in open condition. When the temperature range exceeds 80º C, then the relay 2 becomes ON. At that time the LCD display as emergency.

Video Sensor Module A low-cost custom made webcam is placed on the driver seat dashboard to capture the driver image. The webcam is connected via USB port, with resolution up to 800×600 VGA 30 frames per second. Several infrared LEDs are built at the left and right side near to the camera used to captured the driver image more clearly during the night time. In order to avoid incorrect information interpretation, the data packet received is first validated by its length, channel number, and identification number before analysis take place. The video sensor is directly connected to the web camera using USB port whereas the biomedical sensor transmits the data packets wirelessly to the smart phone through the transceiver connected by RS232 serial port.

Information Fusion Method Various variables or parameters can be used to reflect the characteristic of the driver attentiveness level. Basically, those parameters are divided into two groups: image and bio-signal. In our implementation, a dynamic Bayesian network paradigm is applied for fatigue analysis. DBN paradigm [10] is a probabilistic graphical model that uses different mathematical techniques to model an object based on the given input data. The foremost reason of adapting DBN is that its ability to integrate distinct categories of Parameters even the extraction methods measurement techniques, and etc. of those parameters are different. A dynamic Bayesian network paradigm is applied for fatigue analysis. DBN paradigm [10] is a probabilistic graphical model that uses different mathematical techniques to model an object based on the given input data. A similar method, blink rate (BR) calculates the blinking speed for each eyes blink. Next a well-known approach known as percentage of eye closure (PERCLOS) which described the amount of time the eyes are closed is considered as well. A method called Average Eye Closure/Open Speed (AECS) it calculates the speed of opening or closing the eyes for a specific time.

Experimental Result The relay 2 will be in the OFF condition, when the temperature ranges below 80º C. In that period, the solenoid valve will be in open condition. When the temperature range exceeds 80º C, then the relay 2 becomes ON. At that time the LCD display as emergency.

A. Nivetha et. al.

6

ADC resolution for temperature sensor For displaying the temperature in the decimal values the following ADC resolution calculation has to be done.

2ଵ Where the power 10 represents the number of bits used in ADC.The given supply voltage is 5v. So while substituting voltage value as 5v, then the ADC resolution value is given by the equation 2 and 3.

To convert normal temperature into ADC resolution value, Temperature = 0.3ºc

This is done by a 10 bit ADC which is already incorpated in PIC microcontroller.

Thus by varying the temperature value the ADC resolution value for different temperature value is measured. Table 3: Temperature Vs ADC Resolution.

S. No Voltage (volts) Temperature (ºc) ADC Resolution (mv) 1. 5 0.3 1.464 2. 5 0.5 2.440 3. 5 0.7 3.416 4. 5 0.9 4.392 5. 5 0.11 5.368

In table 3 the ADC resolution values for the corresponding temperature values were shown. When the temperature increases the resolution also gets increased.

Monitoring the Fatigue Conditions and Controlling the Speed of the Vehicle 7

Figure3: Temperature Vs ADC Resolution.

Figure 3 shows the chart of temperature Vs ADC resolution in which as temperature increases ADC resolution also increases.

Simulation Result Using Proteus Software

Figure4: LCD display for the temperature at 104ºc. Figure 4, shows the LCD display when the temperature is sensed as 104ºc. This is

above the threshold value.

A. Nivetha et. al.

8

Figure 5: LCD Display above the threshold temperature.

In figure 5, the LCD displays ‘Emergency’ because the sensed value is above the threshold value.

Figure6: LCD Display for the pulse rate 109.

Monitoring the Fatigue Conditions and Controlling the Speed of the Vehicle 9

Figure 6 shows the LCD display when the pulse rate is sensed as 109. This is above the threshold value.

Figure 7: LCD Display above the threshold pulse rate.

Figure 7 shows the pulse rate has exceeded the threshold value (100) and so

‘Emergency’ is displayed in the LCD.

Conclusion The leading causal factor driver drowsiness can be prevented by this hardware module. Thus achieve driver alertness in a simple but effective way. This has the fatigue display unit which shows the current driving aptitude of the driver. Here two relays are used. One is for indicating whether the serial port is in the ON condition or in the OFF condition. The OFF condition means the sensed value is higher than the threshold parameter value. The second relay is for indicating the raise in temperature and heartbeat sensor above its threshold value. When the value increases above the threshold value “emergency” is displayed in LCD. Hence driver is alerted. In second phase, drowsiness image recognition of driver is done. Fusion of this image and bio signal is utilized to open/close the solenoid valve if the fatigue is detected to be sleeping.

A. Nivetha et. al.

10

References [1] “Driver Alertness Monitoring Using Fusion ofFacial Features and Bio-

Signals”, Boon-Giin Lee and Wan-Young Chung, Member, IEEESENSORS JOURNAL, VOL. 12, NO. 7, JULY 2012

[2] N. Xu, “A survey of sensor network applications,” IEEE Commun. Mag.,vol. 40, no. 8, pp. 1–9, Aug. 2002.

[3] M. Ying, “Application of sensor networks,” Dept. Comput.Sci., Friedrich-Alexander-Univ. Erlangen-Nuremberg, Nuremberg, Germany, May 2005.

[4] R. L. Lai and C. L. Liu, “A fuzzy control massage seat for awaking drowsy drivers,” in Proc. 7th Ind. Eng. Manage. Syst. Conf., Bangkok, Thailand, 2006, pp. 618–623.

[5] L. M. Bergasa, J. Nuevo, M. A. Sotelo, R. Barea, and M. E. Lopez, “Real-time system for monitoring driver vigilance,” IEEE Trans. Intell. Transport. Syst., vol. 7, no. 1, pp. 63–77, Mar. 2006.

[6] T. Kasukabe, M. Hiraoka, O. Yamamoto, M. Yamada, and T. Nakano, “Development of system for comprehensively measuring driving ability for elderly safe driving,” in Proc. Conf. Mach. Vis. Appl., Yokohama, Japan, May 2009, pp. 443–446.

[7] J. Pauwelussen and P. J. Feenstra, “Driver behavior analysis during ACCactivation and deactivation in a real traffic environment,” IEEE Trans. Intell. Transport. Syst., vol. 11, no. 2, pp. 329–338, Jun. 2010.

[8] J. D. Lee, J. D. Li, L. C. Liu, and C. M. Chen, “A novel driving pattern recognition and status monitoring system,” in Proc. Pacific-Rim Symp. Image Video Technol., Hsinchu, Taiwan, Dec. 2006, pp. 504–512.

[9] S. F. Zhao, G. H. Xu, and T. F. Tao, “Detecting driver’s drowsiness Using multi wavelet packet energy spectrum,” in Proc. Int. Congr. Image Signal Process., Tianjin, China, Oct. 2009, pp. 1–5.

[10] Y. S. Lee and W. Y. Chung, “Video sensor based eye tracking and blink detection to automated drowsy driving warning system using image processing,” in Proc. 13th Int. Meet. Chem. Sensors, Perth, Australia, Jul. 2010, p. 358.

[11] [11] J. H. Yang, Z. H. Mao, L. Tijerina, T. Pilutti, J. F. Coughlin, and E. Feron, “Detection of driver fatigue caused by sleep deprivation,” IEEE Trans. Syst. Man Cybern. Part A.: Syst. Humans, vol. 39, no. 4, pp. 697–705, Jul. 2009.

[12] J. Wang and Y. Gong, “Recognition of multiple drivers’ emotional state,” inProc. 19th Int. Conf. Pattern Recognit., Tampa, FL, Dec. 2008, pp. 1–4.

[13] [13] H. S. Shin, S. J. Jung, J. J. Kim, and W. Y. Chung, “Real time car driver’scondition monitoring system,” in Proc. IEEE Sensors, Nov. 2010,pp.951–954.

[14] P. Bouchner, R. Pieknik, S. Novontny, J. Pekny, M. Hajny, and C. Borzová, “Fatigue of car drivers - detection and classification based on experiments on

Monitoring the Fatigue Conditions and Controlling the Speed of the Vehicle 11

car simulators,” in Proc. 6th Int. Conf. Simul., Model., Optim., Lisbon, Portugal, Sep. 2006, pp. 727–732.

[15] [15] R. N. Khushaba, S. Kodagoda, S. Lal, and G. Dissanayake, “Driver drowsiness classification using fuzzy wavelet packet based feature extraction algorithm,” IEEE Trans. Biomed. Eng., vol. 58, no. 1, pp 121–131, Jan. 2011.