Real-time Drowsiness Detection System for Drivers & Industrial Workers.

Driver fatigue has become one of the key reasons for road accidents in modern days. Various surveys prove that if a driver is correctly identified as fatigued, and he or she is timely alarmed regarding the same, the cases of accidents can be remarkably reduced. There have been various techniques adopted to identify a drowsy driver. Through this project, an in-depth study of various existing techniques of fatigue in a driver is studied, followed by developing a deep learning-based model to accurately identify a driver’s state using a novel technique of using spatiotemporal features of the face. It can be
determined that the accuracy will remarkably increase when this technique is used.

A solution to this problem is to identify when the driver is falling asleep and alarming the passengers about the situation so that appropriate measures can be taken.

From the incoming live video stream, a frame is extracted for processing. Then Histogram Oriented Gradient (HOG) algorithm is applied to that frame to extract dense features from the image. The advantage of the HOG algorithm is that it can extract all features from a region of interest (ROI) irrespective of local features extraction algorithms such as LBP, SIFT. HOG algorithm captures the shape of the object by dividing an image into small size of blocks such as 8 x 8 or 4 x 4. The facial landmarks feature extracted from the frontal face image using the HOG algorithm is shown in Figure 2. For drowsiness detection we need only the eye region, therefore only eye landmark features are extracted from whole facial features. Figure 3 represents the data flow diagram (DFD) of the system.

Landmark feature of a face image
DFD of the drowsiness detection system

After extracting eye landmark features from the face image (as shown in Figure 2) algorithm calculates the co-ordinate of the left and right eye. An eye aspect ratio (EAR) is calculated using Euclidean distance. EAR is the ratio of the maximum horizontal length to the maximum vertical length of the eyes. Based on the threshold, the system raise alarm is it detects drowsiness. Figure 4,5 and 6 shows the alarm status of the system when the driver is awake, blinks an eye and when closes an eye to sleep.

Figure 4: When a driver is awake
When a driver closes the eye to sleep.

Video shows real-time drowsiness detection using a webcam, Bandicam is used to record desktop activity. Please use a headphone to hear the alarm.

FUTURE WORK

In the last decades, many researchers worked on drowsiness detection and got significant success. However, still, research is going on in this field to make the system more reliable. There are some challenges with this application. For example, if a driver is wearing spectacles, having a partial face that can not be seen by a camera or sensor.

1) For future work, we will extend our application to detect drowsiness even when the subject is using sunglasses or colour spectacles. Another future study that can be done to detect drowsiness is to develop a system that can detect drowsiness at night or with very low illumination.

2) Real-time data are always unconstrained and with the unconstrained nature of the subject/driver, it becomes very difficult to find facial landmarks. Therefore, to overcome this constraint a more reliable application can be developed which can detect face even when the user/subject is not friendly.

3) To increase the sensitivity of application, a multimodal system can be developed which not uses only eye feature, but also facial expression, mouth feature and combine all features to make the decision where driver/subject is sleeping or not.

You can find the source code at my GitHub repository.

Multifaceted Geek | Brand Ambassador @ IEEE | Strategist | Analyst | Engineer | Writer