Various research efforts have used image and video processing to approach the problem of monitoring human driver behavior. Driver recklessness, tiredness, and distraction can be detected using a Driver Monitoring System (DMS), a future-proof, smart driving assistance system that monitors the driver. The process involves capturing and analyzing the driver's face and vehicle dynamics, and using those results to avoid accident causing conditions. The DMS solution looks for signs of sudden illness, such as heart attacks or reckless driving scenarios, based on which autonomous driving can take over. A DMS recognizes signals of drowsiness, and forces drivers to pull over to rest. A DMS also prevents driver distraction (such as cell phone use) by detecting when a driver loses focus and sounding an audio alarm warning them to keep their eyes on the road.
Artificial intelligence (AI), the capability of a machine to imitate human behavior, has emerged as the next-generation technology that can deliver multiple benefits with minimal effort. Since AI has progressed and gained widespread acceptance, future advancements in driver monitoring has the potential to make our roadways safer. Deep learning (DL) is gaining traction for identifying driver behaviors, and substantial research continues utilizing neural network algorithms to anticipate and evaluate driver behavior or action-related data. It also focuses on using DL models for predicting various traffic states, such as traffic speed, traffic flow prediction, and travel time estimation, ultimately leading to the development of Intelligent Transportation Systems (ITS).
The fundamental building blocks for driver safety applications are face detection, head pose estimation, gaze estimation, and eye status (blink rate, blink duration, open/close) detection. A state-of-the-art DMS can detect distractions and drowsy drivers by accurately identifying eye movement and head position to determine attention and fatigue levels. DL, being computationally intensive, will typically require an embedded platform using a GPU or high-end CPU to run in real-time. Object detection and classification is a much-researched field in DL. Features that can be implemented as DL matures include identification alerts, detecting driver drowsiness and distraction, speed alerts, real-time location feed, emotion detection, semi-autonomous driving assistance, and visibility for fleet owners. Face detection typically involves training a neural network with a large database of face annotated images. A face detector must be robust against lighting changes, ethnicities, expressions, and occlusions, including sunglasses.
A DMS needs to be calibrated to the individual driver. When a driver enters the car, a non-intrusive calibration procedure runs, detecting the driver’s blink rate and duration. Eye open/close is a binary classification problem. Blink detection is a bit more complex and involves the analysis of several video frames.
A DMS is an in-cabin sensing automotive AI solution that identifies driver drowsiness, distraction, mood levels, and other emotional states. Emotions such as aggression are detected using near-infrared (NIR) and thermal cameras with analysis done based on convolution neural networks (CNN). NIR cameras detect facial feature points and measure their changes, while thermal cameras can measure temperature changes in a driver's body, invisible to the naked eye.