Real-Time Driver Monitoring Systems through Modality and View Analysis
If neighbouring in-cabin frames are almost identical, do driver-monitoring systems need 3D video models and every available sensor stream?
01 / Problem
What is this work trying to solve?
Contemporary systems run 3D CNNs over synchronized multi-view, multi-modal clips even though adjacent frames in DAD have near-identical histograms and high structural similarity. That redundancy multiplies computation, while the original DAD test split supplies only normal/anomalous labels and cannot evaluate which distraction occurred.
02 / Idea
What is the central technical intuition?
The proposed MobileNet-V2 and ResNet-18 systems discard the temporal dimension and classify the final frame with channel attention. For multiple sources, an extended attentional fusion module combines top/front and infrared/depth features. The study also relabels DAD's test videos into activity classes, including a pooled 'unseen' category, so individual sensor choices can be evaluated per activity.
03 / Evidence
What does the current evidence show?
The image-based MobileNet model reaches 97.5% AUC-PR with both views in infrared; as a single source, top infrared gives 92.4% binary accuracy and about 96% AUC-PR. On the tested M1 Pro and AMD 5900X platforms, the 2D models are reported roughly 10x faster than the 3D baselines and satisfy the dataset camera's real-time interval.
04 / Limits
Where are the limits of this work?
The efficiency result is strong for binary anomaly detection, but fine-grained classification peaks at 75.7% accuracy and several distractions are confused with normal driving. The preferred top-infrared placement is a DAD-specific finding; other cabin geometries, sensors, and activity distributions may change the ranking.