Sequential Capsule Networks for Driver Behavior Recognition
DOI:
https://doi.org/10.5281/zenodo.22111293Keywords:
Intelligent Transport Systems, Driver Behavior Recognition, Driver Decision-Making, In-Vehicle Communication Technology, Privacy-Preserving Analytics, External Behavior Sensing, Spatiotemporal Feature Learning, Sequence Modeling Framework, Active Driver Behavior Recognition, 3D Capsule Networks, Driver Intention Classification, Multi-Vehicle Interaction Modeling, Motion State Analysis, Heading Direction Estimation, Lane Distance Features, Latent Action Representation, Spatiotemporal Dynamics, Privacy-Aware Modeling, Traffic Participant Characterization, Experimental Validation.Abstract
Understanding driver action and decision-making is important for a variety of intelligent transport systems. Recently, the use of in-vehicle information and communication technology to collect data on driver behavior has seen rapid development. However, as these data grow in volume, privacy protection has become a pressing issue. Active behavior recognition based on external data is therefore emerging as a new trend in intelligent transport systems. Existing methods, however, have relied solely on static images, limiting their ability to accurately represent the spatiotemporal dynamics of driver behavior.
A sequence modeling framework capable of learning spatiotemporal features and characterizing the interrelations of driver actions is proposed. Active driver behavior recognition is then viewed as a 3D capsule network classification task, and multiple 3D capsule networks are combined to classify driver intentions on different vehicles. To preserve privacy, traffic participants’ in-vehicle behavior is comprehensively characterized by a variety of external features, including motion state, heading direction, and lane distance. The objectives developed in the framework enable the model to learn a latent representation for action typing. Experimental evaluations and ablation studies demonstrate the feasibility and advantages of the proposed approach.
References
1. Al-Hussein, W. A., Por, L. Y., Kiah, M. L. M., & Zaidan, B. B. (2022). Driver behavior profiling and recognition using deep-learning methods: In accordance with traffic regulations and experts guidelines. International Journal of Environmental Research and Public Health, 19(3), 1470.
2. AlShalfan, K. A., & Zakariah, M. (2021). Detecting driver distraction using deep-learning approach. Computers, Materials & Continua, 68(1), 689–704.
3. Krishna AzithTejaGanti, V., Senthilkumar, K. P., Robinson L, T., Karunakaran, S., Pandugula, C., & Khatana, K. (2024, November). Energy-Efficient Real-Time Hybrid Deep Learning Framework for Adaptive Iot Intrusion Detection with Scalable and Dynamic Threat Mitigation. In Proceedings of the 3rd International Conference on Optimization Techniques in the Field of Engineering (ICOFE-2024).
4. Bertasius, G., Wang, H., & Torresani, L. (2021). Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Machine Learning, 813–824.
5. Bragança, H., Colonna, J. G., Oliveira, H. A. B. F., & Souto, E. (2022). How validation methodology influences human activity recognition mobile systems. Sensors, 22(6), 2360.
6. Annapareddy, V. N. (2024). Leveraging Artificial Intelligence, Machine Learning, and Cloud-Based IT Integrations to Optimize Solar Power Systems and Renewable Energy Management. Machine Learning, and Cloud-Based IT Integrations to Optimize Solar Power Systems and Renewable Energy Management (December 06, 2024).
7. Chen, J., Jiang, Y., Huang, Z., Guo, X., & others. (2021). Fine-grained detection of driver distraction based on neural architecture search. IEEE Transactions on Intelligent Transportation Systems, 22(9), 5783–5801.
8. Chen, S., Yao, H., Qiao, F., Ma, Y., Wu, Y., & Lu, J. (2023). Vehicles driving behavior recognition based on transfer learning. Expert Systems with Applications, 213, 119254.
9. Pandiri, L., Paleti, S., Kaulwar, P. K., Malempati, M., & Singireddy, J. (2023). Transforming financial and insurance ecosystems through intelligent automation, secure digital infrastructure, and advanced risk management strategies. Educational Administration: Theory and Practice, 29(4), 4777-4793.
10. Ezzouhri, A., Charouh, Z., Ghogho, M., & Guennoun, Z. (2021). Robust deep learning-based driver distraction detection and classification. IEEE Access, 9, 168080–168092.
11. Elhenawy, M., Masoud, M., Haworth, N., Young, K., Rakotonirainy, A., Grzebieta, R., & Williamson, A. (2023). Detection of driver distraction in the Australian naturalistic driving study videos using pre-trained models and transfer learning. Transportation Research Part F: Traffic Psychology and Behaviour, 97, 31–43.
12. Forster, Y., Schoemig, N., Kremer, C., Wiedemann, K., Gary, S., Naujoks, F., Keinath, A., & Neukum, A. (2024). Attentional warnings caused by driver monitoring systems: How often do they appear and how well are they understood? Accident Analysis & Prevention, 199, 107684.
13. Davuluri, P. S. L. (2023). AI-Augmented Sanctions Screening: Enhancing Accuracy and Latency in Real Time Compliance Systems. AI-Augmented Sanctions Screening: Enhancing Accuracy and Latency in Real Time Compliance Systems (December 15, 2023).
14. Gu, F., Chung, M.-H., Chignell, M., Valaee, S., & others. (2021). A survey on deep learning for human activity recognition. ACM Computing Surveys, 54(8), Article 177.
15. Gu, M., Chen, K., & Chen, Z. (2024). RFDANet: An FMCW and TOF radar fusion approach for driver activity recognition using multi-level attention based CNN and LSTM network. Complex & Intelligent Systems, 10, 1517–1530.
16. Kolla, S., Meda, R., Balleda, L., & Thimmapuram, C. R. (2024). The utility value of ROX index and modified ROX index in determining the efficiency of HFNC in children admitted with respiratory distress. International Journal of Contemporary Pediatrics, 11(6), 775.
17. Ha, M.-H., & Chen, O. T.-C. (2021). Deep neural networks using capsule networks and skeleton-based attentions for action recognition. IEEE Access, 9, 135615–135628.
18. Sriram, H. K., Challa, S. R., Challa, K., & ADUSUPALLI, B. (2024). Strategic Financial Growth: Strengthening Investment Management. Secure Transactions, and Risk Protection in the Digital Era. Secure Transactions, and Risk Protection in the Digital Era (November 10, 2024).
19. Han, S., & Choi, J.-I. (2021). Study on driver's distraction research trend and deep learning based behavior recognition model. Journal of the Korea Society of Computer and Information, 26(11), 173–182.
20. Singireddy, S. (2024). The Integration of AI and Machine Learning in Transforming Underwriting and Risk Assessment Across Personal and Commercial Insurance Lines. Journal of Computational Analy-sis and Applications (JoCAAA), 33(08), 3966-3991.
21. Jayasundara, V., Roy, D., & Fernando, B. (2021). FlowCaps: Optical flow estimation with capsule networks for action recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 3409–3418.
22. Jiao, S.-J., Liu, L.-Y., & Liu, Q. (2021). A hybrid deep learning model for recognizing actions of distracted drivers. Sensors, 21(21), 7424.
23. Kadar, J. A., Dewi, M. A. K., Suryawati, E., Heryana, A., Zilfan, V., Kusumo, B. S., Yuwana, R. S., Supianto, A. A., Pratiwi, H., & Pardede, H. F. (2023). Distracted driver behavior recognition using modified capsule networks. Journal of Mechatronics, Electrical Power, and Vehicular Technology, 14(2), 177–185.
24. Ikudabo, A. O., & Kumar, P. (2024). AI-driven risk assessment and management in banking: balancing innovation and security. International Journal of Research Publication and Reviews, 5(10), 3573-88.
25. Kaseris, M., Kostavelis, I., & Malassiotis, S. (2024). A comprehensive survey on deep learning methods in human activity recognition. Machine Learning and Knowledge Extraction, 6(2), 842–876.
26. Le, V.-T., Tran-Trung, K., & Hoang, V. T. (2022). A comprehensive review of recent deep learning techniques for human activity recognition. Journal of Computer Networks and Communications, 2022, 8323962.
27. Singireddy, J. (2024). AI-driven payroll systems: Ensuring compliance and reducing human error. American Data Science Journal for Advanced Computations (ADSJAC) ISSN, 3067-4166.
28. Lin, Y., Cao, D., Fu, Z., Huang, Y., & Song, Y. (2022). A lightweight attention-based network towards distracted driving behavior recognition. Applied Sciences, 12(9), 4191.
29. Liu, D., Yamasaki, T., Wang, Y., Mase, K., & others. (2021). TML: A triple-wise multi-task learning framework for distracted driver recognition. IEEE Access, 9, 130216–130229.
30. Yadav, V. (2024). Predictive Analytics for Preventive Medicine: Analyzing how Predictive Analytics is Utilized for Forecasting Patient Health Trends and Preventive Disease. Progress in Medical Sciences. PMS-1126. Prog Med Sci, 8(4).
31. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin Transformer: Hierarchical vision Transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10012–10022.
32. Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022). Video Swin Transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3202–3211.
33. Kummari, D. N. (2022). IoT-enabled additive manufacturing: Improving prototyping speed and customization in the automotive sector. Migration Letters, 19(S8), 2084-2104.
34. Ma, Y., Yuan, L., Abdelraouf, A., Han, K., Gupta, R., Li, Z., & Wang, Z. (2023). M2DAR: Multi-view multi-scale driver action recognition with vision Transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.
35. Ma, Y., Du, R., Abdelraouf, A., Han, K., Gupta, R., & Wang, Z. (2024). Driver digital twin for online recognition of distracted driving behaviors. IEEE Transactions on Intelligent Vehicles.
36. Srinivas Kalisetty, D. A. S. (2024). Leveraging Artificial Intelligence and Machine Learning for Predictive Bid Analysis in Supply Chain Management: A Data-Driven Approach to Optimize Procurement Strategies.
37. Mou, L., Chang, J., Zhou, C., Zhao, Y., Ma, N., Yin, B., Jain, R., & Gao, W. (2023). Multimodal driver distraction detection using dual-channel network of CNN and Transformer. Expert Systems with Applications, 234, 121066.
38. Patrick, M. K., Adekoya, A. F., Mighty, A. A., & Edward, B. Y. (2022). Capsule networks—A survey. Journal of King Saud University—Computer and Information Sciences, 34(3), 1295–1310.
39. Pawan, S. J., & Rajan, J. (2022). Capsule networks for image classification: A review. Neurocomputing, 509, 102–120.
40. Koppolu, H. K. R. (2024). The impact of data engineering on service quality in 5G-enabled cable and media networks. European Advanced Journal for Science & Engineering (EAJSE), 1(1).
41. Ping, P., Huang, C., Ding, W., Liu, Y., Chiyomi, M., & Kazuya, T. (2023). Distracted driving detection based on the fusion of deep learning and causal reasoning. Information Fusion, 89, 121–142.
42. Qu, F., Dang, N., Furht, B., & Nojoumian, M. (2024). Comprehensive study of driver behavior monitoring systems using computer vision and machine learning techniques. Journal of Big Data, 11, Article 32.
43. Sun, H., & Li, Z. (2024). CDCL-VRE: An ensemble deep learning-based model for distracted driver behavior detection. Journal of Intelligent & Fuzzy Systems, 46(1), 1–16.
44. Sheelam, G. K. (2024). AI-driven spectrum management: Using machine learning and agentic intelligence for dynamic wireless optimization. European Advanced Journal for Emerging Technologies (EAJET), 2(1), 3050-9742.
45. Tang, X., Chen, Y., Ma, Y., Yang, W., & others. (2024). A lightweight model combining convolutional neural network and Transformer for driver distraction recognition. Engineering Applications of Artificial Intelligence, 132, 107910.
46. Wang, X., Xu, R., Zhang, S., Zhuang, Y., & Wang, Y. (2022). Driver distraction detection based on vehicle dynamics using naturalistic driving data. Transportation Research Part C: Emerging Technologies, 136, 103561.
47. Xiao, W., Liu, H., Ma, Z., Chen, W., & others. (2022). Attention-based deep neural network for driver behavior recognition. Future Generation Computer Systems, 132, 152–161.
48. Xiao, W., Xie, G., Liu, H., Chen, W., & Li, R. (2024). FDAN: Fuzzy deep attention networks for driver behavior recognition. Journal of Systems Architecture, 147, 103063.
49. Yang, W., Tan, C., Chen, Y., Xia, H., Tang, X., Cao, Y., Zhou, W., Lin, L., & Dai, G. (2023). BiRSwinT: Bilinear full-scale residual Swin-Transformer for fine-grained driver behavior recognition. Journal of the Franklin Institute, 360(2), 1166–1183.
50. Yang, X., Qiao, Y., Han, S., Feng, Z., & Chen, Y. (2024). Appearance-posture fusion network for distracted driving behavior recognition. Expert Systems with Applications, 257, 124883.
51. Zhao, L., Zhao, X., Wu, F., Chen, D., & Wang, C. (2024). A lightweight and efficient distracted driver detection model fusing convolutional neural network and vision Transformer. IEEE Transactions on Intelligent Transportation Systems, 25(12), 19962–19978.
52. Ramanakar Reddy Danda, Z. Y., Mandala, G., & Maguluri, K. K. (2024). Smart Medicine: The Role of Artificial Intelligence and Machine Learning in Next-Generation Healthcare Innovation.
53. Zhao, X., Li, Z., Zhao, C., Fu, R., & Wang, C. (2024). Distraction-level recognition based on stacking ensemble learning for IVIS secondary tasks. Expert Systems with Applications, 244, 122849.
54. Zhu, S., Chen, W., Liu, F., Zhang, X., Han, X., & others. (2023). Human activity recognition based on a modified capsule network. Mobile Information Systems, 2023, 8273546.
55. Jegham, I., Alouani, I., Ben Khalifa, A., & Mahjoub, M. A. (2023). Deep learning-based hard spatial attention for driver in-vehicle action monitoring. Expert Systems with Applications, 225, 119629.
56. Abdulmolla Abdulghafor, R. A., Hanafi, F. M., Nasir, M. F., Wani, S., Gulzar, Y., & Hamid, Y. (2021). A real time deep learning based driver monitoring system. International Journal on Perceptive and Cognitive Computing, 7(1), 79–84.
57. Liu, Y., Zhang, Y., Zhuang, Y., Wang, Y., & others. (2021). Identification of driver distraction based on SHRP2 naturalistic driving study. Mathematical Problems in Engineering, 2021, 6699327.
58. Fitri Mohd Hanafi, M. F., Faiz Md. Nasir, M. S., Wani, S., Abdulmolla Abdulghafor, R. A., Gulzar, Y., & Hamid, Y. (2021). A real time deep learning based driver monitoring system. International Journal on Perceptive and Cognitive Computing, 7(1), 79–84.
59. Bragança, H., Colonna, J. G., Oliveira, H. A. B. F., & Souto, E. (2022). How validation methodology influences human activity recognition mobile systems. Sensors, 22(6), 2360.
60. Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022). Video Swin Transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3202–3211.
Additional Files
Published
Data Availability Statement
None
Issue
Section
License
Copyright (c) 2024 Alexander Miller (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons
Attribution 4.0 International License (CC BY 4.0).
You are free to:
- Share: copy and redistribute the material
- Adapt: remix, transform, and build upon the material
for any purpose, even commercially.
Under the following terms:
Attribution — You must give appropriate credit to
the original author(s) and source.
Full license text: https://creativecommons.org/licenses/by/4.0/
