Intelligent Knowledge Exploration and Processing

Intelligent Knowledge Exploration and Processing

Facial recognition of masked people using MediaPipe Facemesh and deep learning algorithms

Document Type : Original Article

Authors
Mansour Hesabi Moghaddam 1
hamidreza ghaffary 2
Mahdi khazaiepoor 1
1 Computer Engineering Department, Birjand Branch, Islamic Azad University, Birjand, Iran
2 Technical and Engineering Department - Electrical and Computer Faculty - Islamic Azad University, Ferdous Branch - Iran
10.30508/kdip.2024.452512.1101
Abstract
In response to the fundamental need for accurate recognition of tense faces wearing masks, this paper presents an innovative approach that takes advantage of parallel two-stage deep learning methods combined with hybrid meta-heuristic algorithms. The challenges associated with mask-wearing detection are addressed through a comprehensive framework that leverages state-of-the-art technologies and diverse inputs. This method includes a parallel algorithmic strategy, which optimizes the detection of faces with and without masks for accuracy. A special algorithm is used when detecting unmasked faces, while detecting masked faces uses a separate algorithm. In addition, an integration of multiple data sources including masked face images and inputs from temperature sensors increases the recognition accuracy. The main focus of this research is on data clustering, where datasets are organized based on their volume, then classification is performed using a proposed convolutional neural network. Duplicate features are carefully removed from each cluster, and parallel post-processing is performed by differentiating algorithms. In this study, two hybrid algorithms are introduced, and with the increase in data volume, additional algorithms can be easily inserted to provide scalability and increase accuracy. This innovative approach demonstrates the ability to significantly improve the accuracy and efficiency of complex facial recognition systems and addresses an important need in the fields of security to public health and beyond. Also, with the advancement of technology and the progress of research in this field, the possibility of improving the accuracy of detecting contracted faces is still promising.
Keywords
Face Recognition
Mask
MediaPipe
Deep Learning
1.      Avazov, K., Abdusalomov, A., Mukhiddinov, M., Baratov, N., Makhmudov, F., & Cho, Y. I. (2022). An improvement for the automatic classification method for ultrasound images used on CNN. International Journal of Wavelets, Multiresolution and Information Processing, 20(02), 2150054.
2.      Canal, F. Z., Müller, T. R., Matias, J. C., Scotton, G. G., de Sa Junior, A. R., Pozzebon, E., & Sobieranski, A. C. (2022). A survey on facial emotion recognition techniques: A state-of-the-art literature review. Information Sciences, 582, 593-617.
3.      Chen, X., Wang, X., Zhang, K., Fung, K. M., Thai, T. C., Moore, K., ... & Qiu, Y. (2022). Recent advances and clinical applications of deep learning in medical image analysis. Medical image analysis, 79, 102444.
4.      Dzedzickis, A., Kaklauskas, A., & Bucinskas, V. (2020). Human emotion recognition: Review of sensors and methods. Sensors, 20(3), 592.
5.      Guo, K., Soornack, Y., & Settle, R. (2019). Expression-dependent susceptibility to face distortions in processing of facial expressions of emotion. Vision research, 157, 112-122.
6.      Hess, U. (2021). Who to whom and why: The social nature of emotional mimicry. Psychophysiology, 58(1), e13675.
7.      JM, S. (2009). Expressing fear enhances sensory acquisition. Nature Neurosci, 43, 822-829.
8.      Keltner, D., Sauter, D., Tracy, J., & Cowen, A. (2019). Emotional expression: Advances in basic emotion theory. Journal of nonverbal behavior, 43, 133-160.
9.      Ko, B. C. (2018). A brief review of facial emotion recognition based on visual information. sensors, 18(2), 401.
10.  Li, W., Zhang, L., Wu, C., Cui, Z., & Niu, C. (2022). A new lightweight deep neural network for surface scratch detection. The International Journal of Advanced Manufacturing Technology, 123(5), 1999-2015.
11.  Maithri, M., Raghavendra, U., Gudigar, A., Samanth, J., Barua, P. D., Murugappan, M., ... & Acharya, U. R. (2022). Automated emotion recognition: Current trends and future perspectives. Computer methods and programs in biomedicine, 215, 106646.
12.  Mellouk, W., & Handouzi, W. (2020). Facial emotion recognition using deep learning: review and insights. Procedia Computer Science, 175, 689-694.
13.  Mukhiddinov, M., Akmuradov, B., & Djuraev, O. (2019, November). Robust text recognition for Uzbek language in natural scene images. In 2019 International Conference on Information Science and Communications Technologies (ICISCT) (pp. 1-5). IEEE.
14.  Mukhiddinov, M., Jeong, R. G., & Cho, J. (2020). Saliency cuts: salient region extraction based on local adaptive thresholding for image information recognition of the visually impaired. Int. Arab J. Inf. Technol., 17(5), 713-720.
15.  Mukhiddinov, M., Djuraev, O., Akhmedov, F., Mukhamadiyev, A., & Cho, J. (2023). Masked face emotion recognition based on facial landmarks and deep learning approaches for visually impaired people. Sensors, 23(3), 1080.
16.  Mukhamadiyev, A., Khujayarov, I., Djuraev, O., & Cho, J. (2022). Automatic speech recognition method based on deep learning approaches for Uzbek language. Sensors, 22(10), 3683.
17.  Rakhmatillaevich, K. U., Ugli, M. M. N., Ugli, M. A. O., & Nuruddinovich, D. O. (2018). A novel method for extracting text from naturalscene images and TTS. European science review, 1(11-12), 30-33.
18.  Ramdani, C., Ogier, M., & Coutrot, A. (2022). Communicating and reading emotion with masked faces in the Covid era: A short review of the literature. Psychiatry research, 316, 114755.
19.  Saxena, A., Khanna, A., & Gupta, D. (2020). Emotion recognition and detection methods: A comprehensive survey. Journal of Artificial Intelligence and Systems, 2(1), 53-79.
20.  Susskind, J. M., Lee, D. H., Cusi, A., Feiman, R., Grabski, W., & Anderson, A. K. (2008). Expressing fear enhances sensory acquisition. Nature neuroscience, 11(7), 843-850.
21.  Van Kleef, G. A. (2009). How emotions regulate social life: The emotions as social information (EASI) model. Current directions in psychological science, 18(3), 184-188.
22.  Xia, C., Pan, Z., Li, Y., Chen, J., & Li, H. (2022). Vision-based melt pool monitoring for wire-arc additive manufacturing using deep learning method. The International Journal of Advanced Manufacturing Technology, 120(1), 551-562.
Intelligent Knowledge Exploration and Processing
Volume 4, Issue 12
Spring 2024
Pages 88-101