Hand Gesture Recognition Using Optimized Hyperparameters of CNN for Real-Time Control of a Multi-Servo Hand
DOI:
https://doi.org/10.37385/jaets.v7i1.8740Keywords:
Hand gesture, Hybrid CNN, PSO, GWO, BBOAbstract
Gesture recognition has emerged as a promising approach to enhance human-machine interaction, especially in robotics and assistive devices. This study presents a real-time gesture-controlled robotic system that combines deep learning and machine learning techniques to recognize hand gestures and map them to servo motor movements. A convolutional neural network (CNN) was used to classify six predefined hand gestures: a closed fist and five individual finger extensions. To enhance classification accuracy and generalization, CNNs are tuned using hyperparameter tuning techniques, such as Particle Swarm Optimization (PSO), Grey Wolf Optimization (GWO), and Brown Bear Optimization (BBO). These methods efficiently explore the hyperparameter space—such as learning rate, filter size, and batch size—reducing manual trial and error in control. Among these proposed models tested, the BBO-CNN has been achieved the highest performance with a classification accuracy of 99.98%, outperforming both PSO-CNN (99.89%) and GWO-CNN (99.44%). The model CNN without optimization achieved an accuracy of 97.50%. The combination of advanced deep learning models and embedded control demonstrates the feasibility and effectiveness of gesture-based robotics applications.
Downloads
References
s. IEEE Access, 12, 28564–28574. https://doi.org/10.1109/ACCESS.2024.3360857
AL Kafaf, D., Thamir, N. N., & AL-Hadithy, S. S. (2024). Malaria disease prediction based on convolutional neural networks. Journal of Applied Engineering and Technological Science (JAETS), 5(2), 1165–1181. https://doi.org/10.37385/jaets.v5i2.3947
Alabduallah, B., Al Dayil, R., Alkharashi, A., & Alneil, A. A. (2025). Innovative hand pose-based sign language recognition using hybrid metaheuristic optimization algorithms with a deep learning model for hearing-impaired persons. Scientific Reports, 15(1), 9320.
Ashraf, H., Waris, A., Gilani, S. O., Shafiq, U., Iqbal, J., Kamavuako, E. N., & Niazi, I. K. (2024). Optimizing the performance of convolutional neural networks for enhanced gesture recognition using sEMG. Scientific Reports, 14(1), 2020. https://doi.org/10.1038/s41598-024-52405-9
Chen, B., Cao, L., Chen, C., Chen, Y., & Yue, Y. (2024). A comprehensive survey on the chicken swarm optimization algorithm and its applications: State-of-the-art and research challenges. Artificial Intelligence Review, 57(7), 170. https://doi.org/10.1007/s10462-024-10786-3
Desai, V., Kharde, A., Deochake, S., Nakwa, C., Patil, S., Wadile, H., & Avasare, O. (2025). Hand gesture-based servo motor control using edge computing. International Journal of Advanced Research in Science, Communication and Technology, 5(8). https://doi.org/10.48175/IJARSCT-25576
Devecioglu, I., & Karakulak, E. (2024). Three sliding probes placed on forelimb skin for proprioceptive feedback differentially yet complementarily contribute to hand gesture detection and object-size discrimination. Annals of Biomedical Engineering, 52(4), 982–996. https://doi.org/10.1007/s10439-023-03434-4
Faris, H., Aljarah, I., Al-Betar, M. A., & Mirjalili, S. (2018). Grey wolf optimizer: A review of recent variants and applications. Neural Computing and Applications, 30(2), 413–435. https://doi.org/10.1007/s00521-017-3272-5
Farooq, F., Ali, Z. A., Shafiq, M., Israr, A., & Hasan, R. (2025). Intelligent planning of UAV flocks via transfer learning and multi-objective optimization. Arabian Journal for Science and Engineering. Advance online publication. https://doi.org/10.1007/s13369-025-10064-6
Fatima, B., Mushtaq, B., Iqbal, M. A., & Ahmed, A. (2024). IoT-based smart home automation using gesture control and machine learning for individuals with auditory challenges. ICCK Transactions on Internet of Things, 2(4), 74–82.
Fukano, K., Iiazawa, K., Soeda, T., Shirai, A., & Capi, G. (2021). Deep learning for gesture recognition based on surface EMG data. In Proceedings of the 2021 International Conference on Advanced Mechatronic Systems (ICAMechS) (pp. 41–45). IEEE. https://doi.org/10.1109/ICAMechS54019.2021.9661533
Harman, E., Singh, P., & Kaur, A. (2015). Statistical analysis of velocity update rules in particle swarm optimization. International Journal of Artificial Intelligence Applications and Smart Devices, 3, 11–22..
Jaddoa, I. A. (2024). Integration of convolutional neural networks and grey wolf optimization for advanced cybersecurity in IoT systems. Journal of Robotics and Control, 5(4), 1189–1202. https://doi.org/10.18196/jrc.v5i4.22178
Meem, A. A., & Roy, S. (2025). Hand gesture controlled pick and place robot. In AIP Conference Proceedings (Vol. 3307, No. 1, Article 020004). AIP Publishing LLC. https://doi.org/10.1063/5.0262065
Mehta, P., Kumar, S., & Tejani, G. G. (2024). MOBBO: A multi-objective brown bear optimization algorithm for solving constrained structural optimization problems. Journal of Optimization, 2024(1), Article 5546940. https://doi.org/10.1155/2024/5546940
Mujahid, A., Awan, M. J., Yasin, A., Mohammed, M. A., Damasevicius, R., Maskeliunas, R., & Abdulkareem, K. H. (2021). Real-time hand gesture recognition based on deep learning YOLOv3 model. Applied Sciences, 11(9), 4164. https://doi.org/10.3390/app11094164
Nasir, M., Sadollah, A., Mirjalili, S., Mansouri, S. A., Safaraliev, M., & Rezaee Jordehi, A. (2024). A comprehensive review on applications of grey wolf optimizer in energy systems. Archives of Computational Methods in Engineering. Advance online publication. https://doi.org/10.1007/s11831-024-10214-3
Papalkar, R. R., Jadhav, J., Pattewar, T., Thorat, V., Morey, P., Deshmukh, M., & Jagdale, R. (2025). WACSO: Wolf crow search optimizer for convolutional neural network hyperparameter optimization. Neural Processing Letters, 57(2), 1–22. https://doi.org/10.1007/s11063-025-11740-2
Patil, N., Ansari, M. W., Jadhav, S. R., Mhaske, M. G., & Rohit, K. K. (2024). Gesture voice: Revolutionizing human–computer interaction with an AI-driven virtual mouse system. Turkish Online Journal of Qualitative Inquiry, 15(3). https://doi.org/10.53555/tojqi.v15i3.10282
Qi, J., Ma, L., Cui, Z., & Yu, Y. (2024). Computer vision-based hand gesture recognition for human–robot interaction: A review. Complex & Intelligent Systems, 10(1), 1581–1606. https://doi.org/10.1007/s40747-023-01173-6
Rezaee, K., Khavari, S. F., Ansari, M., Zare, F., & Roknabadi, M. H. A. (2024). Hand gesture classification of sEMG signals based on BiLSTM–metaheuristic optimization and a hybrid U-Net–MobileNetV2 encoder architecture. Scientific Reports, 14(1), 31257. https://doi.org/10.21203/rs.3.rs-4254210/v1
Rezaei, H., Bozorg-Haddad, O., & Chu, X. (2017). Grey wolf optimization (GWO) algorithm. In O. Bozorg-Haddad (Ed.), Advanced optimization by nature-inspired algorithms (pp. 81–91). Springer. https://doi.org/10.1007/978-981-10-5221-7_9
Saeed, Z. R., Ibrahim, N. F., Zainol, Z. B., & Mohammed, K. K. (2025). A hybrid improved IRSO–CNN algorithm for accurate recognition of dynamic gestures in Malaysian Sign Language. Journal of Electrical and Computer Engineering, 2025(1), Article 6430675. https://doi.org/10.1155/jece/6430675
Sakinala, U. C., & Abinaya, S. (2025). Enhanced detection of hand gestures from sEMG signals using stacking ensemble with particle swarm optimization and meta-classifier. IEEE Access. https://doi.org/10.1109/ACCESS.2025.3559182
Sayed, G. I., Hassanien, A. E., & Basha, S. H. (2024). Neutrosophic set and optimized deep learning for classification of chicken Eimeria species: A practical solution for the poultry industry. Environment, Development and Sustainability, 1–28. https://doi.org/10.1007/s10668-024-05478-5
Shi, H., Jiang, X., Dai, C., & Chen, W. (2024). EMG-based multi-user hand gesture classification via unsupervised transfer learning using unknown calibration gestures. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 32, 1119–1131. https://doi.org/10.1109/TNSRE.2024.3372002
Shin, J., Kaneko, Y., Miah, A. S. M., Hassan, N., & Nishimura, S. (2024). Anomaly detection in weakly supervised videos using multistage graphs and deep learning-based spatiotemporal feature enhancement. IEEE Access, 12, 65213–65227. https://doi.org/10.1109/ACCESS.2024.3395329
Sümbül, H. (2024). A novel MEMS- and flex sensor-based hand gesture recognition and regenerating system using a deep learning model. IEEE Access, 133685–133693. https://doi.org/10.1109/ACCESS.2024.3448232
Sun, Q., Zhang, T., Gao, S., Yang, L., & Shao, F. (2024). Optimizing gesture recognition for seamless UI interaction using convolutional neural networks. In 2024 International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) (pp. 1838–1842). IEEE. https://doi.org/10.1109/ICICML63543.2024.10958166
Sun, Y., Huang, J., Cheng, Y., Zhang, J., Shi, Y., & Pan, L. (2024). High-accuracy dynamic gesture recognition: A universal and self-adaptive deep-learning-assisted system leveraging high-performance ionogel-based strain sensors. SmartMat, 5(6), e1269. https://doi.org/10.1002/smm2.1269
Utama, A. B. P., Wibawa, A. P., Muladi, M., & Nafalski, A. (2022). PSO-based hyperparameter tuning of CNN multivariate time-series analysis. Jurnal Online Informatika, 7(2), 193–202. https://doi.org/10.15575/join.v7i2.858
Wang, X., Veeramani, D., Dai, F., & Zhu, Z. (2024). Context-aware hand gesture interaction for human–robot collaboration in construction. Computer-Aided Civil and Infrastructure Engineering, 39(22), 3489–3504. https://doi.org/10.1111/mice.13202
Yu, Y., Zhou, Z., Xu, Y., Chen, C., Guo, W., & Sheng, X. (2025). Toward hand gesture recognition using a channel-wise cumulative spike train image-driven model. Cyborg and Bionic Systems, 6, Article 0219. https://doi.org/10.34133/cbsystems.0219
CITEDNESS IN SCOPUS
CITEDNESS IN WOS
