Human Activity Recognition Using Inertial Sensors in a Smartphone: Technical Background (Review)
DOI:
https://doi.org/10.22401/370q9638Keywords:
Deep Learning , HAR Approach, Machine Learning , Technical BackgroundAbstract
Human Activity Recognition (HAR) stands at the intersection of machine learning, deep learning, and sensor technology, primarily focusing on leveraging inertial sensors in smartphones and wearable devices. This paper presents a comprehensive technical overview of HAR, examining the amalgamation of machine learning and deep learning systems while considering the data inputs from mobile and wearable inertial sensors. The review encompasses a broad spectrum of methodologies applied to HAR, ranging from classical machine learning algorithms to cutting-edge deep learning architectures. Emphasis is placed on the nuanced challenges and opportunities posed using inertial sensors in smartphones and wearables. This includes discussions on data preprocessing strategies, feature extraction methods, and model architectures, accounting for the unique characteristics of sensor data, such as noise, variability, and power consumption. The paper explores recent advancements, scrutinizing state-of-the-art approaches, innovative model architectures, and emerging trends in HAR. Through a comparative evaluation of various machine learning and deep learning techniques, the review aims to guide researchers and practitioners in selecting the most appropriate methods for HAR applications across diverse scenarios. In conclusion, this paper serves as an inclusive guide to the technical landscape of HAR, incorporating insights from both mobile and wearable inertial sensors. By synthesizing existing knowledge and addressing future research directions, it aims to propel advancements in developing robust and efficient systems for recognizing human activities, accommodating the evolving landscape of sensor technologies in mobile and wearable devices.
References
Ahmadi, M.N.; Pfeiffer, K.A.; Trost, S.G.; “Physical Activity Classification in Youth Using Raw Accelerometer Data from the Hip,” Meas. Phys. Edu. Exe. Sci. 24(2): 129–136, 2020.
Wang, Y.; Cang, S.; Yu, H.; “A survey on wearable sensor modality centered human activity recognition in health care,” Exp. Sys. Appl. 137: 167–190, 2019.
Voicu, R.-A.; Dobre, C.; Bajenaru, L.; Ciobanu, R.-I.; “Human Physical Activity Recognition Using Smartphone Sensors,” Sensors 19(3): 458, 2019.
Minh Dang, L.; Min, K.; Wang, H.; Md. Jalil P.; Lee, C.H.; Moon, H.; “Sensor-based and vision-based human activity recognition: A comprehensive survey,” Pat. Recogn. 108: 107561, 2020.
Jung, M.; Chi, S.; “Human activity classification based on sound recognition and residual convolutional neural network,” Auto. Const. 114: 103177, 2020.
Ma, M.; Marturi, N.; Li, Y.; Leonardis, A.; Stolkin, R.; “Region-sequence based six-stream CNN features for general and fine-grained human action recognition in videos,” Patter. Recogn. 76: 506–521, 2018.
Vidya, B.S.P, “Wearable multi-sensor data fusion approach for human activity recognition using machine learning algorithms,” Sens. and Actu. A: Physical 341: 113557, 2022.
Jordao, A.; “Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art,” arXiv.org, 2018. doi: 10.48550/arXiv.1806.05226.
Dang L.M.; Piran, M.; Han, D.; Min, K.; Moon H. et al., “A survey on internet of things and cloud computing for healthcare”, Electronics 8 (7): 768, 2019.
Alshwely, M.K.; AlSaad, S.N.; “Image splicing detection based on noise level approach,” Al-Mustansiriyah J. Sci. 31(4): 55–61, 2020.
Maharana, K.; Mondal, S.; Nemade, B.; “A review: Data pre-processing and data augmentation techniques,” Glob. Trans. Proc. 3(1): 91–99, 2022.
Chen, L.; Nugent, C.D.; Chen, L.; Nugent, C.D.; “Time-Window Based Data Segmentation. Human Activity Recognition and Behaviour Analysis”: Cyber-Physical Systems in Smart Environments, pp.103-126, 2019.
Wang, Y.; Li, M.; Shang, Y.; Li, N.; Zhao, Y.; Shang, Z.; "Analysis and Classification of Time Domain Features of Pulse Manifestation Signals between Different Genders". 2019 Chinese Automation Congress (CAC), Hangzhou, China, 2019, pp. 3981-3984, doi: 10.1109/CAC48633.2019.8997260.
Fan, C.; Gao, F.; “Enhanced Human Activity Recognition Using Wearable Sensors via a Hybrid Feature Selection Method,” Sensors 21(19): 6434, 2021.
Gaikwad, N.B.; Tiwari, V.; Keskar, A.; Shivaprakash, N.C.; ”Efficient FPGA implementation of multilayer perceptron for real-time human activity classification”. IEEE Access, 7: 26696–26706, 2019.
D’Arco, L.; Wang, H.; Zheng, H.; ”Assessing impact of sensors and feature selection in smart-insole-based human activity recognition”. Meth. Proto. 5(3): 45, 2022.
Guo, J.; Mu, Y.; Xiong, M.; Liu, Y.; Gu, J.; ”Activity feature solving based on TF-IDF for activity recognition in smart homes”, Complexity: 2019.
Venkatesh, B.; Anuradha, J.; “A Review of Feature Selection and Its Methods,” Cyber. Info. Technol. 19(1); 3–26, 2019.
Khaire, U.M.; Dhanalakshmi, R.; “Stability of feature selection algorithm: A review,” J. King Saud Uni. Comp. Info. Sci. 34(4): 1060–1073, 2022.
Tawhid, M.A.; Ibrahim, A.M.; “Feature selection based on rough set approach, wrapper approach, and binary whale optimization algorithm,” Int. J. Mach. Lear. Cyber. 11(3): 573–602, 2019.
Effrosynidis, D.; Arampatzis, A.; “An evaluation of feature selection methods for environmental data,” Ecol. Info. 61: 101224, 2021.
Saha, A.; Sharma, T.; Batra, H.; Jain, A.; Pal, V.; “Human Action Recognition Using Smartphone Sensors,” 2020 International Conference on Computational Performance Evaluation (ComPE), Jul. 2020, Published, doi: 10.1109/compe49325.2020.9200169.
Dewi, C.; Chen, R.-C.; "Human Activity Recognition Based on Evolution of Features Selection and Random Forest," 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, 2019, pp. 2496-2501, doi: 10.1109/SMC.2019.8913868.
Nguyen, H.D.; “Wearable Sensor Data Based Human Activity Recognition using Machine Learning: A new approach,” arXiv.org, May 09, 2019. https://doi.org/10.48550/arXiv.1905.03809
Asmita, N.; Jayita, S.; Chandreyee, C.; Kundan, P.D.S.; “Detailed Human Activity Recognition using Wearable Sensor and Smartphones,” International Conference on Opto-Electronics and Applied Optics, IEEE Conference Publication 2019.
https://doi.org/10.1109/OPTRONIX.2019.8862427.
Athavale, V.A.; Gupta, S.C.; Kumar, D.; Savita; “Human Action Recognition Using CNN-SVM Model”. Adv. Sci. Technol. 105: 282–290, 2021.
Hossain Shuvo, M.M.; Ahmed, N.; Nouduri, K.; Palaniappan, K.; "A Hybrid Approach for Human Activity Recognition with Support Vector Machine and 1D Convolutional Neural Network," 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), Washington DC, DC, USA, 2020, pp. 1-5, doi: 10.1109/AIPR50011.2020.9425332.
Loris, N.; Matteo, I.; Sheryl, B.; Christian, S.; Sergio, P.; Raffaello, N.; Isabella, C.;, “Comparison of Transfer Learning and Conventional Machine Learning Applied to Structural Brain MRI for the Early Diagnosis and Prognosis of Alzheimer’s Disease,” Frontiers in Neurology, vol. 11, Nov. 2020.
Burkart, N.; Huber, M.F.; “A Survey on the Explain ability of Supervised Machine Learning,” J. Art. Intel. Res. 70: 245–317, 2021.
Gamal, G.; Omar, Y.M.; Maghraby, F.A.; “Machine Learning Approaches for Human Activity Recognition Based on Multimodal Body Sensors”. In Intelligent Systems Design and Applications: 20th International Conference on Intelligent Systems Design and Applications (ISDA 2020) held December 12-15, 2020 (pp. 977-987). Cham: Springer International Publishing.
Randhawa, P.; Shanthagiri, V.; Kumar, A.; Yadav, V.; “Human activity detection using machine learning methods from wearable sensors”. Sensor Review 40(5): 591-603, 2020.
Levine, S.; “Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems,” arXiv.org, 2020.
https://doi.org/10.48550/arXiv.2005.01643.
Lorenzo, C.; Gian, C.C.; Luca, D.N.; Rocco, F.; Daniele, G.; Marco, R.; Sergio, S.; “Multi-Agent Reinforcement Learning: A Review of Challenges and Applications”. Appl. Sci. 11(11): 4948, 2021.
Sewak, M.; “Deep Q Network (DQN), Double DQN, and Dueling DQN,” Deep Reinforcement Learning: 95–108, 2019.
Fan, C.; Gao, F.; “Enhanced Human Activity Recognition Using Wearable Sensors via a Hybrid Feature Selection Method,” Sensors 21(19): 6434, 2021.
Kalmár, G.; ”Hardware-Software Co-development for Audio and Video Data Acquisition and Analysis” (Doctoral dissertation, Szte).
Berry, M.W.; Mohamed, A.; Yap, B.W.; “Supervised and Unsupervised Learning for Data Science”. Springer Nature, 2019.
Manduchi, L.; Hüser, M.; Faltys, M.; Vogt, J.; Rätsch, G.; Fortuin, V.; “T-DPSOM,” Proceedings of the Conference on Health, Inference, and Learning, Apr. 2021, Published, doi: 10.1145/3450439.3451872.
Zhao, R.; Ruan, J.; Dong, B.; Meng, L.; Zhang, W.; “Improving unsupervised image clustering with spatial consistency,” Knowledge-Based Systems 246: 108673, 2022.
Jamel, A.A.; Akay, B.; “Human Activity Recognition Based on Parallel Approximation Kernel K-Means Algorithm. Comput”. Syst. Sci. Eng. 35(6): 441-456, 2020.
Sheng, T.; Huber, M.; “Unsupervised embedding learning for human activity recognition using wearable sensor data”. in The Thirty-Third International Flairs Conference, 2020.
Telikani, A.; Gandomi, A.H.; Shahbahrami, A.; “A survey of evolutionary computation for association rule mining,” Info. Sci. 524: 318–352, 2020.
Zhang, Y.; Huang, Q.; Zhao, K.; “Hybridizing association rules with adaptive weighted decision fusion for personal credit assessment”. Sys. Sci. Contr. Eng. 7(3): 135-142, (2019).
Espadoto, M.; Martins, R.M.; Kerren, A.; Hirata, N.S. T.; Telea, A.C.; "Toward a Quantitative Survey of Dimension Reduction Techniques". IEEE Trans. Vis. Comp. Graph. 27(3): 2153-2173, 2021.
Kherif, F.; Latypova, A.; “Principal component analysis,” Mach. Lear.: 209–225, 2020.
Aljarrah, A.A.; Ali, A.H.; "Human Activity Recognition using PCA and BiLSTM Recurrent Neural Networks" in 2019 2nd International Conference on Engineering Technology and its Applications (IICETA), 2019.
Kelleher, J.D.; Deep Learning. MIT Press, 2019.
Oyedotun, O.K; Khashman, A.; “Deep learning in vision-based static hand gesture recognition”, Neur. Comp. Appl. 28(12): 3941–3951, 2017.
Chen, C.; Jafari, R.; Kehtarnavaz, N.; “A survey of depth and inertial sensor fusion for human action recognition”, Multimed. Tool. Appl. 76(3): 4405–4425, 2017.
Zhang, Q.; Yang, L.T.; Chen, Z.; Li, P.; “A survey on deep learning for big data”, Info. Fusion (42): 146–157, 2018.
Mohammadi, M.; Al-Fuqaha, A.; Sorour, S.; Guizani, M.; “Deep learning for IoT big data and streaming analytics: a survey”, IEEE Comm. Surv. Tutor. 20(4): 2923–2960, 2018.
Khassaf, N.M.; Shaker, S.H.; “Image Retrieval based Convolutional Neural Network,” Al-Mustansiriyah J. Sci. 31(4): 43–54, 2020.
Xu, W.; Pang, Y.; Yang, Y.; Liu, Y.; “Human activity recognition based on convolutional neural network”. In 2018 24th international conference on pattern recognition (ICPR) (pp. 165-170) (2018). IEEE.
Bianchi, V.; Bassoli, M.; Lombardo, G.; Fornacciari, P.; Mordonini, M.; De-Munari, I.; “IoT wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment”. IEEE Inter. Thin. J. 6(5): 8553-8562, 2019.
Haqi Al-Tai, M.; Nema, B.M.; Al-Sherbaz, v; “Deep Learning for Fake News Detection: Literature Review,” Al-Mustansiriyah J. Sci. 34(2): 70–81, 2023.
Sherstinsky, A.; “Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network,” Physica D: Nonlin. Phenomena 404: 132306, 2020.
Schmidt, R.M.; “Recurrent Neural Networks (RNNs): A gentle Introduction and Overview,” arXiv.org, 2019.
https://doi.org/10.48550/arXiv.1912.05911.
Bouarara, H.A.; “Recurrent Neural Network (RNN) to Analyse Mental Behaviour in Social Media,” Int. J. Soft. Sci. Comp. Intel. 13(3): 1–11, 2021.
Mekruksavanich, S.; Jitpattanakul, A.; “Lstm networks using smartphone data for sensor-based human activity recognition in smart homes”. Sensors 21(5): 1636, 2021.
Ian, G.; Jean, P.A.; Mehdi, M.; Bing, X.; David, W.F.; Sherjil, O.; Aaron, C.; Yoshua, B.; “Generative adversarial networks,” Communications ACM 63(11): 139–144, 2020.
Hoelzemann, A.; Sorathiya, N.; Van-Laerhoven, K.; "Data Augmentation Strategies for Human Activity Data Using Generative Adversarial Neural Networks," in 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), 2021.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Hade Khalaf, Musaab Riyadh
This work is licensed under a Creative Commons Attribution 4.0 International License.
.jpg)
