Beyond Fitness: Revolutionary Exercise Tracker Combining Pose Recognition with Heart Rate Monitoring using Remote Photoplethysmography (rPPG)
, Sadaf Kauser 2, Maniza Hijab 1, Gouri R Patil 1, Syeda Fouzia Sayeedunissa 1
Adv. Sci. Technol. Eng. Syst. J. 11(1), 33–43 (2026);
DOI: 10.25046/aj110104
Heart rate (HR) is a critical indicator in fitness monitoring, athletic performance evaluation, and injury prevention. However, traditional motion-sensitive wearable devices are highly susceptible to movement artifacts, which degrade measurement accuracy during physical activity. Remote photoplethysmography (rPPG) offers a non-contact alternative for HR measurement, though it too remains sensitive to motion. This study proposes a novel framework that integrates MediaPipe’s BlazePose for bicep-curl motion tracking with rPPG-based real-time HR monitoring. The system leverages pose-estimation data to accurately track arm movements and applies advanced signal-processing techniques to suppress motion-induced noise in the rPPG signal, thereby enhancing HR estimation accuracy. The extracted signal is pre-processed using filtered, normalized, and transformed from the time domain to the frequency domain, enabling reliable HR extraction during continuous exercise. The green color channel is selected for BPM estimation, as it exhibits a strong correlation with cardiac pulse signals. Experimental results demonstrate that motion-corrected rPPG can effectively support real-time, non-contact HR monitoring during exercise. This approach shows strong potential as a platform for personalized fitness coaching and AI-based workout optimization. Future work will focus on extending the system to support multi-exercise tracking and continuous performance monitoring. The proposed framework enhances user safety by enabling early detection of abnormal heart rate patterns during exercise, such as tachycardia or irregular fluctuations caused by overexertion. By correlating physiological signals with biomechanical movement data, the system can adapt workout intensity in real time, ensuring exercises remain within safe cardiovascular limits. This closed-loop feedback mechanism supports individualized training plans tailored to the user’s fitness level and physiological response.
- J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiological Measurement, 28(3), R1–R39, 2007, doi:10.1088/0967-3334/28/3/R01.
- V. Bazarevsky, BlazePose: on-device real-time body pose tracking, 2020.
- Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, Y. Sheikh, “OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1), 172–186, 2021, doi:10.1109/TPAMI.2019.2929257.
- P.H. Charlton, P.A. Kyriacou, J. Mant, V. Marozas, P. Chowienczyk, J. Alastruey, “Wearable Photoplethysmography for Cardiovascular Monitoring,” Proceedings of the IEEE, 110(3), 355–381, 2022, doi:10.1109/JPROC.2022.3149785.
- C. Cheuque, M. Querales, R. León, R. Salas, R. Torres, “An Efficient Multi-Level Convolutional Neural Network Approach for White Blood Cells Classification,” Diagnostics, 12(2), 248, 2022, doi:10.3390/diagnostics12020248.
- Y. Cui, C.-H. Fu, H. Hong, Y. Zhang, F. Shu, “Non-contact time varying heart rate monitoring in exercise by video camera,” in 2015 International Conference on Wireless Communications & Signal Processing (WCSP), IEEE: 1–5, 2015, doi:10.1109/WCSP.2015.7341278.
- A.A. Fedotov, “Selection of Parameters of Bandpass Filtering of the ECG Signal for Heart Rhythm Monitoring Systems,” Biomedical Engineering, 50(2), 114–118, 2016, doi:10.1007/s10527-016-9600-8.
- N. Gayapersad, S. Rocke, Z. Ramsaroop, A. Singh, C. Ramlal,“Beyond Blood Pressure and Heart Rate Monitoring: Towards a Device for Continuous Sensing and Automatic Feature Extraction of Cardiovascular Data,” in 2016 8th International Conference on Computational Intelligence and Communication Networks (CICN), IEEE: 261–265, 2016, doi:10.1109/CICN.2016.58.
- Google AI Blog, On-device real-time body pose tracking, Google AI Blog, 2020.
- G. Guven, H. Gürkan, U. Guz, “Biometric identification using fingertip electrocardiogram signals,” Signal, Image and Video Processing, 12(5), 933–940, 2018, doi:10.1007/s11760-018-1238-4.
- A.G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, MobileNets: efficient convolutional neural networks for mobile vision applications, 2017.
- J. Lee, K. Matsumura, K. Yamakoshi, P. Rolfe, S. Tanaka, T. Yamakoshi, “Comparison between red, green and blue light reflection photoplethysmography for heart rate monitoring during motion,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE: 1724–1727, 2013, doi:10.1109/EMBC.2013.6609852.
- K. Li, H. Rüdiger, T. Ziemssen, “Spectral Analysis of Heart Rate Variability: Time Window Matters,” Frontiers in Neurology, 10, 2019, doi:10.3389/fneur.2019.00545.
- Y.-C. Lin, Y.-H. Lin, “A study of color illumination effect on the SNR of rPPG signals,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE: 4301–4304, 2017, doi:10.1109/EMBC.2017.8037807.
- H. Lu, H. Han, S.K. Zhou, “Dual-GAN: Joint BVP and Noise Modeling for Remote Physiological Measurement,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE: 12399–12408, 2021, doi:10.1109/CVPR46437.2021.01222.
- C. Nadrag, V. Poenaru, G. Suciu, “Heart Rate Measurement Using Face Detection in Video,” in 2018 International Conference on Communications (COMM), IEEE: 131–134, 2018, doi:10.1109/ICComm.2018.8484779.
- M.-Z. Poh, D.J. McDuff, R.W. Picard, “Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam,” IEEE Transactions on Biomedical Engineering, 58(1), 7–11, 2011, doi:10.1109/TBME.2010.2086456.
- F. Taranum, S.F. Sayeedunnisa, G.R. Patil, K.S. Hijab, S.S. Ahmad, “Deep learning framework with optimizations for automatic detection of arrhythmia from ECG data,” Journal of Theoretical and Applied Information Technology, 103(7), 2025.
- D.N. Tran, H. Lee, C. Kim, “A robust real time system for remote heart rate measurement via camera,” in 2015 IEEE International Conference on Multimedia and Expo (ICME), IEEE: 1–6, 2015, doi:10.1109/ICME.2015.7177484.
- A. Toshev, C. Szegedy, “DeepPose: Human Pose Estimation via Deep Neural Networks,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE: 1653–1660, 2014, doi:10.1109/CVPR.2014.214.
- W. Verkruysse, L.O. Svaasand, J.S. Nelson, “Remote plethysmographic imaging using ambient light,” Optics Express, 16(26), 21434, 2008, doi:10.1364/OE.16.021434.
- P. Viola, M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, IEEE Computer Society: I–518, 2001, doi:10.1109/CVPR.2001.990517.
- S.-E. Wei, V. Ramakrishna, T. Kanade, Y. Sheikh, “Convolutional Pose Machines,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE: 4724–4732, 2016, doi:10.1109/CVPR.2016.511.
- T. Wu, V. Blazek, H.J. Schmitt, “Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes,” in: Priezzhev, A. V and Oberg, P. A., eds., 62, 2000, doi:10.1117/12.407646.
- F. Xu, Q.-W. Zhou, P. Wu, X. Chen, X. Yang, H. Yan, “Heart rate measurement based on face video sequence,” in: Hadjiiski, L. M. and Tourassi, G. D., eds., in Medical Imaging: Computer-Aided Diagnosis, SPIE: 941435, 2015, doi:10.1117/12.2083315.
- Q. Zhu, C.-W. Wong, C.-H. Fu, M. Wu, “Fitness heart rate measurement using face videos,” in 2017 IEEE International Conference on Image Processing (ICIP), IEEE: 2000–2004, 2017, doi:10.1109/ICIP.2017.8296632.