Detecting Exercise and Calories Burned Using Human Pose Estimation

Abstract
The overarching idea of this research project is to develop a mobile application that can independently track the human body, detect exercise and workout, and count calories burned while doing various activities without any external devices. With the use of the Human Pose Estimation technique, we can track different joints and their position in real-time. A machine learning classifier algorithm will be used to predict the exercise being performed using these joint positions. Then calories burned can be calculated using various factors like what joints are being used, the duration of the workout, its intensity, etc. The main idea of the research is to perform all these tasks in a mobile device without the use of devices like smartwatches, heart rate sensors, and depth cameras. The research also comprises of in-depth study in differences between machine learning algorithm and their accuracy in predicting (classifying) the correct workout being performed by the user. Both, the research and practical applications stretch from- future development in mobile phone’s capabilities in health and medical fields like joint therapy, correcting exercise posture, to a commercial ready application that can be used by millions of people who cannot afford expensive tracking devices.

LOESS smoothed-out data showing X and Y time-series data for all the 14 joints with respect to time
as tracked by pose estimation

Dynamic Time Warping (DTW) visualization for comparing two different time-series data of ‘L_ELBOW_X’

Dynamic Time Warping (DTW) visualization for comparing two different time-series data of ‘L_KNEE_X’