Pose tracking and error detection models for safe dumbbell workouts developed from scratch.

• ChallengeImplement workout error detection based on limited data
• SolutionCustom video pose estimation models for dumbbell weight workouts
• Technologies and toolsPyTorch, Tensorflow, Torchvision, Albumentations, OpenCV, FFmpeg, Scikit-video, Tensorboard, Numpy, Matplotlib, Flask, Redis

Client

The client works in the sports industry. The client is an entrepreneur, acclaimed heavyweight champion bodybuilder, and the creator of a fitness product. They’ve made exclusive dumbbells for effective and injury-free workouts. The product targets Olympic athletes, sports teams, premium sports clubs and gyms, universities, military bases, etc. The dumbbells boast of maximum stability and control, so workouts are safe.

Challenge: implement workout error detection based on limited data

They wanted to hire a technology vendor with hands-on experience in video analytics and AI. The client wanted to bring some innovation to the project and make it technology-wise. They were seeking to add tech that’d allow the user to track their workouts while using dumbbells. They also wanted to automate motion estimation during workouts (detecting and tracking pose estimation in videos), since it was still a manual task and required a lot of time and effort.

InData Labs has got broad experience in AI video analytics software and fitness and physical therapy application development. So, the client asked us to take on the project.

The first challenge was working with limited data. We only had a few videos given by the client. We used open-source data to train pose estimation models, and the client’s videos for error detection.

Solution: custom video pose estimation models for dumbbell weight workouts

To do human pose estimation in videos, our data scientists needed to analyze poses with computer vision. The most challenging part of the project for our tech team was to define the repetitions boundaries because human video pose estimation in sports is complex. If the person is doing the exercise the wrong way or their position is hard to define, estimating their posture and providing analytical insights might be difficult.

To define workout repetitions on video, we’ve built a custom RNN model. It enabled position estimation and error detection.

Repetitions Boundaries Detection:

The plots below represent predictions of the RNN model on two videos as well as the final results of repetitions detection obtained after applying several steps to RNN output. True repetitions boundaries are also given.

Then we moved on to developing a cloud-based solution where the client could upload their dumbbell workout videos in various formats.

Result: AI video analytics for safe dumbbell workouts

As a result, the client’s got bespoke AI software for video analytics. It’s fast, precise, and helps the user understand if their fitness routine is working or not. The solution provides AI and deep learning-powered video workout analytics. With AI-based video analytics, the user can analyze their workout mistakes and fix them.

The client’s benefited from the collaboration with InData Labs in the following ways:

• bespoke artificial intelligence-based video analytics
• pose tracking and error detection models for efficient movement analysis developed from scratch