In this blog series, I attempt to bring two of my passions together … AI and rock climbing.
In the first blog, I defined some use cases for a camera capable of being aware of climbing activity.
http://avnet.me/rock-climbing-ai-part1
In the second blog, I described the challenges and limitations of common pose estimation models used with this "odd" climbing use case.
http://avnet.me/rock-climbing-ai-part2
My first experiments struggled at simply detecting a human in these unusual conditions (hanging on wall, hanging from ceiling, ...).
At that time, I concluded that I would need to re-train my own model, but that would have to wait for another time ...
This week, I saw several posts from Learn OpenCV, describing the pose estimation model included with the latest YOLOv7, and how it compared with the MediaPipe pose estimation model.
https://learnopencv.com/yolov7-object-detection-paper-explanation-and-inference/
https://learnopencv.com/yolov7-pose-vs-mediapipe-in-human-pose-estimation/
The YOLOv7 pose estimation model was doing a great job with unusual human poses, such as this example of a human riding a horse:
https://learnopencv.com/wp-content/uploads/2022/10/yolov7-vs-mediapipe-occlusion-example.mp4
I decided to re-try my experiments on some rock climbing footage.
The first experiment, which failed in the previous blog, generated very encouraging results.
Applying AI to Climbing - Experiment 01
The second experiment, once again a side-by-side comparison, also generated very encouraging results.
Applying AI to Climbing - Experiment 02
Although there are still some false pose estimations in the video, the general results are very impressive.
I also need to understand how to interpret the keypoints for the head for this use case where the human is almost always facing away:
Applying AI to Climbing - Experiment 03
This motivates me to continue on my journey to apply AI to climbing 
With the stationary video (background not moving), I was able to get a blank background (with some photoshop edits), and overlay the two climbers as stick men to get a comparison of the two climbers:
The mechanics of climbing - 5.11b Second début, Champlain
Doing this with a moving background will require video stabilization to be performed first. Sounds like a great topic for the next blog ...
Please share your feedback:
- In your opinion, what has increased the accuracy of pose estimation in YOLOv7 ?
- the model architecture ?
- the dataset(s) used to train the model ?
- Now that we can detect human pose in climbing video, what next ?
References:
- YOLOv7 Object Detection Paper Explanation and Inference : https://learnopencv.com/yolov7-object-detection-paper-explanation-and-inference/
- YOLOv7 Pose versus MediaPipe in Human Pose Estimation : https://learnopencv.com/yolov7-pose-vs-mediapipe-in-human-pose-estimation/
