Group Actions
Engagement
  • Author Author: albertabeef
  • Date Created: Date Created
  • Views 3338 views
  • Likes 12 likes
  • Comments 10 comments
Related
Recommended

Applying AI to Climbing ... Part 3

albertabeef

In this blog series, I attempt to bring two of my passions together … AI and rock climbing.

image

In the first blog, I defined some use cases for a camera capable of being aware of climbing activity.

http://avnet.me/rock-climbing-ai-part1

In the second blog, I described the challenges and limitations of common pose estimation models used with this "odd" climbing use case.

http://avnet.me/rock-climbing-ai-part2

My first experiments struggled at simply detecting a human in these unusual conditions (hanging on wall, hanging from ceiling, ...).

At that time, I concluded that I would need to re-train my own model, but that would have to wait for another time ... 

This week, I saw several posts from Learn OpenCV, describing the pose estimation model included with the latest YOLOv7, and how it compared with the MediaPipe pose estimation model.

https://learnopencv.com/yolov7-object-detection-paper-explanation-and-inference/

https://learnopencv.com/yolov7-pose-vs-mediapipe-in-human-pose-estimation/

The YOLOv7 pose estimation model was doing a great job with unusual human poses, such as this example of a human riding a horse:

https://learnopencv.com/wp-content/uploads/2022/10/yolov7-vs-mediapipe-occlusion-example.mp4

image

I decided to re-try my experiments on some rock climbing footage.

The first experiment, which failed in the previous blog, generated very encouraging results.

Applying AI to Climbing - Experiment 01

Applying AI to Climbing - Experiment 01

The second experiment, once again a side-by-side comparison, also generated very encouraging results.

Applying AI to Climbing - Experiment 02

Applying AI to Climbing - Experiment 02

Although there are still some false pose estimations in the video, the general results are very impressive.

I also need to understand how to interpret the keypoints for the head for this use case where the human is almost always facing away:

Applying AI to Climbing - Experiment 03

Applying AI to Climbing - Experiment 03

This motivates me to continue on my journey to apply AI to climbing Slight smile


With the stationary video (background not moving), I was able to get a blank background (with some photoshop edits), and overlay the two climbers as stick men to get a comparison of the two climbers:

The mechanics of climbing - 5.11b Second début, Champlain

image

Doing this with a moving background will require video stabilization to be performed first.  Sounds like a great topic for the next blog ... 

Please share your feedback:

  • In your opinion, what has increased the accuracy of pose estimation in YOLOv7 ?
    • the model architecture ?
    • the dataset(s) used to train the model ?
  • Now that we can detect human pose in climbing video, what next ?

References:

Parents Comment Children