Posture detection using OpenCV and Raspberry Pi HQ Camera kit

<<<<Previous Blog     Next Blog>>>>

Posture detection using OpenCV and Raspberry Pi HQ Camera Kit

Overview of the project

This pandemic has made most of us work from home. While we may not have an ergonomic chair to sit for long enough at home, we may need to sit in a straight posture (without like slouching) such that it does not affect our body's posture or our health in the long term.

So, in this project, we'll use Raspberry Pi HQ Kit and Computer Vision (CV) to monitor the posture of the person sitting on a chair and alert when the person's posture is bad and not as per standards (refer image below).

Source: Sitting positions: Posture and back health (medicalnewstoday.com)

This way we can prevent some of the below detrimental effects on our body.

• Back, neck and shoulder pain
• Poor circulation
• Impaired lung function
• Poor digestion
• Constricted Nerves
• Misaligned Spine
• A curvature of the spine

Source: 7 Detrimental Effects of Bad Posture | BackFit Health + Spine

Unboxing of the kit

Thanks to  tariq.ahmad  and Element14 community for selecting my proposal and providing me with a Raspberry Pi HQ Camera kit. I was pleased to see my proposal selected for the additional kits than it was originally planned for the 'Photography' Project14 contest with an extended deadline (July 28, 2021).

On July 16, 2021, I got a call from UPS delivery person, that the kit parcel has arrived at my office address (which I used before for getting Element14 parcels), unfortunately, I was in a different location (updated in my profile) by then due to the pandemic situation here. So, there was a week's delay in receiving it and I got it on July 24, 2021. Hence couldn't make it to the contest deadline July 28, 2021, as planned. However, as proposed, I would like to complete and submit this project.

Let's now see what we got in the Raspberry Pi HQ Camera kit in the video below,

Edit media

Dimensions x Small Medium Large Custom

Subject (required) Brief Description Tags (separated by comma) Video visibility in search results Visible Hidden

Parent content

Posture detection using OpenCV and Raspberry Pi HQ Camera kit

Poster

Upload Preview

Ok

Cancel

Getting started

The contents of the Raspberry Pi HQ Camera kit are,

1. Raspberry Pi 4

Let's first get started with Raspberry Pi 4. For this you'll need,

* These are optional if you wish to control your Raspberry Pi through your personal computer or laptop via VNC (Virtual Network Computing) - Raspberry Pi Documentation

Step 1: Flash the SD card with OS (Refer Setting up your Raspberry Pi - Set up your SD card | Raspberry Pi Projects)

Step 2: Mount the SD card in its slot and connect the power supply.

Step 3: Turn on the power supply and that's it! you have your Pi up and running.

People at Raspberry Pi have done a wonderful job with the getting started documentation which you can find here - Setting up your Raspberry Pi - Introduction | Raspberry Pi Projects

2. Raspberry Pi HQ Camera

Now let's jump onto the Raspberry Pi HQ Camera and Telephoto Lens. Again, people at Raspberry Pi have documented clearly the steps required to get started,

Step 1: Remove the dust caps from both HQ Camera and lens.

Step 2: Turn off your Pi and connect the HQ camera to Raspberry Pi using the ribbon connector.

Step 3: Fit the 16mm C-mount lens onto the C-CS adapter that comes with the HQ Camera (Refer HQ Cam Lens Diagrams (raspberrypi.org))

Step 4: Turn on your Pi and enable Camera. For this go to Raspbian Menu -> Preferences -> Interfaces. Select the radio button below enable and next to camera and OK. Reboot your Pi.

Step 5: Enter the below line in the Pi's terminal. A preview will open, and you should be able to view an image of object/ scene the camera and lens is pointed to. After 5 seconds, preview will close, and an image will be saved to your Home folder with name 'test.jpg'.

raspistill -o test.jpg

Voila! you now have setup Raspberry Pi & HQ Camera. Let me know in the comments section if you face any trouble following the steps or while getting started.

Source/ Reference:

     1. Official Guide - hq-camera-getting-started.pdf (raspberrypi.org)

     2. Get started with the High Quality Camera — The MagPi magazine (raspberrypi.org)

3. Raspberry Pi 4 and accessories

Here are some photos of Raspberry Pi 4 and accessories - Heat sink, SD Card, Micro HDMI to standard HDMI cable and Power supply.

4. Raspberry Pi 4 Case

Here are some photos of the case for Raspberry Pi 4 with fan which I bought and an animation of fitting Pi into the case.

OpenCV - Posture detection

Initially I was waiting for the Raspberry Pi HQ Camera kit to arrive. So, without wasting any time, I started exploring OpenCV in my laptop (with Linux OS) and this is how it went...

OpenCV has been in my to-do list for a long time. I am happy that finally I got a nice excuse to start learning and making projects using OpenCV. From OpenCV's Website we can understand that (About - OpenCV),

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library.

In this project, we are going to use OpenCV as the brains to analyze the photos/ videos that Raspberry Pi HQ Camera kit would capture and estimate the posture of the person who is sitting. Then we'll alert the person if required to correct his posture to prevent any long-term detrimental effects on his/ her body.

There is a project called OpenPose which will help to detect key points of body. Then based on the position of key points we can identify whether ideal sitting posture is followed or not. In this blog, we'll cover up to key point detection and the remaining will be covered in upcoming blog posts.

1. Installation of OpenCV in Linux - I had followed instructions in the following link to install OpenCV in Linux OpenCV: Installation in Linux

I faced an issue while importing OpenCV library (command given below) even after following the required steps in the web page mentioned above,

import cv2 as cv

This was resolved by using the below command in terminal. Thanks to the following post - python - Cannot find module cv2 when using OpenCV - Stack Overflow

pip install opencv-python

After installing OpenCV in Linux I ran a basic project given in OpenCV website - OpenCV: Getting Started with Images. It ran as expected, so now let's move on to the next step.

2. Selecting a Model - After searching for a while, I landed on this beautifully written article on human pose estimation - Deep Learning based Human Pose Estimation using OpenCV (learnopencv.com). I could see two data sets that can be used for pose estimation or key point detection. One is MPII and another is COCO model. But based on the key points generated I can see COCO model covers more facial details like ear and eye whereas MPII detects only as head (and other body key points as well). So, I picked up and proceeded with the COCO model - COCO - Common Objects in Context (cocodataset.org). Since it can help to better estimate the position of head like titled or straight.

3. Model Weights - We'll need to now get model weights which will be used in the OpenPose project. To do the same, first you need the project from here - openpose/models at master · CMU-Perceptual-Computing-Lab/openpose · GitHub. Then you can run the below commands in terminal.

sudo chmod a+x getModels.sh 
./getModels.sh

Or it can be downloaded from here as well - http://posefs1.perception.cs.cmu.edu/OpenPose/models/pose/coco/pose_iter_440000.caffemodel

4. OpenPose Project - This is the last step, for this you need project from here - learnopencv/OpenPose at master · spmallick/learnopencv · GitHub. Place the weights file generated or downloaded from previous step inside 'pose/coco' folder. You can then place the image/ video that needs to be processed and key points identified into the project home directory. I used Visual Studio Code to run the 'OpenPoseImage.py' or 'OpenPoseVideo.py' files. A screenshot of the same is given below. On successful run, output skeleton and output keypoints detection files will be placed in the same project home directory.

First, I was using images and videos available online or captured using my mobile phone to test the OpenPose project. And then the Raspberry Pi HQ Camera kit arrived which was then used to capture images and video to provide input to the OpenPose project.

Below are the images and video of the camera kit in action. I made a case out of foam board to temporarily support the camera module and to hold the lens.

Output Skeleton

Output KeyPoints

Video processed using OpenPose project - Output Skeleton

Edit media

Dimensions x Small Medium Large Custom

Subject (required) Brief Description Tags (separated by comma) Video visibility in search results Visible Hidden

Parent content

Posture detection using OpenCV and Raspberry Pi HQ Camera kit

Poster

Upload Preview

Ok

Cancel

Source/ Reference:

     1. OpenCV: OpenCV Tutorials

     2. Deep Learning based Human Pose Estimation using OpenCV (learnopencv.com)

     3. learnopencv/OpenPose at master · spmallick/learnopencv · GitHub

Next Steps

Next is to use the output from OpenPose project to estimate the posture of the person sitting and alert him/ her to correct his/ her posture to avoid any long-term effects on his/ her body.

<<<<Previous Blog     Next Blog>>>>