I've put the code for the project on Github as it's now starting to get interesting.
Testing on something simpler
So that I knew that OpenCV was working correctly I created a simple test script.
import numpy as np import cv2 print cv2.__version__
I also decided to do my testing and development on a Windows laptop with a lot more power than the Dragonboard. This will be particularly important when it comes to the number crunching of a new classifier.
I had a few false starts on getting OpenCV working on my laptop. I downloaded and installed the 3.1 version of OpenCV but then used the wrong version of the Python extensions, once I'd picked the right version to go with OpenCV3.1 and Python 2.7 (opencv_python-3.1.0-cp27-cp27m-win32.whl) things started working correctly. I also found that I needed to copy the classifier XML files into my project folder. I modified an example file so that it ran without creating a window.
# A simple test file for OpenCV
import numpy as np
import cv2
import datetime
def detect(img_color):
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
detected = 0
gray = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY)
# Now we find the faces in the image. If faces are found, it returns the positions of detected faces as Rect(x,y,w,h).
# Once we get these locations, we can create a ROI for the face and apply eye detection on this ROI (since eyes are always on the face !!! ).
faces = face_cascade.detectMultiScale(gray)
for (x, y, w, h) in faces:
cv2.rectangle(img_color, (x, y), (x + w, y + h), (255, 0, 0), 2)
roi_gray = gray[y:y + h, x:x + w]
roi_color = img_color[y:y + h, x:x + w]
eyes = eye_cascade.detectMultiScale(roi_gray)
if len(eyes) > 0:
detected = detected + 1
for (ex, ey, ew, eh) in eyes:
cv2.rectangle(roi_color, (ex, ey), (ex + ew, ey + eh), (0, 255, 0), 2)
print(str(detected) + " people")
return detected
img = cv2.imread('TestPicture.JPG')
if detect(img) > 0:
newFileName = "TestOutput" + datetime.datetime.now().isoformat().replace(":","") + ".jpg"
cv2.imwrite(newFileName, img);
The code reads a test file and then passes that to the detect function. The detect function creates two classifiers, one for faces and one for eyes based on the example XML provided with OpenCV. It grayscales the image for faster processing and then detects faces, for each face found it draws a rectangle and the checks for eyes. If it detects eyes on the face then it's a hit and we have found a person.
I tested my classifier with some astronauts and it detected three of them although interestingly spotted a face on one of the crumpled sleeves. On my laptop it takes about 1s to load and process the (615x425) pixel file.
The same script on the Dragonboard takes 0.6s to run.
Getting Webcam images
Capturing from the webcam with OpenCV is very simple. Most of the tutorials assume you want to display video output but these can be simplified to capture a single frame. In this example we capture a single frame from the camera and then save it to the disk. In the finished version that frame would be passed on the classifer.
import cv2
import datetime
cap = cv2.VideoCapture(0)
# Capture single frame
ret, frame = cap.read()
cap.release()
if ret:
newFileName = "CaptureOutput" + datetime.datetime.now().isoformat().replace(":", "") + ".jpg"
cv2.imwrite(newFileName, frame)
else:
print "Capture failed"
When I tested this with the dragonboard I received the following error:
VIDEOIO ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV
Capture failed
There seemed to be some anicdotal evidence to fix this but they did not seem to work for me so I swapped the script out with one that called the command line "streamer" app instead.
That also caused me some trouble with the parameters not getting passed to the streamer app correctly, an streamer in turn complaining that the format could not be determined. After some experimentation, the following approach worked and the file could be opened by OpenCV.
import cv2
import datetime
from subprocess import call
capture = "CaptureInput" + datetime.datetime.now().isoformat().replace(":", "") + ".jpeg"
cmdline = "streamer -c /dev/video0 -b 32 -f jpeg -o " + capture
call(cmdline, shell=True)
img = cv2.imread(capture)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
newFileName = "CaptureOutput" + datetime.datetime.now().isoformat().replace(":", "") + ".jpg"
cv2.imwrite(newFileName, gray)Training a classifier
I followed this tutorial to create my classifier xml data, althought I had to drop the featureType parameter as that caused it to crash on my system. The training application is very hungry for memory and used about 2.5GB on my system for image sizes of 50 x 50. It ran one of my CPUs at between 50% and 100%. When I repeated the test with a 100 x 100 image the memory usage shot upto 8GB, although it should be possible to control this with the buffer size settings. However, I reverted to the 50 x 50 and increased the number of cycles of training as that is apparently what gives the quality results rather than the size. The training programme does seem to crash rather than report sensible errors, it also crashed for me when I put in really large image sizes. After 30 minutes I had my first prototype classifier.
opencv_createsamples.exe -info Dragons.info -num 90 -w 50 -h 50 -vec Dragons.vec Info file name: Dragons.info Img file name: (NULL) Vec file name: Dragons.vec BG file name: (NULL) Num: 90 BG color: 0 BG threshold: 80 Invert: FALSE Max intensity deviation: 40 Max x angle: 1.1 Max y angle: 1.1 Max z angle: 0.5 Show samples: FALSE Width: 50 Height: 50 Create training samples from images collection... Done. Created 90 samples opencv_traincascade.exe" -data data -vec Dragons.vec -bg Negative.info -numPos 89 -numNeg 765 -numStages 5 -w 50 -h 50 PARAMETERS: cascadeDirName: data vecFileName: Dragons.vec bgFileName: Negative.info numPos: 89 numNeg: 765 numStages: 5 precalcValBufSize[Mb] : 1024 precalcIdxBufSize[Mb] : 1024 acceptanceRatioBreakValue : -1 stageType: BOOST featureType: HAAR sampleWidth: 50 sampleHeight: 50 boostType: GAB minHitRate: 0.995 maxFalseAlarmRate: 0.5 weightTrimRate: 0.95 maxDepth: 1 maxWeakCount: 100 mode: BASIC ===== TRAINING 0-stage ===== <BEGIN POS count : consumed 89 : 89 NEG count : acceptanceRatio 765 : 1 Precalculation time: 9.468 +----+---------+---------+ | N | HR | FA | +----+---------+---------+ | 1| 1| 1| +----+---------+---------+ | 2| 1| 1| +----+---------+---------+ | 3| 1| 1| +----+---------+---------+ | 4| 1| 0.330719| +----+---------+---------+ END> Training until now has taken 0 days 0 hours 12 minutes 54 seconds. ===== TRAINING 1-stage ===== <BEGIN POS count : consumed 89 : 89 NEG count : acceptanceRatio 765 : 0.407783 Precalculation time: 9.62 +----+---------+---------+ | N | HR | FA | +----+---------+---------+ | 1| 1| 1| +----+---------+---------+ | 2| 1| 1| +----+---------+---------+ | 3| 1| 0.426144| +----+---------+---------+ END> Training until now has taken 0 days 0 hours 22 minutes 45 seconds. ===== TRAINING 2-stage ===== <BEGIN POS count : consumed 89 : 89 NEG count : acceptanceRatio 765 : 0.26127 Precalculation time: 9.143 +----+---------+---------+ | N | HR | FA | +----+---------+---------+ | 1| 1| 1| +----+---------+---------+ | 2| 1| 1| +----+---------+---------+ | 3| 1| 0.488889| +----+---------+---------+ END> Training until now has taken 0 days 0 hours 32 minutes 18 seconds.
Testing
My initial testing provided too many false positives so I found a 1000 more negative images and added them to the training, I also set the maxFalseAlarmRate to a smaller value and set my training going again.
This time the training took a lot longer, nearly 15hrs later the classifier was trained and it worked a whole lot better than my first version.
I think the next task is to look at hooking some proper hardware to the Dragonboard using the level shifters.
Reference
Getting Started with Videos — OpenCV-Python Tutorials 1 documentation
Coding Robin Train Your Own OpenCV Haar Classifier
Top Comments