Picasso Adapting Art to Viewers: OpenCV Face Detection, Blog #2

Art is often made to appeal to a specific set of viewers.  This is the second blog in a series exploring the idea of using AI to recognize a viewer and adapt art to their age or interest.  The intent is to encourage early art appreciation while being attractive to all household members.

In this post the loading of software and recognition of faces using OpenCV on the Raspberry Pi 3B+ and Raspberry Pi Camera V2 are discussed.  A subjective comparison to the Raspberry Pi 3B and original Raspberry Pi Camera is made.  Shortcomings in face recognition on the Raspberry Pi to date and possible ways to overcome are discussed.  A video is presented which demonstrates face recognition and plans for next steps are made.

Requirements

Raspberry Pi 3B+

Raspberry Pi Camera V2

5.1V, 2500 mA power supply

SD Card with Rasbian

Raspberry Pi 3B+ and Raspberry Pi Camera V2

Thanks to Element14 for providing the new Raspberry Pi 3B+, Kit, and Camera for this challenge.  The RPi Zero W (left) was not tried but it would probably struggle to manage OpenCV for this application.  I have successfully used a Raspberry Pi 3B with the original camera (middle in the photo above).  But the new Camera V2 on right gives better color balance in mixed lighting and for my use is a substantial upgrade over the original.  And the 1400 MHz clock in the RPi 3B+ improves over the 1200 MHz RPi 3B for facial recognition where update frame rate speed is important.  So the current plan is to use the RPi 3B to display and control art while the RPi 3B+ does facial recognition.

I am using the Debian Stretch OS with Raspberry Pi Desktop November 2018 release, Kernel version 4.9.  Help on getting started with the Raspberry Pi can be found here.

Loading OpenCV Face Detection on the Raspberry Pi 3B+

Sharib Hasan gives an excellent step by step description of Raspberry Pi Facial Recognition  and how to load it here on Element14 which I will not repeat.  One small difference is that I use the following simple Python script which takes 5 photos and then select the one(s) I want to use.  The selected photos are then placed into their individually named folders in the dataset folder.

from picamera import PiCamera
from time import sleep
camera = PiCamera()
camera.resolution = (800, 600)
# camera.rotation = 180
for i in range(5):
 camera.start_preview()
 sleep(3)
 camera.capture('/home/pi/Desktop/image%s.jpg' % i)
 camera.stop_preview()
 sleep(0.1)

If your photos are upside down, uncomment the line "camera.rotation = 180".  I am still playing with camera resolution and 800 x 600 is probably higher than needed.  Higher resolution photos slow down the face learning part of OpenCV.

Raspberry Pi 3B+ OpenCV Face Detection Performance

To start, a limited number of photos has been used to train the facial recognition AI.  The subjects are myself and a doll my granddaughter plays with when she is at the house.  The doll is being used since there are no grandchildren at hand to serve as  unwilling subjects. The photos below are representative and there are 3 of each subject. A photo of one of my granddaughters was also placed  in the dataset (not shown below) to see if it would misidentify her.

The following video gives an idea of frame rate and errors encountered thus far with facial recognition using OpenCV on the Raspberry Pi 3B+.  Keep a lookout for where it identifies the ice cream cone as me.

Edit media

Dimensions x Small Medium Large Custom

Subject (required) Brief Description Tags (separated by comma) Video visibility in search results Visible Hidden

Parent content

Picasso Adapting Art to Viewers: OpenCV Face Detection, Blog #2

Poster

Upload Preview

Ok

Cancel

The frame update rate is acceptable for what I want to but there is room for improvement on identification.  The problems are

• misidentification (The doll is not Margot, and the ice cream cone is not me)
• unknown identification (not so much a problem)

It seems to recognize me fairly quickly when facing head on.  This isn't too surprising since all the dataset photos are head on.  It is slower to pick up the doll and misidentifies her as my granddaughter.  This is more difficult to explain.  It is really hard to explain how it mistakes me for a cone head.  This is the preliminary list of things to explore for improving identification:

• Take photos and set up the art exhibit in better lighting
• Place some photos with faces less head on into the training dataset
• Find optimal photo resolution for training OpenCV (large photos are slow to train and don't seem to give better results)
• Experiment with background to see if it makes a difference
• Use actual people instead of the doll whose head may be anatomically incorrect - or perhaps the underlying algorithm does not recognize babies

Next Steps

Investigations on how to improve facial image recognition will continue.  Failing improvement, it may be just as entertaining to watch it misidentify people .

The second Raspberry Pi can now be connected and different displays and / or actions be taken when an identification is made.

Other Posts in this Series

Picasso Adapting Art to Viewers: Cabinet Design, Blog #3

PiCasso Design Challenge:  Adapting Art to Ages Blog #1

Useful Links

Raspberry Pi Facial Recognition by Sharib Hasan on Element 14

Getting Started on Raspberry Pi by Raspberry Pi Organization