Group Actions
Engagement
  • Author Author: fmilburn
  • Date Created: Date Created
  • Views 1600 views
  • Likes 9 likes
  • Comments 5 comments
Related
Recommended

Picasso Adapting Art to Viewers: OpenCV Face Detection, Blog #2

fmilburn

Art is often made to appeal to a specific set of viewers.  This is the second blog in a series exploring the idea of using AI to recognize a viewer and adapt art to their age or interest.  The intent is to encourage early art appreciation while being attractive to all household members.

 

image

In this post the loading of software and recognition of faces using OpenCV on the Raspberry Pi 3B+ and Raspberry Pi Camera V2 are discussed.  A subjective comparison to the Raspberry Pi 3B and original Raspberry Pi Camera is made.  Shortcomings in face recognition on the Raspberry Pi to date and possible ways to overcome are discussed.  A video is presented which demonstrates face recognition and plans for next steps are made.

 

Requirements

image

Raspberry Pi 3B+

Raspberry Pi Camera V2

5.1V, 2500 mA power supply

SD Card with Rasbian

 

Raspberry Pi 3B+ and Raspberry Pi Camera V2

image

Thanks to Element14 for providing the new Raspberry Pi 3B+, Kit, and Camera for this challenge.  The RPi Zero W (left) was not tried but it would probably struggle to manage OpenCV for this application.  I have successfully used a Raspberry Pi 3B with the original camera (middle in the photo above).  But the new Camera V2 on right gives better color balance in mixed lighting and for my use is a substantial upgrade over the original.  And the 1400 MHz clock in the RPi 3B+ improves over the 1200 MHz RPi 3B for facial recognition where update frame rate speed is important.  So the current plan is to use the RPi 3B to display and control art while the RPi 3B+ does facial recognition.

 

I am using the Debian Stretch OS with Raspberry Pi Desktop November 2018 release, Kernel version 4.9.  Help on getting started with the Raspberry Pi can be found here.

 

Loading OpenCV Face Detection on the Raspberry Pi 3B+

 

Sharib Hasan gives an excellent step by step description of Raspberry Pi Facial Recognition  and how to load it here on Element14 which I will not repeat.  One small difference is that I use the following simple Python script which takes 5 photos and then select the one(s) I want to use.  The selected photos are then placed into their individually named folders in the dataset folder.

from picamera import PiCamera
from time import sleep
camera = PiCamera()
camera.resolution = (800, 600)
# camera.rotation = 180
for i in range(5):
 camera.start_preview()
 sleep(3)
 camera.capture('/home/pi/Desktop/image%s.jpg' % i)
 camera.stop_preview()
 sleep(0.1)

 

If your photos are upside down, uncomment the line "camera.rotation = 180".  I am still playing with camera resolution and 800 x 600 is probably higher than needed.  Higher resolution photos slow down the face learning part of OpenCV.

 

Raspberry Pi 3B+ OpenCV Face Detection Performance

To start, a limited number of photos has been used to train the facial recognition AI.  The subjects are myself and a doll my granddaughter plays with when she is at the house.  The doll is being used since there are no grandchildren at hand to serve as  unwilling subjects. The photos below are representative and there are 3 of each subject. A photo of one of my granddaughters was also placed  in the dataset (not shown below) to see if it would misidentify her.

image

The following video gives an idea of frame rate and errors encountered thus far with facial recognition using OpenCV on the Raspberry Pi 3B+.  Keep a lookout for where it identifies the ice cream cone as me.

The frame update rate is acceptable for what I want to but there is room for improvement on identification.  The problems are

  • misidentification (The doll is not Margot, and the ice cream cone is not me)
  • unknown identification (not so much a problem)

 

It seems to recognize me fairly quickly when facing head on.  This isn't too surprising since all the dataset photos are head on.  It is slower to pick up the doll and misidentifies her as my granddaughter.  This is more difficult to explain.  It is really hard to explain how it mistakes me for a cone head.  This is the preliminary list of things to explore for improving identification:

  • Take photos and set up the art exhibit in better lighting
  • Place some photos with faces less head on into the training dataset
  • Find optimal photo resolution for training OpenCV (large photos are slow to train and don't seem to give better results)
  • Experiment with background to see if it makes a difference
  • Use actual people instead of the doll whose head may be anatomically incorrect - or perhaps the underlying algorithm does not recognize babies

 

Next Steps

Investigations on how to improve facial image recognition will continue.  Failing improvement, it may be just as entertaining to watch it misidentify people image.

The second Raspberry Pi can now be connected and different displays and / or actions be taken when an identification is made.

 

image

Other Posts in this Series

 

Picasso Adapting Art to Viewers: Cabinet Design, Blog #3

PiCasso Design Challenge:  Adapting Art to Ages Blog #1

 

Useful Links

Raspberry Pi Facial Recognition by Sharib Hasan on Element 14

Getting Started on Raspberry Pi by Raspberry Pi Organization

  • in reply to genebren

    Hi Gene,

     

    That is definitely the part I am most concerned with. My Linux / RPi skills are basic and OpenCV is brand new to me.

  • Frank,

     

    Nice update on your project.  Good to see that you are jumping into the Pi side of things and getting your feet wet.  That is something that I should get started on sooner than later.

     

    I look forward to hearing more on your project and your progress.

     

    Gene

  • Frank,

     

    You seem to be making good progress, it is only a matter of time and training before it will be (almost) correctly recognising you and the doll.

     

    I have used a web based face recognition demonstration system before which was amazing good so I'm sure you'll get it going. Either that or you are a Cone Head!

     

    Dubbie

  • in reply to shabaz

    Hi Shabaz,

     

    Thanks for the encouragement!  I am very much a beginner with OpenCV so comments are greatly appreciated.  You picked up on it but I should have clarified in the writeup that the reason I started with a small training dataset and am looking for a lower limit to photo resolution is that interpreted Python on the RPi is pretty slow in this application.  Doing more than a handful of training photos at the start was taking too much time and keeping me from just getting something working.  I need to figure out how to compile the training portion or somehow load it faster for the final version.

     

    That is a good idea to do voting on incoming data!  Great idea.  I have it on the list to do an online search on how others are handling it but haven't really dug into it yet.  An initial search combining "OpenCV"  "facial recognition" and "Raspberry Pi" turns up quite a few tutorials and such.  I have MATLAB and thought I would look into what could be done with that also.

  • Hi Frank,

     

    This is great to see, you're getting good results!!

    Very cool that you did this with the Python API. I've never tried this with OpenCV, I'm a beginner with it, and had an easier task to just identify humans (not individuals). I used very low-res images, so that did help speed things up as you suggest. In your case there's a limit of course, since you still need face area detail, and it's a wide-angle lens.

    I guess if the training content is dozens or hundreds of images then it will provide even more accuracy, and also perhaps for the intended application, a voting system could be used, e.g. it waits for the results from a dozen frames from the video, and then adds up the results, to get a percentage probability that it was you or the doll, and then acts if the result is (say) 70% likely, otherwise reattempts from the next dozen frames.

    One other idea could be to double-check how other sources online may have implemented such recognition in OpenCV, because some people may have implemented differently (e.g. data-reducing or filtering or other manipulation procedures in a different order). I had to try different combinations to get something working enough to get moving forward.

    Anyway, it's really great that the system is in principle already working and now just needs tweaks.