Introduction

A couple of months ago (November 2019) I'v been asked by e14phil to test and troubleshoot using the Omron HVC on the Raspberry Pi 4 using their python scripts (https://www.element14.com/community/docs/DOC-93443#comment-239308 ). There were already amazing Similar posts from ralphjy  (Omron HVC-P2 Roadtest - Getting Started), as a follow up on https://www.element14.com/community/message/229828/l/looking-a-for-facial-recognition-hardware-tester#229828. Goal was to interact with the product using USB and the SDK in a Windows Operating system environment.

In my case Phil was looking for someone to take it the python route. The plan was to publish a blog before Christmas 2019, but in the mean time I was selected to roadtest the 3 Series MDO Mixed Domain Oscilloscope. I immediately emailed Phil to cancel the HVC trouble shooting project, but the equipment was already sent, and arrived at my home at the same date as the scope. So after working very hard to review the scope before the deadline of 29 January, I now want to fulfill my promise by writing a blog about the Omron HVC on the Raspberry Pi 4 project.

Hardware

Let's first have a look at the hardware. Ralf has a interesting post on all hardware versions for Omron HVC technology: Omron HVC-P2 Roadtest - OKAO Vision Technology.

Mine is the HVC-P2 Camera and processor on separate boards. Communicates over serial UART. Here is a picture. Raspberry Pi on top, HVC-P2 connected to an USB2 port.

As you can see the power consumption (measured with board in action) is not very much for the challenging task the board performs.

When the HVC-P2 is connected to an usb port, a serial device is created. Below you see the list of tty devices before and after the unit is connected.

The units serial port is /dev/ttyACM0.

Software

OMRON has Python sample code available at GitHub (https://github.com/omron-devhub/B5T-007001_HVC-P2_for-Python).

This code provides B5T-007001(HVC-P2) python API class and sample code using that API class.

Instructions on how to use this code are in the readme.txt file. The code is written in Python 2.7, the nowadays more common Python3 is not supported,

Luckily Python 2.7 is standard installed on Raspberry Pi's most recent Raspbian Buster distribution.

For running the code pySerial and Python Imaging Library(PIL)  are required.

So I started with the installation of pySerial and Pillow.

Next I cloned the code from GitHub:

And finally I started execution.py on the serial port with a very high speed (921600 bd) and using the STB library.

It works! mission accomplished!.

That was easy, less than an hour work. So I decided to play a bit more with the device, and make a simple graphical user interface on top of execution.py using PySimpleGUI.

I made a copy of execution.py to execution_gui.py and added a function to create a guide window:

def creategui():
    sg.ChangeLookAndFeel('BlueMono')

    frame_layout = [
                  [sg.T('Elapsed', size=(60, 1), key='-ELAPSED-')],
                  [sg.Multiline('', size=(60,12), autoscroll=True, key='-ML-')],
               ]
    
    # define the window layout
    layout = [[sg.Text('OMRON HVC P2 Demo GUI', size=(50, 1), justification='center', font='Helvetica 20')],
              [sg.Image(filename='', key='image'),
               sg.Frame('Result', frame_layout)],
              [sg.ReadButton('Exit', size=(10, 1), pad=((0, 0), 3), font='Helvetica 14'),
               sg.RButton('Pause', key='-RUN-PAUSE-', size=(10,1), font='Any 14')]]

    # create the window and show it without the plot
    window = sg.Window('OMRON HVC P2 Demo Application',
                       location=(400,200))
    #window.Layout(layout).Finalize()
    window.Layout(layout)

    return window

Next the main loop is extended with code to handle the 'exit' and 'run' buttons and to process the tracking results. For now the camera image is displayed and the detected faces are annotated with a box. The rest of the information (gaze, gender, expression etc.) are just shown in a text window.

The boxes are yellow when the gender is not known, blue for men and pink for woman.

pilim = img.getpilim()
                pilim = pilim.convert("RGB")

                window['-ELAPSED-'].Update('==== Elapsed time:{0} [msec] ===='.format(elapsed_time))
                window['-ML-'].Update(hvc_tracking_result)
                #print hvc_tracking_result
                if hvc_tracking_result.faces:
                    for f in (0, len(hvc_tracking_result.faces)-1):
                        pos_x = hvc_tracking_result.faces[f].pos_x/5
                        pos_y = hvc_tracking_result.faces[f].pos_y/5
                        size = hvc_tracking_result.faces[f].size/10
                        box = [pos_x-size,pos_y-size,pos_x+size,pos_y+size]
                        draw = ImageDraw.Draw(pilim)
                        genderstring = hvc_tracking_result.faces[f].gender
                        boxcolor = (255,255,0)
                        if 'Male' in genderstring.__str__():
                            boxcolor = (64,128,255)
                        if 'Female' in genderstring.__str__():
                            boxcolor = (255,64,255)
                        draw.rectangle(box, outline=boxcolor, width=2)

                bio = io.BytesIO()  # a binary memory resident stream
                pilim.save(bio, format= 'PNG')  # save image as png to it
                imgbytes = bio.getvalue()  # this can be used by OpenCV hopefully
                window.FindElement('image').Update(data=imgbytes)

In order to display the image, I needed a pointer to it, so therefore I added a function to greyscale_image.py:

def getpilim(self):
        w = self.width
        h = self.height

        # if no data, no save.
        if w == 0 or h == 0:
            return False

        img = Image.new("L", (w, h), 0)
        x = 0
        y = 0
        for y in range(h):
            for x in range(w):
                img.putpixel((x, y), ord(self.data[w * y + x]))
        
        return img

And here is the result:

I will add more functionality in the future.

You can find all my adjustments in a fork that I made from the B5T-007001_HVC-P2_for-Python repository.

This fork can be found here: https://github.com/AgriVision/B5T-007001_HVC-P2_for-Python .