Bluetooth Unleashed : AAPSAD #5 : Audio and Visuals

Hello guys,

Before I start, apologies for not responding to your comments, although I've been going through them I couldn't reply  to you guys have been running on a tight schedule.

Previously in Bluetooth Unleashed....

With no clue on recognizing facial expressions our hero was struggling to find a good API and finally ended in way to detect smiles and proceeded further.....

Now on Bluetooth Unleashed

Still unhappy with results and working with APIs in Parallel, I found out Google Vision to be satisfactory.

They have a very good documentation and found this YouTube Video to be much easier.

What I've done here is:

1) I've used OpenCV and a python script to capture an image and save it locally as emo.jpg, and call another python script (the Vision API).

2) This one Upload to the GCloud and returns a json response from which emotions could be fetched.

capture.py

import cv2
import numpy as np
import sys
import time
import os
facePath = "haarcascade_frontalface_default.xml"
faceCascade = cv2.CascadeClassifier(facePath)
cap = cv2.VideoCapture(0)
cap.set(3,640)
cap.set(4,480)
sF = 1.05

while True:
  ret, frame = cap.read()
  img = frame
  gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
  faces = faceCascade.detectMultiScale(
    gray,
    scaleFactor= sF,
    minNeighbors=8,
    minSize=(55, 55),
    flags=cv2.CASCADE_SCALE_IMAGE
  )
  if (len(faces)):
     cv2.imwrite(filename = 'emo.jpg', img=frame)
    print('opening API')
    os.system('python emotion.py')
    time.sleep(10)
  c = cv2.waitKey(7) % 0x100
  if c == 27:
        break
cap.release()

This calls for the emotion.py which in turn return emotion values.

The next part is to play a soothing visual with a audio according. Will be using omxplayer to run them, I'm yet to find the proper visuals and audios. For now 3 music videos each emotions.

modified the emotion.py to play videos.

emotion.py

import io
import os

# Imports the Google Cloud client library
from google.cloud import vision
from google.cloud.vision import types

# Instantiates a client
client = vision.ImageAnnotatorClient()

# The name of the image file to annotate
file_name = 'emo.jpg'

# Loads the image into memory
with io.open(file_name, 'rb') as image_file:
    content = image_file.read()

image = types.Image(content=content)

# Performs label detection on the image file
response = client.face_detection(image=image)
faces = response.face_annotations
# 0-'UNKNOWN',1-'VERY_UNLIKELY',2-'UNLIKELY',3-'POSSIBLE',4-'LIKELY',5- 'VERY_LIKELY')
for face in faces:
    anger = face.anger_likelihood
    joy = face.joy_likelihood
    sorrow = face.sorrow_likelihood
#print(anger) print(joy) print(sorrow)
if (anger >= joy) and (anger >= sorrow):
   emo = 'a'
elif (joy >= anger) and (joy >= sorrow):
   emo = 'j'
else :
   emo = 's'

if emo == 'a' :
  os.system('omxplayer Anger.mp4')
  os.system('killall omxplayer.bin')
if emo == 'j' :
  os.system('omxplayer Joy.mp4')
  os.system('killall omxplayer.bin')
if emo == 's' :
  os.system('omxplayer Sorrow.mp4')
  os.system('killall omxplayer.bin')

I just ran the capture.py from putty and within 2seconds this came up on the Rpi screen

Here's a short clip on the output...

Edit media

Dimensions x Small Medium Large Custom

Subject (required) Brief Description Tags (separated by comma) Video visibility in search results Visible Hidden

Parent content

Bluetooth Unleashed : AAPSAD #5 : Audio and Visuals

Poster

Upload Preview

Ok

Cancel