Smart Home Energy Management System - Blog Post #3 - Collecting audio data with Nicla Vision Board and Edge Impulse

I have won the Nicla Vision board from one of the Element14 events. I was excited since it is my first Arduino Pro's item, and I am getting started with boards that can use WiFi or Bluetooth. I really wanted to test the camera but at second thought using the microphone would be better. Three reasons:

• Power consumption
• Easily creating a model with higher accuracy even with more classes
• Privacy.

I checked on Arduino page the power consumption for the microphone and the camera. The datasheet mentions the power consumption for the microphone (which is very low) but no the power consumption for the camera. I just assume the camera has an higher power consumption since it has more processing to do. Also, since I am mounting the SHEMS inside my bathroom, I just believe not using the camera would give more privacy and trust in the system. Lastly, processing images from a camera require a lot of computational power and I might get a model with less accuracy than with using a model based on the microphone. Audio data seems easier to process than visual data.

Nicla Vision | Arduino Documentation

As I mention in blog #1, I followed the instructions on Edge impulse to connect to the platform so I can collect label data and create models all from their platform. I followed the following link:

Arduino Nicla Vision - Edge Impulse Documentation

For the Arduino CLI, I installed by error the latest version which was very unstable, and I couldn't connect with Edge impulse, so I had to download the 0.18.3 version which work. Still, while following instructions I had issues with the process but nothing that couldn't be fixed by just unplugging and then plugging back in the cable or by pressing the RESET bottom on the Arduino board.

Once connected, I followed the following tutorial to create a model to respond to my commands:

Responding to your voice - Edge Impulse Documentation

Apparently, they have already a dataset with few labels (like "yes", "no", noise, and unknown). This helped me because I could download the dataset and the upload noise and unknown. I am uploading labels data like "Hey Bath" (just to start the SHEMS), "Turn off 1", "Turn on 1", "Turn off 2", "Turn on 2", "Half 1", and "Half 2" to adjust the level of the RGB LED strips.

You can see the dashboard on Edge impulse. It is really cool because you can easily personalize each data entry and visualize it really easily and quickly.

Here is a sample of command of "Turn off 1". It is cool that I can immediately visualize the data and I can have an idea too. Right now, I am uploading more data, I want at least ten minutes of each command just for the training, then 2 minutes for the test dataset. It is taking a lot of time especially because many times it fails to download the data, so I continuosly lose many sessions. For now, I do sessions of 30 seconds where I say multiple times the command and then I split session with the many commands. So from a session of 30 seconds I can get 15 commands of 1 seconds. I do 30 seconds because increasing the time might increase the possibility of failure and I would be losing a lot of time. Also, it takes more time to download logner sessions.

For a future blog post, I will update you how my dataset is going and progress on the Arduino Nicla Vision board.