I received my hardware 4 days ago, but haven't had the time or opportunity to use it. The MKR WAN 1310 box was crushed, but it seems the board was not damaged.
Live Classification
The first thing that I want to try with the Nicla Vision is live classification with the Edge Impulse model. This would allow me to verify that the microphone hardware and software are operating and that the model works with the microphone before I deploy it.
The Nicla Vision is fully supported by Edge Impulse, so this task should have been easy. I use Edge Impulse with other Arduino boards, so I already have the Edge Impulse CLI and the Arduino CLI installed and operating on my PC. Unfortunately, because I use a lot of different boards with these tools - there is always the possibility of running into version conflicts with sotware and firmware.
Install Edge Impulse firmware on Nicla Vision
Edge Impulse provides firmware that you can flash onto the Nicla Vision to provide access to all of its onboard sensors from your Edge Impulse project using the edge-impulse-daemon with the USB connection to the board.
Download the latest Edge Impulse firmware arduino-nicla-vision-firmware.zip and unzip the file. I am using Windows, so after connecting the Nicla Vision via USB, just need to run the flash_windows.bat script. Unfortunately, that failed.
Not sure why the flash script is poking through all the board libraries, but it ran into a problem in the ESP Arduino library with a Heltec board that I don't even use (neither does Edge Impulse). There was some corruption in that library - but even after I reinstalled it and eliminated the error - I could not get the firmware flashed. It turns out that I am using version 0.19.0 of the Arduino CLI and it wants 0.18.x. Funny, because the latest version is actually 0.30.0. Rather than ruminate on that, I just loaded an archived version 0.18.3 and that worked.
Edge Impulse CLI
The next step is to run the edge-impulse-daemon (part of the Edge Impulse CLI) and connect to your Edge Impulse project in the cloud. Unfortunately, that didn't work because the Nicla Vision is a newer board and apparently the CLI version I was using (v1.13.12) was out of date.
Updated CLI to v1.17.1 and then was able to connect to my Bee_Present project.
There is an alternate method to connect sensors which is to load an ingestion program onto the Nicla Vision. I also tried that with the microphone program nicla_vision_ingestion_mic.ino and it worked, but the the disadvantage of this approach is that you don't get access to all of the sensors in a single program.
Now on the Data acquisition window on the project I have access to the sensors:
Likewise I have those options on the Live Classification window:
And I'm ready to try it:
Audio Source
I decided to use a USB speaker to play audio files from my computer. I printed a spacer clip to position the Nicla at a fixed distance so that I could get some measurement repeatability.
I
Now, what to use as a data source? I wanted to use audio captured from a honey bee hive and I found some YouTube videos by Frederick Dunn and decided to try those.
Here is a classification of a 5 second video segment where it classified 7 bees. I have verified that there are segments where it predominantly classifies noise. Obviously, for this video it does not find any crickets. I am going to have to see if I can stitch together some audio test files where I have specific expectations as I will have to admit that I can't discriminate specifically what it is classifying in this video. It would be easier with solitary bees, but the training data is from hives. Unfortunately, I don't have the unsegmented data.
I'm satisfied that the hardware is operating correctly, so the next step is to deploy the model as an Arduino library and try inferencing on the Nicla Vision board.