Blog List:
1. Bee Monitoring: Introduction
2. Bee Monitoring: System Design & Workflow
3. Bee Monitoring: Setting up Nicla Vision
4. Bee Monitoring: Training the Model & Inferencing
5. Bee Monitoring: Bee Hive Environmental Monitoring using Arduino MKR 1310
Introduction:
In this blog, I will carry forward from the last blog and work on completing the Bee Detection and counting model. I will also deploy the model on the Nicla Vision for offline inferencing. Other than that, I will work on and complete the bee hive sound model. Let's get started!
Bee Detection & Counting Model:
Impulse Design:
I used this post, to learn more about detecting and counting objects using Edge Impulse. It uses the FOMO model to do the task. FOMO (Faster Objects, More Objects) is a brand-new approach to run object detection models on constrained devices. FOMO is a ground-breaking algorithm that brings real-time object detection, tracking and counting to microcontrollers for the first time. FOMO is 30x faster than MobileNet SSD and can run in <200K of RAM.
I designed an impulse which detects the objects(bees) and provides the count as the output.
Generating features:
Next, I used the generate features function and the parameters to let edge impulse reduce the dimensions of the dataset.
Training:
Next part was the training of the model. The first training results were just better than 53% which wasn't too great. I tried it in the live classification and the results were all over the place.
Improved model:
For making some improvements and to make the model usable, I did some hyperparameter tuning. The results got satisfactorily better though I believe there is scope to make it better with both, more data and more tuning.
Test the model:
Considering the small dataset of only 24 images, it was expected that the results would not be great. I was not surprised by the poor results.
Offline Inferencing:
Once I design, train, and verify my impulse, I can easily deploy it to my Arduino Nicla Vision with Edge Impulse. This allows my model to run locally, without an internet connection, minimizing latency and power consumption. The entire impulse, including the signal processing code, neural network weights, and classification code, can be packaged into a single firmware that I can run directly on the Nicla Vision.
Finally done with this part!
Classifying sound around the bee hive:
For this part, I uploaded 6 sound files. I used a 50-50 split between training and testing for this case.
The 3 types of sounds were:
1. Active Day
2. Missing Queen
3. Queen Present
Impulse Design:
The process of impulse design was similar though the model was changed to Audio MFE and a classifier for the neural network. The ideas were taken from this post.
Feature Generation:
This was mostly about dimensionality reduction and pre processing data for finding patterns. This makes it easier for the neural network to clean and converge well.
Training:
The training results were pretty good. I kept a low learning rate to prevent overfitting and added some random noise to make it generalize better.
Testing & Evaluation:
This model produced pretty decent results though it struggled to identify the Missing Queen audio. This could be improved with more data and some more hyperparameter tuning. Though adding more data would increase the training times significantly so I couldn't do that here.
Live Classification also worked okayishly well. I run a few audio files that were not provided to the model and let them run through my PC's speaker and turned on live classification. The results were good mostly though the Missing Queen problem was still present in this scenario and no audio file was recognized as Missing Queen.
As previously, I built the offline build for this model as well and updated the firmware for the Nicla Vision.
Conclusion:
So, we are finally done with 2 models - one detecting and counting bees in images or video and the other classifying sounds around the bees to detect if the queen bee is there, they are having an active day or the queen is missing. Both models were tested and deployed to the Nicla Vision.
Though I don't have a real bee hive near me so I won't be able to test it out in the real life scenarios. But I tried to test those models out by running them at my PC and using the camera and inbuilt microphone to get the results. It was a fun good project to build. It needs some fine tuning for proper real time use though looks good enough for testing out in the real world.
Now, I will the conclude this blog series with building the Bee Environment monitoring system with Arduino MKR WAN 1310. Hoping to see you in the next blog!
Connect with Me:
Github - aaryan2134