RoadTest: Nordic Thingy:53 with Edge Impulse TinyML Solution
Author: tmarcin
Creation date:
Evaluation Type: Development Boards & Tools
Did you receive all parts the manufacturer stated would be included in the package?: True
What other parts do you consider comparable to this product?: SensorTile Box - STEVAL-MKSBOX1V1, TINY Machine Learning KIT with Arduino Nano 33 BLE Sense
What were the biggest problems encountered?: lack of helpful hints if errors occur when building a machine learning model; breaking bluetooth connection after record new data (Android app); no battery charge information
Detailed Review:
I was very lucky :) when I was selected to write a review of the Nordic Thingy:53 module, which is a modern IoT platform. I will not give technical details of this device, because the Nordic:Thingy:53 parameters can be found on the website:
https://www.nordicsemi.com/Products/Development-hardware/Nordic-Thingy-53
The module contains several sensors, is aesthetically made, has a simple and nice housing. Due to miniaturization, the ON-OFF switch is also small, which is subjectively quite inconvenient, because quite often you have to use it to resume Bluetooth communication.
I started getting to know the module by watching the presentation "Say hello to Thingy:53 - The prototyping platform for embedded ML" https://webinars.nordicsemi.com/say-hello-to-thingy53-the (as a side note, it must be said that the resolution of the video is sometimes too low and the font on the presented smartphone is hardly visible.) As shown in the video, I installed two programs nRF Programmer and nRF Edge Impulse.
For testing, I used a Samsung Galaxy S7 Edge smartphone and an iPad. Please note that the nRF Edge Impulse app for each of these devices is slightly different: Version 1.0.0 for Android, Version 1.0.4 for iOS.
In the case of iOS, it is more convenient to set the data acquisition time (you can enter the desired value, e.g. 1500 seconds). In the case of Android, the time (given in milliseconds) must be increased/decreased using the +/- buttons, which is very time consuming. For this reason, I rather used the iPad, although the iPad software does not allow (?) to create new projects.
Of course, I also set up an account at https://studio.edgeimpulse.com/
The general concept of system elements and communication can be shown schematically
Below I present the preparation of two projects, the first concerns a classification of illuminance and the second an assessment of bicycle paths based on data from the accelerometer and the lighting sensor.
Because I'm interested in lighting sensors, I decided to test the use of the color sensor first. The platform includes the BH1749 sensor. Information about the use of this sensor can be found at https://infocenter.nordicsemi.com/index.jsp?topic=%2Fug_thingy53%2FUG%2Fthingy53%2Fintro%2Ffrontpage.html
BH1749 senses Red, Green, Blue (RGB) and Infrared and converts them to digital values.
I started by creating a new project. I did it on the computer on the website https://studio.edgeimpulse.com/studio/profile/projects The name of the project, of course, appeared in the nRF Edge application.
During the acquisition of reference data, I compared the measured values with the BH1750 sensor connected to the Arduino UNO with a display. I prepared data for 3 classes: dark (sensor covered), indoors, outdoors.
The acquisition went quite smoothly, the data appeared on my edgeimpulse account. I thought it was possible to train the network right away directly in the mobile application,
but I had to go back to the edgeimpulse website. The website allows you to view the data and trim them if necessary. Below are illustrations of the data for each of the 3 classes.
Now it's time to move on to training. The next steps are precisely defined, and the user has the ability to specify the method of machine learning. You can say that the support is semi-automatic, but it definitely simplifies the process of preparing the model.
I chose RAW data (input R,G and I channels of BH1749) and classification.
In the screenshot (on the right), we can see that the data is easily separable, so we can proceed to the process of training a simple network, which of course can be changed.
For the default number of epochs = 30, the confusion matrix is not very satisfactory
I increased the number of epochs to 100 and the result has already improved a bit, now the model is sometimes wrong for two classes (indoor and outdoor).
After changing input data to RGB only
I received 100 % accuracy :)
Sometimes problems may occur during training, and a bit of machine learning programming knowledge is useful :)
I didn't go further into performance analysis as that's not our goal in this review. In the next steps, you can test off-line data from a file or data downloaded from the device.
Finally, we can also install the model on the device itself and observe the classification.
To sum up: The above screenshots show that the process of data acquisition and model preparation is quite easy, although obtaining high efficiency requires the selection of good parameters. The entire project can be found at https://studio.edgeimpulse.com/public/201416/latest
Ultimately, I would like to use the Nordic Thingy:53 platform to test the quality of bicycle paths, especially in the evening. Using a combination of the measurement from the color sensor (light intensity) and the accelerometer, we can assess whether the path is safe or whether it is better to go a little slower.
I bought a special phone holder for the Nordic Thingy:53 to mount it to the bike handlebar.
As part of the preliminary tests, I have recorded 4 classes:
Create Impulse:
Feature explorer:
Model training results:
The confusion matrix indicates that 3 classes are correct. In the case of the "good-dark" class, it is assigned to "poor-dark", maybe it means that it is better not to ride a bike in the dark? :)
I believe that EDGE IMPULSE studio in combination with Nordic Thingy:53 is an interesting tool to further experiment and analyze such tasks.
The Nordic Thingy:53 is fantastic tool for fast ML experiments. Wide range of sensors gives the possibility of quick design of ML classifiers.
