RoadTest: Nordic Thingy:53 with Edge Impulse TinyML Solution
Author: aaryan2134
Creation date:
Evaluation Type: Development Boards & Tools
Did you receive all parts the manufacturer stated would be included in the package?: True
What other parts do you consider comparable to this product?: Syntiant TinyML Board, Arduino Nano 33 BLE Sense, Raspberry Pi Pico, ESP32
What were the biggest problems encountered?: No real problems as such. Worked great
Detailed Review:
Hi Readers!
This is my Roadtest Review for Nordic Thingy:53 with Edge Impulse TinyML Solution.
I am a 3rd year student at Delhi Technological University pursuing Computer Engineering. I have been building projects in IOT, Machine Learning, Web Development and love to take on new challenges.
Email: aaryan2134@gmail.com
GitHub - aaryan2134
The Nordic Thingy:53 is a prototyping platform for Matter, embedded machine learning, and wireless IoT products. It is based on the nRF5340 SoC and has integrated sensors for motion, sound, light, and environmental factors. The compact square shape and slim-profile design of the Thingy:53 feature a door for easy accessibility to the on/off switch and external connectors. The Arm Cortex-M33 processor application core ensures heavy computational tasks of embedded machine learning can be handled without affecting wireless connectivity. Wireless connectivity is handled separately by another Arm Cortex-M33 core. The Thingy:53 comes with a debug- and current-measurement board in the box for troubleshooting purposes.
| {gallery}Unboxing |
|---|
|
Box: The kit |
|
Unboxed: All contents |
|
IO: Input output ports and the debugger |
|
Top view: with debugger |
|
Taking it apart: Inside view |
| {gallery}The insides |
|---|
|
Board with battery: inside view |
|
Top view: The chip |
|
Angled view: The chip & battery |
The kit included:
The Thingy: 53 comes in a cool carton box with the Nordic Semiconductor colors. The Thingy:53 is built around the nRF5340 SoC, Nordic’s flagship dual-core wireless SoC. It is enclosed in a Red and Blue cool looking protective enclosure. The enclosure has an indicator LED for charging and for showing when the Thingy is on and connected to the Edge Impulse. It also has all the required IO enclosed with a cover - Power On switch, Type C port, Current and Debug Port, Programming Port and an external 4-pin JST connector for connecting to external board controllers.
Key features
Battery powered prototyping platform for Matter and machine learning on the nRF5340 SoC
Support for multiple wireless standards, Bluetooth LE, Bluetooth mesh, Thread, Zigbee
Environmental sensor for temperature, humidity, air quality and air pressure
Color and light sensor
Low-power accelerometer and 6-axis inertial measurement unit (IMU)
Buzzer and PDM microphone
Connector for additional external boards and accessories
USB-C rechargeable 1350 mAh Li-Po battery
nRF Edge Impulse mobile app for embedded machine learning
nRF Programmer mobile app for easily flashing firmware on the go
nRF5340 SoC
High performance 128 MHz Arm Cortex-M33 application core
Ultra-low power 64 MHz Arm Cortex-M33 network core
Multi-protocol radio with support for Bluetooth LE, Bluetooth mesh, Thread and Zigbee
nPM1100 PMIC
Highly efficient PMIC for better battery life
Full power path for seamless switching between charging and battery operation
nRF21540 FEM
RF front end for extended range and increased link robustness
nRF Edge Impulse machine learning app
The nRF Edge Impulse machine learning app is pre-installed on every Nordic Thingy:53, allowing for wireless transfer of training data via Bluetooth LE to the cloud through the app. The Edge Impulse Studio can be used to create an embedded machine learning model that can be wirelessly deployed to the Thingy:53 for inferencing. Results can be viewed through the app, making use of the advanced sensors of the Thingy:53 for voice recognition or movement pattern detection. The low-power accelerometer and PDM microphone can wake the SoC from sleep on motion or sound events, making it ideal for low-power embedded machine learning applications that can save power when not registering or reacting to anything.
nRF Programmer app
The nRF Programmer app for the Thingy:53 offers a new level of simplicity in a prototyping platform, enabling users to select from pre-made firmware and flash it directly over-the-air from an iOS or Android device. This feature allows for uploading new firmware to the nRF5340 SoC and utilizing the built-in sensor capabilities anywhere, without requiring a PC connection.
nRF Connect SDK
The Nordic Thingy:53 has full support in the nRF Connect SDK for programming custom firmware applications. This SDK is a scalable and unified software development kit for building products based on all the nRF52, nRF53 and nRF91 series wireless devices. It provides developers with an extensible framework for building firmware for devices and applications, from lightweight to computational heavy tasks, and integrates the Zephyr RTOS and a wide range of samples, application protocols, protocol stacks, libraries, and hardware drivers. The nRF Connect SDK offers a single code base for all devices and software components, simplifying the process of porting modules, libraries, and drivers from one application to another, and resulting in high memory efficiency.
It is pretty easy to get started using the Thingy 53. Let's see how we can try out the inbuilt Edge Impulse model which is present in all Thingy 53's by default. It is basically a motion detection model that tells about idle state, snake motion, up down and wave motion.
1. Download and install the nRF Edge Impulse App from Play Store or App store
2. Create an account or log in.
3. Create new Project -> Enter a name for your project
4. Turn on Bluetooth for searching your Thingy 53. Also turn on the Thingy 53. You will see a flashing blue and green light.
5. You will see "EdgeImpulse" in add devices. Just Click on that and your device will get connected. The Thingy 53 will show a constant Red Light.
6. Now time to start testing out the default model. Go to the Inferencing tab and click on start.
7. You will see "idle" if your Thingy 53 is just kept stable. Now try various motions like snake, up down or a wave motion. You will see the values changing on the app.
8. A blue led will flash every time it sends a response to the Edge Impulse App
9. Click on stop once you are done testing out
| {gallery}Interface |
|---|
|
Searching the device |
|
Device Added |
|
Device Information |
|
Sensor information |
|
Starting Inferencing |
|
Results |
We can also see all the details about the sensors once we get connected to the device by clicking on it.
That's how easy it is to use a model deployed on the Thingy 53!
Now, time to go ahead and dig deeper into the Thingy 53 and build our own model.
Getting started guide and the documentation makes it easy to try out various things. All these resources are linked down below at the end of the review blog.
I also tried the nRF Programmer App to change the firmware of the Thingy 53. I tried the nRF Machine Learning Firmware to test it out and moved back to Edge Impulse for building the model which I will show in the next section.
I have created a video trying out the nRF Machine Learning default model. This firmware can be used along with the nRF Connect SDK and Data Forwarder for making forwarding data from custom ML models as well.
More details about nRF Machine Learning can be found here. Also, to know more about the nRF Connect SDK go here.
So, I cloned this already publicly available project to test out the building process. This model basically detects "Hello, world", unknown sound and noise using the mic on the Thingy 53. It has 34 mins of collected data for training the model. It uses 80/20 split for dividing between training and testing. The most interesting part is the impulse design. We can select various models for preprocessing of the data like for keywords in Audio Files. Also, we can build a Keras based Neural Network. It has a cool looking, easy to use GUI for adding different layers. Along with running the model, we get to see the data results and accuracy of the model in a very visual manner.
The features for building ML models and optimizing them are pretty much endless in the edge impulse portal. From customizing various layers and model to tuning it and retraining the model. Also, it allows live testing of the created model through the portal itself. Best part of all if that we don't need to create any environment on our PCs to transfer the deployed model. We can straight away deploy it to the target device, the Thingy 53 in our case. It gets uploaded over bluetooth to the Thingy 53 using the Edge Impulse App.
Have a look on some images depicting the process.
Data Exploration:
Impulse Builder (ML Model):
Results & Parameter Tuning:
EON Tuner:
Testing:
Deployment:
The model real life performance wasn't that great as such. So, I guess we could try to collect some data(which is super easy using the Thingy 53) and also tune and retrain the model for better results. Also, I believe we need to handle some overfitting as the accuracy on the dataset was pretty high at 93%.
Have a look at this video to easy it yourself:
Here the main goal is to recognize a sound in an environment. This could be used to detect if you have any running faucets, TV, etc that you forgot to turn off. So, in this case I detected whether a kitchen faucet is running or not using the Thingy 53 as depicted in the tutorial.
So, I collected some of my data for this one. So, I left the Thingy 53 near my kitchen faucet. Then I used the edge impulse app to record samples for 1 min duration(which is the maximum allowed due to memory). To mimic real life scenarios, I recorded 1 minute samples of the faucet running under different circumstances -> silent, under fan noise, under someone cooking/walking in the kitchen, with the TV turned on in the other room. This allowed me to collect me around 6 minutes of training samples when the faucet is running.
After that I collected samples for when the faucet is not running under similar circumstances and collected another 6 minutes of data. This data collection process was a bit time consuming due to the upload time. Though it was pretty easy thanks to option of using Edge Impulse App and Thingy 53. I could just run the data collection process and get the data on Edge Impulse without doing any additional steps.
After that I just did some data exploration to have a look on the data. I didn't go too much into optimizing and the used the standard model ideas given in the tutorial. I was done in about 3-4 hours where majority of the time was still in the data collection process.
The results were pretty good at 99% accuracy on the first go itself. I tried some live testing and it was able to detect if the faucet is running or not every time.
The best part is the ease of use. It basically streamlines the process of building out your own ML models along with IOT. From data collection, preprocessing, building a model to evaluating and deploying doesn't require using anything complicated at all. Just use the edge impulse studio along with the Thingy 53 and you are done within a few hours with your own ML model. It is easy to use it to control something or send that data and makes it really cool.
The one thing I believe needs to be taken in care is that data collection is key in building a good machine learning model. Even though the thingy 53 makes it super to collect data (kind of makes you lazy enough to just not take care at this step) we must be careful and try our best to get the most real life situations covered in the dataset. Also, the dataset should be balanced according to the real life scenario. If you believe it is more likely that one condition will be satisfied more then have more training samples for that category. Otherwise equally dividing the training samples goes well.
Overall, the Thingy 53 along with its Edge Impulse capabilities is a great tool which makes it very easy to use and perfect tool for prototyping. It hardly took me much time to build a simple model and test out my approaches. This makes it great for applications like testing out your projects, for hobbyists, building a minimal viable product etc. Once you have the idea, the thingy 53 makes it pretty much straight forward to build and test it out in the real world.
Datasheet
nRF Connect SDK
nRF Programmer
nRF Edge Impulse
Edge Impulse
Downloads
Developing with Thingy:53
nRF Connect SDK Fundamentals Course
WEBINAR Say hello to Thingy:53 - The prototyping platform for embedded ML
