RoadTest: Nordic Thingy:53 with Edge Impulse TinyML Solution
Author: embeddedguy
Creation date:
Evaluation Type: Development Boards & Tools
Did you receive all parts the manufacturer stated would be included in the package?: True
What other parts do you consider comparable to this product?: Some Bluetooth device such as ESP32 based boards with onboard sensors which can do ML tasks.
What were the biggest problems encountered?: The Thingy:53 is excellent piece of hardware. There are sensors and connectivity options such as BLE, NFC, Zigbee etc. In comparison, software is somewhat time taking and needs good knowledge about embedded systems and programming. Flashing the board is little different. There is 10-pin Jtag connector which is recommended to program for applications which are not based on nRF connect SDK i.e. Zephyr RTOS applications. Though, I have tried to flash Zephyr applications with programmer app as well and it works fine.
Detailed Review:
First, I would like to start this road test by saying thanks to Element14 community and Nordic Semiconductor for giving me this opportunity to test their product.
Nordic semiconductor is a company from Norway which offers wireless communication products for IoT and industrial application as well as software for these devices.
They offer devices in the form of SoC(System on Chips) which supports Bluetooth, Cellular, WiFi and other well known wireless communication protocols such as NFC, Zigbee, Thread, matter etc.
There are also some propriety protocols designed by Nordic semi.
This roadtest is specifically about their development platform in Thingy series. Which contains Thingy:52, Thingy:53 and Thingy:91 devices. The number represents SoC which the device is based on.
Thingy:53 is based on nRF5340 SoC which is low-power dual core micro controller with Bluetooth, Thread and Zigbee support. nRF5340 SoC has an Application core and a Network core.
The Application core is ARMCortex-M33 with 128 and 64 MhZ operation. It has 1MB Flash and 512kb of RAM memory and various other features. Similarly the network core has ARM-Cortex-M33 processor with 64MhZ operation.
The net core has 256 kb flash and 64 kb of low leakage RAM. Both of these cores have several other features which are like AoA(Angle of Arrival) and AoD(Angle of Departure) features for direction finding with Bluetooth, ARM trust zone support for differentiation between secure and not secure parts of the memory. There are various other mentioned features in the datasheet, however, this document is not about mentioning features of nRF5340 SoC. So in the next paragraph I will write about Thingy:53 device itself.
Thingy:53 is an excellent piece of hardware. It comes with wealth of onboard sensors and comes with nRF5340 dual core processor which supports various wireless communication protocols. It has very nice design and comes in plastic casing with battery and buzzer.
It has various peripherals pins exposed on the outside for user. These pins are used for uploading the binary, supply power and switch for the device to turn on/off.
To program the Thingy:53 Nordic provides various options. One of the official supported SDK is nRF connect SDK by Nordic. It is open-source software suite for Nordic device series such as nRF52, nRF53 and nRF91 series devices. Though there are some parts which are not open source such as some low level radio API.
Other option is to use Zephyr RTOS to write code for Nordic devices. Zephyr has support for various mcu vendor development boards including Nordic devices. I can say it that Nordic devices are quite good supported by Zephyr SDK.
The third and last option I know is to use some open-source Os's like Apache Mynewt to program the device.
It would be a nice idea in the beginning to have a look at the schematic provided by Nordic for Thingy:53. Thingy:53 schematic has some of the important information for further development using the nRF connect SDK or Zephyr based RTOS API's. It will give us details about how the sensors are connected. For example it can give us information about which communication interface a given sensor uses. What are the other features such as regarding power management and charging circuit and last but not least about nRF21540 range extension chip and how antenna is connected.
The following schematic contains the RF front-end module circuit. It has two different Antenna selections for A1 and A2. Nordic Getting started and developing with Thingy:53 guide suggests the following and I Quote
ANT1 is connected to the nRF5340 through the nRF21540 RF FEM and supports TX gain of up to +20 dBm.
ANT2 is connected to the nRF5340 through the RF switch and supports TX output power of up to +3 dBm.
The +20 dBm TX output power can increase the range of communication by 6 to 10 times the normal range. nRF21540's +20 dBm Tx power increase and +13dBm RX gain ensures superior link budget.
The SoC's RF output connects to the nRF21540's RF input which then can be tuned as per requirement. nRF21540 has more than one antenna outputs to support more than one selection. It offers selection of these configuration using interfaces such as SPI, GPIO etc. The device consumes +110mA for TX gain of +20dBm and down to only 45nA in power down mode. It has +39mA current for TX gain of +10dB and +2.9mA for RX.
The gain selection for TX and RX is dynamic and can be changed during runtime or using Kconfig option CONFIG_MPSL_FEM_NRF21540_TX_GAIN_DB
option.
In schematic PE4259 is RF Switch which can select between ANT1 or ANT2. To use ANT2 disable
CONFIG_MPSL_FEM
Kconfig option in network core's build options.
Thingy:53 has wealth of sensor including microphone, accelerometer, magnetometer, buzzer and gas sensors, These sensors are useful for developing machine learning applications really fast. The data from these sensors can be used to train a machine learning model which latter can be flash on to the board. The Thingy:53 is supported by edge impulse and ML models can be trained and deployed really fast.
Let’s first look at the sensors and their schematic diagrams. The first one is BME688 sensor from Bosch Sensortech. The sensor is first of a kind that can measure Outdoor gas along with temperature, humidity and pressure. It supports I2C and SPI communication protocols. The sensor is AI capable and BME studio can be used to customise the sensor for particular application. The Gas sensor can detect Volatile Organic Compounds(VOCs), Volatile Sulphur Compounds(VSCs) and other gases such as carbon monoxide and hydrogen in parts per billion(ppb) range.
One of the interesting thing while reading the sensor datasheet that I found was that the sensor element should be/can be heated to different temperature ranges to detect the various gases surrounding the sensor environment. There are programmable registers to set this temperature ranges. Moreover, the sensor can be used to get the IAQ index for surrounding air, which is the standard measure for the quality of the air in the surroundings.
One more interesting sensor that the Thingy:53 is Bosch BMM150 Magnetometer. The BMM150 can provide magnetic field information in all three axes. It can be used for real-time heading related information. For example can be used to calibrate the Gyroscope or provide the direction to the user on the map navigation.
The sensor has interrupts which can be used to detect preset threshold of particular magnetic axes or if the sensor has new data. This sensor also supports both I2C and SPI interface as BMM688. This is exiting as such a tiny dev board contains so much interface options as well.
I explained two sensors above but how can a board be without a Accelerometer and Gyroscope sensor and have Magnetometer? So here is a Bosch BMI270 sensor with Accelerometer and Gyroscope. The sensor supports I2C and SPI interfaces. One of the exiting thing about this sensor is that it has 2KB of On-chip FIFO buffer for Accelerometer, Gyroscope and timestamp as well as Aux sensor data.
BH1749NUC is a digital color sensor which can detect Red, Blue, Green and Infrared and converts it to digital values. It makes it possible to obtain accurate illuminance and color temprature of ambient light. It operates on I2C bus interface.
The VM3011 is high performance low noise MEMS microphone. It features Vespers’s Adaptive ZeroPower Listening which keeps the microphone in hibernate mode when a background noise is below certain level and only turns on the system when certain threshold is passed for noise. It uses I2C interface for communication with microcontroller. It is really a nice sensor for making ML projects such as wake word detection etc.
ADXL362 is ultra low-power digital MEMS accelerometer. It consumes less than 2uAms at 100Hz output rate which can run it on a small coin cell battery. I think that it is included in Thingy:53 to make application to wake up on motion detection and save power. The sensor can operate in the range of +/- 2g, 4g and 8g ranges.
Zephyr is open source Rtos(Real-time operating system) with small foot print kernel for running on tiny resource contained devices such as microcontrollers and embedded systems. Some people call it “Linux for microcontrolers”, also because it is backed by Linux foundation.
Now, nRF Connect SDK is a software suite from Nordic Semi. For their devices such as nRF52, nRF53 and nRF91 series of devices. The nRF connect SDK runs Zephyr as a software module. So think of it as a part of nRF Connect SDK but the SDK has some more features in addition to Zephyr. I am still new to nRF Connect SDK and hence can not mention more about it, I just have compiled tiny application and flashed it successful till now.
But for Zepher in this road test I will mention how to install it and flash an application to the Thingy:53. The process to install Zephyr following steps.
Setup command-line Zephyr development platform for specific OS
Get the source code for the devices
Build, flash and run the application
As I am using Linux these instructions are for Linux OS
sudo apt update
sudo apt upgrade
Download and install the Kitware apt repository to your sources list.
wget https://apt.kitware.com/kitware-archive.sh sudo bash kitware-archive.sh
Install the required dependencies using apt
sudo apt install --no-install-recommends git cmake ninja-build gperf ccache dfu-util device-tree-compiler wget python3-dev python3-pip python3-setuptools python3-tk python3-wheel xz-utils file make gcc gcc-multilib g++-multilib libsdl2-dev libmagic1
Check that correct versions are installed using the following commands. For example for Cmake and other packages
cmake -–version python3 -–version dtc –version Next part is to install the Zephyr SDK in a virtual environment. Virtual environment is recommendation instead of global installation because it saves us from python dependancy and incompatibility issues with other installed software in our computer. So following are the instrutions for the same. sudo apt install python3-venv python3 -m venv ~/zephyrproject/.venv source ~/zephyrproject/.venv/bin/activate install the west. pip install west Get the Zephyr source code west init ~/zephyrproject cd ~/zephyrproject west update export the Zephyr Cmake package. This allows Cmake to automatically load boilerplate code for building Zephyr application. west zephyr-export There are additional python dependency and requirements.txt has them all. So install them using pip. pip install -r ~/zephyrproject/zephyr/scripts/requirements.txt
Now it is time to actually install the Zephyr SDK source code. The Zephyr SDK contains the required tool-chains for all the supported boards by Zephyr. It also contains Compilers, Linkers, assembler and other programs to build Zephyr applications.
cd ~
wget -O - https://github.com/zephyrproject-rtos/sdk-ng/releases/download/v0.16.0/sha256.sum | shasum --check –ignore-missing
for RaspberryPi replace the architecture x86-64 with aarch64
Now extract the archive:
tar xvf zephyr-sdk-0.16.0_linux-x86_64.tar.xz
Run the SDK bundle setup script:
cd zephyr-sdk-0.16.0
./setup.sh
Uninstall the udev rules which allow you to flash Zephyr boards as regular user
sudo cp ~/zephyr-sdk-0.16.0/sysroots/x86_64-pokysdk-linux/usr/share/openocd/contrib/60-openocd.rules /etc/udev/rules.d
sudo udevadm control –reload
Now the installation process is finished and we can build and run a simple application such as Blinky or Hello world. To check the installation of Zephyr you can use any supported board.
cd ~/zephyrproject/zephyr
west build -p always -b <your-board-name> samples/basic/blinky
Then use west to flash the board
west flash
The LED should blink on your specific board. The -p always and -p auto option is to do pristine build which means when building a sample Zephyr will rebuild everything related to that particular sample and board files.
Nordic recommends flashing the board at first after installing the required software such as nRF connect SDK or even Zephyr RTOS. The idea is that if the flashing of the board works then one is sure that things will be smooth after build process and hardware and flashing software is working fine.
There are different methods to flash the Thingy:53
Flash using the on-board usb connector
Flash using the external 10-pin jtag connector
Flash using Bluetooth Low Energy protocol based application
In this roadtest we will see them one by one.
To flash the device using the usb cable one needs to connect the device to development PC via usb-c cable. After that press the switch SW2 and hold it. While SW2 is pressed turn the SW1 to ON position from OFF.
This process will enumerate the device on the pc. After that open the nRF connect programmer from the nRF connect for desktop.
In the programmers app select the device from the top left corner which will list the device you have connected. After that click on Add file which will have option to browse to the folder where pre-compilled applications are stored.
Select any one the pre-compilled firmware in a .zip format. The nRF Connect Programmer will automatically select the right firmware for application and network core while uploading. Make sure that Enable MCUBoot option is selected. The app will show the Write option available to flash the application. Click on Write and a dialog will show up with some information about MCUBoot process. Again, click Write and device will start to flash.
The next option is to flash the Thingy:53 using the the 10-pin jtag connector. The side cover of the Thingy:53 has 10-pin jtag slot. This can be used to program the device via external jtag. The development kit from Nordic such as nRF5340-dk can work as external jtag device. It has 10-pin connector which can be used to program the Thingy:53 from the DK.
To do that first make sure that SW1 on Thingy:53 is OFF. Then connect the 10-pin connector from the DK to Thingy:53 as shown in the below image. Once done that, connect the DK to the PC via usb cable and turn the Thingy:53 to ON position. This will allow us to program the device via DK.
For actual flashing one can use either Zephy’s west flash command or use the nRF connect programmer application. Choose the right firmware and flash on the board. For nRF5340 SoC based boards such as Thingy:53 one needs to flash Application core firmware and Network core firmware separately.
The command to flash the application core and network core firmware as as follows respectively.
west build -p always -b nrf5340dk_nrf5340_cpuapp samples/sensor/bme280/
west build -p always -b nrf5340dk_nrf5340_cpunet samples/sensor/bme280/
The third option and one of the interesting feature that Thingy:53 has to flash is via BLE communication. There is an app called nRF Programmer in Android play store which can be used to do the flash. Thingy:53 comes with a firmware which directly allows to update new Firmware via this application.
As we saw in previous section how to install Zephyr RTOS now it is time to actually run an application on the board to see how the real application is built, flash and run on the Thingy:53. Though the first sample is normally hello world or blink and led one but in this write up I will explain how to flash an application which fetch BME688 Gas, Temperature sensor data from the board.
The BME688 is connected to the I2C interface of the board. So in order to enable and use I2C sensor we need to look at how the hardware is actually wired. For example which pins SCL and SDA lines of I2C sensor is connected.
In Zephyr this is done through a file called .dtc. These files describe how hardware is wired. For every board there is a folder called boards/<arch>/board folder. This contais .dts and .dtsi files with all the settings described for supported peripherals and GPIOs.
For example following is a code for Thingy:53’s board.dts file which describes the I2C1 peripheral. On the left hand side there is a property and a value of a particular property on the right hand side. For I2C1 the status is “okey” means that it is working as normal. If it is “disabled” means we may not use it as normal or we have to enable it in the Application’s .overlay files.
&i2c1 {To override the default configuration create a .overlay file in your applications folder boards/<boardname.overlay> file. So for example following is the file in /boards directory of my application for Thingy:53. So create thingy53_nrf5340_cppuapp.overlay file in your boards directory and paste the following content. If for example the BME688 is connected to other bus than 0x76 such as 0x77, the .overlay file can override the basic configuration. The final version of the .dts file will be in the build folder of application where Zephyr.dts file is stored.
In project’s prj.conf file add the following lines to enable the sensor interface.
Now, it is time to add some code in the main.c file of the application to fetch sensor values. To do that add the following code inside a main loop of the application.
void main(void)
{
int ret;
const struct device *bme_1 = device_get_binding("BME688_1");
/* Variables for raw and float data */
struct sensor_value temp_raw, press_raw, humidity_raw, gas_res_raw;
double temp, press, humidity, gas_res;
printf("Device %p name is %s\r\n", bme_1, bme_1->name);
while(1){
k_sleep(K_SECONDS(1));
sensor_sample_fetch(bme_1);
sensor_channel_get(bme_1, SENSOR_CHAN_AMBIENT_TEMP, &temp_raw);
sensor_channel_get(bme_1, SENSOR_CHAN_PRESS, &press_raw);
sensor_channel_get(bme_1, SENSOR_CHAN_HUMIDITY, &humidity_raw);
sensor_channel_get(bme_1, SENSOR_CHAN_GAS_RES, &gas_res_raw);
temp = sensor_value_to_double(&temp_raw);
press = sensor_value_to_double(&press_raw);
humidity = sensor_value_to_double(&humidity_raw);
gas_res = sensor_value_to_double(&gas_res_raw);
printf("BME688_1 T: %f; P: %f; H: %f; G: %f\r\n", temp, press, humidity, gas_res);
}
}
To build the project and flash to application core and network core use the following commands.
Great.!
Now the application is flashed to the board and if you are able to open the serial terminal you will be able to see the sensor values for temperature, pressure, humidity and gas.
To open serial terminal use the following commands for your serial terminal
If the above steps are followed carefully you will see the output on the console as in the picture below.
It will be a good idea to look the final .dts file of the build/zephyr/zephyr.dts file. It will be a long list of different peripheral settings but for I2C1 it will look as following.
The device driver development is one of the interesting tasks one can do with the Zephyr. When I started to work on Thingy:53, I realized that some drivers are not yet implemented or some of them are not fully complete. For example, the driver for the color sensor BH1749 is not available yet in Zephyr and the Driver for BME688 is not available for VOC and gas sensor measurements. Though, there are some implementations available on the internet but yet not integrated properly, hence, we could not guarantee the performance and how it works. But, I thought of creating one by myself for the Rohm semiconductors color sensor IC available on Thingy:53. It could be that nRF connect SDK version of Zephyr has this driver already.
So here in this document, I will mention how to develop a driver for sensors based on i2c communication such as BH1749.
For the first step, it is required to create a folder inside zephyr/drivers/sensor. This folder is necessary to have all the source files for the sensor communication and required settings files such as Kconfig and devicetree related files.
Inside the sensor driver folder I have a file called bh1749.c with various C functions for communicating with the sensor through i2c interface API of Zephyr. bh1749.h header file for defining various variables i.e. register address, configuration data structures etc. Zephyr has a generic driver model means there are standard functions for interface to the sensor and there are defined channels over which sensors can send data. For this example, there are four channels already defined in Zephyr API for RGB color values and infrared sensor color values.
The sensor API defines various functions and macros required to communicate with the device. Let’s look at it one by one.
bh1749_init() is to initialize the sensor via i2c bus and send all the default configurations to fetch sensor values via i2c interface.
int bh1749_init(const struct device *dev){
const struct bh1749_config *cfg = dev->config;
//struct bh1749_data *data = dev->data;
uint8_t id;
if(!device_is_ready(cfg->i2c.bus)){
LOG_ERR("Device is not ready!");
return -ENODEV;
}
/*check the chip-id */
if(i2c_reg_read_byte_dt(&cfg->i2c, CHIP_ID_VAL, &id) < 0){
LOG_ERR("Wrong chip detected");
return -EIO;
}
if(i2c_reg_write_byte_dt(&cfg->i2c, SYS_CONTROL, 0x0D) < 0){
LOG_ERR("Error writing SYS control");
return -EIO;
}
/*keep sensor in highest gain and measurement time*/
if(i2c_reg_write_byte_dt(&cfg->i2c, MODE_CNTL1, 0x2A) < 0){
LOG_ERR("Error wtinting control reg1");
return -EIO;
}
if(i2c_reg_write_byte_dt(&cfg->i2c, MODE_CTRL2, 0x10) < 0){
LOG_ERR("Error wtinting control reg2");
return -EIO;
}
if(i2c_reg_write_byte_dt(&cfg->i2c, INTERRUPT, 0x00) < 0){
LOG_ERR("Error wtinting control Interrupt reg");
return -EIO;
}
if(i2c_reg_write_byte_dt(&cfg->i2c, PERSISTANCE, 0x01) < 0){
LOG_ERR("Error wtinting Persistance reg");
return -EIO;
}
/* Write thresold values */
return 0;
}
bh1749_sample_get() is the function required for the sensor for fetching the RGB color sensor values. Similarly bh1749_channel_get() is for converting the fetched sensor values to required channel and storing the value in bh1749_data structure fields. These are all mentioned data structures and functions required for sensor driver development.
static int bh1749_sample_fetch(const struct device *dev, enum sensor_channel chan)
{
struct bh1749_data *data = dev->data;
const struct bh1749_config *cfg = dev->config;
uint16_t buf[3];
uint16_t buff[2];
uint16_t th[2];
if(i2c_burst_read_dt(&cfg->i2c, RED_DATAL, (uint8_t *)buf, 6) < 0){
LOG_ERR("Failed to read data sample");
return -EIO;
}
data->red = sys_be16_to_cpu(buf[0]);
data->green = sys_be16_to_cpu(buf[1]);
data->blue = sys_be16_to_cpu(buf[2]);
if(i2c_burst_read_dt(&cfg->i2c, IR_DATAL, (uint8_t *)buff, 4) < 0){
LOG_ERR("Failed to read data sample");
return -EIO;
}
if(i2c_burst_read_dt(&cfg->i2c, TH_HIGHL, (uint8_t *)th, 4) < 0){
LOG_ERR("Failed to read data sample");
return -EIO;
}
data->ir = sys_be16_to_cpu(buff[0]);
data->green2 = sys_be16_to_cpu(buff[1]);
data->th_high = sys_be16_to_cpu(th[0]);
data->th_low = sys_be16_to_cpu(th[1]);
return 0;
}
static int bh1749_channel_get(const struct device *dev, enum sensor_channel chan, struct sensor_value *val)
{
struct bh1749_data *data = dev->data;
switch(chan) {
case SENSOR_CHAN_RED:
val->val1 = (int)data->red * 0.0125;
val->val2 = (int)data->red % 80;
break;
case SENSOR_CHAN_BLUE:
val->val1 = (int)data->blue * 0.0125;
val->val2 = (int)data->blue % 80;
break;
case SENSOR_CHAN_GREEN:
val->val1 = (int)data->green * 0.0125;
val->val2 = (int)data->green % 80;
break;
case SENSOR_CHAN_IR:
val->val1 = (int)data->ir * 0.0125;
val->val2 = (int)data->ir % 80;
break;
default:
break;
}
return 0;
}
Apart from just the definition of functions, the Zephyr driver development requires a few macros to be defined as follows.
The DT_DRV_COMPAT macro is required to define the device as defined in the compatible property of dts/bindings/sensor/rohm_bh1749.yaml file. Then registering the module with macro LOG_MODULE_REGISTER() follows. After that, there is another #define which needs to be as your source file defines various functions related to a particular sensor.
#define DT_DRV_COMPAT rohm_bh1749
LOG_MODULE_REGISTER(BH1749, CONFIG_SENSOR_LOG_LEVEL);
#define BH1749_DEFINE(inst) \
static struct bh1749_data bh1749_data_##inst; \
\
static const struct bh1749_config bh1749_config_##inst = { \
.i2c = I2C_DT_SPEC_INST_GET(inst), \
}; \
\
SENSOR_DEVICE_DT_INST_DEFINE(inst, bh1749_init, NULL, \
&bh1749_data_##inst, &bh1749_config_##inst, \
POST_KERNEL, CONFIG_SENSOR_INIT_PRIORITY, \
&bh1749_driver_api); \
DT_INST_FOREACH_STATUS_OKAY(BH1749_DEFINE)
There is a Kconfig file that has all the Kconfig settings mentioned that can be enabled or disabled while writing an application. For this sensor, it will be CONFIG_BH1749 enabled by default.
config BH1749
bool "BH1749 with i2c interface"
default y
depends on GPIO
select I2C
help
Enable the color sensor bh1749
The last file inside the folder is the CmakeLists.txt file which instructs the build system to include the source file for the build.
zephyr_library()
zephyr_library_sources(bh1749.c)
For the full details of how to develop the driver look into the attached .zip file and also refer to the link to PR for integrating the driver on GitHub.
TBD
The Thingy:53 comes with exiting features. Though from my perspective it has exciting hardware and on-board sensors and related features, for example, to measure the current and to debug the on-board software. But on the software side, also there are many features and there is an API for every supported protocol and different sensor there is still some lack of documentation and features such as i.e. Bluetooth LE notification not working etc. A new user may find it difficult do deal with configuration files and .dts, .dtsi and .overlay files which describe hardware.
Overall for this road test it was really nice experience but indeed it was somewhat time-consuming compared to other dev. Boards.
Top Comments
Thanks for publishing the review.