Building Edge Impulse Library
After I’ve built the model to classify posture in my previous blog post Trakcore #2 AI Assisted Posture Modification - Acquiring Data and Designing a Model the next step was to build an Arduino library. It is very simple process on Edge Impulse. I've selected default options to enable Edge Impulse EON
compiler (same accuracy, up to 55% less memory and 35% less flash use according to the vendor) and TensorFlow Lite Quantized (int8), 8-bit quantization approximates floating point values, which is the optimization recommended for best performance.
This build process generated a compressed zip file ei-trakcore-arduino-1.0.1.zip (3.54MB). This library can run on any Arm-based Arduino enough RAM, including Arduino Nano 33 IoT.
It includes Edge Impulse DSP and Inferencing SDK, source code for the library, Arduino examples and other metadata.
Adding Edge Impulse Library to Arduino IDE and adding sketch example
Than I've added the library to Arduino IDE. Once library was added I was able to use provided example on integrating the model under Files > Examples > Your project – Edge Impulse > static_buffer
Getting sample
Adding raw feature data to sketch
I've extracted raw data sample from Edge Impulse Studio -> Live Classification -> Raw Data -> Raw Features and added them to my Arduino sketch.
Compiling
It took a lot of time for me to sort out compiling for Arduino Nano 33 IoT as it was failed initially.
So I've looked at Edge Impulse forum.
There are several posts about compilation/build issues:
https://forum.edgeimpulse.com/t/filename-or-extension-too-long-arduino-nano-33-iot/843
But than I found a good document that helped me to resolve all these issues https://docs.edgeimpulse.com/docs/running-your-impulse-arduino#code-compiling-fails-under-windows-os
One of steps that helped me resolve compilation problems was to download and use of Arduino CLI Windows 32 Nightly Build instead of Arduino IDE for compilation/build/deployment.
There was additional step to add SAMD platform ( The Atmel
SMART SAM D21 is a series of low-power microcontrollers using the 32-bit ARM Cortex-M0+ processor):
And then I run build
Build failed again. It was complaining on permissions. I've than I run the same command as admin and it was completed without any errors.
Deployment
Deployment process was quite smooth and simple.
Testing
I've connected Arduino IDE serial monitor to MCU and start getting empty output instead of float values.
I was not the first who got this issue and found workaround on Edge Impulse forum https://forum.edgeimpulse.com/t/arduino-33-iot-no-inference/950
After I followed instruction on the forum it was working as expected. But inference was quite slow and taking 398 ms in DSP.
Again using the same forum post instruction, I've updated my code and it start working twice faster:
Here is the code for my example sketch, which is using my model:
/* Edge Impulse Arduino examples
* Copyright (c) 2021 EdgeImpulse Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
/* Includes ---------------------------------------------------------------- */
#include <cstdarg>
#define EIDSP_USE_CMSIS_DSP 1
#define EIDSP_LOAD_CMSIS_DSP_SOURCES 1
#define __STATIC_FORCEINLINE __attribute__((always_inline)) static inline
#define __SSAT(ARG1, ARG2) \
__extension__ \
({ \
int32_t __RES, __ARG1 = (ARG1); \
__ASM volatile ("ssat %0, %1, %2" : "=r" (__RES) : "I" (ARG2), "r" (__ARG1) : "cc" ); \
__RES; \
})
#include <trakcore_inference.h>
static const float features[] = {
// copy raw features here (for example from the 'Live classification' page)
// see https://docs.edgeimpulse.com/docs/running-your-impulse-arduino
-0.0471, -6.7973, -7.0936, -0.0419, -6.7974, -7.0844, -0.0516, -6.7751, -7.0975, -0.0489, -6.7710, -7.0559, -0.0414, -6.7769, -7.0562, -0.0223, -6.7814, -7.0813, -0.0319, -6.7882, -7.0806, -0.0295, -6.7852, -7.0659, -0.0304, -6.8054, -7.0666, -0.0600, -6.7974, -7.0752, -0.0723, -6.7929, -7.0792, -0.0765, -6.7822, -7.0720, -0.0946, -6.7962, -7.0491, -0.0802, -6.7831, -7.0415, -0.0462, -6.7763, -7.0484, 0.0120, -6.7647, -7.0809, 0.0175, -6.7602, -7.1066, -0.0212, -6.7614, -7.1274, -0.0943, -6.7787, -7.1112, -0.1148, -6.7897, -7.1012, -0.1146, -6.7849, -7.1017, -0.1284, -6.7895, -7.0505, -0.1134, -6.8031, -7.0325, -0.0881, -6.8099, -7.0499, -0.0926, -6.7829, -7.0734, -0.1040, -6.7708, -7.0916, -0.1001, -6.7675, -7.0852, -0.0774, -6.7835, -7.0686, -0.0569, -6.7844, -7.0801, -0.0289, -6.7841, -7.0906, -0.0338, -6.7780, -7.1064, -0.0295, -6.7793, -7.0934, -0.0055, -6.7829, -7.0891, 0.0055, -6.8075, -7.0415, 0.0165, -6.8084, -7.0259, 0.0368, -6.7992, -7.0482, 0.0429, -6.8058, -7.0472, 0.0453, -6.7876, -7.0604, 0.0391, -6.8013, -7.0527, 0.0331, -6.7992, -7.0596, 0.0365, -6.8019, -7.0868, 0.0376, -6.7950, -7.0864, 0.0398, -6.8091, -7.0639, 0.0693, -6.7967, -7.0864, 0.0823, -6.7935, -7.0702, 0.0866, -6.7973, -7.0618, 0.0685, -6.7849, -7.0749, 0.0401, -6.7784, -7.0630, 0.0398, -6.7759, -7.0747, 0.0465, -6.7765, -7.0907, 0.0365, -6.7793, -7.0895, 0.0376, -6.7771, -7.1096, 0.0742, -6.7701, -7.0871, 0.0850, -6.7720, -7.0761, 0.0684, -6.7684, -7.0820, 0.0317, -6.8130, -7.0589, 0.0057, -6.8075, -7.0557, 0.0030, -6.7995, -7.0593, 0.0013, -6.7940, -7.0665, -0.0215, -6.7756, -7.0449, -0.0031, -6.7759, -7.0527, 0.0033, -6.7837, -7.1180, -0.0275, -6.7850, -7.1064, -0.0410, -6.7751, -7.0861, -0.0290, -6.7681, -7.0894, -0.0049, -6.7701, -7.0609, 0.0013, -6.7633, -7.0770, -0.0106, -6.7470, -7.1075, -0.0181, -6.7617, -7.1012, -0.0239, -6.7678, -7.0897, -0.0157, -6.7548, -7.1024, -0.0150, -6.7689, -7.0970, -0.0036, -6.7627, -7.0987, 0.0117, -6.7765, -7.0717, -0.0009, -6.7808, -7.0752, 0.0013, -6.7817, -7.0686, 0.0492, -6.7971, -7.0602, 0.0371, -6.8278, -7.0300, 0.0335, -6.8299, -7.0119, 0.0506, -6.8721, -7.0256, 0.0209, -6.8585, -7.0440, 0.0214, -6.8646, -7.0422, 0.0358, -6.8612, -7.0632, 0.0588, -6.8711, -7.0318, 0.0491, -6.8601, -7.0301, 0.0518, -6.8090, -7.0533, 0.0543, -6.7858, -7.0490, 0.0163, -6.7377, -7.0559, -0.0121, -6.7038, -7.0861, -0.0184, -6.7097, -7.1075, -0.0328, -6.7081, -7.1444, -0.0063, -6.7112, -7.1109, -0.0007, -6.7121, -7.1257, 0.0328, -6.7080, -7.1126, 0.0581, -6.7282, -7.1338, 0.0645, -6.7249, -7.1516, 0.0563, -6.7717, -7.1418, 0.0468, -6.7919, -7.1159, 0.0450, -6.8093, -7.0849, 0.0455, -6.7733, -7.0711, 0.0266, -6.7991, -7.0598, 0.0175, -6.8019, -7.0261, 0.0296, -6.7929, -7.0342, 0.0283, -6.8160, -7.0283, 0.0307, -6.8190, -7.0409, 0.0048, -6.8190, -7.0387, 0.0021, -6.8218, -7.0434, 0.0180, -6.8310, -7.0508, 0.0042, -6.8137, -7.0639, -0.0018, -6.8006, -7.0695, -0.0118, -6.7998, -7.0675, -0.0242, -6.7915, -7.0729, -0.0156, -6.7792, -7.0812, -0.0085, -6.7715, -7.0713, 0.0341, -6.7500, -7.1030, 0.0359, -6.7503, -7.0978, 0.0414, -6.7536, -7.0759, 0.0401, -6.7551, -7.0931, 0.0278, -6.7729, -7.1055, 0.0441, -6.7710, -7.1102, 0.0726, -6.7796, -7.1138, 0.0398, -6.7757, -7.0891, 0.0329, -6.7693, -7.0810, 0.0175, -6.7599, -7.0850, 0.0263, -6.7487, -7.1189
};
/**
* @brief Copy raw feature data in out_ptr
* Function called by inference library
*
* @param[in] offset The offset
* @param[in] length The length
* @param out_ptr The out pointer
*
* @return 0
*/
int raw_feature_get_data(size_t offset, size_t length, float *out_ptr) {
memcpy(out_ptr, features + offset, length * sizeof(float));
return 0;
}
/**
* @brief Arduino setup function
*/
void setup()
{
// put your setup code here, to run once:
Serial.begin(115200);
Serial.println("Edge Impulse Inferencing Demo");
}
/**
* @brief Arduino main function
*/
void loop()
{
ei_printf("Edge Impulse standalone inferencing (Arduino)\n");
if (sizeof(features) / sizeof(float) != EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE) {
ei_printf("The size of your 'features' array is not correct. Expected %lu items, but had %lu\n",
EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, sizeof(features) / sizeof(float));
delay(1000);
return;
}
ei_impulse_result_t result = { 0 };
// the features are stored into flash, and we don't want to load everything into RAM
signal_t features_signal;
features_signal.total_length = sizeof(features) / sizeof(features[0]);
features_signal.get_data = &raw_feature_get_data;
// invoke the impulse
EI_IMPULSE_ERROR res = run_classifier(&features_signal, &result, false /* debug */);
ei_printf("run_classifier returned: %d\n", res);
if (res != 0) return;
// print the predictions
ei_printf("Predictions ");
ei_printf("(DSP: %d ms., Classification: %d ms., Anomaly: %d ms.)",
result.timing.dsp, result.timing.classification, result.timing.anomaly);
ei_printf(": \n");
ei_printf("[");
for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
ei_printf_float(result.classification[ix].value);
//ei_printf("%.5f", result.classification[ix].value);
#if EI_CLASSIFIER_HAS_ANOMALY == 1
ei_printf(", ");
#else
if (ix != EI_CLASSIFIER_LABEL_COUNT - 1) {
ei_printf(", ");
}
#endif
}
#if EI_CLASSIFIER_HAS_ANOMALY == 1
ei_printf_float(result.anomaly);
//ei_printf("%.3f", result.anomaly);
#endif
ei_printf("]\n");
// human-readable predictions
for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
ei_printf(" %s: ", result.classification[ix].label);
ei_printf_float(result.classification[ix].value);
ei_printf("\n");
}
#if EI_CLASSIFIER_HAS_ANOMALY == 1
ei_printf(" anomaly score: ");
ei_printf_float(result.anomaly);
ei_printf("\n");
#endif
delay(1000);
}
/**
* @brief Printf function uses vsnprintf and output using Arduino Serial
*
* @param[in] format Variable argument list
*/
void ei_printf(const char *format, ...) {
static char print_buf[1024] = { 0 };
va_list args;
va_start(args, format);
int r = vsnprintf(print_buf, sizeof(print_buf), format, args);
va_end(args);
if (r > 0) {
Serial.write(print_buf);
}
}
void ei_printf_float(float f)
{
float n = f;
static double PRECISION = 0.00001;
static int MAX_NUMBER_STRING_SIZE = 32;
char s[MAX_NUMBER_STRING_SIZE];
if (n == 0.0) {
strcpy(s, "0");
}
else {
int digit, m, m1;
char *c = s;
int neg = (n < 0);
if (neg) {
n = -n;
}
// calculate magnitude
m = log10(n);
if (neg) {
*(c++) = '-';
}
if (m < 1.0) {
m = 0;
}
// convert the number
while (n > PRECISION || m >= 0) {
double weight = pow(10.0, m);
if (weight > 0 && !isinf(weight)) {
digit = floor(n / weight);
n -= (digit * weight);
*(c++) = '0' + digit;
}
if (m == 0 && n > 0) {
*(c++) = '.';
}
m--;
}
*(c) = '\0';
}
//EtaCspUartPuts(&g_sUart1, s);
Serial.write(s);
}
Next steps
I'll need to integrate IMU data and validated that my model created based on accelerometer from a phone still can be used for inference using data from Arduino Nano 33 IoT IMU module. If it is not the case I may need to recollect data set and create a new model.
