Trakcore #2 AI Assisted Posture Modification - Acquiring Data and Designing a Model

vlasov01

20 Mar 2021

Design for A Cause 2021Trakcore #1 AI Assisted Posture Modification - Project Introduction

#1 - Project Introduction

#2 - Acquiring Data and Designing a Model

#3 - Deploying and Testing Edge Impulse ML Model on Arduino Nano 33 IoT

#4 - Enabling IMU and BLE with Edge Impulse ML on Arduino Nano 33 IoT

#5 - Using Cordova Bluetooth (BLE) Plugin to Connect Arduino Nano 33 IoT with Android Phone

#6 - Arduino Nano 33 IoT Wearable Design

#7 - Adding Pedometer to Reduce False Positives

#8 - Refining User Experience AKA Bugs Fix

Changing the plan

As we've got some delay with shipping of Arduino Nano 33 IoT in the supply chain I've adjusted my project plan. I'm starting with data collection, model design and test instead of starting with MVP (minimal viable product). I was planning to collect data for the model using MCU.

Platform decision

My device should be able to recognize human activities - if person is sitting or active (walking, running). And if a person is sitting it should detect straight and curved positions.

I've took a very good Coursera Introduction to Embedded Machine Learning training in preparation for this project. This course used Edge Impulse platform for data collection, model design and testing. It used a phone and Arduino Nano 33 BLE for its projects. I found the platform easy to use and quite effective. And there is no additional cost for most of developers. It provides a quite complete and streamlined approach to AI in embedded world. They have several examples related to continuous movement use case, where data collected from accelerometer. Edge Impulse doesn't directly supports Arduino Nano 33 IoT, but it seems the generated Arduino library can be used with Arduino Nano IoT. I've considered as well AWS IoT Greengrass in combination with AWS IoT Greengrass ML and AWS SageMaker. It is very powerful and has a lot of capabilities like device management, security, inference at the Edge, But I found the learning curve a bit too steep and I'm concerned about its memory requirements as I was not able to find any examples for Arduino Nano 33 IoT or Arduino Nano 33 BLE. As well use of AWS IoT Greengrass on edge requires a more powerful compute, capable of running Linux like RPi. Alternative option to perform inference in the AWS cloud will require sending sensor data from MCU over radio. It will drain batteries pretty fast in continuously running mode. As well it may have some negative privacy implications.

So I've decided to use Edge Impulse platform for my project for inference. I may use AWS IOT for the device management.

Options to collect accelerometer sensor data

It was recommended in my Coursera Introduction to Embedded Machine Learning training to use the same sensor for training model data collection as for inference. But Jan Jongboom, co-founder and CTO at Edge Impulse, suggested that for my specific use case I can use a phone as a sensor as it translates surprisingly well to on-device inference. I'll verify it once MCU will be delivered.

Collecting accelerometer sensor data from a phone

Edge Impulse provides a very convenient tool for data collection on a phone. It is very easy to load and use. It doesn't require any app installation or credentials. I've just scanned a provided QR code. Once a mobile phone load a web page related to QR code it as well loads WebAssembly. After that I've granted permission to access accelerometer it was ready to collect data.

I've attached the phone to my back on a cardigan using a running armband.

But I've got some minor challenges too. My phone was going into lock mode after capturing several 10-seconds samples. Sometimes the size of the sample was 0 seconds. So I've deleted them. Sometimes the WebAssembly was not able to upload data.

Data collection for Curved vs. Straight position

I've collected a training data set for position. This data set is used to build a model. Each data sample must be labeled with a straight or curved position label.

Then I've collected a test data set. This data set is used to validate the model. The size of my test is 30% of the training data set.

I've selected two components for my model. Flatten to classify and Spectral Analysis for anomaly detection. I've added anomaly detection based on recent recommendation by an engineer Jenny Plunkett from Edge Impulse during recent webcast Industrial-Grade TinyML Applications with Silicon Labs and Edge Impulse! (Win a Thunderboard Sense 2!). I may revisit my choice later.

Than I trained model. The accuracy on the training set was 100%. And the size of the model (1.5k peak RAM usage, 15.8kb ROM usage) should easily fir into MCU, which has 256kb of flash and 32kb of SRAM. But I'll still need to account for the network stack.

I've decided to add anomaly detection block to account for unclassified cases. After some exploration I've adjusted minimum score before tagging as anomaly to 0.50. And I still need to learn more about it.