This is a continuation of my previous post Using SensiML with Arduino Nano 33 BLE Sense - Data Capture .
In preparation to building a model in Analytics Studio, I needed to capture a larger dataset using Data Capture Lab. I am going to try to recognize motion along each accelerometer axis using data from the accelerometers and gyros.
I have 4 data classes that I showed previously:
- Idle - no motion
- UpDown - motion along the Z-axis
- LeftRight - motion along the Y-axis
- ForwardBack - motion along the X-axis
I captured 20 data sets with each of the data classes within each set and then I segmented and labeled the data. I added an additional metadata tag "Data Set" to differentiate data used for training and data used for testing. I tagged 4 of the 20 sets (20%) for test as shown in the Project Explorer of DCL.
All of the data sets were 20 seconds in duration with a 119 Hz sample rate. The amount of motion and spacing was somewhat variable. Below is a representative sample.
SensiML Analytics Studio
Analytics Studio is a web browser based application that you log into with your SensiML account.
Projects that you create in Data Capture Lab will show up in Studio, as will the captured data.
Here's a diagram of the tool flow:
And a diagram of the Analytic Studio Flow:
The Summary page for the project - I'm just starting so the only thing currently showing is the 20 Captured data sets plus a picture that I added:
Prepare the data
The first step to building a model is to prepare the training data set. This is done by creating a query to extract the data from the data set. Here is the query that I created. You can see that 16 instances of each class was extracted from the data set.
Build the model
It turns out that you can build the model within Analytics Studio using AutoML or you can also build it externally using Google Colab and TensorFlow Lite Micro. SensiML recently did a webinar on using the external process https://sensiml.wistia.com/medias/l72tofdz9j , but I don't have access to the Colab Notebook so I'll use the AutoML flow.
The SensiML models are called "pipelines" as opposed to Edge Impulse models which are appropriately called "impulses".
The pipeline uses the data query as its input. Then you select a segmenter - I chose an approx. 2 second window in which the data is classified. Then select an optimization metric - I chose f1-score.
Optimization metric choices:
And then a target model size to define the SRAM usage. It defaults to 32K, but I could increase this parameter if necessary as the BLE Sense board has 256K of SRAM.
There are also some advanced settings to optimize pipeline parameters. This is something that I'll need to learn more about:
After setup, you press the "Optimize" button and it runs AutoML and presents the results of the top 5 models.
Here is my pipeline with the optimization results:
Here is the Model Visualization showing the feature separation:
And the Confusion Matrix:
And Feature Summary:
Test the Model
Now to select the 4 Test data sets and verify performance.
And the results are not that good. But the reason is apparent when looking at the Confusion Matrix:
The problem is with the Idle class. I'll need to go back and look at the training and test data and maybe try different window sizes. I'm testing with data sets that have all 4 classes within the set and variable spacing between the classes. I think I'll make sure there isn't any issues with simpler test data.
So much to learn, but I think using these tools might actually help me figure out how this all works 
......
Top Comments