ZinnBee Blog#06: Arduino Nicla Vision Module Lego Lab

26 Apr 2023

As mentioned in ZinnBee Blog#04, I did a LEGO lab via Nicla Vision. I found that the performance of ML from Edge Impulse classification was limited and the results were not quite good enough for ZinnBee project. I paused the LEGO lab until now. I still cannot find Bees in my neighborhood this week. Even Cellophane bees are very hard to find due to delayed Spring at MN, US in 2023, so the LEGO lab would be my placeholder for Bees in the design challenge in April 2023.

Initial bench testing report on LEGO lab

Edge Impulse® is a fun platform to work with, however, it is also a big black box for me to learn. It is my first time to work with Edge Impulse®. The learning curve is deeper than I anticipated. I even trained myself to work with Edge Impulse® on Rasperry Pi Pico platform before I received Arduino Nicla Vision module from Element 14 community. Long story short, Edge Impulse® is complicated if a project plans to pursue a good reliable ML model.

I decided to use pre-process images to do image classification for LEGO lab because this way can save me time from preparing data sets. The flow of Edge Impulse® from the beginning of 2013 until the first week of April in 2023 were same as what I presented in my ZinnBee Blog#04, however, the flow from Edge Impulse® was updated this week. I do not know when the update happened this month. Hopefully, it will not make the performance different between now and my previous LEGO lab testing.

Updated step at Edge Impulse®: Welcome to your new Edge Impulse project as Fig 1.

Fig 1. Welcome to your new Edge Impulse project at Edge Impulse® before April 15^th, 2023

Which is disappeared at Edge Impulse® under my account now.

Fig 2 shows the configuration of how to create impulse.

Fig 2. Create Impulse on Edge Impulse®

I did more than 5 iterations on LEGO lab last month. The last iteration is reported in this blog since it was the best among all initial tests.

The data sets I used at initial LEGO lab tests were shot from a Sony camera under nature light on my coffee table, where there was some wood fixture background as Fig 3 in grayscale for ML of Edge Impulse® . Here are my 4 LEGO objects from left to right in Fig 3: bigDinosaur, boy, mice and tinyDinosaur.

Fig 3. LEGO Lab Data Set Samples for Edge Impulse®

I tried color type as Fig 4 shew which there was a parameter setting in Edge Impulse®. Color type setting did make the ML model worse.

Fig 4. LEGO Lab Data Set Samples for Edge Impulse® in Color Type

Here is the video of the initial LEGO lab iteration #5 testing results. LEGO lab 1stRound testing report

The deployed ML on Arduino Nicla Vision module had a lot of fault detection about LEGO objects. The estimated fault rate was almost over 50%.

2. Updated bench testing report on LEGO lab

How can I improve ML performance on Arduino Nicla Vision module from previous lab? I think there were probably a few factors needed to be considered.

Background of the data sets

My bench testing lab is set up at Fig 5.

Fig 5. LEGO Lab Bench Testing Setup

I did not use wooden surface of my lab bench because it did not improve the initial LEGO lab testing performance. So I used a solid color as the background to increase the contrast between LEGO objects and the background. Then I think the wooden fixture of the background in the data sets could be noisy signals for Edge Impulse® ML model training process. I changed all the data sets with the same background as Fig 5. The number of data sets are still same as previous labs.

Object size ration among all objects

The big dinosaur is bigger and the small dinosaur is smaller. But my initial data sets did not honestly present that for Edge Impulse® ML model training process, which could be another misleading input.

Edge Impulse® parameter tuning

I learned how to tune parameters at Edge Impulse® in my ZinnBee Blog#05. More number of training cycles could make ML model more accurate.

I finally got pretty reliable LEGO lab testing results after I implemented above 3 factors into Edge Impulse® under the same fundamental ML process: pre-process images for image classification.

Here are the updated data set samples in Fig 6.