As promised here is the Q&A for recent Constrained Object Detection webinar with Edge Impulse. If you missed the original event then you can check it out using the link below:
Constrained Object Detection with Edge Impulse
Q & A
Do you use any open source projects to determine the best suitable model? Do you contribute to open source projects?
Lots of our work is open source, yes! Firmware, projects, etc. : https://github.com/edgeimpulse
What has been your favourite project to work on?
Stay tuned for my perfect toast machine project using embedded machine learning!
Models sometimes feels like a black box where we do not really know whats going on inside them. Is there a way to have some insights of whats the model doing to make the classifications?
Check out https://github.com/mlcommons/tiny
Will it be able to detect transparent objects like glass?
That might not work too well, sorry
how well FOMO can work on esp32 cam
Works great, that is a supported platform
Can you give an id to the object you are tracking?
FOMO is not great for uniquely identifying objects (e.g. one face vs. many faces). You would need other systems or models to do that.
How good is fomo for facial detection?
Works great for face detection - but not possible to identify "who" the face is. Just the fact, that it is a face.
Models sometimes feels like a black box where we do not really know whats going on inside them. Is there a way to have some insights of whats the model doing to make the classifications?
For some, yes. Things like saliency and grad-cam maps can help us gain insights into what a convolutional neural network is looking at.
Does FOMO work with images only or any other sensor input?
FOMO can work with anything so long as it can be configured as a 3-channel, 2D array.
Is there a benchmark comparing the performance with an ESP32 S3 vs other MCU or the standar ESP32? ESP32 S3 introduced some new special instructions for running ML models
Check out https://github.com/mlcommons/tiny
How good is it when detecting objects through translucent glass?
I'm not sure this has been specifically tested, that I am aware of
Is it capable of detecting moving objects ?
Yes, that will work fine
Can we identify people individually with the good accuracy with the FOMO?
FOMO is not great for uniquely identifying objects (e.g. one face vs. many faces). You would need other systems or models to do that.
Can we execute TinyML based task on this FOMO?
FOMO is perfect for TinyML. :-)
Hey Shawn, i had take your cource of edge impulse on courcera & i learnt how we can use anomaly detection usin accelerometer. My question is can we do anomaly detection using images. I am thinking for using image sensor to determine anomaly condition process industry to detect anomaly condition. As such conditions are hard to re-create for to get training data. If we collect health condition and define classes & others then that class data to be identified as anomaly.
There are a number of ways to do anomaly detection with images. One of my favorites is to use autoencoders with image inputs.
how many signals/channels apart from camera signals, can Edge impulse can handle in single frame
FOMO is currently configured for 3 channels. At the moment, this cannot be changed.
Can FOMO mode be split for computational efficiency, so some parts of the model are executed in one different cores, like cores of Raspberry Pi Pico?
Right now, no. This is an active area of research. I know a few embedded engineers (e.g. Pete Warden) who are working on a way to parallelize the neural network layers during inference.
FOMO: Does it work work tiny objects like small tiny defective devices
Works great on small objects Muzamil, similar to the little screws Shawn is showing us right now.
How about fps , how we can improve FPS
This will depend a lot upon the hardware you are using.
Will this work for fast moving objects also...?
You will need faster hardware, with a faster camera most likely...but it *could* work yes.
What is the fastest framerate for FOMO if you are using a raspberry pi?
Can do up to 60fps: https://www.youtube.com/watch?v=o2-o3wEmxaU
How good is FOMO when it comes to solving Captcha?
FOMO is not designed for solving Captchas. :-)
Can FOMO be used to count objects?
FOMO can be used to identify the objects, then you do need a bit of logic to do the counting, but that is a great use case yes!
Is it possible to use FOMO in RISC-V based MCUs?
As long as you have a C++ for your architecture and enough RAM and flash memory, yes.
Can we use FOMO with Raspberry Pi as well?
You sure can! https://www.youtube.com/watch?v=o2-o3wEmxaU
What is the FoV of the built-in camera?
That would depend upon the device being used, this could vary depending on which hardware you are using.
Can FOMO be used to test facial expression especially to people with autism?
FOMO is designed for object localization, not really facial features. It might be trained to locate facial features, but there will be lots of limitations. You will likely get better results from something like DAN trained on AffectNet (https://arxiv.org/pdf/2109.07270.pdf).
Can we use FOMO itself as a controller as well like Raspberry Pi and Arduino as well?
Not sure what you mean by Controller, but it will work on Arduino or RaspPi, yes!
are we using any pruning technique or any quantization method in FOMO.
You can select to download the quantized or floating point version of the model, which allows you to choose to do quantization. I do not believe we are doing any pruning on the model.
When it is necessary to involve sound from built-in microphone in FOMO model what is possible options, incorporate in the FOMO model or run independent sound detection model?
FOMO does not have any sound inputs. You would need to use a second model to do sound classification independently of FOMO right now.
For the use cases, when the bees are closing and overlapping to each other, will FOMO model fail to identify and counting?
If bees (or cars) are too close together, they certainly could overlap and only be detected as one.
It is possible to export our FOMO model as a c/c++ library?
Yes, you can export FOMO as a C/C++ library.
If we consider the heatmap as a gaussian distribution among the neighboring cells, then the detection of larger objects(more than the size of cell) is possible. right?
Yes, you can detect larger objects. However, neighboring cells will likely be ignored and considered to be part of the background class.
What's a good way to improve scale invariance ?
FOMO struggles some with objects of different scale. If scale invariance is important in a model, you're likely better off with something like YOLO or MobileNet-SSD right now.
How FOMO is tolerant to replacing of video camera to other vendor having the same resolution, but different sensitivity and white balance?
Changing the colors means changing the values in the image channels. Depending on the training data used and how close the different objects (including background) are in appearance, this will likely have at least some affect on the ability to detect objects.
Which IDE will we use for FOMO for objects detection tasks?
For training FOMO, you'll need to use Edge Impulse (the graphical Studio or through the web API). For deployment, you can use any IDE you wish.
Can we use FOMO for leave disease detection in windy scenario on multiple leaves in frame.
Yes, but remember that if the leaves are close together, FOMO may miss some.
Can Fomo work with the brainchip aikida chip
It's in the works (https://www.edgeimpulse.com/blog/bringing-development-of-brainchip-akida-neuromorphic-models-to-all-using-edge-impulse).
As FOMO is not giving the object boundaries, can we say FOMO is an object localizer rather than object detector?
Yes, but since it can also detect objects of different classes, it's a very limited detector and localizer.
can we find Queen Bee from a Bee hive using Tiny ML ?
Probably. You'll just need to train a model to identify "worker bee" vs. "queen bee" :)
Can FOMO detect your face if it is a super close-up on say, your eye?
Yes, but be careful. It might think anything that is flesh-colored to be a "face" unless you trained it otherwise. You might be better off training FOMO to detect "eye" if you want to work with close-ups.
Could I train FOMO on heatmap images?
Yes. Anything that you can fit into a 3-channel 2D array should work (in theory). FOMO might struggle to train, as it is pre-trained on ImageNet.
is FOMO good for eye tracking (inside a car for eg.)?
Possibly. Depending on the resolution, etc. you can have FOMO identify the location of a driver's eyes.
Can FOMO give me perfect results for vehicle traffic when playing around with it using openCV
I'd be careful expecting anything to be "perfect" in the world of machine learning. Only a vastly overtrained model will give "perfect" results on the training set. The real question is "how much is good enough" when it comes to ML. If you need "perfect," ML might not be the best tool.
Is there a potential to do a quantization and reduce number of bits during inference for FOMO?
Yes. When you go to deploy your FOMO project, you have the option of selecting 8-bit quantization or floating point for your model parameters.
How well would FOMO work on the TinyML Arduino Sense kit (OV7675 camera)
It's technically possible, but it's slow. I've seen a demo of FOMO working on the Arduino Nano 33 BLE Sense at around 2 fps with a 64x64 resolution input.