The following webinar is now available On Demand:
Smart Agriculture: Pest Monitoring with AI and Arduino
As invasive insects can cause great damage to the vines and grape harvest, monitoring pests in vineyards is a crucial element within wine production. In this webinar we show you how it's possible to remotely evaluate the pest population in vineyards, using Machine Vision and Machine Learning.
During this webinar, Andrea Richetta, head of Customer Success in the Pro/IoT market at Arduino discusses how Machine Vision and Machine Learning can be used with smart agriculture and prepared a live demo using the Nicla Vision board and the MKR 1310, focusing on LoRa and the available solutions from Arduino.
Survey & Poll Results:
Here are some of the results from the survey we did during the presentation:
| {gallery}My Gallery Title |
|---|
 |
|
|
|
|
 |
Q&A:
There were a lot of great questions asked. As promised here are some of the ones we could not get to during the event:
How fast (generally) is the ML on the array of boards you provide?
Typically it is in the range of milliseconds, depending on the model size and number of classes to be analysed.
Does Arduino Pro have a wider operating temperature range VS classic Arduino s?
All PRO boards are ranged in temperatures that go from 0 to 40.
What are the cost ranges for using Arduino Cloud?
50 EUR for 50 devices
Are there any specific challenges that arise when designing and deploying IoT solutions in agriculture, and how does the Nicla Vision board address these challenges through its hardware and software features?
The harsh environment is one of the typical challenges. A proper enclosure is a must. In terms of power, if the solution should be battery powered, this could be another part to take into account
What advice would you give to students interested in pursuing a career in IoT and machine learning, and what are some resources or learning opportunities you would recommend for getting started in this field?
There are plenty of sources, I suggest to look at a platform like EDX and Coursera.
How does the Nicla Vision board utilize Arduino technology to implement machine vision and machine learning algorithms, and what advantages does this approach offer over other hardware platforms?
The arduino platform is open to different AI / ML providers. To name someone, Edge impulse, queexo, edge ML.
Is there a cap for the memory card size that can be used? eg. 64 GB, 128 GB etc
Usually we suggest 64 Gb.
Do the Nicla boards have battery management given their form factor please?
Yes, they do.
Are you having plan to develop drones equipped with Arduino boards with Nicla and Cat.M1/NB IoT GNSS Shield to map field and trasnfer data to cloud?As you know connectivity will be poor in fields and also huge data to collect in large fields.
No, catM1 or nbiot are not suitable for those applications.
What is the range of view/detection range if I want to use Nicla for Fire detection?
80 degrees at 1 mt.
Is Nicla Vision camera frame rate impacted by use of other on board sensors ?
Usually not , because they have a totally different signal path
While arduino is doing awesome and smaller and miniature boards are coming up, for a maker its getting a bit more difficult to hack around components or build an own clone with tiny parts. Do you have equivalent reference designs or suggestions for the same with larger hand solderable components? (just a thought)
Breakout board or portenta max carrier are good example for blueprint / ref design
Is it possible to use Arduino OPTA as a replacement PLC for small company projects like startups?
yes sure
Wow... Can we use OPTA for creating Demo Models like Lego IOT Factory setups?
Yes, OPTA supports 24V and MODBUS
Are there any potential security risks associated with using LoRa to transmit data from the Nicla Vision board to the Arduino IoT Cloud, and what measures are in place to mitigate these risks?
LoRa is a LPWAN protocol, so is possible to send some data, small packets via LoRa.
Is there a way where the included arduino library functions can be accessed or redirected through the main code in Arduino IDE which can possibly make the debug process easy?
Some of the boards have debug pins to connect to an ext. Debugger. Portenta H7 comes with a free license from segger.l
Would it be possible to run a factory using arduino portenta ? does it sustain the work environment ?
Depending on the needs, yes. Multiple boards for multiple operations. OPTA is a good option to start
Thanks you for the presentation. I am starting a swarm robotics project with ROS-2. There will be 200 differential-drive-robots and all of them will be wireless-connected
to a master computer. What is the best product you can suggest for this 200 differential-drive-robots? Each robot will be a ROS-node that will do motion-planning and some sensoring(IR sensor/Led etc) and will communicate with mater computer with wifi.
Portenta X8 can run ROS , so that can be a good starting point
What inspired you to focus on the intersection of machine vision, machine learning, and smart agriculture, and how do you see this field evolving in the coming years??
Smart agriculture is a challenging sector. By adding technology with direct ROI and fast adoption are the key to processes.
Could you walk us through the process of training a model for pest identification using the Nicla Vision board, and what are some best practices for optimizing the model's accuracy?
Collect data, analyze, train and deploy are the main steps. Generally speaking, when it comes to vision recognition, different lighting, and various conditions of images to analyze is a must. Testing with lot of pictures in different conditions produce the best results.
How does the Arduino IoT Cloud enable users to visualize and analyze the data collected from the Nicla Vision board, and what are some potential applications of this data beyond pest monitoring?
You can collect, display and eventually connect to external services that can also provide more vertical applications
As the use of LoRa for remote monitoring of pest populations becomes more widespread, what safety considerations should farmers and other users be aware of with respect to the transmission and storage of data?
There are no specific topics regarding safety, in terms of security, data is encrypted all over the transmission.
Could you discuss the reliability of LoRa connectivity for remote monitoring of pests in vineyards, and what measures are in place to ensure that data is transmitted accurately and securely?
LoRA protocol can be set to different level of QOS. Here's an interesting paper https://webthesis.biblio.polito.it/25414/1/tesi.pdf
Given the potential for interference from other devices or radio signals, what steps are taken to ensure that the LoRa transmission from the Nicla Vision board does not interfere with other wireless communications in the vicinity?
LoRA uses ISM Bands, so it safe enough to avoid overlapping other frequencies
Which module will be the best for hydrophonic or aerophonic farming?
Arduino Edge control
In the event of a malfunction or other issue with the Nicla Vision board or the LoRa transmission, what safety protocols are in place to prevent damage to crops or other unintended consequences, and how are these protocols communicated to users of the technology?
Local intelligence is a must. You should never rely on cloud or connected services when there ìs an automation local to be done
How far does the ToF sensor/camera reach?
around 8mt for tof and the angle is around 80 degrees FOX at 1mt
Can the Nicla Vision board and the MKR 1310 detect any type of insect, or are they limited to specific types of pests?
it depends on the model that can be trained for recognize different species
I saw a "crypto chip" on the Nicla's image you showed earlier - What is its main purpose please?
to ensure security on data, either locally
Will time come that ML will be local instead of in the cloud, for application like pH control for hydroponic nutrient reservoir? pH control being very difficult even for PID, and some systems are using MatLab/MathWorks in the cloud.
An application could be a local analysis of the Ph value that determine if a possible trend is identified. More time / computational computing resources can be don on the cloud
How do you train the model to recognize flies? it's a pre-trained model or you have the training option with local data?
local pictures taken directly from the camera using OpenMV
I was wondering if the IoT cloud step of onboarding the board had anything to do with onboarding the device to the TTN network? Or was it connecting a different Lora Network please?
We have a dedicated instance of TTN to connect our Lora Boards to the Arduino iot Cloud
For vision applications and industrial predictive maintenance applications, can the machine learning model run directly on the Arduino devices independently from IOT Cloud?
Yes sure, leveraging tiny ML, all the models are running on the boards, without the need of any internet connection
Can you connect the system to a bug dataset, not only to count the bugs, but also to recognize the kind of bugs? I guess each bug detection is timestamped, so we can compute bug density?
Yes, you can train the model to recognize different set of bugs and make math around the result
Has LoRa enough data rate for this type of applications (Machine learning, vision)?
Yes, for sending the result of the AI , not for sending any image.
i have heard of a similar application for the recognition of desert locust to predict plagues. Would the board be able to differentiate between different instars / stages of growth? i imagine this would be quite difficult as the shapes and colours are similar
Dimensions can be hard to determined only using vision. Small bugs from adults, if they change in size, based on the blob size definition, you can try to estimate the size.
How did the Vineyard benefit by using the bug recognition system ?
By counting the number of bugs, you can determine the status of an infestation
Do you think that the infrastructures are in need of a transformation for the smart agriculture to be viable? The farming community must be trained for the upcoming technology. How can you achieve these two big factors?
It is not a quick path, but by democratizing the access of the technology is the first step. Then, the farmers need solutions that bring real values in order to be adopted
Is Nicla Vision powerful enough to count moving people?
Yes
I am an owner of a pest control company and I am developing an AI product to identify pests within food manufacturing facilities. What other Arduino products would be recommended to pair with the Nicola Vision for my project?
Nicla Vision can pair with almost every board of the PRO line, but having local signal wired communication and wifi or BT, you can pair almost with every board. Depending on the application, some boards are more useful for specific needs
Considering that the Nicla Vision has multiple sensors like IMU and microphone, is the board powerful enough for a multi-modal neural network model? For example, a model that uses the camera and microphone to detect pests.
Depending on the model itself or better, on the ML training.
In addition to pest monitoring, are there any other potential applications for the Nicla Sense ME device in smart agriculture?
Air quality is another good examples to monitoring the health of animals.
How can farmers and growers integrate this technology into their existing practices? Is there a learning curve involved?
Some learning curve is involved, depending on the complexity of the project. Our main goal is to provide all the building blocks to start the project, than of course can be complicated at discretion :)
Can the Nicla Vision family can be interfaced with SBCs like Raspberry Pi ?
Via Wifi for example.
Do you know any cheap multispectral camera solutions for fixed stations with Arduino?
No, unfortunately
If I had a recognition model for bugs in my local device, I have to upload to Edge Impulse to export to Arduino? Or, is there some tool to make this process offline?
the ML starts on Edge impulse and once you are satisfied with the test, you can download the model to the board
What kind of support or training is required for farmers or agricultural workers to effectively utilize AI and Arduino technology for pest monitoring?
it really depends on the farmer of course. My suggestion is to understand the issue and provide to them a solution, taking care of all the technical aspects. The life of the farmer is quite busy, so they usually prefer to stay in the field instead in front of a desk :)
Considering the integration of emerging technologies, such as Internet of Things (IoT), what further advancements can we expect in smart agriculture for pest monitoring in the future?
I see integration of different technologies from different manufacturers that talk together to provide the best scenario for the farmer
Is it possible to integrate external cameras to Arduino devices? for example higher resolution or 3d camera?
Not now
Dear Sir if the Arduino Nicla Vision get over heated during its image processing in AI what should is the best way to handle it.
The overheating in normal condition is not an issue. When is in hard conditions, some thermal pad can mitigate the problem
How many pictures do you typically need to train the AI?
It really depends, but I would say that a consistent model is based on around 1000 images
How can nicla vision be useful in the agrivoltaic field to ensure through machine learning the maintenance/cleaning of solar panels so that they always work at 100% of their possibilities?
good question, probably can be used to determine the cleanliness of the photovoltaic panel to ensure the maximum efficiency
Is it possible to catch video and compress on different way depending on the connectivity capability?
No, video manipulation is a task too expensive in terms of computational power
What makes the Edge product special for it to be recommended for agritech (outdoors) applications?
The Edge AI is recommend not only for agritech. When you need to have some data analysis before sending data, this is when the AI can help
Are there any limitations when using a third party cloud service with the Arduino Pro series?
no limitations
The Nicla Vision has two powerful cores, Do you have any advice to run two models running on different cores? i.e. keyword spotting and image recognition?
No, right now the model runs on the M7, but we are looking for to have them running on the 2 cores
Did you try to mix bugs and leaves or other images different of bugs and check the accuracy of the models? Thanks
Yes, the model was tested against different bugs, mosquitos and flies.
Is there a way to capture a picture of bugs that are farther away, and moving?
it depends on the camera, the one on the Nicla vision can take pictures and the Frame rate is around 20 fps, so depending on the speed of the object, it could be achieved
We are designing smart pheromone traps for orchards. One of the main problems we face is the battery durability? How much power it consumes when we take 3-4 snapshot a day?
Typically it consumes up to 100 mA when taking the picture and it last for some seconds. In deep sleep is around 400 uA.
How hot does the Portenta and Nicla boards get when running? Would a heatsink be necessary?
In normal condition there's no need of thermal dissipation.
Does Nicla vision ML interpret a movement or only shapes ? For example someone raising the hand , can the ML detect the movement rather than tell it is only a person?
Yes, recognizing faces or movement pattern is possible.
Can the we detect multiple types of objects using this ML models? Because other than the moth in concern, we might have other random bugs which are not counted.
Yes, you can train the model to recognize different set of bugs and make math around the result
IS it possible to connect other image sensors on it? What is the Interface for the image sensor?
mipi interface , you can connect other models that shares the same core cmos . Data is available here https://docs.arduino.cc/resources/datasheets/ABX00051-datasheet.pdf
Could you please shed light into power consumption of the bug counter hardware that is shown in the demo? ..or average power consumption to take one image and analyze it before going to deep sleep?
It is around 100 mA when camera is on, and around 400uA when in deep sleep.
Can the Arduino Pro Boards (LTE enabled ones) be brought up to a VPN(Virtual Private Network) natively, like as VPN require a set of libraries to be installed and non industrial Arduinos do not support those ? or do we need a separate hardware for making it happen ?
With the X8 that runs linux, every compatible VPN is runnable.
How can I connect HD camera(10MP or more) to Nicla Vision?
It is not possible because the RAM would not be sufficient to manage this datastream coming form 10 mp cmos.
How would you compare processing speed and algorithm execution of computer vision with respect to an FPGA or a ZYNQ with respect to Nicla Vision? Why should I use Nicla Vision rather than FPGA and ZYNC for computer vision?
FPGA or ZYNC are more sophisticated hardware products, so it's difficult to compare.
Is 2MP the only resolution camera the Nicla board use?
Yes.
Can Nicla Vision be used as a distance/height sensor on drones? Like we need our drones to have collision avoidance. So for this use case are the boards fast enough?
no, I don't think
How much data is necessary for the object detection model to be able to detect with accuracy above 80%?
There's no fixed number of images, even with 100 picture you can achieve over 80% of good tests. I suggest a model with around 500 to 1000 pictures.
What is Nicla Vision night vision capability, IR sensitivity?
there's no night vision option on the sensor
What is the range for LoRa?
in rural context, around 15 km depending on the terrain hills, buildings can reduce the range. In urban context is around 1 - 3 km
With the nicla vision and the current code are you able to differentiate pest bugs from polionizers and categorize them?
probably yes if you can determine the differences
How does the combination of real-time data monitoring and AI-driven decision-making improve overall crop yield and quality?
Multiple ways. Manage it in different data like weather, soil conditions, temperature, can be enough to build a dataset useful for providing useful informations
Is it possible to use hardware accelerators with Arduino such as, for example, the Coral TPUs to increase AI performance?
yes with a mini pcie module using the Arduino x8
If I use 10000 mah powerbank How long can I use the Nicla vision?
it depends on the usage . Taking a picture requires around 100mA , in deep sleep is around 400 uA
Is it possible to interface LoRa SX1276X module with arduino pro?
with the module only it's required to have a custom pcb around it.
What is Nicla Vision's temperature range? Im planning to use it for outdoor use here in the philippines. thanks
It is around 25 celsius typically: https://docs.arduino.cc/resources/datasheets/ABX00051-datasheet.pdf
