element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • About Us
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
Save The Bees Design Challenge
  • Challenges & Projects
  • Design Challenges
  • Save The Bees Design Challenge
  • More
  • Cancel
Save The Bees Design Challenge
Blog Bees Monitor With Predators Repellent # 5 - Testing The Machine Learning Model With OpenMV
  • Blog
  • Forum
  • Documents
  • Leaderboard
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join Save The Bees Design Challenge to participate - click to join for free!
  • Share
  • More
  • Cancel
Group Actions
  • Group RSS
  • More
  • Cancel
Engagement
  • Author Author: guillengap
  • Date Created: 14 Apr 2023 5:59 AM Date Created
  • Views 497 views
  • Likes 7 likes
  • Comments 0 comments
  • save the bees
  • savethebeesch
  • nicla vision
  • lora
  • mkr1310
  • arduino
Related
Recommended

Bees Monitor With Predators Repellent # 5 - Testing The Machine Learning Model With OpenMV

guillengap
guillengap
14 Apr 2023
Bees Monitor With Predators Repellent # 5 - Testing The Machine Learning Model With OpenMV

Table of Contents

  • Introduction
  • Getting Started
  • Edge Impulse
  • Improving Edge Impulse Model
  • Testing The Machine Learning Model With OpenMV
  • Adding The Water Sprayer System
  • Testing The Water Sprayer System
  • IoT Ambient Monitoring System | Part1
  • IoT Ambient Monitoring System | Part2
  • Summary

**********************************************************************************************************************

In this chapter I will show you the procedure to use OpenMV when we are going to test the machine learning model created with Edge Impulse on our Nicla Vision board.

Build the Firmware

Since the Nicla Vision doesn't have any on-board SRAM we need to build the machine learning model into the firmware and load it from the flash. To do so, go to https://github.com/openmv/openmv and fork the repository. 

image

Rename the machine learning model and the label file downloaded in Edge Impulse, in my case I renamed to bee_or_spider.tflite and bee_or_spider.txt respectively.

image

In your fork, replace the built-in machine learning model under src/lib/libtf/models with the model you downloaded from Edge Impulse Studio. Commit the files and push the commit to the repository. It will build a new firmware automatically.

image

You can inspect the build process under "Actions". Once the firmware for NICLAV, has been built you can download it from the firmware link.

image

Flash the Firmware

We can now return to OpenMV and flash the new firmware to the Nicla Vision.

image

Put the Nicla Vision in bootloader mode by double-clicking the reset button – the green LED will start flashing. Click the Connect button in the IDE – the dialogue to Load a firmware shown below will open.

image

Click OK, and navigate to bin file produced in the previous step and click Run.

image

The Nicla Vision will be flashed with the new firmware, which includes the Edge Impulse model.

image

In my case I uploaded my best machine learning model, which is 1.81 MB in size. In other words, I am using 90.5% of the flash memory of the Nicla Vision.

Run the Script 

The next step is to write a Python script in OpenMV to control the Nicla camera and use the ML library to classify the image stream and try to detect our target objects. The video stream is just a series of image frames which are passed to a TensorFlow object which classifies the frame using the model and calculates a confidence prediction. The complete script of the classification example is as follows:

# AUTHOR: GUILLERMO PEREZ GUILLEN

import sensor, image, time, os, tf, pyb

redLED = pyb.LED(1) # built-in red LED
greenLED = pyb.LED(2) # built-in green LED

sensor.reset()                         # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565)    # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)      # Set frame size to QVGA (320x240)
sensor.set_vflip(True)
sensor.set_hmirror(True)
sensor.set_windowing((240, 240))       # Set 240x240 window.
sensor.skip_frames(time=2000)          # Let the camera adjust.

labels, net = tf.load_builtin_model('bee_or_spider_v2')
found = False

def flashLED(led): # Indicate with LED when target is detected
    found = True
    led.on()
    pyb.delay(2000)
    led.off()
    found = False

clock = time.clock()

while not found:
    clock.tick()
    img = sensor.snapshot()
    for obj in tf.classify(net, img, min_scale=1.0, scale_mul=0.8, x_overlap=0.5, y_overlap=0.5):
        print("**********nPredictions at [x=%d,y=%d,w=%d,h=%d]" % obj.rect())
        img.draw_rectangle(obj.rect())
        predictions_list = list(zip(labels, obj.output()))
        for i in range(len(predictions_list)):
            confidence = predictions_list[i][1]
            label = predictions_list[i][0]
            print("%s = %f" % (label, confidence))
            if confidence > 0.8:
                if label == "bee":
                    print("It's a BEE")
                    img.draw_string(5, 12, label)
                    flashLED(greenLED)
                if label == "spider":
                    print("It's a SPIDER")
                    img.draw_string(5, 12, label)
                    flashLED(redLED)

    print(clock.fps(), "fps")

How does it work?

  1. The bee, spider and unknow prediction scores are printed through the serial port;
  2. The conditional to activate the detection of a bee or a spider must be greater than 0.8;
  3. If the camera detects a bee, then the green LED lights for 2 seconds and prints the message "It's a BEE" on the serial port; and
  4. If the camera detects a spider, then the red LED lights for 2 seconds and prints the message "It's a SPIDER" on the serial port.

Test

Below I show you an image capture when the camera has detected a bee.

image

Also, below I show you an image capture when the camera has detected a spider.

image

Below I show you the tests carried out with my model created with Edge Impulse and OpenMV. As you can see, I did the tests with images of a bee and a spider printed on a cardboard card.

You don't have permission to edit metadata of this video.
Edit media
x
image
Upload Preview
image

Conclusion:

  1. So far, I am reaching my goals of using artificial intelligence to detect objects.
  2. Working with Machine Learning has its easy and difficult points because there is no formula, but you build your own model and it must have an acceptable prediction rate.
  3. Now that this model works, the next step is to build the water sprayer system.
  • Sign in to reply
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube