Table of Contents
- Introduction
- Getting Started
- Edge Impulse
- Improving Edge Impulse Model
- Testing The Machine Learning Model With OpenMV
- Adding The Water Sprayer System
- Testing The Water Sprayer System
- IoT Ambient Monitoring System | Part1
- IoT Ambient Monitoring System | Part2
- Summary
**********************************************************************************************************************
In this section I will show you the programming of the Nicla Vision, and the MKR WAN 1310 in the Water Sprayer system. We will also test the prototype.
Programming The Nicla Vision
Below I show you the code for the Nicla Vision board:
# AUTHOR: GUILLERMO PEREZ GUILLEN
import sensor, image, time, os, tf, pyb
from machine import Pin
pin0 = Pin("PG1", Pin.OUT_PP, Pin.PULL_UP)
pin1 = Pin("PG12", Pin.OUT_PP, Pin.PULL_UP)
redLED = pyb.LED(1) # built-in red LED
greenLED = pyb.LED(2) # built-in green LED
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.set_vflip(True)
sensor.set_hmirror(True)
sensor.set_windowing((240, 240)) # Set 240x240 window.
sensor.skip_frames(time=2000) # Let the camera adjust.
labels, net = tf.load_builtin_model('bee_or_spider_v2')
found = False
def flashLED1(led): # Indicate with LED when target is detected
found = True
led.on()
pin0.on()
img.draw_string(5, 12, label)
pyb.delay(1500)
led.off()
pin0.off()
found = False
def flashLED2(led): # Indicate with LED when target is detected
found = True
led.on()
pin1.on()
img.draw_string(5, 12, label)
pyb.delay(1500)
led.off()
pin1.off()
found = False
clock = time.clock()
while not found:
clock.tick()
img = sensor.snapshot()
for obj in tf.classify(net, img, min_scale=1.0, scale_mul=0.8, x_overlap=0.5, y_overlap=0.5):
print("**********nPredictions at [x=%d,y=%d,w=%d,h=%d]" % obj.rect())
img.draw_rectangle(obj.rect())
predictions_list = list(zip(labels, obj.output()))
for i in range(len(predictions_list)):
confidence = predictions_list[i][1]
label = predictions_list[i][0]
print("%s = %f" % (label, confidence))
if confidence > 0.8:
if label == "bee":
print("It's a BEE")
#img.draw_string(5, 12, label)
flashLED1(greenLED)
#pin0.on()
if label == "spider":
print("It's a SPIDER")
#img.draw_string(5, 12, label)
flashLED2(redLED)
#pin1.on()
print(clock.fps(), "fps")
How does it work?
- This code is similar to the code from chapter 5. The bee, spider and unknow prediction scores are printed through the serial port;
- The conditional to activate the detection of a bee or a spider must be greater than 0.8;
- If the camera detects a bee, then the green LED lights for 2 seconds and prints the message "It's a BEE" on the serial port;
- Additionally, pin0 (PG1 port) turns ON for 1.5 seconds and then turns OFF;
- If the camera detects a spider, then the red LED lights for 2 seconds and prints the message "It's a SPIDER" on the serial port; and
- Additionally, pin1 (PG12 port) turns ON for 1.5 seconds and then turns OFF;
Troubleshooting and solution:
To make this script with MicroPython, I faced the problem of programming the output ports of the Nicla Vision since there is not enough information. Fortunately in the OpenMV discussion forum I found the solution: https://forums.openmv.io/t/controlling-gpio-pins-of-arduino-nicla-vision-board-using-openmv-ide/7533
Programming The MKR WAN 1310
Below I show you the code to upload to the MKR WAN 1310 board, but you can use any arduino board.
/*
AUTHOR: GUILLERMO PEREZ GUILLEN
*/
// We include the library to be able to control the servo
#include <Servo.h>
int Pin1 = 4;
int Pin2 = 5;
// We declare the variable to control the servo
Servo servoMotor;
int bee = 0;
int spider = 0;
void setup() {
// We start the serial monitor to display the result
Serial.begin(9600);
// We start the servo so that it starts working with pin 6
servoMotor.attach(6);
pinMode(Pin1,INPUT);
pinMode(Pin2,INPUT);
// initialize the servo
servoMotor.write(100);
Serial.println("ITS BEE OR SPIDER?");
}
void loop() {
if (digitalRead(Pin1)==HIGH){
Serial.println("It's a BEE");
bee++;
Serial.print("BEE = ");
Serial.println(bee);
Serial.println("--------------------");
delay(1500);
}
else if (digitalRead(Pin2)==HIGH){
Serial.println("It's a SPIDER");
spider++;
Serial.print("SPIDER = ");
Serial.println(spider);
Serial.println("--------------------");
servoMotor.write(175);
delay(1500);
servoMotor.write(100);
}
else {
delay(200);
}
}
How does it work?
- Pin1 (D4) is used to detect the bees, so a counter keeps track of the bees and prints it on the screen each time it is updated;
- Pin2 (D5) is used to detect the spiders, and in the same way a counter keeps track of the spiders and prints it on the screen each time it is updated;
- Additionally, the servo (D6) is activated to press the water sprayer for 1500 milliseconds and then release it.
Test
Below I show you a screenshot during the test carried out with this interesting prototype. You can appreciate the bee and spider counts displayed on the serial port.
And a zoom to the water spray bottle when it emits a waterjet.
Below I show you the video during the tests carried out.
I also show you a video capture on PC so you can see what is happening in detail.
Conclusion
- For me the most important thing is to verify that this works before taking it to the next level, and it's satisfying for me to see that this prototype works;
- This project has potential applications, because it not only predicts something, but also acts, and that is difficult, at least for me; and
- The architecture of the MKR WAN 1310 board is made for other things, so in the final part of this project I will try to use it in a different way.
