Intro
This project has been a whole education into bees and and a real highlight is that the other contestants have been a great source of knowledge on the subject. I love it when a project results in this much new knowledge. My project was ambitiously focused on two different but complete systems, neither of which I had experience with, so it was always going to be a learning experience on the technical front as well. One of those systems involved doing something that nobody else in the world seems to be doing - sending images over LoRa, so that was going to be a self-taught learning experience.
System 1 consisted of 2 nodes, a base station to monitor results and a field node to monitor and report insect activity using machine vision to recognize different insects. Both nodes featured voice synthesis and color LCDs to keep the blogs entertaining. Because some of the technical details of these systems can be a pretty dry and pedantic. Here is a block diagram of System 1 and an image of the hardware:
Here is the video demonstrating System 1 working (replay from a previous blog):
System 2 also consisted of 2 nodes, a similar base station to monitor images and a field node to capture images and send them back to base. This one captured my imagination because it is so unusual to try and send images through a low bandwidth channel. Here is a block diagram of that system and an image of the hardware:
Here is the video demonstrating System 2 working (replay from a previous blog):
System Objectives
System 1 was intended to explore applications of interest to farmers, such as understanding bee activity in their crops, by providing such info as how many bees are in the area and when are they active. The system allows remote monitoring and automated data collection. Hopefully this type of data provides insight into how much ambient conditions affect bee activity.
System 2 was intended to explore applications of interest to bee keepers, such as beehive health and presence of parasites without needing to physically spend as much time at the hives.
Both systems were demonstrated to be technically feasible at reasonable cost.
Project Achievements
This project turned out very well in creating working systems that showcase how technology can be used to help bees and the people concerned with bees. My particular objectives are always to make my designs work and I am really stoked that both the machine vision and LoRa image transmission worked so well, especially since both concepts were new to me. It has taken 9 blogs and 13 videos to explain all the work that was done, and they generated over 22,000 views so far, but that is really only the tip of the iceberg. My project video directory alone has 89 videos in it. This project seemed to go in spurts of accomplishment interleaved with periods of researching, learning, pondering and planning.
I always like to design PCBs for my projects and although they play havoc with schedules, it is a real accomplishment to get them working. They definitely add to the workload - I had to build 2 breadboard systems to get the project started while waiting for PCBs to arrive. In this case I designed a single PCB to be reconfigurable for all four subsystems. This is much trickier than designing 4 separate PCBs since there are a lot more interdependencies and a lot more to go wrong. The card can support 3 different cameras, 3 different LCDs, 2 different MCUs, 2 different speech synthesizers, an audio amp, and even several headers for future expansion. And all of those interfaces work. Any project with a custom PCB that works properly is a big plus in my personal scoresheet. In this case using 4 PCBs in one project is just awesome.
I also enjoy mechanical design, actually I enjoy almost any kind if design work, but on the mechanical front this project used eleven custom 3D printed parts, which represents a significant amount of effort (not mention printing time), but like I indicated, I enjoy it.
On the software side, I like designing algorithms, but struggle with language syntax and compiler constraints. However I did manage to publish six firmware programs for this project, and I wrote a blog about how to process images for those interested in displaying images on small LCDs.
So from a design perspective I got do electronic design, mechanical design, software design and system design. And because there were four subsystems, there was lots of design work and I also got to exercise creativity in using replica insects, and adding speech to insects, I even tried a few new things on the videography front. That always takes more time than expected. Hopefully it injected some measure of multimedia entertainment to the blogs.
From a maker perspective, building more than one system was a bit of a stretch, but it worked out. It was made a lot easier that the PCB made everything electrically plug-and-play and the 3D printer made mechanical assembly easy.
One of the key objectives in element14 projects is forum participation, in this case there is a leaderboard to provide some indication of participation. Good performance there speaks for itself.
Save The Bees Participation Leaderboard
What didn't go so well?
Well getting so sick during the last critical weeks of the challenge was a near catastrophe. I want to talk a bit about the Nicla Vision which I had bigger plans for. Although it wasn't necessary for my project, it could have done the job of the HuskyLens in a much smaller form factor. I did quite a bit of work on the Nicla Vision, including building it right onto the custom PCB. Sadly the big push to implement a Nicla solution was planned to occur after the 2 main systems were fully functional, but I got hijacked by illness. That schedule was partly because I see it as a bigger risk for me at my level to make it do something impressive with a Nicla. I think it packs a phenomenal amount of power into an impressively small form factor, including computing horsepower, camera and sensor suite. Ultimately my Nicla is destined to be used in an application where its strengths are needed, but I had planned to try it out on my "bees". I did some research on ESLOV and actually bought some ESLOV cables because the MKR1310 also has an ESLOV connector.
The ESLOV channel seems to have been placed pretty squarely on the Arduino back burner. I am curious as to why - it made a bit of a splash when it was introduced but all has gone quiet. They are still putting it on modules, so are there plans for it or are they waiting for users to find its potential? I'm rambling but I had expected to find some ESLOV activity ...anyway my progress is still at the "learning the examples" stage. I didn't have enough gas in the tank to implement something impressive after getting my primary systems working with one day to go before the challenge deadline. My Nicla will definitely find a good use, although I'm not convinced the MKR was a great pairing for it. The 2 modules are worlds apart in functional density and the Nicla already has a radio that shouldn't be wasted. I do have the PCB already to go with LCD and the Nicla footprint is in my CAD library. The Nicla series in general are very interesting - the Nicla Voice in particular looks phenomenal.
One other stretch goal I had was to package a field node system in a waterproof enclosure with a local solar assisted power supply. Although I bought some suitable enclosures, this was abandoned early when I decided to use replica insects. I don't consider this aspect of the project to be very innovative, or informative so I didn't mind eliminating it.
What could be done to improve the project?
The technologies in this project are just taken to a working proof-of-concept demonstration stage, if these technology demonstrations prove interesting:
- the LoRa communications software should have error checking
- the LoRa network could be expanded to have more nodes
- a database could be added to log all the info received
- weather data should be recorded and correlated with bee activity
- the field nodes should be packaged in suitable housings with suitable power supply systems
Wrap-up Summary
Other than getting sick, this project was a blast. it had ambitious objectives and multiple systems, I learned a lot about bees and machine learning, got to do lots of design, exercise lots of creativity, overcame a lot of problems and in the end everything worked. I put a lot of effort into making interesting blogs, by learning and using more colorful multimedia techniques, which should help to improve my blogs going forward.
The system was successful in visually recognizing bees and other insects and it was successful in transmitting this information back to base via LoRa. It was successful in providing each insect with a voice and displaying its image on an LCD. And it was successful in transmitting images via LoRa, which is a really unique capability that does not seem to have ever been done before.
Getting sick was a major bummer. It wasn't just hard on me and my projects, it was hard on everyone around me, especially my wife. But I have good people around me who are pulling for a recovery. Thanks to the members here who are also supportive.
As mentioned above, this project achieved its objectives in creating working systems that showcase how technology can be used to help bees and the people most concerned with bees.
This project stretched me in new directions and tested my resolve, and even though it was a wild ride, I am glad I tackled it.
Links:
Save the Bees - Machine Learning
Image Conversion to Integer Array for LCD Display
