Group Actions
Engagement
  • Author Author: Gough Lui
  • Date Created: Date Created
  • Views 1056 views
  • Likes 14 likes
  • Comments 6 comments
Related
Recommended

BeeWatch: LoRaWAN Environmental Monitoring – Blog 1: README.TXT

Gough Lui

On 17th January 2023, I was given the rather unexpected news that I have been chosen as one of the challengers for the Save the Bees element14 Design Challenge. This is the first design challenge which is not of the experimenting-with type that I had entered and I had made some simple proposals, not expecting to get selected. While the kit has not arrived yet (and I haven’t received a shipping confirmation at this time), it’s probably good to start off this series of blogs with some information about me and what I proposed.

Table of Contents

About Me

My name is Dr. Gough Lui and I am an engineer and electronics hobbyist from Sydney, Australia. image

I currently work as a postdoctoral researcher in the Biomedical Engineering space with Western Sydney University and as a research assistant with Civil and Environmental Engineering in the University of New South Wales. I have an interdisciplinary background, which includes an undergraduate degree commencing in Electrical Engineering, finishing in Solar Photovoltaics Engineering with first-class honours and undertaking a Taste-of-Research in Spatial and Surveying Information Systems. I have a PhD in Civil and Environmental Engineering, specifically in Water Research. In my work, I support the design, assembly, testing and deployment of embedded electronics for applications in health, medicine and humanitarian fields.

Outside of work, I run my own personal blog site at https://goughlui.com where I blog about random things including product reviews, repair logs, computing, networking, retro-technology, radio, television, satellite, random observations and going on holidays. I am a licensed foundation-level radio amateur (VK2FGYL).

I have spent much time over the years at element14 as a Top Member, mostly lurking but occasionally responding to a comment here or there. I have benefited greatly from the RoadTest program which has yielded plenty of useful and interesting equipment for my bench. I have contributed a whopping 33 reviews as of this posting (with a 34th in the works) and have learned plenty from the experience. I have participated in a few Project 14 builds as well although not so much as of late. This is my first time entering a “plain” Design Challenge. Previously, I entered three “experimenting with” style Design Challenges, and was awarded the Grand Prize for the Experimenting with Thermal Switches and Experimenting with Thermistors; and Runner Up Prize for Experimenting with Current Sense Amplifiers.

What About the Bees?

Before going any further, I’ll have to admit that I’m absolutely terrified of insects. As much as bees are vital to the food chain and ecosystem as a critical pollinator, just the mere sound of a bee buzzing around the room is enough to elevate my heart rate and get me into a frenzy – I don’t fancy getting stung and I’m sure the bee would prefer not to risk their life, so we’ve spent many years avoiding each other as much as possible.

But that being said, I remember days from my youth in primary school in the mid-90s when I would go into fields of flowering clovers which were completely abuzz with a blanket of bees. Armed with a water bottle, the lunchtime activity (not officially sanctioned, of course) was to collect as many bees into the bottle as you could. Back then, nobody could have guessed the dire situation that has befallen bees worldwide.

By the mid-2000s, Colony Collapse Disorder (CCD) had become big talk amongst workers in the agriculture industry, especially as many outback families in Australia rely on farming to make their living. Beekeepers were finding their hives quickly moved from being buzzing to being completely dead. Having contended with floods, droughts and fires, this would put another dent into crop yields even though the population of bees worldwide as estimated from honey seem to suggest that the population still grows in spite of CCD. Anecdotally, I have never seen fields of clovers abuzz in the same way that I have as a kid. In suburban areas, bees are now few and far between even though they occasionally visit the garden, you’d have to be looking quite hard to find even one.

The cause of CCD is not conclusively known, but pesticide usage is a leading theory. It makes me feel a little guilty for using insecticide spray cans – especially if used outside, it can get into water and have widespread effects on both land and aquatic creatures. Another is the Varroa mite, a pest which targets bees and transmits diseases including one that causes wing deformities to pupae, which can be ultimately fatal. Being in Australia, the threat of Varroa seemed faraway as we were famous for being Varroa-free. However, as sentinel hives near shipping ports have now detected the presence of Varroa and localised instances have been detected at other hives which have been genetically traced back to this initial importation, it seems our ability to remain Varroa-free may not be for long. This is especially important as the bee industry is a significant part of the Australian economy, being worth AU$14bn annually, producing products such as honey and wax aside from pollinating our crops.

There is no doubt as to the importance of bees in the broader context of the world and despite my phobia of insects, I still want to do something to help. When I saw this challenge, there was some confusion from the contestants as to the time-frame as it may be much too cold in some places for bees to be active. Despite being in the southern hemisphere where it’s still a little warmer, I don’t know of any beekeepers and I still don’t fancy getting stung. So, for this challenge, I’ll be avoiding actual live bees and will instead focus on the technical aspects instead. I suppose that’s for the best – at least, that way I can say that no bees were harmed in the process!

What I Propose to Do

My original proposal was very straightforward – although it didn’t have a name. After some thought, I’ve decided to call it BeeWatch, as a play on Baywatch … the idea is that the system will monitor environmental conditions to help keep bee hives safe. This would include things like temperature, humidity (e.g. via a BME280/BME680 or similar) and particulate matter (e.g. via a SPS30). The monitoring station would be PV/battery-powered (as much as possible) and self-contained, using only LoRaWAN for connectivity thanks to the Arduino MKR WAN 1310 provided.

A stretch-goal would be for me to experiment with the Arduino Pro Nicla Vision to detect noise as a proxy of hive activity, as the board has an onboard microphone. If time permits, I will try to get some image processing going based on being pointed at a monitor showing sample images, but I suspect this may take a lot of time for someone who has no experience with AI computer vision development. It also may not prove so useful, given that bees are agile and fly rapidly, which means images may not be particularly clear (assuming you could lure them in with a flower-analogue and some nectar).

Instead, the project will focus on some of the more electronics-related aspects – things like measuring and optimising the power consumption of the MKR WAN 1310 board, designing a simple power supply set-up for PV power, RF-considerations for LoRaWAN, setting up a LoRaWAN gateway (as I have one on hand) using The Things Stack (Cloud, Discovery Free Tier) and linking that via MQTT to a server for logging of data. If time permits, I’ll try to find a way to visualise this (e.g. via NodeRed), although I suspect that time will run out before this is achieved. I’ll also do some electronics-related measurements on the Nicla Vision board as well – as I’m curious just how efficiently AI vision processing can be done. I’ve heard the board runs hot – we’ll see about that!

To keep things moving, I won’t have the time to design and get PCBs made – the turnaround can easily take 4 weeks which I just can’t afford. Instead, I’ll be doing things the old-fashioned way – with breadboards and Veroboard.

Conclusion

Bees are an important part of the world’s ecosystem and as much as I’m skittish and fear insects, I still would like to help. Even though I proposed a rather simple project revolving primarily around the monitoring of environmental parameters using LoRaWAN, I am also hoping to explore a little into what the Nicla Vision can do. Most of my experiments will focus on the technical, electronics and RF aspects and will not involve the use of live bees, but I hope that it’s helpful to others. Things will have to be done “quick and dirty” given the time-frame involved, which will be quite a bit of fun. It’s been way too long since I’ve whipped out the Veroboard!

I really didn’t expect to be chosen with such a simple proposal, but now that I have been, I hope you will join along with me in exploring the Arduino MKR WAN 1310 and Arduino Pro Nicla Vision for this application.

[[BeeWatch Blog Index]]

  • in reply to Gough Lui

    Indeed, and no-one said any of this would be easy. Slight smile

    One issue with garden flowers is that they only last a relatively short period of time in bloom, so to have continuous blooms of bee-friendly flowers for up to nine months, can take a bit of gardening and bee know-how. If doing a vision project then your cameras potentially need to move every few weeks to follow the activity, which may be occurring in different locations or at different heights.

    Before running processor intensive ML image classification then I suspect that one could aim to use a simpler image processing algorithm to first filter and crop the captured image to a specific blob that potentially could be an insect of interest and thus require to be run through a classifier, significantly reducing the data to be processed but retaining the fine features.

    Automating outdoor photography is often a challenge but there are things people can do to make it more predictable. Including a 18% gray card reference somewhere in the frame and you can automate your exposure and white balance by monitoring those reference pixels as the lighting changes throughout the day. Choosing your subject and camera position and controlling lighting can all assist with improving the chances of getting better image to work with.

    If you have a means to data log locally, then not everything is required to be processed on-device. Some things may benefit from off-loaded processing on a higher-powered machine. I used to do this with some IP camera type projects where the on-device control was set up to trigger on certain live image criteria but the hi-res images captured were stored to local storage, which were then pulled off wirelessly off-peak or via a manual visit and the hi-res image data was then processed off-site. It also depends on the urgency of access to the data however.

    I suspect that until any ML algorithm can be thoroughly put through its paces, then any AI generated data will also require captured evidence to support its accuracy. With that in mind I'd see any AI data being used in the early stages as more of a statistical tool to suggest what to look at first. Instead of looking through 12hours of mostly uninteresting video sequentially, the AI data is more like an index which perhaps lists 12minutes of video where the ML algorithm suggests that something of interest did or didn't happen, which is worth looking at first. Often though you find that the trigger point isn't the most important factor and you need to know what was going on just before or just after that point. Also with tuning the ML algorithm you often need to know what was missed in the AI output data.

    It will be very interesting to see the outcome of vision versus sound in this challenge. Also the outcome of targeting honey bees vs wild bees.

  • Nice plan, I look forward to your build and data.

    Bees are very important, both in pollinating plants, but I am also a Mead connoisseur.

  • in reply to beacon_dave

    I've not had such plants in the garden that are perhaps attractive enough in the long term - hence the suggestion for possibly using a flower-analogue.

    Regardless, even though the Nicla Vision has a 2MP camera, I suspect that for good AI performance, that the images may need to be scaled down which makes it more difficult to see fine features. The lens seems to be fixed focus, which means that the bee will have to be at a relatively consistent distance to get a sharp image. To collect enough images to account for plant changes, weather differences, sun-glare, etc would be a challenge too. The power consumption of a vision-based solution is probably going to be fairly high as well.

    This is perhaps why I'm not as enthused about the possibility that vision brings, especially when one is aware of how limited LoRaWAN communication can be - everything will need to be done on-device (hopefully accurately) and only the result can really be codified into the transmission. As a result, it will be more of a stretch goal for me to get practical vision working - I think audio is perhaps less complicated and potentially more valuable.

    - Gough

  • "...It also may not prove so useful, given that bees are agile and fly rapidly, which means images may not be particularly clear (assuming you could lure them in with a flower-analogue and some nectar)..."

    I've noticed that when I have some bee-friendly Allium Sphaerocephalon growing in the garden, the bees will land on them, crawl over them for a bit then hop over to the next un-occupied flower, crawl over it and so on. So, you could potentially get images something like this: 

    https://community.rspb.org.uk/resized-image/__size/400x600/__key/communityserver-blogs-components-weblogfiles/00-00-00-25-81/7127.bumble-CU-on-allium-sphaero.jpg

    As there are 100's of petals on each head, the bees will spend quite some time working through them all before moving onto the next one, and in this case, you probably get a better chance of getting them from multiple angles than from a single flower they crawl into. So choosing your flowers carefully might help keep a captive audience. Also, apparently different bees have different lengths of tongue, which affects which flowers they can feed from.    

    Caning the stalk to reduce sway in the breeze would help keep things in camera frame and perhaps tying multiple stocks together to get more stability and more petals to fill the frame might help.

    I suspect you could do some classification type stuff but ideally you need to get the thorax, abdomen and tail in view to get all the colour markings for identification. You could also monitor date, time of day, amount of activity, etc.

    The plant changes its appearance over time however as it gets visited by the bees so would likely need to consider this with training images.  

  • Sounds very interesting.  I hope you find a way to help the Bees stay happy and healthy.  They need all the help that we can give them.