Group Actions
Engagement
  • Author Author: javagoza
  • Date Created: Date Created
  • Views 1882 views
  • Likes 10 likes
  • Comments 4 comments
Related
Recommended

Sensor Fusion for Firefighters. Summary Blog

javagoza
Sensor Fusion for Firefighters. Summary Blog

Sensor Fusion for Firefighters Summary

At the end of October 2022 I was selected to participate in the Experimenting with Sensor Fusion Challenge sponsored by element14 and Xilinx.

Since we received the SP701 evaluation kit, it has been two months of learning many new things and being very satisfied with the results.

Xilinx SP701 Sensor Fusion

It has not been a very hard road, but it has required many hours of work. Not because of the complexity of the solution, but because of the long times waiting for the synthesis of the hardware and the downloads of the development tools and reading the extensive documentation.

The developed system is quite complex, however the tools, libraries, drivers and IPs provided by Xilinx make such a complex solution become a Lego game.

The concept project we have worked on is a device that provides heads up display information for firefighters. It is a display that allows real-time viewing of video, air quality, temperature and relative humidity, a compass for orientation and a low-resolution thermal image.

image

We built the system in stages as we experimented with new sensors and integrated them into the system. We have experimented with the Pcam 5C camera, 5MP fixed focus color camera module, various HDMI displays, the Pmod NAV module, 9-axis IMU plus barometer, the Pmod HYGRO module, digital humidity and temperature sensor, the Pmod CMPS2 Module, 3-Axis Compass, the SparkFun Environmental Combo Breakout Module(CCS811 Equivalent CO2 and Total Volatile Organic Compounds and BME280, Humidity, Temperature and Barometric Pressure) and finally the SparkFun IR Array Breakout: 110 degree FOV, MLX90640, Sensor Array Thermopile 32x24.

We have designed hardware, a heads display block IP, using high level synthesis tools like Vitis HLS, we have also used different IP blocks provided by Xilinx and Digilent within the Vivado Block Designer and we have built other software drivers with Vitis IDE.

We have been documenting all this work in seven blogs published over the past two months. We have tried to make the blogs serve as a tutorial to be able to replicate the system. There is also a github repository with the code.

We were setting up the final stage with a 360 degree thermal panorama when the MLX90640 thermal array sensor got locked up, probably just resetting the EEPROM is enough to unlock it. We will try to recover the module and publish the results later.

The year 2022 has been our first contact with FPGA design and we have been lucky enough to be chosen for two design challenges around the Spartan-7 FPGA which has allowed us to experiment with FPGA-based design for almost 6 months. Enough time to have the necessary base to understand the challenges to face in the design of embedded systems using FPGAs.

We are surprised by the versatility of the Spartan-7 FPGA, its low consumption and the possibilities it offers. The SP701 board makes prototyping solutions based on the Spartan-7 FPGA really easy. To highlight the MIPI interfaces for cameras and displays, the MIPI-CSI Camera Serial Interface and the MIPI-DSI Display Serial Interface as well as the HDMI port. Even more so with the latest versions of Vivado that already include free licenses for the IP blocks that communicate with these interfaces.

The connection with the Digilent Pcam 5C camera is very simple. Xilinx provides a very well documented reference design that has allowed us to adapt to our solution in a very short time.

We have also tinkered with the two Ethernet connections using the dual TI PHY device DP83867IRPAP for Ethernet communications at 10 Mb/s, 100 Mb/s, or 1000 Mb/s. The evaluation kit includes a one-year license of the IP block needed for Ethernet communications including drivers with a TPC/IP stack for developing network applications.

The deadlines for this design challenge have been pretty tight, not because time was short, but cause the learning curve without a mentor can be very steep. It is enough time to make a decent application thanks to the entire ecosystem that supports Xilinx solutions. 

We have had the invaluable help of Adam Taylor's workshops and blogs: https://www.adiuvoengineering.com/microzed-chronicles-archive . We thank him for all the outreach effort he has made both at element14 and at hackster.io

Lastly, thank the entire element14 team for this great learning opportunity.

The Complete "Sensor Fusion for Firefighters" Blog Series

Github repository