Table of Contents
Z7_Color_AID Project Background
Detecting color is important for a lot of applications. In self-driving cars, AI can help detect the traffic signals. For many industry applications, product color recognition can be used to sort products by a robot. PYNQ_BNN can be used for fabric color/texture analysis. There is a paper about color recognition with CNN algorithm[Reg[1]] in 2024. ML can be a good tool for color recognition related applications.
For this design challenge, I am going to explore color recognition from simple projects to PYNQ _BNN FPGA accelerator aided approach. The deadline of the challenge is the end of this week. After a short self-educated PYNQ_BNN training at home in the past month, I think I will need more time to fully understand the deep knowledge about PYNQ_BNN acceleration on FPGA platform and how to prepare datasets for a reliable PYNQ_BNN AI platform. There will be two phases for Z7_Color_AID project. Phase one is a basic project which presenting what I learned on PYNQ platform under OpenCV framework. Phase two is an advanced project which will use PYNQ_BNN/DPU or other suitable algorithm/classifier accelerator on Zynq FPGA platform for color recognition and classification. I don’t think I can finish phase two project by end of this week. So The ML enabled Z7_Color_AID project will be implemented in the near future.
Z7_Color_AID Project Introduction
The initial Z7_Color_AID project idea was defined to recognize multiple colors for color-blind people. After learning more knowledge and features of PYNQ/OpenCV/ML, Z7_Color_AID project specification is evolved. I would like to share my two project phases as below.
Initial Z7_Color_AID project specification: Multiple color detection in real time on PYNQ
I like LEDs, as well as color related projects. OpenCV has over 150 color space conversion methods in OpenCV (Ref [2]). Exploring color space conversion or recognition became my initial idea to join this design challenge.
RGB color space is the default color space. In OpenCV, RGB color space is represented as BGR. This means that the order of the color channel in Blue, Green, and Red. The initial Z7_Color_AID project was defined to use Arty-Z7-20 PYNQ framework to detect an object via a webcam. I planned to use what I learned from OpenCV framework on FPGA platform to extract the color information of the objects and label the color in the real time.
The basic architecture of Z7_Color_AID phase one project is illustrated below.
AMD/Xilinx does have Vitis Vision library for OpenCV. Element14 community provided quite a few blogs on this topic (Ref[3,4]). However, the overall flow of integrating AMD/Xilinx Vitis Vision library or HW accelerator into PYNQ does need manual work. From what I read to accelerate color space manipulation on PYNQ, DPU is mostly needed. I do have strong interests to explore more on this path so that I can use Zynq FPGA as an accelerator for OpenCV related algorithm. I hope I will find time to present another blog of this topic soon.
Updated Z7_Color_AID project specification: Traffic light detection
A good application of color recognition with ML is traffic light detection. I found that Boash Small Traffic Lights dataset (Ref[5]) can be a good candidate of the dataset for the project. There were already some ML projects targeting traffic light detection/color recognition via Yolo_v3 or Yolo_v8 (Ref [6.7]).
I did attend online ML classes this month. I found that I did need more training of algorithms of BNN or YoLo_v3 before I could go through my customized ML training networking on Zynq FPGA. I prefer to use AMD/Xilinx HLS flow to create a customized BNN on the FPGA platform eventually. Implementing DPU on Zynq FPGA can be very challenging from what I learned during this challenge journey. However, I think I did figure out a path to overcome the challenge. I only need more time to learn the flows in order to achieve my goal.
Next Step of the Design Challenge
I will stick with my plan to finish phase one project of Z7_Color_AID on time.
References
[1] Color Recognition in Challenging Lighting Environments: CNN Approach Nizamuddin Maitlo, Nooruddin Noonari, Sajid Ahmed Ghanghro, Sathishkumar Duraisamy, Fayaz Ahmed 2024 IEEE 9th International Conference for Convergence in Technology
[2] OpenCV Document, V4.10.0, https://docs.opencv.org/4.x/db/d64/tutorial_js_colorspaces.html 2024
[3] ZYNQ Stereo Camera Platform - Part2 stereolbm with Vitis Vision libraries, /technologies/fpga-group/b/blog/posts/zynq-stereo-camera-platform---part2-stereolbm-with-vitis-vision-libraries 2020
[4] Learning Xilinx Zynq: Try to make my own Accelerated OpenCV Function - 1: Vitis HLS, /technologies/fpga-group/b/blog/posts/learning-xilinx-zynq-try-to-make-my-own-accelerated-opencv-function---1-vitis-hls 2021
[5] https://github.com/bosch-ros-pkg/bstld 2017
[6] Detecting Traffic Lights in Real-time with YOLOv3. https://github.com/berktepebag/Traffic-light-detection-with-YOLOv3-BOSCH-traffic-light-dataset 2019
[7] traffic-lights-detection-and-color-recognition-using-yolov8 , https://github.com/farukalamai/traffic-lights-detection-and-color-recognition-using-yolov8 2023
Previous Blogs:
Z7_Color_AID-Blog#01:First Impression of PYNQ on Arty-Z7-20