Image Processing using the Zynq 7000 Series APSoC

1. Introduction

In the era of digital technology, image processing has become an essential component across various applications, ranging from industrial automation and medical imaging to augmented reality and smart surveillance systems. The need for real-time processing capabilities has driven the development of advanced hardware solutions that can efficiently handle complex image processing tasks.

The Zynq 7000 Series All Programmable System on Chip (APSoC) by Xilinx represents a significant leap in this domain, seamlessly integrating a powerful ARM Cortex-A9 dual-core processor with programmable FPGA (Field-Programmable Gate Array) fabric. This unique architecture allows developers to leverage the strengths of both hardware and software, enabling customized solutions that meet the specific demands of image processing applications.

By harnessing the flexibility and performance of the Zynq 7000 Series, engineers and developers can create sophisticated image processing systems capable of executing high-performance algorithms while maintaining low latency. This blog post will explore the features, capabilities, and practical applications of the Zynq 7000 Series APSoC in the realm of image processing.

2. Key Features of Zynq-7020 for Image/Video Processing

The Zynq-7020 offers a range of impressive features that make it a great choice for image processing applications. Its architecture provides several compelling benefits, making it a powerful platform for both beginners and advanced users.

2.1 Zynq 7020's Processing System (PS)

The Zynq-7020 features a powerful dual-core ARM Cortex-A9 MPCore processor running at up to 866MHz. This processing system is perfect for handling high-level image processing algorithms, user interfaces, and system control. The processor includes NEON SIMD engine extensions and floating-point units per core, which are crucial for accelerating image processing operations like filtering and color space conversions. With 512KB L2 cache and 256KB on-chip memory, it provides fast access to frequently used image data and processing kernels.

2.2 Zynq 7020's Programmable Logic (PL)

The real power for image processing comes from the Artix-7 FPGA fabric on the Zynq 7020 SoC, which provides a robust set of resources tailored for high-performance applications. The Zynq 7020 is equipped with 85K logic cells, serving as the fundamental building blocks for digital circuits. These logic cells enable the implementation of complex logic functions and combinational circuits, allowing developers to create sophisticated processing algorithms that can operate in parallel. This large number of logic cells ensures that the device can handle multiple tasks simultaneously, making it ideal for real-time image processing applications.

Additionally, the FPGA fabric includes 53,200 LUTs and 106,400 flip-flops, which are essential for building complex digital logic circuits. LUTs can be configured to perform various logical operations, while flip-flops provide storage elements for data. This combination allows designers to create intricate parallel processing pipelines that can efficiently handle multiple data streams, such as processing different aspects of an image concurrently, thereby enhancing throughput and reducing latency.

The Zynq 7020 features 4.9Mb of Block RAM, organized into 140 blocks of 36Kb each. This memory is well-suited for storing frame buffers and line buffers, which are critical for image processing tasks that require temporary storage of image data during manipulation. The fast access times of Block RAM enable rapid read and write operations, which are essential for maintaining high-speed processing of video or image data.

Finally, the device includes 220 DSP slices, optimized for performing high-speed mathematical operations. These DSP slices are particularly useful for executing complex algorithms such as convolutions, transforms (like Fourier transforms), and filtering operations that are commonly used in image processing. By leveraging these specialized hardware resources, developers can achieve significantly faster processing times compared to software implementations running on a general-purpose processor, making the Zynq 7020 a powerful platform for demanding image processing applications. Together, these programmable logic resources empower the Zynq 7020 to perform advanced image processing tasks efficiently, providing developers with the tools necessary to implement real-time applications that require high performance and flexibility.

2.3 Zynq 7020's Memory Interfaces

The Zynq-7020 supports a variety of external memory interfaces, including DDR3, DDR3L, DDR2, and LPDDR2, which are crucial for efficiently storing multiple video frames and large image datasets. These memory technologies offer a range of performance and power consumption characteristics, allowing developers to select the most suitable option for their specific application requirements. The ability to interface with these different types of memory ensures that the Zynq 7020 can accommodate a wide range of use cases, from high-bandwidth video processing to low-power embedded systems.

In addition to the memory types, the Processing System (PS) of the Zynq 7020 features 8 DMA (Direct Memory Access) channels, with 4 of these channels dedicated to the Programmable Logic (PL). This DMA capability is essential for enabling efficient data transfer between memory and processing elements without burdening the CPU. By allowing peripherals to access memory directly, DMA significantly reduces latency and increases throughput, which is particularly beneficial in applications that require real-time processing of video and image data. This architecture facilitates smooth and fast data movement, enabling the system to handle high data rates typical in video processing tasks.

For instance, in the context of the Arty Z7 development board, we have 512 MB of DDR3 Memory that is directly connected to the Zynq-7020's PS. This substantial amount of memory provides ample capacity for storing large datasets, such as multiple video frames, which is critical for applications that involve image processing, computer vision, and machine learning. The direct connection to the PS ensures that the memory can be accessed quickly and efficiently, which allows to implement complex algorithms that require significant memory bandwidth.

The combination of flexible memory interface options, dedicated DMA channels, and ample memory capacity makes the Zynq 7020 an ideal choice for high-performance applications that demand robust data handling capabilities.

2.4 Zynq 7020's I/O Capabilities

The Zynq 7020 is equipped with an extensive array of I/O capabilities that facilitate the integration of various imaging sensors, displays, and peripheral devices, making it an ideal choice for diverse embedded applications.

• Up to 200 HR I/O Pins Supporting Various Imaging Sensors and Displays: The Zynq 7020 features a robust set of high-range (HR) I/O pins, allowing for direct interfacing with a wide variety of imaging sensors and display devices. This capability is essential for applications such as camera interfaces, LCD displays, and other peripherals that require high-speed data transfer and real-time interaction. The extensive HR I/O count ensures flexibility in design, enabling developers to connect multiple sensors or devices simultaneously.
• 128 PS I/O Pins for Peripheral Connections: The device includes 128 processing system (PS) I/O pins, which are designed for connecting to external peripherals. These multi-use I/O (MIO) pins support various voltage levels and communication standards, making it easy to interface with components such as memory cards, additional sensors, and user interface devices. This versatility simplifies the integration of multiple peripherals into a single system, enhancing the overall functionality of the application.
• Multiple High-Speed Interfaces: The Zynq 7020 is equipped with several high-speed communication interfaces that facilitate rapid data exchange between the device and external components. These interfaces include 2x USB 2.0, which allow for easy connection to a variety of devices, including storage devices, cameras, and other peripherals. They support both host and device modes, enabling flexible connectivity options for data transfer and device control. Additionally, the 2x Gigabit Ethernet ports provide high-bandwidth connectivity for networked applications, which is particularly beneficial for streaming high-definition video data or connecting to remote servers, making it suitable for applications such as video surveillance and IoT devices. The 2x SD/SDIO interfaces support secure digital (SD) memory cards and other SDIO devices, enabling high-capacity storage options for image data and application files. These ports allow for easy data logging and retrieval, which is crucial for applications that generate large amounts of data. Furthermore, the inclusion of 2x SPI and 2x I2C for Sensor Control allows for efficient communication with various sensors and control devices. SPI (Serial Peripheral Interface) is ideal for high-speed data transfers, while I2C (Inter-Integrated Circuit) is useful for connecting multiple low-speed peripherals with minimal wiring. This flexibility supports a wide range of sensor types, like the PMOD ALS, to gather ambient light data, enhancing the Zynq 7020's capability to interact with diverse sources and improve application performance.

3. Development Workflow

First I'll start off by using the PYNQ Based Design Flow for the configuration of the Arty Z7 for Video Processing Applications Since this is an versatile and more adaptable and flexible for beginners without any knowledge on the SoC FPGA design flows and also is easier to access it without any need for excessive setup(as shown in the below figure) on the Work Station / PC.

What is PYNQ ?

PYNQ (Python Package for Zynq) is an framework that allows users to develop and deploy embedded software applications on Xilinx Zynq-7000 SoCs. It provides a high-level interface to interact with the FPGA, enabling developers to create custom hardware accelerators and interfaces without requiring in-depth knowledge of Verilog or VHDL.

In this section, we'll focus on using PYNQ on the Arty Z7-20 board, which is a popular development platform for the Xilinx Zynq-7000 SoC. We'll explore how to set up PYNQ on the Arty Z7-20 and use it to create custom hardware accelerators for image processing applications.

Setting Up PYNQ on Arty Z7-20

First download the PYNQ v3.0.1 image for PYNQ-Z1

Why downloading the PYNQ-Z1's image ?

Well, the PYNQ Z1 and the ARTY Z7-20 have the same SoC and the hardware connected the SoC on the development board is also the same except for the microphone (of course we are missing a power switch too !  Which I hope may had been included ). So the PYNQ Z1 image is perfectly compatible with the Arty Z7-20.

Then flash the image on to the SD card of 32GB (or greater) capacity.After flashing on the image on the SD card then insert the SD card into the Micro SD card slot present on the bottom of the Arty Z7-20.Then set the JP4 jumper to SD card configuration since we want the board to boot from the SD card.

Then you can follow the steps listed in the PYNQ Z1 setup guide to access the  Jupyter Notebooks hosted on Arty Z7.

Note : Just ignore the instructions regarding the power switch in the PYNQ Z1 setup guide if you're following it to use PYNQ on Arty Z7-20 since the power switch is not present in the Arty Z7-20 and by default if you connect the mirco USB cable to the board the board gets the power through the micro USB port and powers up.

References : The images used above are sourced from ZynqBook