Person Detection with yolo #3 Vitis AI and DPU behind Arty Z7

#3 Vitis AI and DPU behind Arty Z7

1 DPU behind Arty Z7

Vitis AI is Integrated Development Environment that fasilating ML development . This is diagram for Vitis AI 3.5, the Arty Z7 is missing from the diagram. According to feedback from AMD community, the Zynq-7000 is not offcially supported since Vitis 1.4. While the Arty Z7 is still applicable as costuminzed board, just like Pynq-Z1 board.

Vitis AI is make developer's  work easy,  using high-level programming tools like tensorflow in python to drive AI applications.

DPU, which is core of Vitis AI, make deploying AI application in Arty Z7 quick and easy.

2 Vitis AI to DPU

Deep Learning Processing Units ( DPUs) is the FPGA binary accelerator for Machine Learning. As of version 3.5 different DPU support different hardware platform

Vitis AI stack support DPU with stack and samples

Another feature of Vitis AI support prune dense model into sparse model with smaller parameters for faster computation with minimum performance lose

DPU is propriety IP imported in vivado IDE, then export the hardware files for petalinux to create image files.

The problem for Using dpu in Arty Z7 is that Zynq-7000 is not officially supported since version 1.4. Here is the screenshot from Vitis AI User Guide v1.4

Porting DPU in new board is obviously difficult for normal developers with limit technical support. And Arty Z7 can support DPU with right version

3 Go deep into DPU 

DPU varies in size and paremeters with name conventions

DPU with ZD in third part support Zynq DDR is minimum size DPU can fit for the LUTs for Zynq-7000. It work like Accelerators

 refer to Introduction • DPUCZDX8G for Zynq UltraScale+ MPSoCs Product Guide (PG338) • Reader • AMD Technical Information Portal

DPU IP IP is available in multiple variants, scaling both in terms of logic resource utilization and parallelism.  Configurations include: B512, B800, B1024, B1152, B1600, B2304, B3136, and B4096, where the nomenclature indicates the total number of MACs per DPU clock cycle.

The DPU work with Vitis AI stack in the following flow

In vivado design block, the position of DPU can be

one example is like

Other DPU runs with more cores and more data throughout fit for other hardware platform.

4 Model for DPU 

The dpu is accelerator and only support normal CNN operators. The following list highlights key supported operators for the DPUCZDX8G :

• Supports both Convolution and transposed convolution
• Depthwise convolution and depthwise transposed convolution
• Max pooling
• Average pooling
• ReLU, ReLU6, Leaky ReLU, Hard Sigmoid, and Hard Swish
• Elementwise-sum and Elementwise-multiply
• Dilation
• Reorg
• Correlation 1D and 2D
• Argmax and Max along channel dimension
• Fully connected layer
• Softmax
• Concat, Batch Normalization

That is enough for most ML models. In Vitis AI Library User Guide,  many model is supported

For classification, object detection and more complex model. There model can be got from AMD model zoo. If customized model is needed, the design flow for compile model is available. .The model is the compiled in .xmodel format for loading into the program.

5 Summary 

This blog brief the algorithm and design flow behind the machine learning for Zynq-7000 SoC platform. Especially how DPU is build and operated. It can help understand how programming language works, such as C++ or python.

The python environment PYNQ is open sourced, and it is high level synthesis over FPGA features including DPUs, very good platform for ML projects.