Person Detection with yolo #7 Integrated DPU with DNNDK - finally

#7 Integrated DPU with DNNDK - finally

1 DNNDK and DPU

Deep learning Processor Unit is developed by Deephi Tech and acquired by Xilinx in 2018. Then it integrated with Vitis AI for fast development. But New version of Vitis AI may not support Arty Z7, so I roll back to previous version of toolset to complete this  blog.  Early version of DPU need DNNDK to bridge the gap of information sharing, so DNNDK shall be installed as well.

2 Getting start with Vivado 2018.3 and create the project

Install previous Vivado 2018.3 

Select Zynq 7000 

wait for completion.

create one vivado project and add DPU IP as design resouces,

Now it added in the project

source tcl file to create block design with DPU,

here is the dpu in the block design

After check the design , generate bitstream for hardware and exported hardware design file

Then configurate the hardware design files and create petalinux project with petalinux 2018.3, which is compatible with older version of ubuntu .

The petalinux project is built and export to SD image file for DPU to work.

3  Getting start to Pynq 2.5 with DPU 

Insert the SD card and booting the Arty Z7, open putty terminal to supervise the startup process, 

DPU is  loaded 

check the ipconfig

By then, the ether0 is opened for putty SSH to access, first time authentication, for 192.168.1.99

login with xilinx account with password xilinx,

here is the pynq directory,

4 Running the DPU

open the port on browser, login with xilinx

running the ipython notebook with resnet50

the image classification with resnet50 is running well.

With other model like yolo, the model can be built with vitis AI Compiler into binary elf file. The VItis AI compiler can use tensorflow or pytorch, the python code is similar.

Here is the source file for ipython.

img_path = '/home/xilinx/val5000/ILSVRC2012_val_00000020.JPEG'
img = plt.imread(img_path)
plt.imshow(img)
img = cv2.imread(img_path)

KERNEL_CONV = "resnet50_0"
KERNEL_CONV_INPUT = "conv1"
KERNEL_FC_OUTPUT = "fc1000"

# Attach to DPU driver and prepare for runing
n2cube.dpuOpen()
# Create DPU Kernels for ResNet50
kernel = n2cube.dpuLoadKernel(KERNEL_CONV)
# Create DPU Tasks from DPU Kernel
task = n2cube.dpuCreateTask(kernel, 0)
# Get the output tensor channel from FC output
channel = n2cube.dpuGetOutputTensorChannel(task, KERNEL_FC_OUTPUT)

FCResult = [0 for i in range(channel)]
mean = [104,107,123]


# Load image to DPU
dputils.dpuSetInputImage(task, KERNEL_CONV_INPUT, img, mean)
# Model run on DPU
n2cube.dpuRunTask(task)
# Get the output from FC output
n2cube.dpuGetOutputTensorInHWCFP32(task, KERNEL_FC_OUTPUT, FCResult, channel)
# Get the label
label = FCResult.index(max(FCResult))

print(label, wordlist[label])

rtn = n2cube.dpuDestroyKernel(kernel)
n2cube.dpuClose()

The crane is detected 

5 Summary

This project show how to deploy the Eye intelligence project in Arty Z7 with DPU. Previous blog show how to detect person with opencv, which is common choice for any edge equipment, but not show the capability of FPGA.

Further work shall dig how to use vitis AI in this Arty Z7.