Creating a Custom Kria App - Part 4

This blog is part of a series of blogs which describe the development steps for an in-depth project tutorial.

• http://avnet.me/kv260-vvas-sms-2021-1-blog

This part of the tutorial will re-compile the sixteen (16) supported models used by the smart_model_select application for our targeted DPU architecture (B3136).

Understanding the 16 supported models

The 16 models used in the smart_model_select example are all part of the Xilinx Model Zoo.

The type column refers to the smart_model_select application, and corresponds to the video files name used with each model.

The “compiled model name” corresponds to the name specified during compilation.

The “Xilinx Model Zoo name” corresponds to the name used in the Xilinx Model Zoo.

The Xilinx Model Zoo is part of the Vitis-AI repository, which we cloned in part 1:

Vitis-AI/models/AI-Model-Zoo

Understanding the Xilinx Model Zoo 

The Xilinx Model Zoo contains over 100 models, which are defined in the following sub-directory:

Vitis-AI/models/AI-Model-Zoo/model-list/{modelzoo_name}

The Xilinx Model Zoo name uses the following naming convention:

F_M_(D)_H_W_(P)_C_V

• F specifies training framework:
  cf is Caffe, tf is Tensorflow, tf2 is Tensorflow 2, dk is Darknet, pt is PyTorch
• M specifies the model
• D specifies the dataset.
  It is optional depending on whether the dataset is public or private
• H specifies the height of input data
• W specifies the width of input data
• P specifies the pruning ratio, it means how much computation is reduced.
  It is optional depending on whether the model is pruned or not
• C specifies the computation of the model: how many Gops per image
• V specifies the version of Vitis AI

For example, cf_refinedet_coco_360_480_0.8_25G_1.4 is a RefineDet model trained with Caffe using COCO dataset, input data size is 360*480, 80% pruned, the computation per image is 25Gops and Vitis AI version is 1.4.

The source files and pre-compiled models are not provided in the Xilinx Model Zoo’s github repository.

Instead, a yaml file is provided, which contains the links that can be used to download the content for each model.

As an example, for the cf_refinedet_coco_360_480_0.8_25G_1.4 model, the model.yaml file specified the following:

...

description: refinedet pedestrian detector.

input size: 360*480

float ops: 5.08G

task: detection

framework: caffe

prune: '0.96'

version: 1.4

files:

- name: cf_refinedet_coco_360_480_0.96_5.08G_1.4

  type: float & quantized

  board: GPU

  download link: https://www.xilinx.com/bin/public/openDownload?filename=cf_refinedet_coco_360_480_0.96_5.08G_1.4.zip

  checksum: e9120a3ad8994cad38339fee6d856188

- name: refinedet_pruned_0_96

  type: xmodel

  board: zcu102 & zcu104 & kv260

  download link: https://www.xilinx.com/bin/public/openDownload?filename=refinedet_pruned_0_96-zcu102_zcu104_kv260-r1.4.0.tar.gz

  checksum: 6a525d07db96f25776c4057127bf28be

...

Several archives are specified, including the following two:

• 96_5.08G_1.4.zip
  • this file contains the source with which the model can be compiled for any DPU architecture
• refinedet_pruned_0_96-zcu102_zcu104_kv260-r1.4.0.tar.gz
  • this file contains pre-compiled models for use with ZCU102, ZCU104, and KV260

NOTE : the pre-compiled models for KV260 are meant to be used with the “benchmark” overlay we saw earlier, which has a B4096 DPU architecture.  Since our design has a B3136 DPU architecture, we cannot use these pre-compiles models in our design.

Compiling the models

In order to facilitate the model compilation process, we will use the following script:

• https://github.com/Avnet/vitis/blob/2021.1/app/zoo/compile_modelzoo.sh 

This script will automatically scan the model-list directory, interpret each model.yaml file, and compile the model accordingly.  A lot of time went into the creation of this compilation script, and will save considerable time during the model compilation process.

Download the compile_modelzoo.sh script and copy to the following location:

Vitis-AI/models/AI-Model-Zoo

Edit the compilation script (compile_modelzoo.sh) as follows:

ARCH=arch.json

TARGET=models.b3136

CACHE=cache

This will specify to use the “arch.json” file for the DPU architecture specification, as well as the directory names for the cache (downloaded archives) and models (model compilation output).

We need to copy our arch.json as follows:

$ cd $PROJ_DIR

$ cp VVAS/ivas-examples/Embedded/smart_model_select/binary_container_1/sd_card/arch.json  Vitis-AI/models/AI-Model-Zoo/.

We also need to create the two directories we specified previously:

$ cd $PROJ_DIR/Vitis-AI/models/AI-Model-Zoo

$ mkdir -p cache

$ mkdir -p models.b3136

Since the compilation script will scan the model-list, we need to create a version of the model-list that only contains the models that we want to compile:

$ mv model-list model-list-backup

$ mkdir model-list

$ cd model-list

$ cp –r ../model-list-backup/cf_densebox_wider_320_320_0.49G_1.4 .

$ cp –r ../model-list-backup/cf_densebox_wider_360_640_1.11G_1.4 .

$ cp –r ../model-list-backup/cf_inceptionv1_imagenet_224_224_3.16G_1.4 .

$ cp –r ../model-list-backup/cf_mobilenetv2_imagenet_224_224_0.59G_1.4 .

$ cp –r ../model-list-backup/cf_refinedet_coco_360_480_0.96_5.08G_1.4 .

$ cp –r ../model-list-backup/cf_resnet18_imagenet_224_224_3.65G_1.4 .

$ cp –r ../model-list-backup/cf_resnet50_imagenet_224_224_7.7G_1.4 .

$ cp –r ../model-list-backup/cf_ssdadas_bdd_360_480_0.95_6.3G_1.4 .

$ cp –r ../model-list-backup/cf_ssdmobilenetv2_bdd_360_480_6.57G_1.4 .

$ cp –r ../model-list-backup/cf_ssdpedestrian_coco_360_640_0.97_5.9G_1.4 .

$ cp –r ../model-list-backup/cf_ssdtraffic_360_480_0.9_11.6G_1.4 .

$ cp –r ../model-list-backup/dk_tiny-yolov3_416_416_5.46G_1.4 .

$ cp –r ../model-list-backup/dk_yolov2_voc_448_448_0.77_7.82G_1.4 .

$ cp –r ../model-list-backup/dk_yolov2_voc_448_448_34G_1.4 .

$ cp –r ../model-list-backup/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4 .

$ cp –r ../model-list-backup/tf_yolov3_voc_416_416_65.63G_1.4 .

$ cd ..

Xilinx provides a docker container with contains the tools required for model compilation.  Launch the vitis-AI 1.4 docker container as follows:

$ cd $PROJ_DIR/Vitis-AI

$ ./docker_run.sh xilinx/vitis-ai:1.4.916

Launch compilation script

> cd models/AI-Model-Zoo

> source ./compile_modelzoo.sh

> exit

$

Notice during compilation that the DPU architecture is reported as DPUCZDX8G_ISA0_B3136_MAX_BG2.

Models will be in the following location

Vitis-AI/models/AI-Model-Zoo/models.b3136

Let’s copy this models.b3136 directory to the smart_model_select directory:

$ cd $PROJ_DIR

$ cp -r Vitis-AI/models/AI-Model-Zoo/models.b3136 VVAS/ivas-examples/Embedded/smart_model_select/.

The smart_model_select also provides additional .json files that contain the labels for the detection models.  These must be copied to the models sub-directories, as follows:

$ cd $PROJ_DIR/VVAS/ivas-examples/Embedded/smart_model_select

$ cp src/jsons/label_ssd_adas_pruned_0_95.json  models.b3136/ssd_adas_pruned_0_95/label.json

$ cp src/jsons/label_ssd_mobilenet_v2.json  models.b3136/ssd_mobilenet_v2/label.json

$ cp src/jsons/label_ssd_pedestrian_pruned_0_97.json  models.b3136/ssd_pedestrian_pruned_0_97/label.json

$ cp src/jsons/label_ssd_traffic_pruned_0_9.json  models.b3136/ssd_traffic_pruned_0_9/label.json

$ cp src/jsons/label_tiny_yolov3_vmss.json  models.b3136/tiny_yolov3_vmss/label.json

$ cp src/jsons/label_yolov2_voc.json  models.b3136/yolov2_voc/label.json

$ cp src/jsons/label_yolov2_voc_pruned_0_77.json  models.b3136/yolov2_voc_pruned_0_77/label.json

$ cp src/jsons/label_yolov3_adas_pruned_0_9.json  models.b3136/yolov3_adas_pruned_0_9/label.json

$ cp src/jsons/label_yolov3_voc_tf.json  models.b3136/yolov3_voc_tf/label.json

Next Steps

The following blogs cover the previous development steps for this in-depth project tutorial.

• http://avnet.me/kv260-vvas-sms-2021-1-part1
• http://avnet.me/kv260-vvas-sms-2021-1-part2
• http://avnet.me/kv260-vvas-sms-2021-1-part3

The following blogs will cover the remaining development steps for this in-depth project tutorial.

• http://avnet.me/kv260-vvas-sms-2021-1-part5
• http://avnet.me/kv260-vvas-sms-2021-1-part6
• http://avnet.me/kv260-vvas-sms-2021-1-part7