Group Actions
Engagement
  • Author Author: albertabeef
  • Date Created: Date Created
  • Views 4614 views
  • Likes 2 likes
  • Comments 0 comments
Related
Recommended

Creating a Custom Kria App - Part 5

albertabeef
Creating a Custom Kria App - Part 5

This blog is part of a series of blogs which describe the development steps for an in-depth project tutorial.

This part of the tutorial describes how to build the petalinux project for the Kria KV260 Vision AI Starter Kit. 

Also, we will create the yocto recipes to create the packages needed for our custom Kria app.

image

 

Building the petalinux project

The petalinux project is based on the official KV260 petalinux BSP from Xilinx:

Download this bsp to same directory where the github repositories were cloned.

Use the following commands to create and configure the petalinux project:

$ cd $PROJ_DIR

$ petalinux-create –t project –s xilinx-k26-starterkit-v2021.1-final.bsp

$ cd xilinx-k26-starterkit-2021.1

$ petalinux-config --silentconfig

 

Make the following change to the petalinuxbsp.conf file, specifically for the KV260 board.

$ echo 'BOARD_VARIANT = "kv"' >>  project-spec/meta-user/conf/petalinuxbsp.conf

 

Use the following command to build the petalinux project:

$ petalinux-build

 

Use the following command to create the SD card image:

$ petalinux-package --wic --bootfiles "ramdisk.cpio.gz.u-boot boot.scr Image system.dtb"

$ gzip images/linux/petalinux-sdimage.wic

 

This will create the following image, without our custom Kria app, which can be programmed using Balena Etcher:

xilinx-k26-starterkit-2021.1/images/linux/petalinux-sdimage.wic.gz

 

Let’s keep a copy for reference:

$ mv images/linux/petalinux-sdimage.wic.gz ../kv260-2021-1-sdimage.wic.gz

 

Creating the custom Kria app

Using the petalinux project created in the previous section, we can create the custom Kria app.  This will consist of four packages:

  1. firmware package (kv260-vvas-sms): containing the bitstream, .xclbin, and device tree content our custom Kria app
  2. models package (kv260-vvas-sms-models): containing the pre-compiled models for the smart_model_select application
  3. app package (kv260-vvas-sms-app): containing the smart_model_select application
  4. packagegroup (packagegroup-kv260-vvas-sms): containing all the first level dependencies required for our custom Kria app

 

packagegroup

rpm packages

first level dependencies

packagegroup-kv260-vvas-sms

ap1302-ar1335-single-firmware

kv260-vvas-sms

kv260-vvas-sms-models

kv260-vvas-sms-app

git

dnf

libdrm, libdrm-tests, libdrm-kms

xrt, xrt-dev

zocl

opencl-headers

packagegroup-petalinux-vitisai

packagegroup-petalinux-vitisai-dev

packagegroup-petalinux-gstreamer

packagegroup-petalinux-opencv

packagegroup-petalinux-v4lutils

packagegroup-petalinux-x11

gstreamer-vcu-examples

htop

jansson

target-factory

unilog vart, vitis-ai-library, xir

ivas-accel-libs, ivas-gst, ivas-utils

 

 

 

Creating the firmware package

The firmware package contains the content used by the XMUTIL utility and dynamic function exchange (DFX-MGR) service.

This content will be placed in the “/lib/firmware/xilinx/kv260-vvas-sms”, as required by the DFX-MGR service:

/lib/firmware/xilinx/kv260-vvas-sms

├── kv260-vvas-sms.bit.bin

├── kv260-vvas-sms.dtbo

├── kv260-vvas-sms.xclbin

└── shell.json

 

In the kv260-firmware directory that we cloned in Part 1, we will create the following sub-directory structure for the firmware files of our custom app:

kv260-firmware

└── vvas-sms

    ├── kv260-vvas-sms.bit

    ├── kv260-vvas-sms.dtsi

    ├── kv260-vvas-sms.xclbin

    └── shell.json

 

The following commands will create the above directory structure:

$ cd $PROJ_DIR/kv260-firmware

$ mkdir -p vvas-sms

$ cp ../VVAS/ivas-examples/Embedded/smart_model_select/binary_container_1/sd_card/kv260_ispMipiRx_vcu_DP_wrapper.bit vvas-sms/kv260-vvas-sms.bit

$ cp ../VVAS/ivas-examples/Embedded/smart_model_select/binary_container_1/sd_card/dpu.xclbin vvas-sms/kv260-vvas-sms.xclbin

$ cp smartcam/kv260-smartcam.dtsi vvas-sms/kv260-vvas-sms.dtsi

$ cp smartcam/shell.json vvas-sms/shell.json

 

Edit the kv260-vvas-sms.dtsi file as follows:

// SPDX-License-Identifier: GPL-2.0

/*

 * dts file for Xilinx KV260 VVAS smart_model_select

 *

 * (C) Copyright 2022, Avnet, Inc.

 *

 */

 

/dts-v1/;

/plugin/;

 

&fpga_full {

     #address-cells = <2>;

     #size-cells = <2>;

     firmware-name = "kv260-vvas-sms.bit.bin";

     resets = <&zynqmp_reset 116>, <&zynqmp_reset 117>, <&zynqmp_reset 118>, <&zynqmp_reset 119>;

};

...

 

Create the custom firmware with petalinux-create, using the “--t apps” option:

$ cd $PROJ_DIR/xilinx-k26-starterkit-2021.1

$ petalinux-create -t apps --template fpgamanager -n kv260-vvas-sms --enable --srcuri "../kv260-firmware/vvas-sms/kv260-vvas-sms.bit ../kv260-firmware/vvas-sms/kv260-vvas-sms.dtsi ../kv260-firmware/vvas-sms/kv260-vvas-sms.xclbin ../kv260-firmware/vvas-sms/shell.json" --force

 

This will create the yocto recipe for our custom firmware in the following location:

xilinx-k26-starterkit-2021.1/project-spec/meta-user/recipes-apps/kv260-vvas-sms

├── files

│   ├── kv260-vvas-sms.bit

│   ├── kv260-vvas-sms.dtsi

│   ├── kv260-vvas-sms.xclbin

│   └── shell.json

├── kv260-vvas-sms.bb

└── README

 

 

Creating the models package

This content will be placed in the “/opt/<vendor>/<package>” location, as defined for add-on application software packages:

The models will be installed at the following location:

/opt/avnet/kv260-vvas-sms/models.b3136

|-- densebox_320_320

|-- densebox_640_360

|-- inception_v1

|-- mobilenet_v2

|-- refinedet_pruned_0_96

|-- resnet18

|-- resnet50

|-- ssd_adas_pruned_0_95

|-- ssd_mobilenet_v2

|-- ssd_pedestrian_pruned_0_97

|-- ssd_traffic_pruned_0_9

|-- tiny_yolov3_vmss

|-- yolov2_voc

|-- yolov2_voc_pruned_0_77

|-- yolov3_adas_pruned_0_9

`-- yolov3_voc_tf

 

 

We need to first create the yocto recipe for the pre-compiled models:

$ mkdir -p project-spec/meta-user/recipes-apps/kv260-vvas-sms-models

$ mkdir -p project-spec/meta-user/recipes-apps/kv260-vvas-sms-models/files

$ touch project-spec/meta-user/recipes-apps/kv260-vvas-sms-models/kv260-vvas-sms-models.bb

 

Copy the models.b3136 content to the package definition:

$ cp -r ../VVAS/ivas-examples/Embedded/smart_model_select/models.b3136 project-spec/meta-user/recipes-apps/kv260-vvas-sms-models/files/.

 

 

Add the following content to the kv260-vvas-sms-models.bb file:

#

# This file is the kv260-vvas-sms-models recipe.

#

 

SUMMARY = "Pre-compiled models for kv260-vvas-sms application"

SECTION = "PETALINUX/apps"

LICENSE = "MIT"

LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

 

SRC_URI = " \

     file://models.b3136/densebox_320_320/md5sum.txt \

     file://models.b3136/densebox_320_320/meta.json \

file://models.b3136/densebox_320_320/densebox_320_320.prototxt \

   file://models.b3136/densebox_320_320/densebox_320_320.xmodel \

     file://models.b3136/densebox_640_360/md5sum.txt \

     file://models.b3136/densebox_640_360/meta.json \

file://models.b3136/densebox_640_360/densebox_640_360.prototxt \

   file://models.b3136/densebox_640_360/densebox_640_360.xmodel \

     file://models.b3136/inception_v1/md5sum.txt \

     file://models.b3136/inception_v1/meta.json \

     file://models.b3136/inception_v1/inception_v1.prototxt \

     file://models.b3136/inception_v1/inception_v1.xmodel \

     file://models.b3136/mobilenet_v2/md5sum.txt \

     file://models.b3136/mobilenet_v2/meta.json \

     file://models.b3136/mobilenet_v2/mobilenet_v2.prototxt \

     file://models.b3136/mobilenet_v2/mobilenet_v2.xmodel \

     file://models.b3136/refinedet_pruned_0_96/md5sum.txt \

     file://models.b3136/refinedet_pruned_0_96/meta.json \

file://models.b3136/refinedet_pruned_0_96/refinedet_pruned_0_96.prototxt \

file://models.b3136/refinedet_pruned_0_96/refinedet_pruned_0_96.xmodel \

     file://models.b3136/resnet18/md5sum.txt \

     file://models.b3136/resnet18/meta.json \

     file://models.b3136/resnet18/resnet18.prototxt \

     file://models.b3136/resnet18/resnet18.xmodel \

     file://models.b3136/resnet50/md5sum.txt \

     file://models.b3136/resnet50/meta.json \

     file://models.b3136/resnet50/resnet50.prototxt \

     file://models.b3136/resnet50/resnet50.xmodel \

     file://models.b3136/ssd_adas_pruned_0_95/label.json \

     file://models.b3136/ssd_adas_pruned_0_95/md5sum.txt \

     file://models.b3136/ssd_adas_pruned_0_95/meta.json \

file://models.b3136/ssd_adas_pruned_0_95/ssd_adas_pruned_0_95.prototxt \

file://models.b3136/ssd_adas_pruned_0_95/ssd_adas_pruned_0_95.xmodel \

     file://models.b3136/ssd_mobilenet_v2/label.json \

     file://models.b3136/ssd_mobilenet_v2/md5sum.txt \

     file://models.b3136/ssd_mobilenet_v2/meta.json \

file://models.b3136/ssd_mobilenet_v2/ssd_mobilenet_v2.prototxt \

   file://models.b3136/ssd_mobilenet_v2/ssd_mobilenet_v2.xmodel \

     file://models.b3136/ssd_pedestrian_pruned_0_97/label.json \

     file://models.b3136/ssd_pedestrian_pruned_0_97/md5sum.txt \

     file://models.b3136/ssd_pedestrian_pruned_0_97/meta.json \

file://models.b3136/ssd_pedestrian_pruned_0_97/ssd_pedestrian_pruned_0_97.prototxt \

file://models.b3136/ssd_pedestrian_pruned_0_97/ssd_pedestrian_pruned_0_97.xmodel \

     file://models.b3136/ssd_traffic_pruned_0_9/label.json \

     file://models.b3136/ssd_traffic_pruned_0_9/md5sum.txt \

     file://models.b3136/ssd_traffic_pruned_0_9/meta.json \

file://models.b3136/ssd_traffic_pruned_0_9/ssd_traffic_pruned_0_9.prototxt \

file://models.b3136/ssd_traffic_pruned_0_9/ssd_traffic_pruned_0_9.xmodel \

     file://models.b3136/tiny_yolov3_vmss/label.json \

     file://models.b3136/tiny_yolov3_vmss/md5sum.txt \

     file://models.b3136/tiny_yolov3_vmss/meta.json \

file://models.b3136/tiny_yolov3_vmss/tiny_yolov3_vmss.prototxt \

   file://models.b3136/tiny_yolov3_vmss/tiny_yolov3_vmss.xmodel \

     file://models.b3136/yolov2_voc/label.json \

     file://models.b3136/yolov2_voc/md5sum.txt \

     file://models.b3136/yolov2_voc/meta.json \

     file://models.b3136/yolov2_voc/yolov2_voc.prototxt \

     file://models.b3136/yolov2_voc/yolov2_voc.xmodel \

     file://models.b3136/yolov2_voc_pruned_0_77/label.json \

     file://models.b3136/yolov2_voc_pruned_0_77/md5sum.txt \

     file://models.b3136/yolov2_voc_pruned_0_77/meta.json \

file://models.b3136/yolov2_voc_pruned_0_77/yolov2_voc_pruned_0_77.prototxt \

file://models.b3136/yolov2_voc_pruned_0_77/yolov2_voc_pruned_0_77.xmodel \

     file://models.b3136/yolov3_adas_pruned_0_9/label.json \

     file://models.b3136/yolov3_adas_pruned_0_9/md5sum.txt \

     file://models.b3136/yolov3_adas_pruned_0_9/meta.json \

file://models.b3136/yolov3_adas_pruned_0_9/yolov3_adas_pruned_0_9.prototxt \

file://models.b3136/yolov3_adas_pruned_0_9/yolov3_adas_pruned_0_9.xmodel \

     file://models.b3136/yolov3_voc_tf/label.json \

     file://models.b3136/yolov3_voc_tf/md5sum.txt \

     file://models.b3136/yolov3_voc_tf/meta.json \

     file://models.b3136/yolov3_voc_tf/yolov3_voc_tf.prototxt \

     file://models.b3136/yolov3_voc_tf/yolov3_voc_tf.xmodel \

     "

 

S = "${WORKDIR}"

 

do_install() {

     install -d ${D}/opt/avnet

     install -d ${D}/opt/avnet/kv260-vvas-sms

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/densebox_320_320

     install -m 0755 models.b3136/densebox_320_320/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/densebox_320_320

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/densebox_640_360

     install -m 0755 models.b3136/densebox_640_360/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/densebox_640_360

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/inception_v1

     install -m 0755 models.b3136/inception_v1/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/inception_v1

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/mobilenet_v2

     install -m 0755 models.b3136/mobilenet_v2/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/mobilenet_v2

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/refinedet_pruned_0_96

     install -m 0755 models.b3136/refinedet_pruned_0_96/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/refinedet_pruned_0_96

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/resnet18

     install -m 0755 models.b3136/resnet18/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/resnet18

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/resnet50

     install -m 0755 models.b3136/resnet50/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/resnet50

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95

     install -m 0755 models.b3136/ssd_adas_pruned_0_95/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2

     install -m 0755 models.b3136/ssd_mobilenet_v2/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97

     install -m 0755 models.b3136/ssd_pedestrian_pruned_0_97/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9

     install -m 0755 models.b3136/ssd_traffic_pruned_0_9/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss

     install -m 0755 models.b3136/tiny_yolov3_vmss/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc

     install -m 0755 models.b3136/yolov2_voc/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77

     install -m 0755 models.b3136/yolov2_voc_pruned_0_77/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9

     install -m 0755 models.b3136/yolov3_adas_pruned_0_9/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf

     install -m 0755 models.b3136/yolov3_voc_tf/* ${D}/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf

}

 

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_320_320/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_320_320/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_320_320/densebox_320_320.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_320_320/densebox_320_320.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_640_360/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_640_360/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_640_360/densebox_640_360.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/densebox_640_360/densebox_640_360.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/inception_v1/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/inception_v1/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/inception_v1/inception_v1.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/inception_v1/inception_v1.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/mobilenet_v2/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/mobilenet_v2/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/mobilenet_v2/mobilenet_v2.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/mobilenet_v2/mobilenet_v2.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/refinedet_pruned_0_96/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/refinedet_pruned_0_96/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/refinedet_pruned_0_96/refinedet_pruned_0_96.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/refinedet_pruned_0_96/refinedet_pruned_0_96.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet18/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet18/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet18/resnet18.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet18/resnet18.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet50/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet50/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet50/resnet50.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/resnet50/resnet50.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95/ssd_adas_pruned_0_95.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_adas_pruned_0_95/ssd_adas_pruned_0_95.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2/ssd_mobilenet_v2.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_mobilenet_v2/ssd_mobilenet_v2.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97/ssd_pedestrian_pruned_0_97.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_pedestrian_pruned_0_97/ssd_pedestrian_pruned_0_97.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9/ssd_traffic_pruned_0_9.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/ssd_traffic_pruned_0_9/ssd_traffic_pruned_0_9.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss/tiny_yolov3_vmss.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/tiny_yolov3_vmss/tiny_yolov3_vmss.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc/yolov2_voc.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc/yolov2_voc.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77/yolov2_voc_pruned_0_77.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov2_voc_pruned_0_77/yolov2_voc_pruned_0_77.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9/yolov3_adas_pruned_0_9.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_adas_pruned_0_9/yolov3_adas_pruned_0_9.xmodel"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf/label.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf/md5sum.txt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf/meta.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf/yolov3_voc_tf.prototxt"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/models.b3136/yolov3_voc_tf/yolov3_voc_tf.xmodel"

 

 

 

Creating the app package

This content will be placed in the “/opt/<vendor>/<package>” location, as defined for add-on application software packages:

The application and supporting files will be installed at the following location:

/opt/avnet/kv260-vvas-sms/app

|-- jsons

|   |-- kernel_bbox.json

|   `-- kernel_dpu.json

|-- setup.sh

|-- smart_model_select

|-- templates

|   |-- file_fake_template.cfg

|   |-- file_file_template.cfg

|   |-- file_kms_template.cfg

|   |-- mipi_fake_template.cfg

|   |-- mipi_file_template.cfg

|   |-- mipi_kms_template.cfg

|   |-- rtsp_fake_template.cfg

|   |-- rtsp_file_template.cfg

|   |-- rtsp_kms_template.cfg

|   |-- usbcam_fake_template.cfg

|   |-- usbcam_file_template.cfg

|   |-- usbcam_kms_template.cfg

|   |-- welcome.cfg

|   `-- welcome_1080.jpg

`-- videos

    |-- CLASSIFICATION.mp4

    |-- FACEDETECT.mp4

    |-- REFINEDET.mp4

    |-- SSD.mp4

    |-- YOLOV2.mp4

    `-- YOLOV3.mp4

 

We start by creating the yocto recipe for the smart_model_select application:

$ mkdir -p project-spec/meta-user/recipes-apps/kv260-vvas-sms-app

$ mkdir -p project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files

$ touch project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/kv260-vvas-sms-app.bb

 

Copy the source code for the application:

$ cp ../VVAS/ivas-examples/Embedded/smart_model_select/src/smart_model_select.c project-spec/meta-user/recipes-apps/kv260-vvas-sms-models/files/.

 

Copy the json configuration files for the application:

$ mkdir project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/jsons

$ cp ../VVAS/ivas-examples/Embedded/smart_model_select/src/jsons/kernel*.json project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/jsons/.

Edit the “kernel_dpu.json” configuration file, to specify the valid “model-path” for the custom Kria app, as follows:

{

  "xclbin-location":"XCLBIN_PATH",

  "ivas-library-repo": "/usr/lib/",

  "element-mode":"inplace",

  "kernels" :[

    {

    "library-name":"libivas_xdpuinfer.so",

    "config": {

    "model-name" : "MODEL",

    "model-class" : "CLASS",

    "model-format" : "FORMAT",

    "model-path" : "/opt/avnet/kv260-vvas-sms/models.b3136/",

    "run_time_model" : false,

    "need_preprocess" : false,

    "performance_test" : false,

    "debug_level" : 1

      }

    }

  ]

}

 

Copy the template files for the application:

$ mkdir project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/templates

$ cp -r ../VVAS/ivas-examples/Embedded/smart_model_select/templates project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/templates/.

 

Copy the video files for the application:

$ mkdir -p project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/videos

$ cp ../videos_smart_model_select/*.mp4 project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/videos/.

 

Copy the setup script for the application:

$ cp ../VVAS/ivas-examples/Embedded/smart_model_select/setup.sh project-spec/meta-user/recipes-apps/kv260-vvas-sms-app/files/.

Edit the “setup.sh” script as follows:

...

 

export XCLBIN_PATH=/lib/firmware/xilinx/kv260-vvas-sms/kv260-vvas-sms.xclbin

 

# Set alpha plane.

modetest -M xlnx -w 40:alpha:0

 

# Force DP resolution to 1080P and display diagonal test pattern

modetest -M xlnx -s 43@41:1920x1080-60@AR24 -P 39@41:1920x1080@NV12  -w 40:alpha:0 &

 

Add the following content to the kv260-vvas-sms-app.bb file:

#

# This file is the kv260-vvas-sms-app recipe.

#

 

SUMMARY = "Simple kv260-vvas-sms-app application"

SECTION = "PETALINUX/apps"

LICENSE = "MIT"

LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

 

SRC_URI = " \

     file://smart_model_select.c \

     file://setup.sh \

     file://jsons/kernel_bbox.json \

     file://jsons/kernel_dpu.json \

     file://templates/welcome_1080.jpg \

     file://templates/welcome.cfg \

     file://templates/file_fake_template.cfg \

     file://templates/file_fake_template_perf.cfg \

     file://templates/file_file_template.cfg \

     file://templates/file_kms_template.cfg \

     file://templates/rtsp_fake_template.cfg \

     file://templates/rtsp_file_template.cfg \

     file://templates/rtsp_kms_template.cfg \

     file://templates/usbcam_fake_template.cfg \

     file://templates/usbcam_file_template.cfg \

     file://templates/usbcam_kms_template.cfg \

     file://templates/mipi_fake_template.cfg \

     file://templates/mipi_file_template.cfg \

     file://templates/mipi_kms_template.cfg \

     file://videos/CLASSIFICATION.mp4 \

     file://videos/FACEDETECT.mp4 \

     file://videos/REFINEDET.mp4 \

     file://videos/SSD.mp4 \

     file://videos/YOLOV2.mp4 \

     file://videos/YOLOV3.mp4 \

     "

 

S = "${WORKDIR}"

 

DEPENDS = "glib-2.0 glib-2.0-native xrt libcap libxml2 bison-native flex-native jansson ivas-utils ivas-gst opencv vitis-ai-library vart"

 

do_compile() {

     ${CC} ${CFLAGS} ${LDFLAGS} -o smart_model_select smart_model_select.c -lpthread

}

 

do_install() {

     install -d ${D}/opt/avnet

     install -d ${D}/opt/avnet/kv260-vvas-sms/app

     install -m 0755 smart_model_select ${D}/opt/avnet/kv260-vvas-sms/app

     install -m 0755 setup.sh           ${D}/opt/avnet/kv260-vvas-sms/app

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/app/jsons

     install -m 0755 jsons/kernel_bbox.json ${D}/opt/avnet/kv260-vvas-sms/app/jsons

     install -m 0755 jsons/kernel_dpu.json  ${D}/opt/avnet/kv260-vvas-sms/app/jsons

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/app/templates

     install -m 0755 templates/welcome_1080.jpg       ${D}/opt/avnet/kv260-vvas-sms/app/templates

     install -m 0755 templates/welcome.cfg            ${D}/opt/avnet/kv260-vvas-sms/app/templates

     install -m 0755 templates/file_*_template.cfg    ${D}/opt/avnet/kv260-vvas-sms/app/templates

     install -m 0755 templates/rtsp_*_template.cfg    ${D}/opt/avnet/kv260-vvas-sms/app/templates

     install -m 0755 templates/usbcam_*_template.cfg  ${D}/opt/avnet/kv260-vvas-sms/app/templates

     install -m 0755 templates/mipi_*_template.cfg    ${D}/opt/avnet/kv260-vvas-sms/app/templates

 

     install -d ${D}/opt/avnet/kv260-vvas-sms/app/videos

     install -m 0755 videos/CLASSIFICATION.mp4 ${D}/opt/avnet/kv260-vvas-sms/app/videos

     install -m 0755 videos/FACEDETECT.mp4     ${D}/opt/avnet/kv260-vvas-sms/app/videos

     install -m 0755 videos/REFINEDET.mp4      ${D}/opt/avnet/kv260-vvas-sms/app/videos

     install -m 0755 videos/SSD.mp4            ${D}/opt/avnet/kv260-vvas-sms/app/videos

     install -m 0755 videos/YOLOV2.mp4         ${D}/opt/avnet/kv260-vvas-sms/app/videos

     install -m 0755 videos/YOLOV3.mp4         ${D}/opt/avnet/kv260-vvas-sms/app/videos

}

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/smart_model_select"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/setup.sh"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/jsons/kernel_bbox.json"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/jsons/kernel_dpu.json"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/welcome_1080.jpg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/welcome.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/file_fake_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/file_fake_template_perf.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/file_file_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/file_kms_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/rtsp_fake_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/rtsp_file_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/rtsp_kms_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/usbcam_fake_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/usbcam_file_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/usbcam_kms_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/mipi_fake_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/mipi_file_template.cfg"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/templates/mipi_kms_template.cfg"

 

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/videos/CLASSIFICATION.mp4"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/videos/FACEDETECT.mp4"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/videos/REFINEDET.mp4"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/videos/SSD.mp4"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/videos/YOLOV2.mp4"

FILES_${PN} += "/opt/avnet/kv260-vvas-sms/app/videos/YOLOV3.mp4"

 

 

Creating the dependencies packagegroup

Next, we want to create a packagegroup that contains all the dependencies for our custom Kria app:

$ mkdir -p project-spec/meta-user/recipes-core/packagegroups

$ touch project-spec/meta-user/recipes-core/packagegroups/packagegroup-kv260-vvas-sms.bb

 

Add the following content to the packagegroup-kv260-vvas-sms.bb file:

 

DESCRIPTION = "KV260 VVAS smart_model_select related packages"

 

inherit packagegroup

 

KV260_VVAS_SMS_PACKAGES = " \

   ap1302-ar1335-single-firmware \

   kv260-vvas-sms \

   kv260-vvas-sms-models \

   kv260-vvas-sms-app \

   git \

   dnf \

   libdrm libdrm-tests libdrm-kms \

   xrt xrt-dev \

   zocl \

   opencl-headers \

   packagegroup-petalinux-vitisai \

   packagegroup-petalinux-vitisai-dev \

   packagegroup-petalinux-gstreamer \

   packagegroup-petalinux-opencv \

   packagegroup-petalinux-v4lutils \

   packagegroup-petalinux-x11 \

   gstreamer-vcu-examples \

   htop \

   jansson \

   target-factory \

   unilog vart vitis-ai-library xir \

   ivas-accel-libs ivas-gst ivas-utils \

   "

 

RDEPENDS_${PN} = "${KV260_VVAS_SMS_PACKAGES}"

 

COMPATIBLE_MACHINE = "^$"

COMPATIBLE_MACHINE_k26-kv = "${MACHINE}"

PACKAGE_ARCH = "${BOARDVARIANT_ARCH}"

 

Building the new packages

We want to make the petalinux project aware of these new packages as follows:

$ echo "CONFIG_kv260-vvas-sms-models" >> project-spec/meta-user/conf/user-rootfsconfig

$ echo "CONFIG_kv260-vvas-sms-apps" >> project-spec/meta-user/conf/user-rootfsconfig

$ echo "CONFIG_packagegroup-kv260-vvas-sms" >> project-spec/meta-user/conf/user-rootfsconfig

 

We also want to enable the new packages as follows:

$ echo "CONFIG_kv260-vvas-sms-models=y" >> project-spec/configs/rootfs_config

$ echo "CONFIG_kv260-vvas-sms-app=y" >> project-spec/configs/rootfs_config

$ echo "CONFIG_packagegroup-kv260-vvas-sms=y" >> project-spec/configs/rootfs_config

 

 

Use the following command to build the petalinux packages:

$ petalinux-build -c kv260-vvas-sms

$ petalinux-build -c kv260-vvas-sms-models

$ petalinux-build -c kv260-vvas-sms-app

$ petalinux-build -c packagegroup-kv260-vvas-sms

 

Use the following command to build the petalinux project:

$ petalinux-build

 

Use the following command to create the SD card image:

$ petalinux-package --wic --bootfiles "ramdisk.cpio.gz.u-boot boot.scr Image system.dtb"

$ gzip images/linux/petalinux-sdimage.wic

 

This will create the following image, this time with our custom Kria app, which can be programmed using Balena Etcher:

xilinx-k26-starterkit-2021.1/images/linux/petalinux-sdimage.wic.gz

 

Let’s keep a copy for reference:

$ mv images/linux/petalinux-sdimage.wic.gz ../kv260-vvas-sms-2021-1-sdimage.wic.gz

 

The rpm packages will be located in the following directories:

 

build/tmp/work/zynqmp_generic-xilinx-linux/kv260-vvas-sms/1.0-r0/deploy-rpms/

└── zynqmp_generic

    ├── kv260-vvas-sms-1.0-r0.0.zynqmp_generic.rpm

    ├── kv260-vvas-sms-dbg-1.0-r0.0.zynqmp_generic.rpm

    ├── kv260-vvas-sms-dev-1.0-r0.0.zynqmp_generic.rpm

    └── kv260-vvas-sms-lic-1.0-r0.0.zynqmp_generic.rpm

 

build/tmp/work/cortexa72-cortexa53-xilinx-linux/kv260-vvas-sms-models/1.0-r0/deploy-rpms/

└── cortexa72-cortexa53

    ├── kv260-vvas-sms-models-1.0-r0.0.zynqmp_generic.rpm

    ├── kv260-vvas-sms-models-dbg-1.0-r0.0.zynqmp_generic.rpm

    ├── kv260-vvas-sms-models-dev-1.0-r0.0.zynqmp_generic.rpm

    └── kv260-vvas-sms-models-lic-1.0-r0.0.zynqmp_generic.rpm

 

build/tmp/work/cortexa72-cortexa53-xilinx-linux/kv260-vvas-sms-app/1.0-r0/deploy-rpms/

└── cortexa72-cortexa53

    ├── kv260-vvas-sms-app-1.0-r0.0.zynqmp_generic.rpm

    ├── kv260-vvas-sms-app-dbg-1.0-r0.0.zynqmp_generic.rpm

    ├── kv260-vvas-sms-app-dev-1.0-r0.0.zynqmp_generic.rpm

    └── kv260-vvas-sms-app-lic-1.0-r0.0.zynqmp_generic.rpm

 

build/tmp/work/all-xilinx-linux/packagegroup-kv260-vvas-sms/1.0-r0/deploy-rpms/

└── noarch

    ├── packagegroup-kv260-vvas-sms-1.0-r0.0.noarch.rpm

    ├── packagegroup-kv260-vvas-sms-dbg-1.0-r0.0.noarch.rpm

    ├── packagegroup-kv260-vvas-sms-dev-1.0-r0.0.noarch.rpm

    ├── packagegroup-kv260-vvas-sms-lic-1.0-r0.0.noarch.rpm

    └── packagegroup-kv260-vvas-sms-ptest-1.0-r0.0.noarch.rpm

 

Next Steps

The following blogs cover the previous development steps for this in-depth project tutorial.

The following blogs will cover the remaining development steps for this in-depth project tutorial.