This blog is part of a series of blogs which describe the development steps for an in-depth project tutorial.

• http://avnet.me/kv260-vvas-sms-2021-1-blog

This part of the tutorial provides a description of the Xilinx repositories that will be used.

Installing the Xilinx tools

The following tools will be used for this tutorial

NOTE : The eSDK update 1 for use with Kria is not compatible with other platforms so should be kept separate, if working on other platforms as well.

Creating a base directory

This project tutorial can be placed in any location on your linux machine. 

To facilitate the flow of the tutorial, we will define an environment variable corresponding to this base directory for the project.

$ export PROJ_DIR=$(pwd)

$ cd $PROJ_DIR

Cloning the Xilinx repositories

The following repositories will be used for this tutorial.

On a linux machine, clone the following repositories to the same location:

$ cd $PROJ_DIR

$ git clone -b release-2021.1 https://github.com/Xilinx/kv260-vitis

$ git clone -b release-2021.1 https://github.com/Xilinx/kv260-firmware

$ git clone -b v1.4 https://github.com/Xilinx/Vitis-AI

$ git clone -b VVAS_REL_v1.0 https://github.com/Xilinx/VVAS

Downloading the Xilinx smart_model_select archives

We will also make use to the following pre-built solution for ZCU104:

• https://www.xilinx.com/member/forms/download/xef.html?filename=vvas_smart_model_select_2021.1_zcu104.zip

We will also make use of the provided video files:

• https://www.xilinx.com/member/forms/download/xef.html?filename=videos_smart_model_select.zip

Download and extract these archives to the same location as the previous repositories.

Downloading the modified source files and pre-built binaries

Throughout this tutorial, some of the source file will be modified, to create new binaries targeting the Kria KV260 Vision AI Starter Kit.

Although not required to complete the tutorial, the modified files and pre-built binaries are provided for reference:

• Modified source files
  • http://avnet.me/kv260-vvas-sms-2021-1-files
    (2022-01-10 - md5sum : 095a29873297295b462dc881768b9f9b)
• Pre-built binaries
  • http://avnet.me/kv260-vvas-sms-2021-1-prebuilt
    (2022-01-10 - md5sum : f3fa7059d657a2f0bd115cafb325ff41)

These archives can be extracted to the same location as the previous content.

Understanding the Vitis platforms and examples

The Vitis Examples (VVAS) and Overlays (kv260-vitis) provided by Xilinx have the following architecture:

The Vitis Examples and Overlays are built on top of Vitis platforms.

The Vitis platforms are composed of:

• Hardware – Vivado project
• Software – Petalinux project, or other

An important observation is that the Vitis platforms may or may not include the petalinux project corresponding to the vivado project. 

For the ZCU104 platforms (VVAS), the petalinux projects are included. 

For the KV260 platforms (kv260-vitis), the petalinux project is not provided.  Instead, a common petalinux project will be used to dynamically load the “overlay”.  We can define that our custom Kria app will be a dynamically loaded “overlay”.

The Vitis platform is a wrapper, that defines the resources that are available, including:

• Resources (LUTs, FFs, BRAMs, URAMs, DSPs, etc…)
• Interconnect
• Clocks
• Resets
• Interrupts

The Vitis Examples and Overlays define and instantiate one or more accelerators, which may include:

• DPU
• HLS-based functions from the Vitis Accelerated Libraries

The “VVAS” and “kv260-vitis” repositories both contain examples of this, as shown below:

The “VVAS” repository includes two (2) platforms, and two (2) examples:

The smart_model_select and multichannel_ml examples both implement the following accelerators:

• preprocess : multi-channel scaler with color space conversion and normalization
• AI engine : B4096 DPU

Each of the VVAS vitis platforms include a combination of the following features:

• vcuDec : VCU, in decode only configuration
• DP : DisplayPort output
• vmixHdmiTx : HDMI output with video mixer
  • zcu104_vcuDec_vmixHdmiTx has nine NV12 layers

Both examples target a ZCU104 platform with VCU (decode only).  

The smart_model_select example provides support for 16 different models from the Xilinx Model Zoo, executing one at a time, and using the DisplayPort as output.

The multichannel_ml example provides support for 4 different models, and uses the platform enabled with Video Mixer and HDMI TX to display up to 8 streams at the same time on the HDMI monitor.

The “kv260-vitis” repository includes four (4) platforms, and five (5) overlays:

Each of the kv260 vitis platforms include a combination of the following features:

• ispMipiRx : MIPI capture pipeline, for use with the AR1335 sensor module
• vcu : VCU, in decode + encode configuration
• vcuDecode : VCU, in decode only configuration
• DP : DisplayPort output
• vmixDP : DisplayPort output with video mixer
  • kv260_vcuDecode_vmixDP has one BGRA8 layer and four NV12 layers
  • kv260_ispMipiRx_vmixDP has one BGRA8 layer and four Y8 layers

We can see that the kv260_ispMipiRx_vcu_DP platform, which includes a MIPI capture pipeline and VCU in decode + encode configuration, is used for two different overlays.

The smartcam overlay implements the following accelerators:

• preprocess : normalization and scaling
• AI engine : B3136 DPU

The benchmark overlay implements the following accelerator:

• AI engine : B4096 DPU

In this tutorial, we will combine the following two components together to create our custom Kria app:

• Vitis Example/Overlay : smart_model_select, from VVAS repository
• Vitis Platform : kv260_ispMipiRx_vcu_DP, from kv260-vitis repository

Next Steps

The following blogs will cover the remaining development steps for this in-depth project tutorial.

• http://avnet.me/kv260-vvas-sms-2021-1-part2
• http://avnet.me/kv260-vvas-sms-2021-1-part3
• http://avnet.me/kv260-vvas-sms-2021-1-part4
• http://avnet.me/kv260-vvas-sms-2021-1-part5
• http://avnet.me/kv260-vvas-sms-2021-1-part6
• http://avnet.me/kv260-vvas-sms-2021-1-part7

Revision History