RoadTest: AMD Xilinx Kria KV260 Vision AI Starter Kit
Author: work14
Creation date:
Evaluation Type: Development Boards & Tools
Did you receive all parts the manufacturer stated would be included in the package?: True
What other parts do you consider comparable to this product?: (1) Stemedu ESP32-CAM Development Board with Flash Ai Thinker Camera Module OV2640 for Arduino. (2) BEAGLEBOARD SBC BeagleBone, AI-64, TDA4VM. (3) Huskylens Kendryte K21 (AI Chip FPGA module, High-end Allwinner V3 system on with 64bit RISC-V (4) OpenMV Cam H7 (5) Arduino Nicla Vision (5) iVS-AWV3-AR0521 (6) TBD
What were the biggest problems encountered?: On-Board development tools are difficult to setup properly. Some on-board update links do not work.
Detailed Review:
the KV260 Vision AI Starter Kit is an out-ofthe-box platform for advanced vision application development. Some links to key information regarding this RoadTest follow.
The Kria KV260 Vision AI Starter Kit is an affordable advanced vision application development without requiring complex hardware design knowledge.
Kria KV260 Vision AI Starter Kit
The results and take away of the Kria KV260 Vision AI Starter Kit Applications for this RoadTest are itemized below:
Appreciation to AMD and element14 for providing the opportunity to review the AMD Xilinx Kria KV260 Vision AI Starter Kit. The RoadTest review program experience allows individuals to learn about new hardware and software solutions, and to provide service to the community by submitting a review of various products.
Special note of gratitude to AMD and element14 for providing a second KV260 board to help troubleshoot a video output issue with the original board. The availability of a second KV260 board helped identify a physical problem with the original board that was subsequently fixed. The second test board was subsequently returned to AMD.
The RoadTest program is the element14 community’s flagship product review program. Testers obtain a product to experiment and to provide element14 a written review of their experiences with the product. The following are important points pertaining to this program.
RoadTesters have 60 days from the receipt of the product to complete testing and review writing on element14.
The review must be filed on the special element14 Roadtest form.
Roadtesters who do not complete their reviews will not be eligible to participate in the program in the future.
This particular RoadTest relates to the Kria KV260 Vision AI Starter Kit (see figure below). This kit intends to facilitate advanced vision application development out-of-the-box. The board is a non-production K26 System on Module (SOM) mounted to an evaluation carrier card optimized for vision applications. It is meant to provide multi-camera support via ON Semiconductor Imager Access System (IAS) and Raspberry Pi connectors.
The initial goal of the out-of-box experience is to get applications up and running in under one (1) hour, with no FPGA experience needed. Customization is possible via preferred design environments from application software to AI model to FPGA design.
This kit provides simplified hardware and software development with the aim to to become the fastest and easiest platform for application development with the goal of volume deployment on Kria K26 SOMs.
Produce a 2 to 3 minute unboxing video. Test the out-of-box experience of the product. Build a project and show how you did it.• Product Brief• Datasheet• User Guide• Getting Started Guide• Github Applications• Kria SOM Github• Avnet Page Brief• Getting Started Video
KR260 User Guide (UG1092)Xilinx SOM Carrier Card Design Guide (UG1091)Kria App Developer GuideKria Starter Kit Boot FW GuideKria Robotics Stack (KRS)KV260 Applications Overview KR260 Applications OverviewThe Getting Started web page provides a pre- built reference image that can be written to a microSD card for out-of-the-box functionality.
The Xilinx® Kria
KV260 Vision AI Starter Kit is comprised of a non-production version of the K26 system-on-module (SOM), carrier card, and thermal solution. The SOM is very compact and includes key components such as a
Zynq® UltraScale+ MPSoC based silicon device, memory module, boot module, security module. The carrier card allows various interfacing options and includes a power solution and network connectors for camera, display, and microSD card. The thermal solution has a heat sink, heat sink cover, and fan. The Kria KV260 Vision AI Starter Kit is designed to provide customers a platform to evaluate their target applications and ultimately design their own carrier card with Xilinx K26 SOMs. Target applications for the Kria KV260 Vision AI Starter Kit include
smart city visionmachine vision, security cameras, retail analytics,other industrial applications.The only sensor provided with the kit is
The kit can work with other USB cameras simultaneously. This RoadTest did not test a multi camera setup.
This particular board did not initially have video output. After much effort, nothing seemed to work still. AMD sent a replacement board to help troubleshoot the issues with the original board. As it turned out the camera socket had a short that Linux detected and kept it from utilizing vido output to the monitor. After cleaning the camera socket with compressed air and alcohol, the video output started to work as expected.
| {gallery}My Gallery Title |
|---|
|
KV260 Vision AI Starter Kit with Flexible Connectivity |
|
KV260 Vision AI Starter Kit |
|
KV260 Starter Kit Block Diagram |
|
Applications |
|
KV260 Starter Ki |
The following link provides a general overview of the Kria KV260 Vision AI Starter Kit in a form of a User Guide.
Kria KV260 Vision AI Starter Kit User Guide
The link below provides a flow for upgrading the firmware and selecting the Linux Operating System (OS)
Getting Started with Kria KV260 Vision AI Starter Kit
The above web page provides a pre- built reference image that can be written to a microSD card for out-of-the-box functionality. This page also contains the Xilinx SOM Starter Linux image to a microSD card. In this website we find the following warning:
Important: Critical Firmware Update Required
To enable complete board functionality, compatibility with latest Operating Systems, and best performance, be sure to install the latest AMD provided boot firmware following the firmware update instructions available on the K26 Wiki
To update the firmware details are found in the following link
The purpose of this page is to provide developers with information and links to collateral available for the Kria K26 SOM, including documentation, pre-built images, firmware updates, and GitHub repositories.
Note: This RoadTest uses Ubuntu to run the latest of several accelerated applications from the App Store to evaluate KV260.
The initial image update is done via the "Image Recovery Tool” which is a stand alone application for manually loading and configurating content of both A and B boot partitions. To install the latest AMD provided boot firmware following the firmware update instructions available on the K26 Wiki. Care must be given to do the initial update from a Firefox browser. Other browsers will fail the firmware update. The latest firmware can be dowloaded from the following link:
Xilinx download - 2022.1_update3_BOOT.BIN
Once the firmware has been updated, the software update detailed in the next section can be done. Once the Linux software is up to date, the other bootable image can be updated via the xmulti tool as specified below:
% sudo xmutil bootfw_update -i BOOT_xilinx-k26-starterkit-v2022.1-09152304_update3.BIN
After the image write is completed issue a power-on reset by pressing the RESET push-button or power cycling the board. After restart it is required by the user verify that Linux fully boots with the new boot FW to verify functionality. This is completed by executing “sudo xmutil bootfw_update -v” to validate successful boot against the new firmware. Note this must be completed on the platform restart immediately following the update, else the platform will fall back to the other boot partition on the next restart.
Once the firmware has been updated we verify the current bootable firmware as such:
$ sudo xmutil bootfw_update -v [sudo] password for ubuntu: Marking last booted image as bootable ubuntu@kria:~$ sudo xmutil bootfw_status Image A: Bootable Image B: Bootable Requested Boot Image: Image A Last Booted Image: Image A XilinxSom_QspiImage_v1.1_20210422 ImageA Revision Info: XilinxSOM_BootFW_20220915 ImageB Revision Info: XilinxSOM_BootFW_20220915
In the case above both image A and B are bootable with the latest available firmware
XilinxSOM_BootFW_20220915
The Ubuntu Desktop 20.04.3 LTS software download page is found in the links below
Getting Started with Kria KR260 Robotics Starter Kit
Ubuntu 22.04 + Ubuntu AI Demos
To access the successfully boot of Ubuntu from a computer running macOS, issue the following command to observe which COM ports appear when you plug in the USB cable attached to the KV260 into your computer:
% ls /dev/tty.* /dev/tty.usbserial-XFL10PJXULJ40 /dev/tty.usbserial-XFL10PJXULJ42 /dev/tty.usbserial-XFL10PJXULJ41 /dev/tty.usbserial-XFL10PJXULJ43
Four serial ports are enumerated where the second numbered port corresponds to the KV260 UART. To access the KV260 type the following:
% screen /dev/tty.usbserial-XFL10PJXULJ41 115200,cs8
Once the firmware update (from the previous section) has successfully completed and Linux successfully boots on the KV260, we have the following at the first terminal login:
Ubuntu 20.04.3 LTS kria ttyPS0 kria login: ubuntu Password: You are required to change your password immediately (administrator enforced) Changing password for ubuntu. Current password: New password: Retype new password: Welcome to Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-1017-xilinx-zynqmp aarch64)
Once logged into the KV260 issue an update/upgrade cycle:
$ sudo apt update $ sudo apt upgrade
Next set up the Xilinx Development & Demonstration Environment for Ubuntu 22.04 LTS by install the xlnx-config snap that is required for system management:
sudo snap install xlnx-config --classic --channel=2.x
Run the xlnx-config.sysinit command to install Kria specific Personal Package Archive (PPA) :
sudo xlnx-config.sysinit
If using in macOS, the screen app may say that there is an issue regarding no pty available. This indicates that there is a zombie screen using the pty which can happen for any number of reasons such as the remote host terminating the connection to screen. To find if there is a zombie screen issue the following command from the terminal:
% screen -ls There is a screen on: 22652.ttys002.2021 (Detached) 1 Socket in /var/folders/r_/h5vt2q1x05b5f1jblnxxdvym0000gn/T/.screen.
To terminate the zombie screen issue the command:
% screen -XS 22652 quit
where the number 22652 is the screen id. The above commands are universal and apply to all platforms running screen.
General documentation for the Xilinx® Kria KV260 Vision AI Starter Kit can be found here:
Kria KV260 Vision AI Starter Kit User Guide
Note: the above link is general in scope. Do not follow the links provided in this page as guide or tutorials for smart applications.
The following link will take the user to a signup place where you can enter some information and in return a link is provided.
Smart Camera Accelerated Application
Once the personal information is accepted, follow the provided links to find the following selections. The landing page (link [1] in the figure) provides links to a number of links pertaining to the Kria SOM support. The [2] Kria KV260 takes the user to the Smart Application Landing page [3] shown in the figure below. Finally link [4] takes the user to the Smart Camera application (one of many).
[2] Kria KV260 Vision AI Starter Kit Applications
[1] Landing Page, [2] Pertaining to the Kria KV260, [3] Landing Page for the Kria kv260 Smart Applications. [4] Specific Links to the various applicable Smart Applications.
There are many choices to try pre-built smart applications. Read carefully to avoid going down a wrong path. Some accelerated applications such as the NLP-SmartVision only run in the Ubuntu 20.04 platform. Some other Accelerated Applications seem to run better in PetaLinux 2021.1 +.
In addition, the documentation does not make it clear that some information found in the various links are outdated or meant for a specific platform. For example, in some instances the instructions indicate to install accelerated applications to your platform by executing a DNF install for the chosen application package group. This bit of information will tell a careful user that the instructions are mean for the PetaLinux platform. Even then, in some instances the above will not work since the PetaLinux distribution will not find the software to install.
The following link takes the user to the Quick Start portion of the how-to
Disable the desktop.
sudo xmutil desktop_disable
Packages compatible with Kv260 are fond dusty:
$ sudo apt search xlnx-firmware-kv260Full Text Search... Donexlnx-firmware-kv260-aibox-reid/jammy 0.8-0xlnx1 arm64FPGA firmware for Xilinx boards - kv260 aibox-reid applicationxlnx-firmware-kv260-benchmark-b4096/jammy 0.8-0xlnx1 arm64FPGA firmware for Xilinx boards - kv260 benchmark-b4096 applicationxlnx-firmware-kv260-defect-detect/jammy 0.8-0xlnx1 arm64FPGA firmware for Xilinx boards - kv260 defect-detect applicationxlnx-firmware-kv260-nlp-smartvision/jammy,now 0.8-0xlnx1 arm64 [installed]FPGA firmware for Xilinx boards - kv260 nlp-smartvision applicationxlnx-firmware-kv260-smartcam/jammy,now 0.8-0xlnx1 arm64 [installed]FPGA firmware for Xilinx boards - kv260 smartcam application
To install a package do:
sudo apt install xlnx-firmware-kv260-smartcam
Verify that it is captured under the listapps function, and to have dfx-mgrd re-scan and register all accelerators in the FW directory tree.
sudo xmutil listapps
Accelerator Accel_type Base Base_type #slots(PL+AIE) Active_slot
k26-starter-kits XRT_FLAT k26-starter-kits XRT_FLAT (0+0) 0, kv260-smartcam XRT_FLAT kv260-smartcam XRT_FLAT (0+0) -1 kv260-nlp-smartvision XRT_FLAT kv260-nlp-smartvision XRT_FLAT (0+0) -1
The above says that the k26-starter-kits application is active on slot 0. This needs to unload first, then switch to kv260-smartcam application.
sudo xmutil unloadapp
Look for print out “remove from slot 0 returns: 0 (Ok)” to know that the firmware was unloaded correctly. Load the kv260-smartcam application next.
sudo xmutil loadapp kv260-smartcam
Look for "kv260-smartcam: loaded to slot 0" to know that the application loaded correctly. Ignore the warning "IRQ index 8 not found". At this point the smart camera application is ready for use.
Visit the following link to read about some debug tips.
Frequently Asked Questions / Common Debug Tips
There are a number of debug information issued after entries, and most of them seem ominous, but they are not relevant. Most importantly:
sudo systemctl restart dfx-mgr.service
xmutil or dfx-mgr runs only in base OS, they do not run in Docker containers.xmutil loadapp to load firmware, it needs several seconds for the whole firmware to be ready. root@xlnx-docker/# exit
ubuntu@kria:~$ sudo xmutil unloadapp
ubuntu@kria:~$ sudo rm -rf /etc/vart.conf/
“ERROR: memory leak” error is a result of upstream Linux device tree framework.xmutil unloadapp
... xilinx-kv260-starterkit-20221 kernel: OF: ERROR: memory leak, ...
... xilinx-kv260-starterkit-20221 kernel: OF: ERROR: memory leak, ...
remove from slot 0 returns: 0 (Ok)” to know that the firmware was unloaded correctly.kv260-smartcam: loaded to slot 0" to know that the firmware was loaded correctly,xmutil loadapp kv260-smartcam
... xilinx-kv260-starterkit-20221 kernel: debugfs: Directory '4-003c' with parent 'regmap' already present!
... xilinx-kv260-starterkit-20221 kernel: zocl-drm axi:zyxclmm_drm: IRQ index 8 not found
... xilinx-kv260-starterkit-20221 kernel: xilinx-video axi:isp_vcap_csi: Entity type for entity 80000000.csiss was not initialized!
IRQ index 8 not found". sudo du -a /var | sort -n -r | head -n 10
docker system prune
At this point the kv260-smartcam smart camera application has been loaded and is ready for use. The instructions on the docker based application work well, and are reproduced here for convenience.
docker images installed with command:docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
xilinx/smartcam 2022.1 aa0270aef908 6 months ago 1.41GB
docker rmi --force <other containers>
sudo du -a /var | sort -n -r | head -n 10
docker pull xilinx/smartcam:2022.1
docker run \
--env="DISPLAY" \
-h "xlnx-docker" \
--env="XDG_SESSION_TYPE" \
--net=host \
--privileged \
--volume="$HOME/.Xauthority:/root/.Xauthority:rw" \
-v /tmp:/tmp \
-v /dev:/dev \
-v /sys:/sys \
-v /etc/vart.conf:/etc/vart.conf \
-v /lib/firmware/xilinx:/lib/firmware/xilinx \
-v /run:/run \
-it xilinx/smartcam:2022.1 bash
2022.1: Pulling from xilinx/smartcam
Digest: sha256:da2e52629011aeec332152a0f468d3ff156917dba9b596cf6d0de958d5dc29d7Status: Downloaded newer image for xilinx/smartcam:2022.1docker.io/xilinx/smartcam:2022.1
smartcam image will launch in a new container (see picture below)root@xlnx-docker/#
Figure kv260-smartcam application loaded in a docker.
kv260-smartcam ApplicationThe Smart Camera kv260-smartcam Application will run via Jupyter notebook. This is a polished method that is suitable to learn and run the application. The instructions in the following link are self explanatory, but are reproduced here for convenience with added comments.
Smart Camera Application Deployment
At this point it is beneficial to monitor the SD disk space usage and availability:
$ df -h
/dev/mmcblk1p2 14G 13G 759M 95% /
The disk space is at 95% full. The docker now contains the smart camera tree:
root@xlnx-docker:/opt/xilinx/kv260-smartcam# ls -al
total 40
drwxr-xr-x 5 root root 4096 Sep 26 2022 .
drwxr-xr-x 3 root root 4096 Sep 26 2022 ..
-rw-r--r-- 1 root root 19198 Sep 26 2022 README_SMARTCAM
drwxr-xr-x 2 root root 4096 Sep 26 2022 bin
drwxr-xr-x 2 root root 4096 Sep 26 2022 lib
drwxr-xr-x 5 root root 4096 Sep 26 2022 share
Run the command smartcam-install.py to install the package shipped notebooks which reside in /opt/xilinx/kv260-smartcam/share/notebooks to the folder /root/notebooks/smartcam
root@xlnx-docker:/opt/xilinx/kv260-smartcam/bin# ./smartcam-install.py
Info: Notebook files under /opt/xilinx/kv260-smartcam/share/notebooks are copied to /root/notebooks/smartcam.
Find the IP address for the board, and next launch Jupyter notebook on the target (note: substitute ip=x.x.1.17 with a proper address):
root@xlnx-docker:~# jupyter-lab --notebook-dir=/root/notebooks/smartcam --allow-root --ip=x.x.1.17 &
root@xlnx-docker:~# [I 02:18:02.593 ServerApp] jupyterlab | extension was successfully linked.[I 02:18:02.653 ServerApp] nbclassic | extension was successfully linked.[I 02:18:02.660 ServerApp] Writing Jupyter server cookie secret to /root/.local/share/jupyter/runtime/jupyter_cookie_secret[I 02:18:03.725 ServerApp] notebook_shim | extension was successfully linked.[I 02:18:03.814 ServerApp] notebook_shim | extension was successfully loaded.[I 02:18:03.820 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.10/dist-packages/jupyterlab[I 02:18:03.820 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab[I 02:18:03.839 ServerApp] jupyterlab | extension was successfully loaded.[I 02:18:03.859 ServerApp] nbclassic | extension was successfully loaded.[I 02:18:03.861 ServerApp] Serving notebooks from local directory: /root/notebooks/smartcam[I 02:18:03.861 ServerApp] Jupyter Server 1.18.1 is running at:[I 02:18:03.861 ServerApp] http://10.137.1.17:8888/lab?token=31ec81178347d467146131c34dd3506e039fa7dc45124804[I 02:18:03.861 ServerApp] or http://127.0.0.1:8888/lab?token=31ec81178347d467146131c34dd3506e039fa7dc45124804[I 02:18:03.862 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).[W 02:18:03.879 ServerApp] No web browser found: could not locate runnable browser.[C 02:18:03.880 ServerApp]
To access the server, open this file in a browser: file:///root/.local/share/jupyter/runtime/jpserver-19-open.html Or copy and paste one of these URLs: http://x.x.1.17:8888/lab?token=31ec81178347d467146131c34dd3506e039fa7dc45124804 or http://127.0.0.1:8888/lab?token=31ec81178347d467146131c34dd3506e039fa7dc45124804
To access the server, go to a remote computer and type the below line to access the Jupyter notebook (see above):
http://x.x.1.17:8888/lab?token=31ec81178347d467146131c34dd3506e039fa7dc45124804
The following interface shown in the figure below runs on the browser:
Figure: Jupyter notebook running on a browser in a remote machine. [1] select and double-click on the smartcam notebook. [2] select, edit and modify a cell to run. [3] click on the forward arrow to run the selected cell and advance to the next one.
At this point the smarcam notebook is active and ready to run. Select each consecutive cell and edit, modify and run as desired. In this particular case we choose the mipi camera supplied by the vendor.
source = "mipi" # choose either 'mipi' or 'usb'
We also choose to have RTSP output to be able to stream from a remote computer.
DP_output=False # True to choose DP output, False to choose RTSP output
After all the cells have been run for the RTSP output branch, the following output indicates what to do next:
Video is now streaming from mipi source.
Run the command "ffplay rtsp://x.x.1.17:5000/test" in another PC which have network access to the SoM board to view the video.From a remote computer run the command line
ffplay rtsp://x.x.1.17:5000/testThe following is a screen capture of the working application:
Figure: Smart Camera kv260-smartcam Application running.
The figure above shows the Smart Camera kv260-smartcam Application capture three faces simultaneously. The app captures the faces in real-time while both the camera itself is in motion and the subject is in motion. The following video demonstrates the application working in real-time.
Video: Smart Camera kv260-smartcam Application running in real-time.
Verify which docker is active by using docker ps command to see it in the running containers list.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4d4e8a0dd2dd xilinx/smartcam:2022.1 "bash" 11 hours ago Up 11 hours optimistic_chandrasekhar
$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZExilinx/smartcam 2022.1 aa0270aef908 6 months ago 1.41GB
Next we stop and then remove the docker
$ docker stop optimistic_chandrasekhar$ docker rm optimistic_chandrasekhar
Verify that the docker is not running any longer
$ docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
The docker image is still loaded
$ docker imagesREPOSITORY TAG IMAGE ID CREATED SIZExilinx/smartcam 2022.1 aa0270aef908 6 months ago 1.41GB
The following command will remove all containers
$ docker rm `docker ps -a -q`
Remove all “Exited” containers:
$ docker rm $(docker ps -a | grep 'Exited' | awk '{print $1}')
Remove all images matching a name (xilinx in this case):
$ docker rmi $(docker images | grep xilinx | awk '{print $3}')
Docker has a single command that cleans up all dangling resources, such as images, containers, volumes, and networks, not tagged or connected to a container.
$ docker system prune
For our particular case, we verify disk space use before we rm the docker image
$ sudo du -a /var | sort -n -r | head -n 10
[sudo] password for ubuntu:
4240500 /var
4022796 /var/lib
2130392 /var/lib/snapd
1506484 /var/lib/docker
1505588 /var/lib/snapd/snaps
1502732 /var/lib/docker/overlay2
636052 /var/lib/docker/overlay2/1d11636fc2d2f5dff498c2c1411948dbb705144ec2567be2e4cb276df534c431
636036 /var/lib/docker/overlay2/1d11636fc2d2f5dff498c2c1411948dbb705144ec2567be2e4cb276df534c431/diff
622340 /var/lib/snapd/seed
622244 /var/lib/snapd/seed/snaps
next we remove the docker image with the command
$ docker rmi $(docker images | grep xilinx | awk '{print $3}')
Finally we verify that the disk space has been returned
$ sudo du -a /var | sort -n -r | head -n 102734428 /var2516716 /var/lib2130392 /var/lib/snapd1505588 /var/lib/snapd/snaps622340 /var/lib/snapd/seed622244 /var/lib/snapd/seed/snaps458588 /var/lib/snapd/snaps/gnome-42-2204_69.snap342056 /var/lib/snapd/snaps/gnome-3-38-2004_138.snap239004 /var/lib/snapd/seed/snaps/gnome-3-38-2004_107.snap221168 /var/lib/snapd/snaps/firefox_2486.snap
And verify the total disk space has returned from 95% utilization after we loaded the docker image to a reasonable 84% disk utilization after we remove the docker image.
$ df -hFilesystem Size Used Avail Use% Mounted on/dev/mmcblk1p2 14G 11G 2.2G 84% /
Xilinx provides a plethora of hardware, AI tools and AI libraries to users. The combination of hardware, tools, models types and samples is very rich. This section will explore the availability of the universe composed of hardware, tools, libraries and experiment with how easily these can be manipulated and utilized. The primary reason to explore with the Vitis AI Libraries is to use pre-trained Xilinx models to quickly build applications.
AI Libraries and PetaLinux 2022.2 Boot FWThe primary link for this section is found below.
Running Vitis AI Library Examples
The original SD card that came with the AMD Xilinx Kria KV260 Vision AI starter Kit had too little room for all the development tools and example images needed to experiment with the Vitis AI Libraries. A larger SD was used to instal a PetaLinux Board Image.
SD Image for xilinx-kv260-dpu-v2022.1-v2.5.0
This image is used exclusively for experimenting with Vitis AI Libraries and quickly building examples projects. Current designs only supports Vitis AI 2.5.0 libraries. At the very least, If a developer or user has custom models that are completely different from the models supported by the Vitis AI Library or have specialized post-processing requirements, the user can use the Vitis AI library implementations for reference. The figure below shows the portal to the Vitis AI 2.5.0 libraries.
The Vitis AI Library contains four parts:
Base librariesThe base libraries provide the basic programming interface with the DPU and the available post-processing modules of each model. dpu_task is the interface library for DPU operations. cpu_task is the interface library for operations that are assigned to the CPU. xnnpp is the post-processing library for each model, with built-in modules such as optimization and acceleration.Model librariesThe model libraries implement most of the open-source neural network deployment including common types of networks, such as classification, detection, segmentation, and others. These libraries provide an easy-to-use and fast development method with a unified interface, which apply to the Xilinx models or custom models.Library test samplesThe library test samples are used to quickly test and evaluate the model libraries.Application demosThe application demos show you how to use the Vitis AI Library to develop applications.The figure below shows the block diagram for the Vitis AI 2.5.0 libraries.
The PetaLinux image contains all the libraries and tools for on-target development. Xilinx provides the resources for programing with cross compilation tools in a host and then transfer the executable to the board. In our case we compile directly on the target board the Kria KV260.
The first step is to install the board image by flashing the SD card with Etcher. The Vitis AI runtime packages and the Vitis AI Library model packages and library packages and other samples and models are built into the board image. Therefore the user does not have to install these on the board separately.
The following is an example of source files for example source code that will pair a neural network model type with model data to generate a result.
root@xilinx-kv260-starterkit-20221:~/Vitis-AI/examples/Vitis-AI-Library/samples/yolov3# ls -altotal 1112drwxr-xr-x 2 root root 4096 Apr 3 11:56 .drwxr-xr-x 53 root root 4096 Apr 3 11:48 ..-rwxr-xr-x 1 root root 1190 Mar 9 2018 build.sh-rw-r--r-- 1 root root 1867 Mar 9 2018 process_result.hpp-rw-r--r-- 1 root root 6258 Mar 9 2018 readme-rw-r--r-- 1 root root 3262 Mar 9 2018 test_accuracy_tiny_yolov3_vmss.cpp-rw-r--r-- 1 root root 3225 Mar 9 2018 test_accuracy_yolov3_adas_pruned_0_9.cpp-rw-r--r-- 1 root root 3271 Mar 9 2018 test_accuracy_yolov3_bdd.cpp-rw-r--r-- 1 root root 4095 Mar 9 2018 test_accuracy_yolov3_coco_416_tf2_mt.cpp-rw-r--r-- 1 root root 4161 Mar 9 2018 test_accuracy_yolov3_mt.cpp-rw-r--r-- 1 root root 3387 Mar 9 2018 test_accuracy_yolov3_voc.cpp-rw-r--r-- 1 root root 3512 Mar 9 2018 test_accuracy_yolov3_voc_tf.cpp-rw-r--r-- 1 root root 1070 Mar 9 2018 test_jpeg_yolov3.cpp-rw-r--r-- 1 root root 865 Mar 9 2018 test_performance_yolov3.cpp-rw-r--r-- 1 root root 1071 Mar 9 2018 test_video_yolov3.cpp
To compile the source code on target (i.e. Kria KV260) the simply run the build.sh command on a terminal.
# sh build.sh
After a short while the executable programs are available for testing. In our case we issue the following command:
# ./test_jpeg_yolov3 yolov3_bdd yolov3/sample_yolov3.jpg
After successful completion the result is shown on the terminal
I0403 12:51:29.856228 2106 demo.hpp:1183] batch: 0 image: /home/petalinux/samples2/samples/yolov3/sample_yolov3.jpgI0403 12:51:29.856499 2106 process_result.hpp:44] RESULT: 2 355.623 143.204 416.623 164.574 0.843508I0403 12:51:29.856628 2106 process_result.hpp:44] RESULT: 2 115.72 139.289 185.977 182.711 0.730628I0403 12:51:29.856691 2106 process_result.hpp:44] RESULT: 2 401.178 135.767 512 245.528 0.730601I0403 12:51:29.856755 2106 process_result.hpp:44] RESULT: 2 323.695 141.1 360.952 158.049 0.705315I0403 12:51:29.856813 2106 process_result.hpp:44] RESULT: 2 299.126 135.785 334.126 153.827 0.698933I0403 12:51:29.856869 2106 process_result.hpp:44] RESULT: 2 164.063 136.371 196.942 159.537 0.592455I0403 12:51:29.856925 2106 process_result.hpp:44] RESULT: 2 -0.840607 131.914 129.028 256 0.561093
A different test run and its output:
# ./test_performance_yolov3 yolov3_bdd test_performance_yolov3.list -t 8I0403 13:02:05.918325 2986 benchmark.hpp:184] writing report to <STDOUT>I0403 13:02:08.667546 2986 benchmark.hpp:211] waiting for 0/30 seconds, 8 threads runningI0403 13:02:18.667785 2986 benchmark.hpp:211] waiting for 10/30 seconds, 8 threads runningI0403 13:02:28.668031 2986 benchmark.hpp:211] waiting for 20/30 seconds, 8 threads runningI0403 13:02:38.668344 2986 benchmark.hpp:219] waiting for threads terminatedFPS=14.2527
These simple tests confirm the following:
AI Libraries provide the bases to quickly build applications. As a review point, the Ubuntu image used is found in the link below:
https://ubuntu.com/download/amd-xilinx
Some other links used in this effort:
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/1641152513/Kria+K26+SOM
https://www.hackster.io/dsp2/easy-machine-learning-on-ubuntu-with-the-xilinx-kria-kv260-c77ed1
sudo apt install vitis-ai-runtime vitis-ai-library
sudo snap install xlnx-vai-lib-samplessudo xmutil desktop_disable
xdputil query
To change the ssh keys:
ssh-keygen -f ~/.ssh/known_hosts -R ip.addr
Please note:
for later version of DPU designs you may check with "xdputil query" instead of "dexplorer -w"!
This below link
Will provide the file:
petalinux-sdimage-2021.1-update1.wic.xz
After initial boot of a new SD card image it is best practice to execute sudo dnf update in order to update core utilities that may have been released following the SD card image release.
The work has been presented above. These are some compact itemized reviews for the documentation and for the hardware/software integration.
The following are results and take away of the AMD Xilinx Kria KV260 Vision AI Starter Kit RoadTest:
existing AMD/Xilinx AI application software is effective and error free.All the RoadTest objectives were met satisfactorily and are summarized in the Results section above this one.
The AMD Xilinx Kria KV260 Vision AI Starter Kit board is a capable hardware that can be the starting point for any number of embedded projects, and it is highly recommended all on its own. Existing documentation is ample and comprehensive. We recommend the AMD Xilinx Kria KV260 Vision AI Starter Kit and the available AI vision software application as an efficient starting platform for AI vision applications.
