Xilinx Vitis-AI - Who is using this framework ?

Hi Mario,

I'm surprised that you haven't gotten a lot of responses to this post as I know that there are a lot of Xilinx users in the E14 community.

I've really appreciated the Vitis-AI posts that you've done on Hackster.  I've been taking a break from the Xilinx FPGAs and been focusing on TinyML, so I haven't tried your latest Vitis-AI 1.3 stuff yet.

I've had problems using the IP cameras with the VCU and GStreamer on the UtraZed-EV.  I'll probably get back to working on this later this summer.  Have you done much with the VCU?  Has anyone replaced Jason Moss after he left Avnet to go to Xilinx?  I think he was the resident expert.  It would be great to see projects integrating the VCU and GPU.

Ralph

HI Ralph,

Always appreciate your input and feedback.

We are still working on a path forward to fill the void Jason left.  I think he was a man of many hats and we are working towards some new VCU designs across a few resources.

He was a man of many hats and talents!

Regardless, the new designs are being targeted for the 2020.2 release, which as of this post is not official yet...so buyer beware!  The 2020.2 branch is almost stable.  As of this exact post, the build scripts for those projects are stable enough for validation testing.

For those in the future:

https://github.com/avnet/

Grab the petalinux, hdl, bdf, repositories.  Then checkout the 2020.2 branch (or in the future, the appropriate tag).  In the petalinux/scripts folder, you should be able to target the make_uz7ev_evcc_hdmi_v.sh or the make_uz7ev_evcc_quadcam_h_v.sh build scripts.  Once completed and the files are put on an SDCARD for testing, I have been told there are scripts that can be run in /home/root/ that setup everything for the tests.

The latter requiring the Quadcam FMC kit.  The former being essentially the VCU design that Jason worked on (on the Xterra2 server).  The design has been extended to include HDMI as inputs/outputs, but still has the original webcam->RTSP capabilities.  Again, not validation tested at this time, so any support will be very limited until this is released.

--Dan

Ralph,

Thank you very much for the reply and feedback.
I am a VCU newbie, so will defer to Dan's response on the status of the VCU.

Can you clarify what you would like to see with the GPU ?

Regards,

Mario.

Thanks Dan.  Helpful as always.  I may actually get around to trying out the HDMI with the VCU .   Any preview as to what functionality you'll have in the new VCU designs?

Probably be bugging you guys mid-summer when I run into problems...

-Ralph

Hi Ralph,

For now, it just spins up the VCU.

4K HDMI RX -> VCU Enc -> VCU dec -> 4K HDMI TX

4K HDMI RX -> VCU Enc -> file

file -> VCU dec -> 4K HDMI TX

Quadcam -> VCU Enc -> VCU dec -> 4K HDMI TX

webcam -> VCU Enc -> RTSP

I am working towards a webinar on this, so anything in particular you think you'd like to see, let me know and I'll see if that will fit in with the messaging on the webinar.

--Dan

I was and still will be trying to implement an AI augmented NVR using multiple IP cameras for detection and identification.  There are the details of handling the decoded VCU output into the GPU and also trying to process multiple streams with the GPU.  Have you done anything using multiple video streams - webcam or recorded video?

I've been interested in Edge AI and now TinyML.  I thought that I could use lower resolution less capable camera modules with TinyML to do detection and use those results to determine which high resolution streams the UltraZed-EV should process for identification.

Not trying to build a product, just like trying out new technologies.  I'm sure the ADAS systems have much more complex configurations working.  Is there a Quadcam example using the GPU?

Ralph,

Which frameworks are you using for your experimentation ?
If you are using gstreamer, you may want to check if there are already gstreamer plug-ins that make use of the MALI GPU.

The only thing I did with graphics was to use the RGBA layer of the video mixer to display a static image with transparent regions,
combined with the other video mixer layers that were displaying live content via gstreamer.

Mario.