Evaluation Type: Development Boards & Tools
Did you receive all parts the manufacturer stated would be included in the package?: True
What other parts do you consider comparable to this product?: From a pure computer vision and machine learning standpoint the market also has some parts from the Nvidia Jetson line, and Intel has the Apollo Lake with Movidius.
What were the biggest problems encountered?: Lack of documentation from the dual camera mezzanine, would be good if there was at least a documented driver or library stating how to manipulate MIPI-CSI mixer, or what parameters can be tweaked on the camera sensors.
When I signed up for this road test I had I hopes of combining some machine vision and machine learning with the seemingly very capable Ultra96v2 and ON Semiconductor dual camera mezzanine. And maybe more depending the speed at which I could develop/integrate new components. I have had some experience with the Xilinx Ultrascale+ ZynqMP devices, but mostly from a software perspective with canned solutions from Xilinx or coworkers.
Out of the Box (OOB)
The out of the box experience is fairly good, as Avnet has provided both an OOB for just the Ultra96v2 that provides a GUI interface, along with a web interface. With this you can be up and running software very quickly. The OOB SD card image for wrapping in the ON Semi dual camera mezz boots right up, but I found there were issues with the video from the two sensors streaming for more than 8-10 seconds. I was able to work around this by just running the commands in run_1920_1080 script by hand on the command line. I would guess there is some timing that isn't quite right (there is a 5 second sleep in the script), which resolved itself by running each command with the speed of a typing turtle between.
Another OOB image that I found very helpful in demonstrating the AI/ML/CV capabilities of this card was the Xilinx Vitis-AI 1.3 image found here (with the addition of a USB Logitech webcam):
With this image and the instructions on the following two sites, you can run through many algorithms without writing any code:
https://www.hackster.io/AlbertaBeef/vitis-ai-1-3-flow-for-avnet-vitis-platforms-cd0c51 (uses videos images from the SD card)
https://www.hackster.io/AlbertaBeef/face-detection-and-tracking-in-python-on-ultra96-v2-02d104 (uses a USB webcam)
These take advantage of Xilinx's PYNQ, DPU, and Vitis-AI (which might just be the embodiment of the previous two).
Scratching Below the Layers
Although Avnet has done a great job of detailing instructions of how to rebuild their OOB images:
Ultra96-V2 ON Semiconductor Dual Camera Mezzanine hardware build instructions
Ultra96-V2 Dual Camera Mezzanine Petalinux Build Instructions
I found the process to be a little weighty(time and hard drive space) and unfortunately not 100% repeatable for me. The OSD license issue when rebuilding the HDL was the biggest set back, and I could never figure out how to enable that license, as it never showed up in the Xilinx license manager website tool for me.
I was able to do two different modifications of the HDL design, where I removed the OSD IP block and either inserted the Video Mixer (as suggested by the OSD web page) or by just removing OSD and TPG and injecting the framebuffer rd directly into the video output chain. Both of these builds made it through Vivado build, but integration with petalinux produced running target images, that wouldn't install a driver on /dev/media0, which would result in the precanned software script to fail.
I think with some patience and some beginning training on Vivado (to fix timing errors reported during build), and on petalinux and yocto a user could get to the point of building some very cool projects with the Ultra96v2. To get the best bang for the buck from the ON Semi dual camera mezz, I think someone will either need to release some documents, or create a very intuitive driver/library API for it.