I wanted to create a quick post explaining a small trick I noticed in the build process that allowed me to efficiently complete a build for Vitis and the Xilinx ZCU104 development platform. I learned that if you abuse a tool, you can still make it behave improperly!
Summary:
- Uncheck the Autoupdate checkbox during an Ubuntu 18.04.1 LTS install
- Make sure to check that the # of JOBS being called out does NOT exceed the # of cores assigned to your Virtual Machine
Details:
I was working on the next version of build instructions for the community. These are used to setup VirtualBox Virtual Machines to aid in setting up Vivado, SDK and SDSoC tools. With the new Vitis all-in-one environment, I am modifying this to be used in setting up Vitis tools.
I wanted to validate the tool configuration and decided to build a ZCU104 platform as that was the CLOSEST thing to the Avnet UltraZed-EV.
After making sure I followed all the online install instructions
[ https://www.xilinx.com/html_docs/xilinx2019_2/vitis_doc/Chunk1858803630.html ]
blending a few tricks that we typically recommend in our VirtualBox setup guide, I had a Virtual Machine that seemed like it was good!
First, I cloned the Xilinx GIT repository for the Embedded Platform Source.
[ https://www.github.com/Xilinx/Vitis_Embedded_platform_Source ]
Next, I moved into the proper folder:
The readme states to run 'make all'
At this point, things would spin for a while, but eventually fail with a very strange error.
After asking around, I discovered that almost everyone (Adam Taylor, coworkers, etc.) were all succeeding with Ubuntu 16 installs, Windows, or other flavors of Linux.
While Ubuntu 18.04.1 LTS IS a supported OS
just to see if there was a difference in my install guide, I repeated this entire exercise on an Ubuntu 16 install. The build would complete, but it would take a VERY long time and ~46.7GB of ram used by the tools - which is likely the source of the slow build time. My machine only had 32GB of ram assigned.
** NOTE that in Ubuntu 18, DURING install it will DEFAULT to upgrade automatically - Ubuntu 18.04.3+ is NOT supported, so make sure to UNCHECK that checkbox!
Apparently what was happening, Ubuntu 16 was MUCH more lax about threads and jobs than Ubuntu 18. Ubuntu 16 would just choke on the jobs until it completed. This I can certainly attest as true, as my PC would become nearly unusable during the build. 100% of the CPU, 150% of the memory, 100% of the hard drive all being used for hours. Once complete, everything would be fine.
I've seen multiple times where Ubuntu 18 caused strange issues with various build tools. That is, this was likely NOT a Xilinx specific issue!
So, back to Ubuntu 18 and all the additional security! Another example of this is when you add new shortcuts to the desktop. In Ubuntu 16, you just ... add them. In Ubuntu 18, you have to add, then authorize the icon.
So what was happening, during the Vivado build, we were getting too many build threads. I believe after a while, they would timeout (or similar) and Ubuntu 18 would...well, kill them off. As this was happening, we would not get proper implementation results, which then had the domino effect of causing a failed build.
After speaking with some of my colleagues over at Xilinx, we were able to work out a solution! First, if you are following guides like the ones from Adam Taylor's MicroZed Chronicles [ https://www.hackster.io/news/microzed-chronicles-getting-up-and-running-with-vitis-8f05eb013289 ], you are using the GUI. Xilinx did a fantastic job protecting a user from putting an incorrect amount of jobs in when using the GUI.
The issue comes from a combination of my using a build script and Ubuntu 18's intolerance for system abuse.
Within the zcu104_base_xsa.tcl file, there is a command that calls out the number of jobs to build. The script engine assumes that the user knows what they are doing!
After seeing that, I changed the jobs to 4 (the number of cores on this build machine).
And instead of failing after many hours, or even taking 1/2 a work day to complete 1 build (under Ubuntu 16), I was pleasantly surprised to see that my build MACHINE running Ubuntu 18 used less than 16GB (OS + Tools) and completed the Vivado build in ~17 minutes
PetaLinux in ~35 minutes and <1 minute for Vitis platform generation (it was copying files)
These time stamps correlate to the time it took to start building and last time the build log was touched - which I validated using:
This means a total of about 50 minutes to create a 'FROM SCRATCH' 7EV MPSoC platform for Vitis. That is really amazing!
What did we learn? If you abuse a tool's parameters, expect funny results, inconsistency or plain old errors. Second, once you follow provided parameters, resource use for Vitis falls DIRECTLY in line with what Xilinx predicts for system resource needs and was REALLY fast for a WHOLE SCRATCH platform build!
Luckily, if you are using the Avnet repository, we've had a way to automatically determined the number of jobs and use THAT value for quite some time, it's probably why some of my testing did not seem to indicate problems when I was messing around with Avnet code.