ZUBoard running PYNQ (Python and PL bit files) with a AVNET SSD/M.2 - works.
No need to setup a build environment, no need to check out source code repositories like PYNQ, Petalinux BSP, in order to modify and build it...
I have placed all needed files in order to "patch" the original ZUBoard PYNQ image 3.0.1 to make the SSD/M.2 working,
plus the instruction steps to modify (see below):
Steps to do:
- Have a regular PYNQ image on SD card:
- boot this regular image
- you should be able to use network in order to copy the needed files to the
PYNQ working (home) directory and to work from there then in an UART terminal
- for instance: have all files from the ZIP extracted in a folder patch_files on $HOME directory
(copy to ZUBoard PYNQ system running via network device) - check if the boot partition of the SD card is mounted (often visible as /boot):
lsblk
if mmcblk0p1 is not mounted - mount it, e.g. as boot
- sudo cp the files:
BOOT.BIN
image.ub
boot.scr
into this boot partition (on mmcblk0p1) - take the tar file modules_pl and extract it into folder:
/lib/modules
- you should have a folder with the name 5.15.36-xilinx-v2022.2 now
- the folder 5.15.19-xilinx-v2022.1 : you could delete later (when all works fine), rename it
for now to make sure it is not used anymore
- extract modules_pl via command:
sudo tar -xvf modules_pl -C /lib/modules
- check if you see now a folder /lib/modules/5.15.36-xilinx-v2022.2 - create an entry in sudoers in order to avoid to provide a password if you do 'sudo':
edit the file /etc/sudoers and add this line:
xilinx ALL=(ALL:ALL) NOPASSWD: ALL - copy the file create_dri.sh into folder:
/etc/profile.d - do the command:
sync && sync - You can reboot:
- stop the U-Boot (any key hit) and change the bootargs:
setenv bootargs "console=ttyPS0,115200 earlycon clk_ignore_unused uio_pdrv_genirq.of_id=generic-uio root=/dev/mmcblk0p2 rw rootwait cma=512M"
saveenv
boot
You should see now the SSD via
lsblk
as nvme0n1 !
if not:
- check if you see it as /dev/nvme0 :
potentially, you might need to create a partition on SSD and to format it (use just ext4 format)As a cross check if all works:
- check if the I2C interrupt is now set to 126:
cat /proc/interrupts
a0030000.i2c should use now 126 (not 121 anymore)!
- check also if a folder
/dev/dri/by-path
was created.
- check also if sudo does not ask anymore for a password, e.g.:
sudo ls /dev/dri/by-pathNow your system should be good to continue with the Python part.
- If you want to use a BITFILE in the Python scripts (as PL Overlay):
sudo cp this BITFILE into folder:
/lib/firmware
remark: in the Python script we will load this via a shell command, but using ol.Overlay(..., download=False),
and before you might need to load the device tree overlay dt.dtbo as well - if you want to use INTerrupts
- keep the dt.dtbo together with your BITFILE in /home/xilinx/pynq/overlays/BITFILE_folder -
Your Python script should look like this:
from pynq import Overlay, Interrupt, devicetree
from cffi import FFI
import os#load the FPGA file: works only manually with SSD module support:
os.system("echo ZUBOARD_SPI.bit > /sys/class/fpga_manager/fpga0/firmware")#load the device tree to add 'fabric', needed if we want to use INTerrupts
dt = devicetree.DeviceTreeSegment("/home/xilinx/pynq/overlays/BITFILE_folder/dt.dtbo")
dt.insert()
if dt.is_dtbo_applied():
print("APPLIED")
else:
print("NOT applied")#create the Overlay but without to load!
ol = Overlay("BITFILE.xsa", download=False) - Now your BITFILE should be loaded and Python should be able to use all its features,
including having INT signals and axi_intc in your design, etc. - as a cross check after your BITFILE was loaded and works in Python... - check if you can use SSD:
mkdir ssd
sudo mount /dev/nvme0n1 ~/ssd
ls ssd
This should work (if SSD is partitioned and formatted). You should see the SSD working even you have
loaded your BITFILE.
(not sure?: if not: maybe we have to enable the PCIe in the user block diagram on the PS block, but I have not
seen an issue so far with my own simple bit files)
REMARK:
There is already a bit file loaded during boot (obviously needed for the PCIe/SSD). It has also blocks
like I2C or GPIO in it (like a system.bit file). On one of the I2C blocks (I guess the second) we have changed the INT vector number!
(we must do so in order to have INT working in our bitfile overlay)
So, one of the I2Cs in system.bit file cannot use anymore INT! (I guess it is intended for reading the I2C DNA chip on an external module).
The files:
dt.dtsi
boot.txt
boot2.txt
are just source files to create the binary file versions needed (via dtc and mkimage, on Linux host).
The files
boot2.txt and boot2.cmd (binary)
are not anymore used (U-BOOT is doing the device tree modification which is now in file boot.scr).
- Options:
You have the option, after you have partitioned and formatted the SSD module (as ext4), to "copy" the entire Linux rootfs file system,
the content of the SD card partition 2, visible as /dev/mmcblk0p2, to the SSD module:
- SSD cannot have a boot partition, so you create just one primary ext4 partition for the Linux files system
(we could not boot anyway from SSD, so no boot partition needed)
- best is not to use cp - instead use tar, at best dd, with keeping symbolic links correct in tar archive or "copy" directly as it is
- ATTENTION:
you cannot tar or dd the entire active root directory, starting from /, when you want to store also the tar file on the same device in use
(a recursive tar!)
==> use an USB memory stick as temporary storage:
- format the USB memory stick as ext4 (for large file size and Linux file system)
- tar the entire rootfs / to a tar file on USB stick
- now you can use tar to unpack the USB stick tar file into the SSD partition
- BTW: instead of tar, better to use:
dd if=... of=... ...
to do it.
- after the SSD has now the same rootfs - check if you see the / and all files on SSD device, nvme0n1, after you have mounted the SSD
you have to boot but stop the U-Boot (by hitting keys right at the beginning of the boot process, watch the UART terminal)
- now you have to set the bootargs for the U-Boot to use the SSD for the rootfs:
setenv bootargs "console=ttyPS0,115200 earlycon clk_ignore_unused uio_pdrv_genirq.of_id=generic-uio root=/dev/nvme0n1 rw rootwait cma=512M"
saveenv
boot
- now, it should boot from SD card but it uses the main Linux filesystem rootfs on SSD - much faster!
The SD card is still needed to boot (partition 1 with BOOT files) but their second partition mmcblk0p2 is now free as a regular device.
REMARK:
For me, only one specific SSD/M.2 module works on ZUBOARD:
- try to find the KingSpec 256G module with B+M key
- it must have a B (+M) key, as PCIe Gen3x2 (2-lanes, not 4! and not just an M key!)
The correct way is actually:
- update the Petalinux BSP in PYNQ
- add the PCIe and NVME to it
- build a new PYNQ image and
- provide a newer ZUBoard PYNQ image - thank you
