Actions
Forum Thread Details
  • Replies 3 replies
  • Subscribers 331 subscribers
  • Views 117 views
  • Users 0 members are here
Related

ZUBoard memory speeds - results

tjaekel

For all interested in the memory speeds possible (what to expect) and comparing the options to use the fastest memory,
e.g. for recording or transferring live data to a fast media.

I have measured the ZUBoard memory speeds (plus ETH speed):

eMMC SD card (boot) USB    1) 4) SSD    2) 5) RAM (as ref.)
wr 4.6 MB/s 21 MB/s - best 26 8 MB/s - best 38 174 MB/s 395 MB/s
rd 85 MB/s 23 MB/s - best 35 7 MB/s - best 40 427 MB/s 805 MB/s - best 883

Ethernet speed:

ETH to host (iperf3)
tx 860 .. 910 MB/s  3)
rx 860 .. 905 MB/s  3)

1) depends on USB stick type, USB 3.x etc.
2) on Petalinux image (not PYNQ), with 256GB SSD and AVNET SSD/M.2 module
3) depends a bit where the rootfs is running on, e.g. a faster SD card vs. eMMC module
4) ATT: even faster as SD card - it is not possible to boot from USB: boot still via SD card (/boot) and change to /dev/sda as rootfs and main filesystem)? - possible?:
    Not possible to change during boot to USB (as /dev/sda) for the rootfs, to continue there!
5) I had a lot of trouble to use an SSD inside an USB enclosure:
    - most of the time the ZUBoard does not see /dev/sda
    - if it sees it: often with size 0
    - if it sees it properly with a valid size - I can "damage" quickly the SSD USB memory
      ATT: it looks also dead as USB SSD on  Windows PC! Hard to recover such a "bricked" USB SDD! (just via Windows host)

Conclusion:

  • the SD card is OK (20 MB/s), but an USB memory can be faster (up to 2x faster as SD card):
    unfortunately, we cannot mount the rootfs to USB (/dev/sda) during boot:
    the USB (/dev/sda) seems to enabled later, boot will hang
  • the eMMC module is faster on READ (85 MB/s) compared to SD card,
    but very bad on WRITE to eMMC (the worst: 4.5 MB/s):
    not an option when we need a lot of WRITE data on rootfs (yes, eMMC can host and run rootfs,
    possible to change from SD card boot to continue on eMMC)
  • USB memory sticks can be faster as SD card (2x faster as SD card):
    but I have trouble to use SSD in USB enclosure as USB stick (often not seen or "bricked" after use)
  • SSD would be the best option:
    BUT: SSD is not supported on PYNQ image (yet)!
    It can also host and run the rootfs: boot from SD card, change bootargs and change to nvme0n1 as rootfs device.
    Remark: it is not possible to image SSD and boot from SSD (directly, via BOOT.BIN, image.ub, still from SD card needed)

Fasted Speed to transfer data:

The best option to bring data with fastest speed is:
use ETH (1G PHY) and transfer the data to a host PC: 900 MB/s is possible (measured with iperf3).

  • Remark:
    the speed on memories in a PYNQ Jupyter script goes down by 50%: expect 1/2 of the max. possible speed inside a Python script.

  • How to measure the memory speed?

    In general, you do this:

    • have the memory mounted already (often it is already mounted, e.g. for your rootfs as / or USB via /run/media/sda)
    • otherwise, mount the device, e.g. for USB, seen as /dev/sda, but potentially not mounted yet:
      SSD should appear as: nvme0n1
      eMMC should appear as: mmcblk1p1 and mmcblk1p2 (use the mmcblk1p2 as media, where the rootfs is located, the larger size available)
      USB should appear as: sda
      SD card (when booted from it): /   (as mmcblk0p2, already your rootfs)
      use command
          lsblk
      in order to see which block devices you have, if they are already mounted...
    • use command dd for a WRITE and READ speed test
    • flash the caches on system before you are doing a read after a write was done
      (otherwise, you could get a very impressive speed)

    Here an example how to measure the WR and RD speed on an USB memory stick, connected on the ZUBoard:

    mkdir usb

    sudo mount /dev/sda ~/usb

    sudo dd if=/dev/zero of=usb/tmpfile bs=1M count=1024 oflag=direct

    Before you read back - clear the caches:

    sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"

    Now do a read (the inverse on dd):

    sudo dd if=usb/tmpfile of=/dev/null bs=1M count=1024

    If you want to measure the speed on RAM (ramdisk) - 1G max. RAM size in total available: change down to 512M as the max. test file size:

    mkdir ram

    sudo mount -t tmpfs -o size=512M tmps ~/ram

    sudo dd if=/dev/zero of=ram/tmpfile bs=1M count=512

    sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"

    sudo dd if=ram/tmpfile of=/dev/null bs=1M count=512

  • In order to make the memory speed comparison complete:
    I have checked also the memory speeds on a Digilent PYNQ-Z1 board vs. AVNET ZUBoard:
    the AVNET ZUBoard is faster! - on onboard DDR memory (and FPGA PL overlays).

    Results:

    SD card
    PYNQ-Z1    		 USB
    PYNQ-Z1    		 RAM
    PYNQ-Z1    		 SD card
    ZUBoard

    USB
    ZUBoard

    		 RAM
    ZUBoard
    wr 19 MB/s 23 MB/s 158 MB/s 21 MB/s 30 MB/s 395 MB/s
    rd 24 MB/s 24.5 MB/s 340 MB/s 23 MB/s 33 MB/s 880 MB/s

    So, no wonder:

    • SD card speeds (boot media and rootfs on SD card) are similar (limited by the SDIO controller config, seems to be similar on both boards,
      similar PYNQ image used?)
    • USB on ZUBoard is a bit faster (but marginal: both boards are slow on USB)
    • the RAM speed differs a lot - sure: ZUBoard uses a newer, faster FPGA (zynqmp), with A53 cores and faster
      FPGA clock speeds (including larger DDR RAM on board, different DDR memory type...)

    So, the AVNET ZUBoard is a slightly better/faster option,
    but:
    having the SSD/M.2 working on PYNQ would make this AVNET ZUBoard outstanding,
    otherwise : no real difference in performance (assuming SD card is the main memory)

    ZUBoard runs 2x faster on onboard DDR memory, but the SD card as boot media and for rootfs slows it down.
    So, at overall: ZUBoard is not really faster as PYNQ-Z1 - even it has all the potentials to be 2x faster!
    (but not used due to missing SSD/PCIe/M.2 support)

    Please:!
    enable SSD/M.2 for AVNET module in PYNQ image for ZUBoard
    (otherwise no real performance benefits: the SSD/M.2 module would make the ZUBoard way faster!
    but running from SD card is slow as well and no benefits from a newer board, faster FPGA version).

    With the AVNET SSD module it would make this AVNET ZUBoard 2x faster compared to PYNQ-Z1!
    (esp. if rootfs is on SSD)