element14 Community
element14 Community
    Register Log In
  • Site
  • Search
  • Log In Register
  • Community Hub
    Community Hub
    • What's New on element14
    • Feedback and Support
    • Benefits of Membership
    • Personal Blogs
    • Members Area
    • Achievement Levels
  • Learn
    Learn
    • Ask an Expert
    • eBooks
    • element14 presents
    • Learning Center
    • Tech Spotlight
    • STEM Academy
    • Webinars, Training and Events
    • Learning Groups
  • Technologies
    Technologies
    • 3D Printing
    • FPGA
    • Industrial Automation
    • Internet of Things
    • Power & Energy
    • Sensors
    • Technology Groups
  • Challenges & Projects
    Challenges & Projects
    • Design Challenges
    • element14 presents Projects
    • Project14
    • Arduino Projects
    • Raspberry Pi Projects
    • Project Groups
  • Products
    Products
    • Arduino
    • Avnet Boards Community
    • Dev Tools
    • Manufacturers
    • Multicomp Pro
    • Product Groups
    • Raspberry Pi
    • RoadTests & Reviews
  • Store
    Store
    • Visit Your Store
    • Choose another store...
      • Europe
      •  Austria (German)
      •  Belgium (Dutch, French)
      •  Bulgaria (Bulgarian)
      •  Czech Republic (Czech)
      •  Denmark (Danish)
      •  Estonia (Estonian)
      •  Finland (Finnish)
      •  France (French)
      •  Germany (German)
      •  Hungary (Hungarian)
      •  Ireland
      •  Israel
      •  Italy (Italian)
      •  Latvia (Latvian)
      •  
      •  Lithuania (Lithuanian)
      •  Netherlands (Dutch)
      •  Norway (Norwegian)
      •  Poland (Polish)
      •  Portugal (Portuguese)
      •  Romania (Romanian)
      •  Russia (Russian)
      •  Slovakia (Slovak)
      •  Slovenia (Slovenian)
      •  Spain (Spanish)
      •  Sweden (Swedish)
      •  Switzerland(German, French)
      •  Turkey (Turkish)
      •  United Kingdom
      • Asia Pacific
      •  Australia
      •  China
      •  Hong Kong
      •  India
      •  Korea (Korean)
      •  Malaysia
      •  New Zealand
      •  Philippines
      •  Singapore
      •  Taiwan
      •  Thailand (Thai)
      • Americas
      •  Brazil (Portuguese)
      •  Canada
      •  Mexico (Spanish)
      •  United States
      Can't find the country/region you're looking for? Visit our export site or find a local distributor.
  • Translate
  • Profile
  • Settings
Eye On Intelligence Challenge
  • Challenges & Projects
  • Design Challenges
  • Eye On Intelligence Challenge
  • More
  • Cancel
Eye On Intelligence Challenge
Blog Blog 2 - FruitVision Scale - PYNQ on the Digilent Arty Z7 - Quickstart
  • Blog
  • Forum
  • Documents
  • Polls
  • Files
  • Members
  • Mentions
  • Sub-Groups
  • Tags
  • More
  • Cancel
  • New
Join Eye On Intelligence Challenge to participate - click to join for free!
  • Share
  • More
  • Cancel
Group Actions
  • Group RSS
  • More
  • Cancel
Engagement
  • Author Author: javagoza
  • Date Created: 18 Sep 2024 9:43 AM Date Created
  • Views 4281 views
  • Likes 13 likes
  • Comments 18 comments
  • video processing
  • python
  • arty z7
  • Face Detection
  • hdmi
  • xilinx
  • FruitVisionScale
  • jupyter
  • fpga
  • vivado
  • Eye on Intelligence Challenge
  • digilent
  • pynq
  • zynq-7000
  • amd
  • webcam
  • arm
  • microblaze
Related
Recommended

Blog 2 - FruitVision Scale - PYNQ on the Digilent Arty Z7 - Quickstart

javagoza
javagoza
18 Sep 2024
Blog 2 - FruitVision Scale - PYNQ on the Digilent Arty Z7 - Quickstart

This is my second entry for the AMD-sponsored “The Eye on Intelligence” experimentation challenge. In my initial post, I provided a brief introduction to my smart fruit scale project and explored the potential of the Arty Z7 development board.

In this follow-up post, I explore PYNQ, an AMD framework designed to simplify the use of adaptive computing systems.

image

Getting started with fruit recognition with PYNQ for smart scales.

What is PYNQ

Applying for this design challenge was partly motivated by my desire to expand my understanding of PYNQ. Although I had experimented with PYNQ and the Kria KV260 Vision AI Starter Kit, I had not yet embarked on a personal project. PYNQ is an AMD open-source project with a large community that simplifies the development of adaptive computing systems. PYNQ offers a flexible and versatile platform for embedded systems development. It combines the power of a programmable logic (PL) device with a user-friendly Python API. This unique combination allows developers to easily interact with a wide range of peripherals and devices.PYNQ enables application development with:

  • Parallel hardware execution: to accelerate computationally intensive tasks.
  • High frame rate video processing: ideal for real-time video applications and analytics.
  • Hardware-accelerated algorithms: optimization of performance-critical algorithms.
  • Real-time signal processing: for applications that require immediate response to input data.
  • High-bandwidth I/O: Facilitates efficient data transfer.
  • Low-latency control: Ensures accurate and timely system responses.

The PYNQ framework extends the Linux kernel with specific APIs that provide PYNQ libraries and drivers.

image

These APIs allow interactive Python (iPython) applications running in a Jupyter notebook to interact with the programmable logic design.

image

Communication between the processor system (PS) and the programmable logic (PL) depends on the interface used. PYNQ implementations offer five primary PS/PL interfaces:

  • Bitstream: Configures the PL for the desired application. The xdevcfg driver is used in PYNQ.
  • GPIO: Provides simple input/output. The sysgpio driver supports this.
  • Interrupts: Enables interrupt generation from the PL to the PS. The Userspace IO driver supports this.
  • Master AXI Interfaces: Used for PS-to-PL data transfer when the PS initiates. The devmem driver is used.
  • Slave AXI Interfaces: Used for PL-to-PS data transfer when the PL initiates. The xlnk driver enables these transfers.

PYNQ applications are developed using a Jupyter notebook via a web interface.

When creating PYNQ designs, a system optimization compiler can be used to generate the PL design and C libraries, which are then wrapped in Python.

PYNQ and Overlays

Overlays, or hardware libraries, are configurable FPGA designs that can be dynamically loaded to the PL to extend the capabilities of the PS. This allows for hardware acceleration of software applications and customization of the hardware platform.

image

PYNQ provides a Python interface for controlling overlays from the PS. This simplifies the process for software developers, as they can leverage pre-designed overlays without needing to understand FPGA design in detail.

A common use case is image processing. Overlays can be created for functions like edge detection, thresholding, and filtering. These overlays can then be loaded and controlled from Python, allowing for efficient and customizable image processing pipelines.

PYNQ overlays, or bitstreams, are dynamically loaded FPGA designs that extend the functionality of the Zynq SoC. By default, a base overlay is loaded at boot time. Additional overlays can be installed and loaded as needed.

Components of an Overlay

  • Bitstream: The configuration file for the FPGA fabric.
  • HWH File: Describes the available IP cores in the overlay.
  • Python API: Exposes the IP cores as attributes for Python interaction.

Loading an Overlay

  • Import the Overlay class: Use from pynq import Overlay.
  • Instantiate the Overlay: Specify the bitstream file name. This automatically downloads the bitstream and parses the HWH file.
from pynq import Overlay
overlay = Overlay("base.bit")

Use the BaseOverlay class: For the base overlay, use BaseOverlay to access pre-defined IP attributes.

image

Installing PYNQ on the Digilent Arty Z7

Prerequisites for the Arty Z7

  • Arty Z7 board
  • Computer with compatible browser
  • Ethernet cable
  • Micro USB cable
  • Micro-SD card with preloaded image, or blank card (Minimum 8GB recommended)

To install PYNQ on the Digilent Arty Z7, visit the official PYNQ boards page. Download the PYNQ image for the PYNQ-Z1; the PYNQ-Z1 and the Arty Z7 have nearly identical schematics, although the PYNQ-Z1 includes a PDM microphone, which the Arty Z7 lacks.

Go to https://www.pynq.io/boards.html

Or try these links:

PYNQ-Z1 v3.0.1 v2.7 v2.6 PYNQ setup guide Digilent Pynq-Z1

Flashing the PYNQ image

I used Balena Etcher, you don't need to unzip the image. https://etcher.balena.io/

Open Balena and select the downloaded zip file

image

Flashing

image

Then validating

image

image

We now have the card ready to be inserted into the card reader of the Arty Z7 development board.

image

The Arty Z7 supports three different boot modes: microSD, Quad SPI Flash, and JTAG. The boot mode is selected using the Mode jumper (JP4), which affects the state of the Zynq configuration pins after power-on. Before booting we must select the boot from the SD card by changing that jumper.

image

After a few seconds, the Yellow/Green LD12 / Done LED will light up to show that the Zynq device is operational.

After a minute you should see two Blue LD4 & LD5 LEDs and four Yellow/Green LD0-LD3 LEDs flash simultaneously. The Blue LD4-LD5 LEDs will then turn on and off while the Yellow/Green LD0-LD3 LEDs remain on. The system is now booted and ready for use.

image

Serial console USB UART Bridge (Serial Port)

The Arty Z7 includes an FTDI FT2232HQ USB-UART bridge (attached to connector J14) that lets you use PC applications to communicate with the board using standard COM port commands (or the tty interface in Linux). Drivers are automatically installed in Windows and newer versions of Linux. Serial port data is exchanged with the Zynq using a two-wire serial port (TXD/RXD). After the drivers are installed, I/O commands can be used from the PC directed to the COM port to produce serial data traffic on the Zynq pins. The port is tied to PS (MIO) pins and can be used in combination with the UART 0 controller.

image

Using Teraterm to acces the serial console: configure Serial POrt communication

image

 tty interface

image

Boot console log

In the following capture you can see the entire log generated by the messages to the tty interface when starting the Ubuntu Linux distribution that supports PYNQ.

U-Boot 2022.01 (Apr 04 2022 - 07:53:54 +0000)

CPU:   Zynq 7z020
Silicon: v3.1
DRAM:  ECC disabled 512 MiB
Flash: 0 Bytes
NAND:  0 MiB
MMC:   mmc@e0100000: 0
Loading Environment from FAT... *** Error - No Valid Environment Area found
*** Warning - bad env area, using default environment

In:    serial@e0000000
Out:   serial@e0000000
Err:   serial@e0000000
Net:
ZYNQ GEM: e000b000, mdio bus e000b000, phyaddr -1, interface rgmii-id
SF: Detected w25q128jv with page size 256 Bytes, erase size 4 KiB, total 16 MiB
eth0: ethernet@e000b000
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
2776 bytes read in 10 ms (270.5 KiB/s)
## Executing script at 03000000
Trying to load boot images from mmc0
6464304 bytes read in 365 ms (16.9 MiB/s)
## Loading kernel from FIT Image at 10000000 ...
   Using 'conf-1' configuration
   Verifying Hash Integrity ... OK
   Trying 'kernel-0' kernel subimage
     Description:  Linux Kernel
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x100000d4
     Data Size:    6442720 Bytes = 6.1 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x00080000
     Entry Point:  0x00080000
     Hash algo:    sha1
     Hash value:   b056acb229d05b806f975396dacae4ad04002e0c
   Verifying Hash Integrity ... sha1+ OK
## Loading fdt from FIT Image at 10000000 ...
   Using 'conf-1' configuration
   Verifying Hash Integrity ... OK
   Trying 'fdt-0' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x106250a8
     Data Size:    19767 Bytes = 19.3 KiB
     Architecture: ARM
     Hash algo:    sha1
     Hash value:   5779ba3e5b1fe87f7fdec42e7688329e3f583819
   Verifying Hash Integrity ... sha1+ OK
   Booting using the fdt blob at 0x106250a8
   Loading Kernel Image
   Loading Device Tree to 1ead4000, end 1eadbd36 ... OK

Starting kernel ...

Booting Linux on physical CPU 0x0
Linux version 5.15.19-xilinx-v2022.1 (oe-user@oe-host) (arm-xilinx-linux-gnueabi-gcc (GCC) 11.2.0, GNU ld (GNU Binutils) 2.37.20210721) #1 SMP PREEMPT Mon Apr 11 17:52:14 UTC 2022
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: xlnx,zynq-7000
Memory policy: Data cache writealloc
cma: Reserved 128 MiB at 0x16800000
Zone ranges:
  Normal   [mem 0x0000000000000000-0x000000001fffffff]
  HighMem  empty
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x000000001fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000001fffffff]
percpu: Embedded 12 pages/cpu s18828 r8192 d22132 u49152
Built 1 zonelists, mobility grouping on.  Total pages: 129920
Kernel command line: root=/dev/mmcblk0p2 rw earlyprintk rootfstype=ext4 rootwait devtmpfs.mount=1 uio_pdrv_genirq.of_id="generic-uio" clk_ignore_unused
Unknown kernel command line parameters "earlyprintk", will be passed to user space.
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 373332K/524288K available (9216K kernel code, 324K rwdata, 2512K rodata, 1024K init, 296K bss, 19884K reserved, 131072K cma-reserved, 0K highmem)
rcu: Preemptible hierarchical RCU implementation.
rcu:    RCU event tracing is enabled.
rcu:    RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
        Trampoline variant of Tasks RCU enabled.
        Tracing variant of Tasks RCU enabled.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
efuse mapped to (ptrval)
slcr mapped to (ptrval)
GIC physical location is 0xf8f01000
L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
L2C-310 erratum 769419 enabled
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 8 ways, 512 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
random: get_random_bytes called from start_kernel+0x364/0x5f8 with crng_init=0
zynq_clock_init: clkc starts at (ptrval)
Zynq clock init
sched_clock: 64 bits at 162MHz, resolution 6ns, wraps every 4398046511101ns
clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0x257a3bfb55, max_idle_ns: 440795207830 ns
Switching to timer-based delay loop, resolution 6ns
Console: colour dummy device 80x30
printk: console [tty0] enabled
Calibrating delay loop (skipped), value calculated using timer frequency.. 325.00 BogoMIPS (lpj=1625000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
CPU: Testing write buffer coherency: ok
CPU0: Spectre v2: using BPIALL workaround
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x100000 - 0x100060
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
CPU1: Spectre v2: using BPIALL workaround
smp: Brought up 1 node, 2 CPUs
SMP: Total of 2 processors activated (650.00 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 512 (order: 3, 32768 bytes, linear)
pinctrl core: initialized pinctrl subsystem
NET: Registered PF_NETLINK/PF_ROUTE protocol family
DMA: preallocated 256 KiB pool for atomic coherent allocations
thermal_sys: Registered thermal governor 'step_wise'
cpuidle: using governor menu
amba f8801000.etb: Fixing up cyclic dependency with replicator
amba f8803000.tpiu: Fixing up cyclic dependency with replicator
amba f8804000.funnel: Fixing up cyclic dependency with replicator
amba f889c000.ptm: Fixing up cyclic dependency with f8804000.funnel
amba f889d000.ptm: Fixing up cyclic dependency with f8804000.funnel
hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
hw-breakpoint: maximum watchpoint size is 4 bytes.
zynq-ocm f800c000.ocmc: ZYNQ OCM pool: 256 KiB @ 0x(ptrval)
e0000000.serial: ttyPS0 at MMIO 0xe0000000 (irq = 32, base_baud = 6250000) is a xuartps
printk: console [ttyPS0] enabled
raid6: int32x8  gen()   117 MB/s
raid6: int32x8  xor()    78 MB/s
raid6: int32x4  gen()   125 MB/s
raid6: int32x4  xor()    88 MB/s
raid6: int32x2  gen()   200 MB/s
raid6: int32x2  xor()   117 MB/s
raid6: int32x1  gen()   180 MB/s
raid6: int32x1  xor()    94 MB/s
raid6: using algorithm int32x2 gen() 200 MB/s
raid6: .... xor() 117 MB/s, rmw enabled
raid6: using intx1 recovery algorithm
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mc: Linux media interface: v0.10
videodev: Linux video capture interface: v2.00
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
EDAC MC: Ver: 3.0.0
FPGA manager framework
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm_global_timer
NET: Registered PF_INET protocol family
IP idents hash table entries: 8192 (order: 4, 65536 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 0 bytes, default 64
armv7-pmu f8891000.pmu: hw perfevents: no interrupt-affinity property, guessing.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
Initialise system trusted keyrings
workingset: timestamp_bits=14 max_order=17 bucket_order=3
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
xor: measuring software checksum speed
   arm4regs        :  1045 MB/sec
   8regs           :   805 MB/sec
   32regs          :   836 MB/sec
xor: using function: arm4regs (1045 MB/sec)
Key type asymmetric registered
Asymmetric key parser 'x509' registered
io scheduler mq-deadline registered
io scheduler kyber registered
zynq-pinctrl 700.pinctrl: zynq pinctrl initialized
dma-pl330 f8003000.dmac: Loaded driver for PL330 DMAC-241330
dma-pl330 f8003000.dmac:        DBUFF-128x8bytes Num_Chans-8 Num_Peri-4 Num_Events-16
brd: module loaded
loop: module loaded
random: fast init done
spi-nor spi0.0: w25q128jv (16384 Kbytes)
spi0.0: error parsing ofpart partition /axi/spi@e000d000/flash@0/partition@0 (/axi/spi@e000d000/flash@0)
tun: Universal TUN/TAP device driver, 1.6
CAN device driver interface
macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000 irq 34 (00:18:3e:04:a1:50)
e1000e: Intel(R) PRO/1000 Network Driver
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
usbcore: registered new interface driver cdc_acm
cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
usbcore: registered new interface driver cdc_wdm
usbcore: registered new interface driver usb-storage
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial support registered for generic
usbcore: registered new interface driver usb_serial_simple
usbserial: USB Serial support registered for carelink
usbserial: USB Serial support registered for zio
usbserial: USB Serial support registered for funsoft
usbserial: USB Serial support registered for flashloader
usbserial: USB Serial support registered for google
usbserial: USB Serial support registered for libtransistor
usbserial: USB Serial support registered for vivopay
usbserial: USB Serial support registered for moto_modem
usbserial: USB Serial support registered for motorola_tetra
usbserial: USB Serial support registered for novatel_gps
usbserial: USB Serial support registered for hp4x
usbserial: USB Serial support registered for suunto
usbserial: USB Serial support registered for siemens_mpi
ULPI transceiver vendor/product ID 0x0451/0x1507
Found TI TUSB1210 ULPI transceiver.
ULPI integrity check: passed.
ci_hdrc ci_hdrc.0: EHCI Host Controller
ci_hdrc ci_hdrc.0: new USB bus registered, assigned bus number 1
ci_hdrc ci_hdrc.0: USB 2.0 started, EHCI 1.00
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
i2c_dev: i2c /dev entries driver
cdns-wdt f8005000.watchdog: Xilinx Watchdog Timer with timeout 10s
device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
EDAC MC: ECC not enabled
Xilinx Zynq CpuIdle Driver started
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
ledtrig-cpu: registered to indicate activity on CPUs
clocksource: ttc_clocksource: mask: 0xffff max_cycles: 0xffff, max_idle_ns: 551318127 ns
timer #0 at (ptrval), irq=47
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
xlnk xlnk: Major 243
xlnk xlnk: xlnk driver loaded
xlnk xlnk: xlnk_pdev is not null
mmc0: SDHCI controller on e0100000.mmc [e0100000.mmc] using ADMA
fpga_manager fpga0: Xilinx Zynq FPGA Manager registered
IPVS: Registered protocols (TCP, UDP)
IPVS: Connection hash table configured (size=4096, memory=32Kbytes)
IPVS: ipvs loaded.
IPVS: [rr] scheduler registered.
Initializing XFRM netlink socket
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
mmc0: new high speed SDHC card at address aaaa
NET: Registered PF_PACKET protocol family
mmcblk0: mmc0:aaaa SE32G 29.7 GiB
can: controller area network core
NET: Registered PF_CAN protocol family
 mmcblk0: p1 p2
can: raw protocol
can: broadcast manager protocol
can: netlink gateway - max_hops=1
Registering SWP/SWPB emulation handler
Loading compiled-in X.509 certificates
Btrfs loaded, crc32c=crc32c-generic, zoned=no, fsverity=no
of-fpga-region fpga-full: FPGA Region probed
of_cfs_init
of_cfs_init: OK
cfg80211: Loading compiled-in X.509 certificates for regulatory database
cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
cfg80211: failed to load regulatory.db
clk: Not disabling unused clocks
ALSA device list:
  No soundcards found.
EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
VFS: Mounted root (ext4 filesystem) on device 179:2.
devtmpfs: mounted
Freeing unused kernel image (initmem) memory: 1024K
Run /sbin/init as init process
systemd[1]: System time before build time, advancing clock.
systemd[1]: Failed to find module 'autofs4'
systemd[1]: systemd 249.11-0ubuntu3 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS -OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP -LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
systemd[1]: Detected architecture arm.

Welcome to PynqLinux, based on Ubuntu 22.04!

systemd[1]: Hostname set to <pynq>.
systemd[1]: Queued start job for default target Multi-User System.
random: systemd: uninitialized urandom read (16 bytes read)
systemd[1]: Created slice Slice /system/modprobe.
[  OK  ] Created slice Slice /system/modprobe.
random: systemd: uninitialized urandom read (16 bytes read)
systemd[1]: Created slice Slice /system/serial-getty.
[  OK  ] Created slice Slice /system/serial-getty.
random: systemd: uninitialized urandom read (16 bytes read)
systemd[1]: Created slice User and Session Slice.
[  OK  ] Created slice User and Session Slice.
systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Started Forward Password R…uests to Wall Directory Watch.
systemd[1]: Condition check resulted in Arbitrary Executable File Formats File System Automount Point being skipped.
systemd[1]: Reached target Local Encrypted Volumes.
[  OK  ] Reached target Local Encrypted Volumes.
systemd[1]: Reached target Remote File Systems.
[  OK  ] Reached target Remote File Systems.
systemd[1]: Reached target Slice Units.
[  OK  ] Reached target Slice Units.
systemd[1]: Reached target Local Verity Protected Volumes.
[  OK  ] Reached target Local Verity Protected Volumes.
systemd[1]: Listening on Syslog Socket.
[  OK  ] Listening on Syslog Socket.
systemd[1]: Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on initctl Compatibility Named Pipe.
systemd[1]: Condition check resulted in Journal Audit Socket being skipped.
systemd[1]: Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket (/dev/log).
systemd[1]: Listening on Journal Socket.
[  OK  ] Listening on Journal Socket.
systemd[1]: Listening on udev Control Socket.
[  OK  ] Listening on udev Control Socket.
systemd[1]: Listening on udev Kernel Socket.
[  OK  ] Listening on udev Kernel Socket.
systemd[1]: Condition check resulted in Huge Pages File System being skipped.
systemd[1]: Mounting POSIX Message Queue File System...
         Mounting POSIX Message Queue File System...
systemd[1]: Mounting Kernel Debug File System...
         Mounting Kernel Debug File System...
systemd[1]: Condition check resulted in Kernel Trace File System being skipped.
systemd[1]: Starting Journal Service...
         Starting Journal Service...
systemd[1]: Starting Restore / save the current clock...
         Starting Restore / save the current clock...
systemd[1]: Starting Set the console keyboard layout...
         Starting Set the console keyboard layout...
systemd[1]: Condition check resulted in Create List of Static Device Nodes being skipped.
systemd[1]: Starting Load Kernel Module configfs...
         Starting Load Kernel Module configfs...
systemd[1]: Starting Load Kernel Module drm...
         Starting Load Kernel Module drm...
systemd[1]: Starting Load Kernel Module fuse...
         Starting Load Kernel Module fuse...
systemd[1]: Started Nameserver information manager.
[  OK  ] Started Nameserver information manager.
systemd[1]: Reached target Preparation for Network.
[  OK  ] Reached target Preparation for Network.
systemd[1]: Starting Load Kernel Modules...
         Starting Load Kernel Modules...
systemd[1]: Starting Remount Root and Kernel File Systems...
         Starting Remount Root and Kernel File Systems...
systemd[1]: Starting Coldplug All udev Devices...
         Starting Coldplug All udev Devices...
systemd[1]: Started Journal Service.
[  OK  ] Started Journal Service.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted Kernel Debug File System.
[  OK  ] Finished Restore / save the current clock.
[  OK  ] Finished Load Kernel Module configfs.
[  OK  ] Finished Set the console keyboard layout.
[  OK  ] Finished Load Kernel Module drm.
[  OK  ] Finished Load Kernel Module fuse.
[  OK  ] Finished Load Kernel Modules.
[  OK  ] Finished Remount Root and Kernel File Systems.
         Activating swap /var/swap...
         Mounting Kernel Configuration File System...
Adding 524284k swap on /var/swap.  Priority:-2 extents:1 across:524284k SS
         Starting Flush Journal to Persistent Storage...
systemd-journald[110]: Received client request to flush runtime journal.
         Starting Load/Save Random Seed...
systemd-journald[110]: File /var/log/journal/b4e19d1e11f24633a2f2e1fe2798cbe4/system.journal corrupted or uncleanly shut down, renaming and replacing.
         Starting Apply Kernel Variables...
         Starting Create System Users...
[  OK  ] Activated swap /var/swap.
[  OK  ] Mounted Kernel Configuration File System.
[  OK  ] Reached target Swaps.
[  OK  ] Finished Apply Kernel Variables.
[  OK  ] Finished Create System Users.
         Starting Create Static Device Nodes in /dev...
[  OK  ] Finished Coldplug All udev Devices.
         Starting Helper to synchronize boot up for ifupdown...
[  OK  ] Finished Flush Journal to Persistent Storage.
[  OK  ] Finished Create Static Device Nodes in /dev.
[  OK  ] Reached target Preparation for Local File Systems.
[  OK  ] Reached target Local File Systems.
         Starting Enable support fo…l executable binary formats...
         Starting Set console font and keymap...
         Starting Create Volatile Files and Directories...
         Starting Rule-based Manage…for Device Events and Files...
[  OK  ] Finished Enable support fo…nal executable binary formats.
[  OK  ] Finished Set console font and keymap.
[  OK  ] Finished Create Volatile Files and Directories.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
         Starting Network Name Resolution...
         Starting Network Time Synchronization...
         Starting Record System Boot/Shutdown in UTMP...
[  OK  ] Started Rule-based Manager for Device Events and Files.
[  OK  ] Finished Record System Boot/Shutdown in UTMP.
[  OK  ] Found device /dev/ttyPS0.
zocl-drm axi:zyxclmm_drm: IRQ index 0 not found
[  OK  ] Started Network Time Synchronization.
[  OK  ] Finished Helper to synchronize boot up for ifupdown.
[  OK  ] Finished Load/Save Random Seed.
[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Host and Network Name Lookups.
[  OK  ] Reached target System Time Set.
[  OK  ] Reached target Hardware activated USB gadget.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Reached target System Initialization.
[  OK  ] Started resolvconf-pull-resolved.path.
[  OK  ] Started Trigger to poll fo…y enabled on GCP LTS non-pro).
[  OK  ] Started Daily apt download activities.
[  OK  ] Started Daily apt upgrade and clean activities.
[  OK  ] Started Daily dpkg database backup timer.
[  OK  ] Started Periodic ext4 Onli…ata Check for All Filesystems.
[  OK  ] Started Discard unused blocks once a week.
[  OK  ] Started Daily rotation of log files.
[  OK  ] Started Daily man-db regeneration.
[  OK  ] Started Message of the Day.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started Ubuntu Advantage Timer for running repeated jobs.
[  OK  ] Reached target Path Units.
[  OK  ] Reached target Timer Units.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Listening on UUID daemon activation socket.
[  OK  ] Reached target Socket Units.
[  OK  ] Reached target Basic System.
         Starting LSB: automatic crash report generation...
         Starting Executing boot.py from the boot partition...
[  OK  ] Started Regular background program processing daemon.
[  OK  ] Started D-Bus System Message Bus.
[  OK  ] Started Save initial kernel messages after boot.
         Starting Remove Stale Onli…t4 Metadata Check Snapshots...
[  OK  ] Started ifup for eth0.
         Starting Jupyter Notebook Server...
         Starting LSB: Load kernel …d to enable cpufreq scaling...
         Starting Dispatcher daemon for systemd-networkd...
         Starting Raise network interfaces...
         Starting Resize Filesystem on SD card...
         Starting resolvconf-pull-resolved.service...
         Starting System Logging Service...
         Starting User Login Management...
[  OK  ] Finished Resize Filesystem on SD card.
[  OK  ] Started System Logging Service.
[  OK  ] Finished resolvconf-pull-resolved.service.
[  OK  ] Started LSB: automatic crash report generation.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started LSB: Load kernel m…ded to enable cpufreq scaling.
         Starting LSB: set CPUFreq kernel parameters...
[  OK  ] Started LSB: set CPUFreq kernel parameters.
[  OK  ] Started Dispatcher daemon for systemd-networkd.
[  OK  ] Started User Login Management.
[  OK  ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
         Starting resolvconf-pull-resolved.service...
[  OK  ] Finished resolvconf-pull-resolved.service.
         Starting resolvconf-pull-resolved.service...
         Stopping Network Time Synchronization...
[  OK  ] Stopped Network Time Synchronization.
         Starting Network Time Synchronization...
[  OK  ] Finished resolvconf-pull-resolved.service.
[  OK  ] Stopped Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Entropy Daemon based on the HAVEGE algorithm.
[  OK  ] Started Network Time Synchronization.
[  OK  ] Finished Raise network interfaces.
[  OK  ] Reached target Network.
[  OK  ] Reached target Network is Online.
[  OK  ] Started ISC DHCP IPv4 server.
[  OK  ] Started ISC DHCP IPv6 server.
         Starting Samba NMB Daemon...
         Starting OpenBSD Secure Shell server...
         Starting Permit User Sessions...
[  OK  ] Started Unattended Upgrades Shutdown.
[  OK  ] Finished Permit User Sessions.
[  OK  ] Started Serial Getty on ttyPS0.
         Starting Set console scheme...

PYNQ Linux, based on Ubuntu 22.04 pynq ttyPS0

pynq login: xilinx (automatic login)

Welcome to PYNQ Linux, based on Ubuntu 22.04 (GNU/Linux 5.15.19-xilinx-v2022.1 armv7l)

Last login: Thu Sep 12 19:40:39 UTC 2024 on ttyPS0
xilinx@pynq:~$

Shutdown console log

And the log generated when performing an orderly shutdown.

Last login: Thu Sep 12 20:51:48 UTC 2024 on ttyPS0
xilinx@pynq:~$ sudo shutdown now
[sudo] password for xilinx:
         Stopping Session c1 of User xilinx...
[  OK  ] Removed slice Slice /system/modprobe.
[  OK  ] Stopped target Multi-User System.
[  OK  ] Stopped target Login Prompts.
[  OK  ] Stopped target Host and Network Name Lookups.
[  OK  ] Stopped target Timer Units.
[  OK  ] Stopped Daily apt upgrade and clean activities.
[  OK  ] Stopped Daily apt download activities.
[  OK  ] Stopped Daily dpkg database backup timer.
[  OK  ] Stopped Periodic ext4 Onli…ata Check for All Filesystems.
[  OK  ] Stopped Discard unused blocks once a week.
[  OK  ] Stopped Daily rotation of log files.
[  OK  ] Stopped Daily man-db regeneration.
[  OK  ] Stopped Message of the Day.
[  OK  ] Stopped Daily Cleanup of Temporary Directories.
[  OK  ] Stopped Ubuntu Advantage Timer for running repeated jobs.
[  OK  ] Stopped target System Time Set.
[  OK  ] Stopped target Hardware activated USB gadget.
         Stopping LSB: automatic crash report generation...
         Stopping LSB: set CPUFreq kernel parameters...
         Stopping Regular background program processing daemon...
         Stopping Getty on tty1...
         Stopping Jupyter Notebook Server...
         Stopping Dispatcher daemon for systemd-networkd...
         Stopping System Logging Service...
         Stopping Serial Getty on ttyPS0...
         Stopping Samba SMB Daemon...
         Stopping OpenBSD Secure Shell server...
         Stopping Load/Save Random Seed...
         Stopping Unattended Upgrades Shutdown...
[  OK  ] Stopped Regular background program processing daemon.
[  OK  ] Stopped Dispatcher daemon for systemd-networkd.
[  OK  ] Stopped System Logging Service.
[  OK  ] Stopped Unattended Upgrades Shutdown.
[  OK  ] Stopped Serial Getty on ttyPS0.
[  OK  ] Stopped Getty on tty1.
[  OK  ] Stopped OpenBSD Secure Shell server.
[  OK  ] Stopped Samba SMB Daemon.
[  OK  ] Stopped Load/Save Random Seed.
[  OK  ] Stopped Session c1 of User xilinx.
[  OK  ] Removed slice Slice /system/getty.
[  OK  ] Removed slice Slice /system/serial-getty.
         Stopping Samba NMB Daemon...
         Stopping User Login Management...
         Stopping User Manager for UID 1000...
[  OK  ] Stopped User Login Management.
[  OK  ] Stopped Samba NMB Daemon.
[  OK  ] Stopped User Manager for UID 1000.
[  OK  ] Stopped LSB: automatic crash report generation.
[  OK  ] Stopped LSB: set CPUFreq kernel parameters.
[  OK  ] Stopped target Network is Online.
         Stopping LSB: Load kernel …d to enable cpufreq scaling...
         Stopping User Runtime Directory /run/user/1000...
[  OK  ] Unmounted /run/user/1000.
[  OK  ] Stopped LSB: Load kernel m…ded to enable cpufreq scaling.
[  OK  ] Stopped User Runtime Directory /run/user/1000.
[  OK  ] Removed slice User Slice of UID 1000.
         Stopping Permit User Sessions...
[  OK  ] Stopped Permit User Sessions.
[  OK  ] Stopped target Network.
[  OK  ] Stopped target Remote File Systems.
         Stopping ifup for eth0...
         Stopping Raise network interfaces...
         Stopping Network Name Resolution...
[  OK  ] Stopped Network Name Resolution.
[  OK  ] Stopped Raise network interfaces.
[  OK  ] Stopped ifup for eth0.
[  OK  ] Stopped target Preparation for Network.
[  OK  ] Stopped Jupyter Notebook Server.
[  OK  ] Stopped target Basic System.
[  OK  ] Stopped target Path Units.
[  OK  ] Stopped resolvconf-pull-resolved.path.
[  OK  ] Stopped Trigger to poll fo…y enabled on GCP LTS non-pro).
[  OK  ] Stopped target Slice Units.
[  OK  ] Removed slice User and Session Slice.
[  OK  ] Stopped target Socket Units.
[  OK  ] Closed Syslog Socket.
[  OK  ] Closed UUID daemon activation socket.
[  OK  ] Stopped target System Initialization.
[  OK  ] Stopped target Local Encrypted Volumes.
[  OK  ] Stopped Dispatch Password …ts to Console Directory Watch.
[  OK  ] Stopped Forward Password R…uests to Wall Directory Watch.
[  OK  ] Stopped target Swaps.
[  OK  ] Stopped target Local Verity Protected Volumes.
         Deactivating swap /var/swap...
         Stopping Restore / save the current clock...
[  OK  ] Stopped Apply Kernel Variables.
[  OK  ] Stopped Load Kernel Modules.
         Stopping Network Time Synchronization...
         Stopping Record System Boot/Shutdown in UTMP...
[  OK  ] Stopped Network Time Synchronization.
[  OK  ] Deactivated swap /var/swap.
[  OK  ] Stopped Restore / save the current clock.
[  OK  ] Stopped Record System Boot/Shutdown in UTMP.
[  OK  ] Stopped Create Volatile Files and Directories.
[  OK  ] Stopped target Local File Systems.
         Unmounting boot.mount...
         Unmounting /run/credentials/systemd-sysusers.service...
[  OK  ] Unmounted boot.mount.
[  OK  ] Unmounted /run/credentials/systemd-sysusers.service.
[  OK  ] Stopped target Preparation for Local File Systems.
[  OK  ] Reached target Unmount All Filesystems.
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  OK  ] Stopped Create System Users.
[  OK  ] Stopped Remount Root and Kernel File Systems.
[  OK  ] Reached target System Shutdown.
[  OK  ] Reached target Late Shutdown Services.
[  OK  ] Finished System Power Off.
[  OK  ] Reached target System Power Off.
reboot: System halted

Checking IP address using my router.

Before I started changing the network configuration parameters, I quickly checked if my router had registered the new connection. To my pleasant surprise, it appeared on the network as PYNQ, no problem. You can also check the assigned IP from the command line, use whatever is fastest for you.

image

Pinging the board

The board responds correctly to ICMP echoes ping.

PS C:\Users\ealbertos> ping pynq

Pinging pynq [192.168.1.41] with 32 bytes of data:
Reply from 192.168.1.41: bytes=32 time<1ms TTL=64
Reply from 192.168.1.41: bytes=32 time<1ms TTL=64
Reply from 192.168.1.41: bytes=32 time<1ms TTL=64
Reply from 192.168.1.41: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.1.41:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

Opening Jupyter

Once your board is setup, to connect to Jupyter Notebooks open a web browser from your PC or laptop and navigate to: http://pynq:9090

image

If your board is configured correctly, you will be presented with a login screen. The username is xilinx and the password is also xilinx.

image

Updating PYNQ files

If you are not sure you have the latest version of the files you can update them from the command line using the serial console.

sudo pip3 install --upgrade --upgrade-strategy only-if-needed pynq

image

xilinx@pynq:~$ cat REVISION
Release 2022_10_22 93ddd21
Board 2022_10_22 93ddd21 https://github.com/Xilinx/PYNQ
xilinx@pynq:~$

Accessing Files on The Board

Samba, a file sharing service, is running on the board. This allows you to access the Pynq home area as a network drive, to transfer files to and from the board.

image

Windows explorer

image

 

Welcome to PYNQ

Included in the PYNQ distribution are several Jupyter notebooks with tutorials, ready to be run on demand.

image

PYNQ Libraries

A significant benefit of PYNQ is its capability to support a wide array of peripherals. It accommodates common interfaces such as USB, Ethernet, and HDMI, as well as specialized ones like Pmods and Arduino devices. To utilize peripherals connected to the PL, one must load an overlay that includes the necessary hardware IP for peripheral control, ensuring smooth hardware-software integration.

The Python API of PYNQ eases the interaction with peripherals. Developers can control devices using the familiar Python syntax, broadening accessibility. Moreover, the API is extendable to include additional IP, offering customized solutions for various applications.

For advanced control, PYNQ provides low-level access to the overlay, featuring memory-mapped IO, memory allocation, and bitstream management, perfect for developers aiming to enhance performance or add custom features.

PYNQ also boasts the PynqMicroBlaze subsystem, facilitating the loading and creation of pre-compiled applications, thus expediting the development workflow.

Arty-Z7 Overlays

The following overlays are included by default in the PYNQ image for the PYNQ-Z1 and the Arty-Z7 board:

  • Base Overlay: The base overlay for the PYNQ-Z1/Arty Z7 board enables PYNQ to utilize its peripherals immediately after booting, thanks to embedded hardware IP that manages these peripherals and connects them to the Zynq’s Processing System. Typical peripherals include GPIO devices (like LEDs, switches, and buttons), video, audio, and other custom interfaces, which can also serve as reference designs for creating custom overlays. Additionally, for general-purpose interfaces such as Pmod or Arduino headers, the base overlay may include a PYNQ MicroBlaze, allowing control of devices with various interfaces and protocols on the same port without modifying the programmable logic design.

image

Mic is not included on the Arty Z7

  • Logictools Overlay: The LogicTools overlay includes programmable hardware blocks that interact with external digital logic circuits. Finite state machines, Boolean logic functions, and digital patterns can be generated using Python. Additionally, a programmable switch links the inputs and outputs of these hardware blocks to external I/O pins. And the LogicTools overlay includes a trace analyzer for capturing and analyzing data from the I/O interface, aiding in debugging and analysis.

image

The Arty-Z7 board has the following features:

  • Zynq XC7Z020-1CLG400C
  • 512MB DDR3
  • 1G Ethernet
  • USB 2.0
  • MicroSD
  • Uart
  • 3.5mm mono audio output jack
  • 2x HDMI (can be used as input or output)
  • 4 push-buttons
  • 2 slide switches
  • 4 LEDs
  • 2 RGB LEDs
  • 2x Pmod ports
  • 1x Arduino header

Exploring the video base overlay sample

The PYNQ Video subpackage provides a comprehensive set of drivers for managing video data on the PYNQ platform. It enables developers to capture video from the HDMI-In port, process it, and display it on the HDMI-Out port.

The underlying hardware consists of three key components: the HDMI-In block, the HDMI-Out block, and the Video DMA. The HDMI-In block captures video data, while the HDMI-Out block displays it. Both blocks support color space conversions and channel adjustments, allowing for flexible video processing.

image

The video library accommodates two distinct video front ends. These front-ends are tasked with transforming the device pin signals into a 24-bit-per-pixel, BGR formatted AXI stream, suitable for the subsequent pipeline stages. The Pynq-Z1 and the Arty Z7 boards feature a DVI-based front-end capable of supporting resolutions up to 1080p; however, the official support is limited to 720p due to the differential pins' speed ratings. This front-end is readily available as a drop-in IP within the PYNQ IP library.

HDMI-In

The Pixel Unpack and the Color Convert block allow conversion between different color spaces at runtime.

image

HDMI-out

The HDMI-Out is similar to HDMI-In. It has a Pixel Pack block (instead of the Unpack block for HDMI-In) and a Color Convert block.

image

The PYNQ video library offers three primary methods for processing video data:

  • Python Processing on the PS: This approach is flexible and straightforward to implement, accommodating complex algorithms. However, it can be computationally intensive, especially for high-resolution tasks or those requiring real-time execution.
    • Frame Handling: Input and output frames are represented as NumPy arrays, allowing for seamless integration with other Python libraries like OpenCV. Frames are read from the HDMI input and written to the HDMI output.
    • Synchronization: The HDMI input will either return the most recent complete frame or wait until a frame is ready. The HDMI output will persist in displaying the last frame written until a new frame is supplied.
  • IP Insertion in the Overlay: This method offers high-performance, hardware-accelerated processing. However, it requires modifying the overlay and recompiling the bitstream, which can be time-consuming.
    • Pipeline Processing: Custom AXI-stream IP can be integrated into the video pipeline, either on the HDMI-In or HDMI-Out side.
    • IP Placement: For optimal performance, consider placing the IP after the pixel_pack block in the HDMI-In section or before the pixel_unpack block in the HDMI-Out section. This allows for efficient color space conversion.
  • IP Connection to an Available Interface: This modular approach allows for easy integration of existing IP cores. However, it may require additional interfaces or adapters.
    • Batch Processing: An IP block can be added to an overlay and connected to an available interface. A DMA is typically used to stream data between the IP and the frame buffers in DRAM.
    • DRAM Bottleneck: The DRAM can be a bottleneck for video processing, especially for high-resolution content. Consider optimizing memory access patterns and potentially using external memory for demanding applications.
    • For the Arty-Z7 with a 512MB DDR3 with 16-bit bus, up to 1080p grayscale (8-bits per pixel) can be processed at approximately 60 FPS, but this is close to the system's total memory bandwidth.

image

The Apple Ipad detects the HDMI sink.

image

HDMI Sink and Source Demo: A Real-Time Look

Initially, I had reservations about sharing these unedited videos; however, their effectiveness in illustrating the examples has persuaded me. Although lengthy, they offer a clear demonstration of real-time image processing utilizing the HDMI sink and source on the Arty Z7 from PYNQ.

What You'll See:

  • Two examples showcasing accelerated and non-accelerated image processing.
  • A live session using the base overlay.
  • No editing, just raw footage.


Setup Breakdown:

  • HDMI Input: Apple iPad's camera app connected via USB to HDMI.
  • HDMI Output: Arty Z7 to HDMI to USB video capture card.
  • Display: Surface workstation with OBS screen capture.
  • Software: Jupyter server (PYNQ 2.0), camera program, TeraTerm (serial console).


Screen Breakdown:

  • Left: Chrome browser connected to Jupyter server.
  • Right: Top: Camera program window. Bottom: TeraTerm serial console.

HDMI introduction

You don't have permission to edit metadata of this video.
Edit media
x
image
Upload Preview
image

Face Detection Demo

You don't have permission to edit metadata of this video.
Edit media
x
image
Upload Preview
image

image

USB Webcam demo

This notebook shows how to use a USB web camera attached to the board. An image is captured using fswebcam. The image can then be manipulated using the Python Image Library (Pillow).

The webcam used is the Ziquian 1080p Webcam and the driver for this webcam has already been installed on the board.

image

Recreating the base overlay from Windows

Clone https://github.com/Xilinx/PYNQ/tree/master/pynq

image

The build_ip.tcl file compiles any HLS IP used in the design. The base.tcl builds the overlay.

In Windows, the two Tcl files can be sourced in Vivado to rebuild the overlay. The Tcl files to rebuild the overlay can be sourced from the Vivado GUI, or from the Vivado Tcl Shell (command line).

From Vivado 2022.1

image

To rebuild from the Vivado GUI, open Vivado. In the Vivado Tcl command line window change to the correct directory, and source the Tcl files:

source build_ip.tcl

source base.tcl

Once completed the process you can review the overlay block diagram and start your customizations.

Overlay Block Diagram:

image

PYNQ resources

  • https://github.com/Xilinx/PYNQ
  • PYNQ | Python Productivity to AMD Adaptive Coompute platforms
  • awai54st/PYNQ-Classification: Python on Zynq FPGA for Convolutional Neural Networks (github.com)
  • Xilinx/BNN-PYNQ: Quantized Neural Networks (QNNs) on PYNQ (github.com)
  • Xilinx/PYNQ-ComputerVision: Computer Vision Overlays on Pynq (github.com)
  • Xilinx/PYNQ_Composable_Pipeline: PYNQ Composabe Overlays (github.com)
  • briansune/PYNQ-2.7-MNIST: PYNQ-Based MNIST with Tensorflow Lite (github.com)

Summary

In this blog I have presented a quick introduction to the PYNQ framework, from its installation on the Arty-Z7 development board and a brief walk through some of the official examples and tutorials. I have also rebuilt the base overlay from Vivado by obtaining the original code from the github repository. I hope this is a blog that will help you get started quickly with the Arty-Z7 for this experimentation challenge.

FruitVision Scale Blogs Series

  •  Blog 1 - FruitVision Scale - Digilent Arty Z7 - AMD Zynq-7000 - Overview 
  •  Blog 2 - FruitVision Scale - PYNQ on the Digilent Arty Z7 - Quickstart 
  •  Blog 3 - FruitVision Scale - Weighing strategies with load cells using the Digilent Arty Z7 
  •  Blog 4 - FruitVision Scale - FPGA-Based HX711 Driver for Precision Weighing with AMD PYNQ 
  •  Blog 5 - FruitVision Scale - Exploring Deep Learning Applications with AMD Zynq 7000 
  •  Final - FruitVision Scale 
  • Sign in to reply

Top Comments

  • Jan Cumps
    Jan Cumps 2 months ago in reply to Saif_Obaid +1
    Your terminal tries to connect to a serial port. SSH is over the network, TCP Looks like your connection setup may be correct, but the terminal is not using it. Title says it's serial to COM7
  • javagoza
    javagoza 2 months ago in reply to Saif_Obaid

    The com port number is assigned by Windows. Go to the device manager and check which Port port is assigned to the board when plugging the USB port

    image

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • Saif_Obaid
    Saif_Obaid 2 months ago

    Dear Javagoza,

    I am a student that just started on DL boards, I was using your blog to get started on this subject, however I am stuck at Tera Term, I don't know why when I start it, and while I am connected to the board, with the USB cable, and ethernet cable, it shows that the port is COM 7, and not COM 4 as seen in this blog, I will try on another laptop, and another possibility I could think of is that I am using the latest version of Tera Term, which is Tera Term 5, and I though maybe you are using an older version? if so can you kindly share with me the software?

    Kind regards,

    Saif Obaid

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • Jan Cumps
    Jan Cumps 2 months ago in reply to Saif_Obaid

    Your terminal tries to connect to a serial port. SSH is over the network, TCP
    Looks like your connection setup may be correct, but the terminal is not using it. Title says it's serial to COM7

    image

    • Cancel
    • Vote Up +1 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • Saif_Obaid
    Saif_Obaid 2 months ago

    Dear Javagoza, 

    I'm facing a problem with the interface ( Tetra Term 5 ), it's not showing the same as the picture you attached to this page. Also, my board is LD's are not flashing as mentioned in  the blog. I will attach the pictures to this comment. Your help will be truly appreciated.

    imageimageimage

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
  • javagoza
    javagoza 8 months ago

    image

    Weighing scale using the Zynq 7000 XADC and weight and barcode overlay.

    • Cancel
    • Vote Up 0 Vote Down
    • Sign in to reply
    • More
    • Cancel
>
element14 Community

element14 is the first online community specifically for engineers. Connect with your peers and get expert answers to your questions.

  • Members
  • Learn
  • Technologies
  • Challenges & Projects
  • Products
  • Store
  • About Us
  • Feedback & Support
  • FAQs
  • Terms of Use
  • Privacy Policy
  • Legal and Copyright Notices
  • Sitemap
  • Cookies

An Avnet Company © 2025 Premier Farnell Limited. All Rights Reserved.

Premier Farnell Ltd, registered in England and Wales (no 00876412), registered office: Farnell House, Forge Lane, Leeds LS12 2NE.

ICP 备案号 10220084.

Follow element14

  • X
  • Facebook
  • linkedin
  • YouTube