Decided to recreate the process of going from a vendor provided reference design to a first version of my project. I've literally gone thru more than a dozen iterations of this process to gain the knowledge of how to do it properly.
The vendor's reference design was done with Vivado 2019 on windoz. I am using Vivado 2022.2 on Ubuntu 22.04. This causes most of the IP to be "out of date". There is a vendor provided doc, which being run under windoz can be misleading...
The first step is to open the reference design, bring it up to date, and then save as our design.
$ file 32_pcie_ram_axilite_usr_bypass.rar 32_pcie_ram_axilite_usr_bypass.rar: RAR archive data, v4, os: Win32 $ unrar x 32_pcie_ram_axilite_usr_bypass.rar UNRAR 6.11 beta 1 freeware Copyright (c) 1993-2022 Alexander Roshal Extracting from 32_pcie_ram_axilite_usr_bypass.rar Creating pcie_ram_axilite_usr_bypass OK ... All OK
Start Vivado and select [Open Project]
This will display a splash screen:
Click [OK]
Then:
Click [Yes]
A splash screen warns of the "out of date" IP:
Click [Report IP Status]
At the bottom of the Vivado GUI an IP Status tab will be added:
Everything requiring update should be selected.
Click [Upgrade Selected]
Then this appears:
Click [OK]
Click [OK]
Click [Generate]
Click [OK]
This will grind along in the background for quite a while...
Ending with:
Click [details]
Causing this popup:
Click [Force up-to-date]
Which may display this:
If so, select the Messages tab at bottom of GUI:
I'm guessing this is a result of limiting the path to what windoz can handle, we'll clean it up when we "save as" to create out project.
At this point we have:
Time to "save as" our project.
Select [File > Project > Save As...]
I renamed the project to dfcSDR, and selected a location with the same name. Leave "Include run results" unchecked, we want to leave the windoz cruft behind us. Edit appropriately and then:
Click [OK]
Select [File > Close Project]
Re-open the new project, it should be at the top of Recent Projects on the right side of GUI. We now have:
As a result of Create project directory we have:
../dfcSDR$ ls -At Blog/ Notes/ dfcSDR/ ../dfcSDR$ ls -At dfcSDR dfcSDR.runs dfcSDR.cache dfcSDR.gen dfcSDR.hw dfcSDR.ip_user_files dfcSDR.sim dfcSDR.srcs dfcSDR.xpr
I added Blog and Notes directories for things I make associated with the project.
Add git version control.
../dfcSDR$ git init ... Initialized empty Git repository in ../dfcSDR/.git/ ../dfcSDR$ l Blog/ dfcSDR/ Notes/ ../dfcSDR$ l -a ./ ../ Blog/ dfcSDR/ .git/ .gitignore Notes/ ../dfcSDR$ git status On branch master No commits yet Untracked files: (use "git add <file>..." to include in what will be committed) .gitignore Blog/ dfcSDR/ nothing added to commit but untracked files present (use "git add" to track) ../dfcSDR$ l Blog addtoBlog ../dfcSDR$ git add . ../dfcSDR$ git status On branch master No commits yet Changes to be committed: (use "git rm --cached <file>..." to unstage) new file: .gitignore new file: Blog/addtoBlog ... new file: dfcSDR/dfcSDR.xpr ../dfcSDR$ find . -type f | wc -l 1487 ../dfcSDR$ git config user.email "foo@bar" ../dfcSDR$ git config user.name "Foo Bar" ../dfcSDR$ git commit -m "Initial commit." [master (root-commit) cb09936] Initial commit. 881 files changed, 3027609 insertions(+) create mode 100644 .gitignore ... create mode 100644 dfcSDR/dfcSDR.xpr ../dfcSDR$ git status On branch master nothing to commit, working tree clean
Now that the base files are under git control its time to synthesize the design:
Click [Run Synthesis]
Note that I had previously set "Number of jobs" to 14 (default is 8) as I have 16 cores.
Click [OK]
This takes awhile...
Click [OK]
Synthesis created a lot of new files for git:
../dfcSDR$ git status On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: Blog/addtoBlog modified: dfcSDR/dfcSDR.cache/wt/synthesis.wdf ... modified: dfcSDR/dfcSDR.xpr Untracked files: (use "git add <file>..." to include in what will be committed) dfcSDR/dfcSDR.cache/ip/ dfcSDR/dfcSDR.runs/ no changes added to commit (use "git add" and/or "git commit -a") ../dfcSDR$ git add -u ../dfcSDR$ git commit -m "After running initial synthesis." [master 888c25b] After running initial synthesis. 59 files changed, 924884 insertions(+), 924856 deletions(-) rewrite dfcSDR/dfcSDR.gen/sources_1/bd/PCIe/ip/PCIe_Reg_wrAXI4lite_0_2/PCIe_Reg_wrAXI4lite_0_2.dcp (95%) ... rewrite dfcSDR/dfcSDR.gen/sources_1/bd/PCIe/ip/PCIe_util_vector_logic_0_0/PCIe_util_vector_logic_0_0.dcp (88%)
Now we have to run the implementation.
Select [Run implementation]
Click [OK]
This takes even more time...
And finally finishes:
Click [OK]
Implementation also creates a lot of new files for git:
../dfcSDR$ git status On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: dfcSDR/dfcSDR.cache/wt/gui_handlers.wdf modified: dfcSDR/dfcSDR.cache/wt/java_command_handlers.wdf modified: dfcSDR/dfcSDR.cache/wt/webtalk_pa.xml modified: dfcSDR/dfcSDR.xpr Untracked files: (use "git add <file>..." to include in what will be committed) dfcSDR/dfcSDR.cache/ip/ dfcSDR/dfcSDR.runs/ no changes added to commit (use "git add" and/or "git commit -a") /dfcSDR$ git add -u ../dfcSDR$ git commit -m "After running initial implementation." [master c50aaf1] After running initial implementation. 5 files changed, 457 insertions(+), 413 deletions(-) ../D/dfcSDR$ git status On branch master Untracked files: (use "git add <file>..." to include in what will be committed) dfcSDR/dfcSDR.cache/ip/ dfcSDR/dfcSDR.runs/ nothing added to commit but untracked files present (use "git add" to track)
Now we create the .bit and .bin files.
Select [Generate Bitstream]
Click [OK]
More churn...
All done.
Click [OK]
Update git:
../dfcSDR$ git status On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: dfcSDR/dfcSDR.cache/wt/project.wpc modified: dfcSDR/dfcSDR.cache/wt/webtalk_pa.xml Untracked files: (use "git add <file>..." to include in what will be committed) dfcSDR/dfcSDR.cache/ip/ dfcSDR/dfcSDR.runs/ no changes added to commit (use "git add" and/or "git commit -a") ../dfcSDR$ git add -u ../dfcSDR$ git commit -m "After running Generate Bitstream." [master ff74bde] After running Generate Bitstream. 3 files changed, 52 insertions(+), 3 deletions(-)
Looking good:
A LOT of Warnings, but no Errors!
Time to test.
Select [Open Hardware Manager]
Select [Open Target > AutoConnect]
Select [Program Device > <your device>]
Select the correct .bit file:
Select [Program]
All looks good:
This step loaded the code directly to the FPGA. To test it we have to restart linux WITHOUT a power-cycle. Running the test immediately after the load fails.
The test:
#! /bin/sh
#
echo "Checking for device..."
lspci | grep Xilinx
if [ $? -eq 1 ]; then
echo FAIL; sleep 50; exit
fi
echo "Loading driver"
cd ../LinuxPCIeDrivers/dma_ip_drivers/XDMA/linux-kernel/tests
./load_driver.sh
if [ $? -eq 1 ]; then
echo FAIL; sleep 50; exit
fi
./run_test.sh
sleep 1000
The result:
Checking for device... 01:00.0 Memory controller: Xilinx Corporation Device 7024 Loading driver interrupt_selection . Loading driver...insmod xdma.ko interrupt_mode=2 ... Error: The Kernel module installed correctly, but no devices were recognized. FAILED FAIL
On Ubuntu select Poweroff > restart
Retest once Ubuntu has restarted:
Checking for device... 01:00.0 Memory controller: Xilinx Corporation Device 7024 Loading driver interrupt_selection . Loading driver...insmod xdma.ko interrupt_mode=2 ... The Kernel module installed correctly and the xmda devices were recognized. DONE Info: Number of enabled h2c channels = 2 Info: Number of enabled c2h channels = 2 Info: The PCIe DMA core is memory mapped. Info: Running PCIe DMA memory mapped write read test transfer size: 1024, count: 1 Info: Writing to h2c channel 0 at address offset 0. Info: Writing to h2c channel 1 at address offset 1024. Info: Wait for current transactions to complete. /dev/xdma0_h2c_1 ** Average BW = 1024, 8.630716 /dev/xdma0_h2c_0 ** Average BW = 1024, 7.923673 Info: Writing to h2c channel 0 at address offset 2048. Info: Writing to h2c channel 1 at address offset 3072. Info: Wait for current transactions to complete. /dev/xdma0_h2c_0 ** Average BW = 1024, 16.706366 /dev/xdma0_h2c_1 ** Average BW = 1024, 29.789957 Info: Reading from c2h channel 0 at address offset 0. Info: Reading from c2h channel 1 at address offset 1024. Info: Wait for current transactions to complete. /dev/xdma0_c2h_0 ** Average BW = 1024, 15.247852 /dev/xdma0_c2h_1 ** Average BW = 1024, 46.265759 Info: Reading from c2h channel 0 at address offset 2048. Info: Reading from c2h channel 1 at address offset 3072. Info: Wait for current transactions to complete. /dev/xdma0_c2h_0 ** Average BW = 1024, 7.360711 /dev/xdma0_c2h_1 ** Average BW = 1024, 14.372131 Info: Checking data integrity. Info: Data check passed for address range 0 - 1024 Info: Data check passed for address range 1024 - 2048 Info: Data check passed for address range 2048 - 3072 Info: Data check passed for address range 3072 - 4096 Info: All PCIe DMA memory mapped tests passed. Info: All tests in run_tests.sh passed.
Now we test the ability to load the .bin file to the board's flash. If its highlighted, you need to click the "Add Configuration Memory Device", and then create the appropriate device for your board. If its grayed out you should right-click the existing <flash device> field:
Then select "Program Memory Configuration Device..." from the popup menu.
Click [OK]
It first erases the part and then programs the flash part.
This warning may pop up. It can safely be ignored. It occurs because Labtools looses communication with the FPGA while the JPEG is connected to the flash.
Click [OK] to dismiss it.
This step loaded the code to the board's flash, which is used to load the FPGA on power-up of the box. To test this we stop Ubuntu with a power-off, followed by a power-on. I've found that having the JPEG attached during power-up sometimes prevents the FPGA from booting, so best to disconnect it before re-powering the box:
Checking for device... 01:00.0 Memory controller: Xilinx Corporation Device 7024 Loading driver interrupt_selection . Loading driver...insmod xdma.ko interrupt_mode=2 ... The Kernel module installed correctly and the xmda devices were recognized. DONE Info: Number of enabled h2c channels = 2 Info: Number of enabled c2h channels = 2 Info: The PCIe DMA core is memory mapped. Info: Running PCIe DMA memory mapped write read test transfer size: 1024, count: 1 Info: Writing to h2c channel 0 at address offset 0. Info: Writing to h2c channel 1 at address offset 1024. Info: Wait for current transactions to complete. /dev/xdma0_h2c_1 ** Average BW = 1024, 8.630716 /dev/xdma0_h2c_0 ** Average BW = 1024, 7.923673 Info: Writing to h2c channel 0 at address offset 2048. Info: Writing to h2c channel 1 at address offset 3072. Info: Wait for current transactions to complete. /dev/xdma0_h2c_0 ** Average BW = 1024, 16.706366 /dev/xdma0_h2c_1 ** Average BW = 1024, 29.789957 Info: Reading from c2h channel 0 at address offset 0. Info: Reading from c2h channel 1 at address offset 1024. Info: Wait for current transactions to complete. /dev/xdma0_c2h_0 ** Average BW = 1024, 15.247852 /dev/xdma0_c2h_1 ** Average BW = 1024, 46.265759 Info: Reading from c2h channel 0 at address offset 2048. Info: Reading from c2h channel 1 at address offset 3072. Info: Wait for current transactions to complete. /dev/xdma0_c2h_0 ** Average BW = 1024, 7.360711 /dev/xdma0_c2h_1 ** Average BW = 1024, 14.372131 Info: Checking data integrity. Info: Data check passed for address range 0 - 1024 Info: Data check passed for address range 1024 - 2048 Info: Data check passed for address range 2048 - 3072 Info: Data check passed for address range 3072 - 4096 Info: All PCIe DMA memory mapped tests passed. Info: All tests in run_tests.sh passed.
Our project is working!
