Actions
Forum Thread Details
  • State Not Answered
  • Replies 6 replies
  • Subscribers 319 subscribers
  • Views 1867 views
  • Users 0 members are here
Related

PCIe root complex Error with PCIe to PCI adapter (7z045)

Former Member

Hi,

I try to use a PCIe to PCI adapter on Zynq Minit-itx board.


Link to PCIe to PCI adapter: http://www.startech.com/Cards-Adapters/Slot-Extension/PCI-Express-to-PCI-Adapter-Card~PEX1PCI1


I use the PCIe root complex design provided at http://zedboard.org/content/zynq-mini-itx-7z045-pcie-design-vivado-20144
I use version 2014.4 for petalinux tools, Vivado and SDK.


After generating bitstream, FSBL, u-boot and kernel I test PCIe with Ethernet device.


PCIe root complex work, I connect an ethernet PCIe device to test it. Below, a log of PCI initialization when the PCIe ethernet is connected:
xaxi_pcie_init_port: LINK IS UP
AXI PCIe Root Port Probe Successful
xaxi_pcie_set_bridge_resource:pci_space: 0x02000000 pci_addr:0x0000000060000000 size: 0x0000000010000000
xaxi_pcie_set_bridge_resource:Setting resource in Memory Space
PCI host bridge /amba_pl/pciex@50000000 (primary) ranges:
MEM 0x0000000060000000..0x000000006fffffff -> 0x0000000060000000
PCI: PHB MEM resource 0 = 0000000060000000-000000006fffffff [200]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [io  0x1000-0xffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
PCI: bus0: Fast back to back transfers disabled
PCI: bus1: Fast back to back transfers disabled
pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
pci 0000:00:00.0: BAR 14: assigned [mem 0x60000000-0x600fffff]
pci 0000:00:00.0: BAR 13: assigned [io  0x1000-0x1fff]
pci 0000:01:00.0: BAR 4: assigned [mem 0x60000000-0x60003fff 64bit pref]
pci 0000:01:00.0: BAR 2: assigned [mem 0x60004000-0x60004fff 64bit]
pci 0000:01:00.0: BAR 0: assigned [io  0x1000-0x10ff]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
pci 0000:00:00.0:   bridge window [mem 0x60000000-0x600fffff]


The kernel boot and I can view the device in a lspci.


When I connect the PCIe to PCI adapter the kernel lock during PCI initialization. Below, a log of PCI initialization when PCIe to PCI adapter is connected:
xaxi_pcie_init_port: LINK IS UP
AXI PCIe Root Port Probe Successful
xaxi_pcie_set_bridge_resource:pci_space: 0x02000000 pci_addr:0x0000000060000000 size: 0x0000000010000000
xaxi_pcie_set_bridge_resource:Setting resource in Memory Space
PCI host bridge /amba_pl/pciex@50000000 (primary) ranges:
MEM 0x0000000060000000..0x000000006fffffff -> 0x0000000060000000
PCI: PHB MEM resource 0 = 0000000060000000-000000006fffffff [200]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [io  0x1000-0xffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci_bus 0000:00: scanning bus
pci 0000:00:00.0: [10ee:0706] type 01 class 0x060400
pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
pci 0000:00:00.0: calling pci_fixup_ide_bases+0x0/0x3c
pci_bus 0000:00: fixups for bus
PCI: bus0: Fast back to back transfers disabled
pci 0000:00:00.0: scanning [bus 01-07] behind bridge, pass 0
pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 1
pci_bus 0000:01: scanning bus
pci 0000:01:00.0: [12d8:e111] type 01 class 0x060400
pci 0000:01:00.0: calling pci_fixup_ide_bases+0x0/0x3c
pci_bus 0000:01: fixups for bus
PCI: bus1: Fast back to back transfers disabled
pci 0000:01:00.0: scanning [bus 00-00] behind bridge, pass 0
pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:01:00.0: scanning [bus 00-00] behind bridge, pass 1
pci_bus 0000:02: scanning bus


After this line the kernel is locked, no other information is printed and no boot.


The PCIe to PCI adapter is OS independant (No additional drivers or software required).
I test it on a standard PC (with BIOS), it worked.


I don't understand what is wrong in this configuration.
Do I need to update my device-tree with information about the PCIe to PCI adapter ?
Or hardware compatibility issue between PCIe to PCI adapter and PCIe root complex Xilinx design ?


Best regards,
Hugo



  • 0

    Just to be sure, are you using the Avnet reference design or are you creating your own HW and SW projects? If you are using your own, would you please try the Avnet ref design.

  • 0 in reply to Former Member

    Yes I use the Avnet design (http://zedboard.org/content/zynq-mini-itx-7z045-pcie-design-vivado-20144).

    First I try with ready_to_test example (zynq_mini_itx_7z045_pcie_design_v2014_4.zip/ready_to_test/sd_image_nic) and I have the same behavior a PCIe ethernet card is detected but if I connect PCIe to PCI adapter the kernel lock during PCI initialization.

    After this, I try to regenrate Bitsrteam FSBL and u-boot from the hardware design. The only modification I made on the design is to make PTP_ETHERNET port external (I need Ethernet connexion). I ran synhtesis, implementation, bitstream generation and hardware export. With the .hdf I create a petalinux project and I generate FSBL u-boot and kernel with petalinux tools. I have encoutered some issues with the auto generated device-tree, the Ethernet and PCIe sections was wrong so I decompile the device-tree blob provided in ready_to_test example to get PCIe and Ethernet sections. I copy those information into the petalinux device-tree source before building (petalinux-build).

    With this build I observe the same behavior as the ready_to_test example (a PCIe ethernet card is detected but when I connect adapter the kernel lock during PCI init).

  • 0

    Yes, I think.

    Below, a lspci -vvv of adapter when it is connected in standard desktop computer:
    02:00.0 PCI bridge: Pericom Semiconductor Device e111 (rev 02) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: 00003000-00003fff
        Memory behind bridge: e1000000-e10fffff
        Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
        Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
            PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [80] PCI-X bridge device
            Secondary Status: 64bit- 133MHz- SCD- USC- SCO- SRD- Freq=conv
            Status: Dev=02:00.0 64bit- 133MHz- SCD- USC- SCO- SRD-
            Upstream: Capacity=16 CommitmentLimit=16
            Downstream: Capacity=16 CommitmentLimit=16
        Capabilities: [a8] Subsystem: Device 0000:0000
        Capabilities: [b0] Express (v1) PCI/PCI-X Bridge, MSI 00
            DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
            DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
                MaxPayload 128 bytes, MaxReadReq 512 bytes
            DevSta: CorrErr+ UncorrErr+ FatalErr- UnsuppReq+ AuxPwr+ TransPend-
            LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <512ns, L1 <1us
                ClockPM- Surprise- LLActRep- BwNot-
            LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk+
                ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
            LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
        Capabilities: [d8] Vital Product Data
            Unknown small resource type 00, will not decode more.
        Capabilities: [f0] MSI: Enable- Count=1/1 Maskable- 64bit+
            Address: 0000000000000000  Data: 0000
        Capabilities: [100 v1] Advanced Error Reporting
            UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
            UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
            UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
            CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
            CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
            AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [150 v1] Virtual Channel
            Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
            Arb:    Fixed- WRR32- WRR64- WRR128-
            Ctrl:   ArbSelect=Fixed
            Status: InProgress-
            VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff

    Thank for your help.

    Best regards,
    Hugo

  • 0

    Sorry it took so long to get to the root cause of the issue you are having with the PCIe Root Complex design, but here is a brief description of what is causing this behavior.

    The root cause of this problem is related to the PCIe wrapper (fabric logic wrapped around the PCIe Hard IP in Zynq PL to implement the PCIe Root Complex IP). The AXI Memory Mapped bridge in the PCIe wrapper can only handle memory transactions to the PCIe Devices.  It can not handle any sort of IO transactions, which would be required to communicate with the PCIe-to-PCI bridge (a legacy device).

    In order to support the PCIe-PCI adapter, the PCIe wrapper must be modified to access the adapter via IO transactions. Unfortunately, Xilinx has no plans on implementing this feature and if needed it must be implemented by the user. If you are willing to implement this feature yourself, I can put you in contact with someone at Xilinx who can provide some limited assistance.

  • 0

    Thank for your feedback.

    I have a working configuration since last evening. I meant to post here this morning.

    If I remove the Multi-core scheduler from kernel configuration the adapter works. (I will post an issue at linux-xlnx github page).

    To sumerize my kernel configuration:
    CONFIG_SCHED_MC must be unset (Kernel Features => Multi-core scheduler support)
    CONFIG_XILINX_AXIPCIE=y (System Type => Xilinx Zynq ARM Cortex A9 Platform => Xilinx AXI PCIe host bridge support)
    CONFIG_PCI=y (Bus support => PCI support)
    CONFIG_PCI_MSI=y (Buss support => Message Signaled Interrupts (MSI and MSI-X))
    CONFIG_PCI_REALLOC_ENABLE_AUTO=y (Buss support => Enable PCI resource re-allocation detection)
    CONFIG_PCIEPORTBUS=y (Buss support => PCI Express Port Bus support)
    CONFIG_PCIEAER=y (Buss support =>  Root Port Advanced Error Reporting support)

    I'm not sure that MSI, REALLOC, PCIEPORTBUS and PCIEAER are necessary, I need to test this.

    I change two configurations in Vivado hardware design, I set MSI Vectors Requested to 1 and COMP TIMEOUT to 50u00B5s, I'm not sure that is necessary.