Actions
Forum Thread Details
  • State Not Answered
  • Replies 5 replies
  • Subscribers 174 subscribers
  • Views 418 views
  • Users 0 members are here
Related

PICe MSI - legacy interrupt issues

Former Member

Weu2019ve starting use the PCIe part of picoZed, xc7z030sbg485-1. The root complex seems to be coming up OK:

[root@alarm ~]# lspci
00:00.0 PCI bridge: Xilinx Corporation Device 7012
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
02:03.0 PCI bridge: PLX Technology, Inc. PEX 8605 PCI Express 4-port Gen2 Switch (rev ab)
03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
04:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

The issue seems to be with the MSI interrupt connections. When the i210 driver loads its indicating using legacy interrupts.

[    1.159680] pci 0000:00:00.0: enabling device (0144 -> 0147)
[    1.164007] pci 0000:01:00.0: enabling device (0140 -> 0143)
[    1.168325] pci 0000:02:02.0: enabling device (0140 -> 0143)
[    1.172663] igb 0000:03:00.0: enabling device (0140 -> 0142)
[    1.210435] igb 0000:03:00.0: added PHC on eth0
[    1.213606] igb 0000:03:00.0: Intel(R) Gigabit Ethernet Network Connection
[    1.219137] igb 0000:03:00.0: eth0: (PCIe:unknown:unknown) 00:a0:c9:00:00:00
[    1.224967] igb 0000:03:00.0: eth0: PBA No: 000300-000
[    1.228719] igb 0000:03:00.0: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)
[    1.235130] pci 0000:02:03.0: enabling device (0140 -> 0143)
[    1.239423] igb 0000:04:00.0: enabling device (0140 -> 0142)
[    1.277118] igb 0000:04:00.0: added PHC on eth1
[    1.280260] igb 0000:04:00.0: Intel(R) Gigabit Ethernet Network Connection
[    1.285838] igb 0000:04:00.0: eth1: (PCIe:unknown:unknown) 00:a0:c9:00:aa:aa
[    1.291626] igb 0000:04:00.0: eth1: PBA No: 000300-000
[    1.295394] igb 0000:04:00.0: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)

Then when we try to connect the Ethernet port we get an error 22:

[root@alarm ~]# ifconfig enp4s0 up
igb 0000:04:00.0: Error -22 getting interrupt
SIOCSIFFLAGS: Invalid argument

I have the MSI vectors req set to 5 and have MSI/MSI-X enabled in the Kernel build, and connected the interrupt from the AXI-PCIe to the Zynq core IRQ_F2P[0]. Weu2019re also using a PLX 8505 switch to connect the Intel MACs to the PCIe root.

Weu2019ve try removing one of the Intel MACs from the PLX switch (so only one is showing). The problem stayed the same. 

Also, tried loading the i210 driver manual via modprobe. The problem stays the same. 

Any ideas on how to debug / fixes the MSI and / or legacy interrupts issues?

  • 0

    Have you tried removing the external switch to see if that sheds any light on the problem?

     

    You might try posting on the Xilinx Community forums for more input on the Xilinx software involved.

     

    -Gary

  • 0

    Yes, we've been trying to remove the PLX switch so we would have direct Zynq-PCIe/root to Intel MAC. So far we ran into several issues. Try using the mini-ITX dev board, but seem to have Linux boot problems (seems like a combo between devicetree and SD card issues). Right now we've having one of our picoZed / SoC (our design) reworked to remove the PLX. Should be done early next week. In the meantime I was hoping to get some better debug info from the Intel MAC driver loading. I've try setting the debug level on the driver load, but the messages are the same. Any other debug ideas?

    Van

  • 0

    I tried an experiment to see if the PLX switch was part of the issues; using the Mini-ITX board to boot using the PCIe reference design and an Intel NIC (i210 part). Iu2019m still seeing error -22 when the IGB driver load and was unable to get the NIC to link up. A couple on points of note; the PCIe reference will not load without changing the how the AXI PCIe interrupt is connected. In the reference design the interrupt is connected in the top level wrapper. Doing this, the device tree generator doesnu2019t know how the interrupt is routed and the AXI PCIe doesnu2019t load. I fixed the connection. For SW, I used the 3.17 kernel with the IGB driver built-in, uBoot from the 2014.4 release, and a RAM RootFS from the reference design NIC-example.

    Van

  • 0

    I updated the Intel MAC driver to 5.2.15 (1/7/15). Iu2019m still seeing the same error -22, but getting some additional debug info. Looks like the driver is trying to connect as MSI but failing:

    [root@alarm src]# modprobe igb debug=16 IntMode=1
    Intel(R) Gigabit Ethernet Network Driver - version 5.2.15
    Copyright (c) 2007-2014 Intel Corporation.
    igb: 0000:03:00.0: igb_validate_option: Interrupt Mode set to 1
    igb: 0000:03:00.0: igb_check_options: RSS is not supported when in MSI/Legacy Interrupt mode, using default of 1
    igb 0000:03:00.0: Failed to initialize MSI interrupts.  Falling back to legacy interrupts.
    igb 0000:03:00.0: added PHC on eth2
    igb 0000:03:00.0: Intel(R) Gigabit Ethernet Network Connection
    igb 0000:03:00.0 enp3s0: renamed from eth2
    igb 0000:03:00.0: enp3s0: (PCIe:unknown:unknown)
    igb 0000:03:00.0 enp3s0: MAC: 00:a0:c9:00:00:00
    igb 0000:03:00.0: enp3s0: PBA No: 000300-000
    igb 0000:03:00.0: LRO is disabled
    igb 0000:03:00.0: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)

    [root@alarm src]# ifconfig enp3s0 10.160.2.10/24 up
    igb 0000:03:00.0: Error -22 getting interrupt
    SIOCSIFFLAGS: Invalid argument
    igb 0000:03:00.0: Error -22 getting interrupt
    SIOCSIFFLAGS: Invalid argument

    Iu2019m building the kernel and IGB driver local to the picoZed now (Linux alarm 3.17.0-xilinx #1 SMP PREEMPT Mon Feb 16 20:34:22 PST 2015 armv7l GNU/Linux).

    Do you know if Xilinx is seeing any other reports of MSI interrupts with the Zynq AXI-PCIe device/driver issues.

    Van

  • 0

    I tested both a Broadcom and Intel NIC on the mini-ITX. Both drivers show MSI interrupt capable but didnu2019t use it. The 3.17 Kernel build has pci_msi enable (Vivado 2014.4). At this point Iu2019m assuming MSI isnu2019t functional in 3.17.  Any idea if this is being addressed in the next release?

    Van