Actions
Forum Thread Details
  • State Suggested Answer
  • Replies 18 replies
  • Answers 6 answers
  • Subscribers 325 subscribers
  • Views 2567 views
  • Users 0 members are here
Related

UltraZed USB 3 IDS Camera Speed Issue

jbspears

We have the UltraZed eval board (EG) and have Xilinx 2017.4 u-boot/kernel with Ubuntu 18.04 running.  USB 3 seems to be setup correctly - all USB 3 devices we've tried are recognized as USB 3.  We are connecting an IDS Ueye camera to the USB port and trying to reach a frame rate that would be approximately 1.1 Gbps throughput.  However, we are only able to get around 780 Mbps before errors start to occur in the stream transfer over USB.  We have ran the USB 3 performance test provided by Avnet with various pen/usb drives and depending on the drive and the test (fixed block/block size, etc.) sometimes to get close to 1 Gbps.  The camera works fine at the high frame rate (and even higher) on an Ubuntu desktop computer with USB 3.

 

Anyone else tried running a USB 3 camera at a high frame rate on these boards?  Or have any idea of maybe a configuration somewhere (hardware/software) that might improve the USB 3 throughput?

 

Thanks!

  • 0

    Hi John,

     

    First, have you seen this blog article I posted?

    Solve USB 3.0 issues with this one weird trick

    Second, do you have a local XFAE you are working with?  If so, you should contact them about this issue.  Please also have them reach out to me as I can help provide some guidance to them.

     

    In the mean time, it is strange that you ask this at this time!  I just responded to another user where I explained that I had used a USB 3.0 camera with an Ultra96V2 (I realize not the same, but it is the same basic circuit).  I recall using that at high resolution, but I admit that I cannot say what the bit rate was.  Only that it was a CU30 (3.4MP) and was likely running at 1080P60.  I realize not apples to apples, but it just seemed strange two posts related to USB and video!

    https://www.e-consystems.com/ar0330-lowlight-usb-cameraboard.asp

     

    --Dan

  • 0 in reply to drozwood90

    Thanks for the info Dan.  Actually that blog article got me as far as I have gotten so far - getting a USB cable that works with the UltraZed (got the cable/adapter from Amazon).  With the right cable the camera was recognized as a USB 3.0 device.  Since I posted this earlier today, I ran the UltraZed performance test with a USB 3 toaster and a SSD on both a Linux desktop and the UltraZed.  Both performed about the same - around 300MB/s for read and write (I need only around 130MB/s).  So that tells me the USB 3 is performing at good speeds (at least for a SSD drive anyway) and the issue may be more camera/driver related.

     

    Camera I'm using:

    https://en.ids-imaging.com/store/ui-3140cp-rev-2.html

     

    I will look through the article again for any other ideas just in case.  Thanks!

  • 0 in reply to jbspears

    John,

     

    Now you got me thinking...watch your processor loading.  I've seen where dropping everything on 1 core can make funny things happen as well.  IIRC, that is why our Webcam interface on the UltraZed-EV VCU TRD Port uses 2 or 3 threads to spread the load across the various processors.  I can't be certain how many threads, but I do recall we split the gstreamer path up to a few cores.

     

    --Dan

  • 0 in reply to drozwood90

    Finally made some progress.  After changing the frame size to a power of 2 (512x512) and using a powered USB hub, we are able to meet framerate (500 fps).  However, we are seeing random failures with the USB.  It can happen after running for 15 minutes or it can happen after running for 2 minutes.  The error messages are:

     

    [   83.289070] xhci-hcd xhci-hcd.0.auto: Stopped the command ring failed, maybe the host is dead

    [   83.313647] xhci-hcd xhci-hcd.0.auto: Abort command ring failed

    [   83.319478] xhci-hcd xhci-hcd.0.auto: HC died; cleaning up

    [   83.319527] xhci-hcd xhci-hcd.0.auto: Error while assigning device slot ID

    [   83.319531] xhci-hcd xhci-hcd.0.auto: Max number of devices this xHCI host supports is 64.

    [   83.319536] usb usb1-port1: couldn't allocate usb_device

    [   83.345402] usb 2-1: USB disconnect, device number 3

  • 0 in reply to jbspears

    Hi John,

     

    Do you have a local XFAE you are working with?  If so, you should contact them about this issue.  Please also have them reach out to me as I can help provide some guidance to them.

     

    --Dan

  • 0 in reply to drozwood90

    Update.  I think the camera works fine on the UltraZed board.  Our goal was to capture and stream frames across the network.  But that would kill the USB controller after a while.  So as a test we have an app that just captures the camera images and does nothing with them.  This will run for days without issue.  We then created an app that just streams dummy frames (totally independent of the camera app).  This app will run alone for days without issue.  But when we run them together, within an hour or so, the USB controller dies with the message below.  We cannot figure out why using the network device causes the USB controller to die.

     

    xhci-hcd xhci-hcd.0.auto: xHCI host controller not responding, assume dead

  • 0 in reply to jbspears

    Hi there,

     

    Wow, what a find.  I will ping a few people, but in the mean time, is there any suspicion on power draw / issues around the USB device?  I have no proof, and this might be a red herring, but my first gut feel with this new information is, could the device be getting marginal power?  Then eventually it just...hiccups?

     

    I recall earlier you saying you resolved some issues by just adding a USB Hub.

     

    Are you also leveraging the VCU?  PL DDR memory?

    Have you checked the power estimator in Vivado?

     

    Again, not trying to side-track you, but I was just musing about this.

     

    --Dan

  • 0 in reply to drozwood90

    I know power definitely was an issue at first.  The camera people suggested a specific ("approved") USB hub for their camera (IDS uEye).  And this solved our initial issues.  I will have our Vivado expert look into your questions.  There must be some low-level resource (power, clocks, etc.) that the network device and USB device share for this to be an issue.  I have even tried the 2020.1 tools (Avnet Petalinux, Xilinx Linux/u-boot, with Ubuntu 18.04, etc.) in hopes there was a driver fix or something, but still same issue.  We are using the UltraZed EG board.

     

    Thanks for your help!!

  • 0 in reply to drozwood90

    FYI, new data this morning related to power.

     

    There are two processes.  The camera process just acquires frames and discards them.  The publisher process is streaming simulated images to N clients at once over the network (TCP).  Each process is totally independent of the other.

     

    I connected a USB power monitor to the camera connection.

     

    Power observations:

     

    Neither process running: 5.09V, 0.280A

    Only camera process: 5.02V, 0.544A

     

    Start camera process after publisher with 5 clients: 5.02V, 0.494A, then suddenly 5.14V, 0.137A

     

    Publisher with 1 client – best case will run for close to 18 hours

    Publisher with 2 clients - best case will run for close to 8 hours

    Publisher with 3 clients - best case will run for a couple hours

    Publisher with 4 clients - best case will run for a few minutes

    Publisher with 5 clients - best case will run for a few seconds

  • 0 in reply to jbspears

    Hey John,


    Dan asked me to take a look at this as well as I'm the power expert on the team. Based on what you saw on the power monitor, it would be reasonable to assume that there could be a decent size transient when the load jumps. There is an experiment you could to try to see if it is a lack of 5V power capacity from the IO carrier causing your issue. J6 on the carrier board has a jumper shorting pins 1-2. If you remove that jumper, it removes 5V power from the generating source and leaves the USB+ rail unpowered. This feature allows both Host and Device mode operation. If this jumper is removed, you could provide 5V from an external supply to pin 2, assuming you have an external supply you could use. That would allow you to use a high current 5V supply as an experiment to see if a power limitation was causing the issue. If the design ran without issue using an external high current source, that would point pretty hard at a power limitation as the source of your problem. Let me know if you think you may be able to try this and need any assistance in setup.

    Thanks
    Chris