avoiding usb

This might well be the worst problem with the Pi.  This Broadcom SoC just wasn't made for networking.

It contrasts rather badly with the TI SoC on the BeagleBone which provides both Ethernet MAC and two USB ports on the chip directly.

I wonder if Broadcom has a better device in this range that might be used for a future version of Pi.

well yes, it is somwhat less than ideal, but that`s what there is to work with.

know any work-arounds?

YT2095 . wrote:

 

know any work-arounds?

Use a Model A (once available) in combination with the Ethernet-over-USB gadget driver.  The on-chip USB controller will then handle the network traffic in the usual interrupt-driven way (an assumption, but reasonable I think), and a downstream class-compliant USB hub will also be handled by the normal interrupt-driven USB driver.

It's only a partial solution since it requires something else to provide the actual Ethernet hardware, but at least it avoids the polling.

Interesting that you should mention polling, Esp since the Pi is open source (for the S/ware at least), it should be perfectly possible to Lower this 8KHz polling rate to something more reasonable surely?

or perhaps make it at least Semi-inteligent whereby it`s scanned at a rate suitable for that which is in use.

I guess you could poll less frequently for keyboard and mouse traffic since their data rate is somewhat low, but remember that the average response latency is half the polling interval, so it would become progressively worse as you decreased the polling rate.  And for Ethernet, that sounds extremely unlikely as a viable solution at all, since you could end up losing data in buffer overruns between polls.

Basically, "polling == bad", period.

My understanding, which might be wrong, is that the 20% CPU overhead from the 8k interrupts/sec is from the USB driver on the on-chip SoC USB port. So it would be just the same in the Model A.  It has been suggested that it might be possible to make a much more efficient USB driver, reducing the interrupt service routine overhead by up to 90%, but even though the code is open-source, not clear if there's enough good documentation to really do this (?)

The 8 kHz interrupt rate is actually mandated by the specific USB hardware they used in the SoC. 8 kHz is apparently the high-speed USB2 "micro-frame" rate, and any lower rate would not work correctly (at least with high-speed USB devices). You can think of it in analogy to a "winmodem" which offloads some of the processing to the CPU, to save complexity/cost in hardware.

jbeale wrote:

 

My understanding, which might be wrong, is that the 20% CPU overhead from the 8k interrupts/sec is from the USB driver on the on-chip SoC USB port.

It's a timer-driven poll of the external device, apparently.  On a Model A, it wouldn't need to poll anything, since the USB driver would just sit there quiescent and only become active when USB data arrives and the SoC's USB controller then generates an interrupt to process it.  Ie. event-driven, just like everything else in the standard *nix model.

Morgaine Dinova wrote:

 

YT2095 . wrote:

 

know any work-arounds?

 

Use a Model A (once available) in combination with the Ethernet-over-USB gadget driver.  The on-chip USB controller will then handle the network traffic in the usual interrupt-driven way (an assumption, but reasonable I think), and a downstream class-compliant USB hub will also be handled by the normal interrupt-driven USB driver.

I don't think going with a Model A with external USB hub and Ethernet device would buy you anything.  The LAN9512 is really just a USB hub plus Ethernet device in a single chip.  The performance problem with RasPi is entirely the host software which runs on the BCM2835 in either case.  It just sounds like it's non-optimized software and could be replaced with something efficient if someone wants to go to the trouble of learning all about writing kernel drivers and USB control structures.  The USB 2.0 standard goes into quite a bit of detail about what's needed.

My understanding of USB host software is that it needs to find out what devices are connected and use device driver information to determine how often it needs to poll each device.  Then it uses a clock interrupt to trigger the polling routine.  ARM actually has very efficient interrupts if you choose to use them properly: ARM uses a different set of processor registers for interrupts so you don't have to save and restore main thread registers in memory.  However, you may have to do this in assembly language level, and it's easier to throw together a C interrupt service routine and accept the overhead, figuring that someone else will optimize it "some day".

I don't know how much polling intelligence is built into USB host hardware.  It may be that it can do a certain amount of this by itself, if one writes the software to configure it properly.  At one place I worked, we had the acronym SAMP: "Simply A Matter of Programming".  It was used both for features that required a small amount of programming and -- with irony -- for those that required a huge amount of programming.

I don't think it can be so John, because if it were, then all machines would be suffering the same USB polling overhead whenever they have downstream hubs and other devices attached.

No sensible engineer designs polled systems like that.

The problem here seems to be compounded by a dreadful driver implementation, but even if the driver were perfect then it would still be appallingly bad if it has to poll a downstream device.  You can't overcome the response latency and the overhead of active polling by writing a perfect driver.  It's the polling model here that's inherently broken.

Although I agree with that polling for the most part isn`t a great use of time or Cycles, and can think of few instances where polling is of any real benefit beyond keyboard scan lines and multiplexing, without the addition of extra hardware like priority encoders and the such, what do You suggest as an alternative?

Morgaine Dinova wrote:

 

I don't think it can be so John, because if it were, then all machines would be suffering the same USB polling overhead whenever they have downstream hubs and other devices attached.

 

No sensible engineer designs polled systems like that.

 

The problem here seems to be compounded by a dreadful driver implementation, but even if the driver were perfect then it would still be appallingly bad if it has to poll a downstream device.  You can't overcome the response latency and the overhead of active polling by writing a perfect driver.  It's the polling model here that's inherently broken.

My understanding of USB 2.0 (I haven't studied 3.0) is that it's exactly a polled system.  USB is great for what it was originally designed for: low-speed devices like keyboards and mice (where it replaced the Apple Desktop Bus), printers (where the vast majority of data is transferred from host to device), and mass storage devices (where all transactions are initiated by the host).  It's also effective for fixed data rate devices like sound I/O when you know how often you need to poll.

USB is a poor way to talk to Ethernet, since you don't know when the frames are coming and long latency can hurt.  (With a keyboard, a 10 msec delay won't be noticed.)  Ethernet frames over USB remind me of Samuel Johnson's comment about a dog walking on its hind legs: "It is not done well; but you are surprised to find it done at all."

You have a similar problem with USB serial (e.g., RS-232RS-232) devices, but it works well at modest speeds like 9600 Baud.

It may be that I don't know about some clever things that hubs do to improve the situation.  USB 2.0 is pretty complex and reading about host controllers is a good cure for insomnia.

Morgaine Dinova wrote:

 

I don't think it can be so John, because if it were, then all machines would be suffering the same USB polling overhead whenever they have downstream hubs and other devices attached.

 

No sensible engineer designs polled systems like that.

 

The problem here seems to be compounded by a dreadful driver implementation, but even if the driver were perfect then it would still be appallingly bad if it has to poll a downstream device.  You can't overcome the response latency and the overhead of active polling by writing a perfect driver.  It's the polling model here that's inherently broken.

My understanding of USB 2.0 (I haven't studied 3.0) is that it's exactly a polled system.  USB is great for what it was originally designed for: low-speed devices like keyboards and mice (where it replaced the Apple Desktop Bus), printers (where the vast majority of data is transferred from host to device), and mass storage devices (where all transactions are initiated by the host).  It's also effective for fixed data rate devices like sound I/O when you know how often you need to poll.

USB is a poor way to talk to Ethernet, since you don't know when the frames are coming and long latency can hurt.  (With a keyboard, a 10 msec delay won't be noticed.)  Ethernet frames over USB remind me of Samuel Johnson's comment about a dog walking on its hind legs: "It is not done well; but you are surprised to find it done at all."

You have a similar problem with USB serial (e.g., RS-232RS-232) devices, but it works well at modest speeds like 9600 Baud.

It may be that I don't know about some clever things that hubs do to improve the situation.  USB 2.0 is pretty complex and reading about host controllers is a good cure for insomnia.

I've been trying to get to the bottom of this, but with only partial success short of actually analysing the USB driver code.

On the one hand, there is no doubt that USB is a polled bus, because despite devices being able to send interrupts, the interrupt can't get to the host without the host polling for the transfer of the interrupt message.  I've read a variety of documents on this now, and the clearest is probably this one -- http://www.embeddedsys.com/subpages/resources/images/documents/InterruptsAndUSB.pdf .  The rate of polling is typically negociated during device ennumeration, and the host may reject a request for a high poll rate that it does not want to handle.

But just because the bus itself uses polling in a manner not too different from the old-fashioned "multi-drop" mechanism over serial links, this doesn't necessarily tell you whether it is the host's USB controller hardware that does the polling after being configured for a particular rate, or whether it is the host CPU that has to do the regular polling by itself, which would be really unwelcome if the rate is 8k.  Most of that effort would be totally wasted except when there is data available to be transferred.

You know, this could well be one area where the Pi will make a major contribution --- I suspect we're going to have a lot of Linux USB driver experts before long, and rewriting that very poor closed source driver may well be the start of the process.

When I receive my Pi, I'll hook up my logic analyser to it and try to see what's going on.  I can do that right now with my BeagleBone --- hopefully the comparison will shed some light.

Morgaine.

PS. Here's SMSC's datasheet for the LAN9512 device for reference --- http://www.smsc.com/media/Downloads_Public/Data_Sheets/9512.pdf .  If you look at the EEPROM table on page 36, it seems to be configured to request Full-Speed [12Mbit/s] Polling Interval for Interrupt Endpoint (1ms) and Hi-Speed [480Mbit/s] Polling Interval for Interrupt Endpoint (4ms), so it seems there is a tradeoff between response latency and data bandwidth.

PS2. I can't make head or tail of what the BCM2835 SoC's USB controller capabilities are.  See page 200 of their ARM Peripherals doc -- http://www.raspberrypi.org/wp-content/uploads/2012/02/BCM2835-ARM-Peripherals.pdf .  The linked Synopsys site is too tied up in logins rubbish for me to bother, maybe someone can BugMeNot it.

PS3.  How sad, I have 3 dead-tree books on the Linux kernel on my shelf, and they all predate the appearance of USB.   There are at least two free online books in this area though --- http://stid.googlecode.com/files/Linux.Kernel.Development.3rd.Edition.pdf and http://lwn.net/Kernel/LDD3/ -- having a browse of the USB sections now.