avoiding usb

This might well be the worst problem with the Pi.  This Broadcom SoC just wasn't made for networking.

It contrasts rather badly with the TI SoC on the BeagleBone which provides both Ethernet MAC and two USB ports on the chip directly.

I wonder if Broadcom has a better device in this range that might be used for a future version of Pi.

well yes, it is somwhat less than ideal, but that`s what there is to work with.

know any work-arounds?

jbeale wrote:

 

My understanding, which might be wrong, is that the 20% CPU overhead from the 8k interrupts/sec is from the USB driver on the on-chip SoC USB port.

It's a timer-driven poll of the external device, apparently.  On a Model A, it wouldn't need to poll anything, since the USB driver would just sit there quiescent and only become active when USB data arrives and the SoC's USB controller then generates an interrupt to process it.  Ie. event-driven, just like everything else in the standard *nix model.

Morgaine Dinova wrote:

 

YT2095 . wrote:

 

know any work-arounds?

 

Use a Model A (once available) in combination with the Ethernet-over-USB gadget driver.  The on-chip USB controller will then handle the network traffic in the usual interrupt-driven way (an assumption, but reasonable I think), and a downstream class-compliant USB hub will also be handled by the normal interrupt-driven USB driver.

I don't think going with a Model A with external USB hub and Ethernet device would buy you anything.  The LAN9512 is really just a USB hub plus Ethernet device in a single chip.  The performance problem with RasPi is entirely the host software which runs on the BCM2835 in either case.  It just sounds like it's non-optimized software and could be replaced with something efficient if someone wants to go to the trouble of learning all about writing kernel drivers and USB control structures.  The USB 2.0 standard goes into quite a bit of detail about what's needed.

My understanding of USB host software is that it needs to find out what devices are connected and use device driver information to determine how often it needs to poll each device.  Then it uses a clock interrupt to trigger the polling routine.  ARM actually has very efficient interrupts if you choose to use them properly: ARM uses a different set of processor registers for interrupts so you don't have to save and restore main thread registers in memory.  However, you may have to do this in assembly language level, and it's easier to throw together a C interrupt service routine and accept the overhead, figuring that someone else will optimize it "some day".

I don't know how much polling intelligence is built into USB host hardware.  It may be that it can do a certain amount of this by itself, if one writes the software to configure it properly.  At one place I worked, we had the acronym SAMP: "Simply A Matter of Programming".  It was used both for features that required a small amount of programming and -- with irony -- for those that required a huge amount of programming.

I don't think it can be so John, because if it were, then all machines would be suffering the same USB polling overhead whenever they have downstream hubs and other devices attached.

No sensible engineer designs polled systems like that.

The problem here seems to be compounded by a dreadful driver implementation, but even if the driver were perfect then it would still be appallingly bad if it has to poll a downstream device.  You can't overcome the response latency and the overhead of active polling by writing a perfect driver.  It's the polling model here that's inherently broken.

Although I agree with that polling for the most part isn`t a great use of time or Cycles, and can think of few instances where polling is of any real benefit beyond keyboard scan lines and multiplexing, without the addition of extra hardware like priority encoders and the such, what do You suggest as an alternative?

Morgaine Dinova wrote:

 

I don't think it can be so John, because if it were, then all machines would be suffering the same USB polling overhead whenever they have downstream hubs and other devices attached.

 

No sensible engineer designs polled systems like that.

 

The problem here seems to be compounded by a dreadful driver implementation, but even if the driver were perfect then it would still be appallingly bad if it has to poll a downstream device.  You can't overcome the response latency and the overhead of active polling by writing a perfect driver.  It's the polling model here that's inherently broken.

My understanding of USB 2.0 (I haven't studied 3.0) is that it's exactly a polled system.  USB is great for what it was originally designed for: low-speed devices like keyboards and mice (where it replaced the Apple Desktop Bus), printers (where the vast majority of data is transferred from host to device), and mass storage devices (where all transactions are initiated by the host).  It's also effective for fixed data rate devices like sound I/O when you know how often you need to poll.

USB is a poor way to talk to Ethernet, since you don't know when the frames are coming and long latency can hurt.  (With a keyboard, a 10 msec delay won't be noticed.)  Ethernet frames over USB remind me of Samuel Johnson's comment about a dog walking on its hind legs: "It is not done well; but you are surprised to find it done at all."

You have a similar problem with USB serial (e.g., RS-232RS-232) devices, but it works well at modest speeds like 9600 Baud.

It may be that I don't know about some clever things that hubs do to improve the situation.  USB 2.0 is pretty complex and reading about host controllers is a good cure for insomnia.

I've been trying to get to the bottom of this, but with only partial success short of actually analysing the USB driver code.

On the one hand, there is no doubt that USB is a polled bus, because despite devices being able to send interrupts, the interrupt can't get to the host without the host polling for the transfer of the interrupt message.  I've read a variety of documents on this now, and the clearest is probably this one -- http://www.embeddedsys.com/subpages/resources/images/documents/InterruptsAndUSB.pdf .  The rate of polling is typically negociated during device ennumeration, and the host may reject a request for a high poll rate that it does not want to handle.

But just because the bus itself uses polling in a manner not too different from the old-fashioned "multi-drop" mechanism over serial links, this doesn't necessarily tell you whether it is the host's USB controller hardware that does the polling after being configured for a particular rate, or whether it is the host CPU that has to do the regular polling by itself, which would be really unwelcome if the rate is 8k.  Most of that effort would be totally wasted except when there is data available to be transferred.

You know, this could well be one area where the Pi will make a major contribution --- I suspect we're going to have a lot of Linux USB driver experts before long, and rewriting that very poor closed source driver may well be the start of the process.

When I receive my Pi, I'll hook up my logic analyser to it and try to see what's going on.  I can do that right now with my BeagleBone --- hopefully the comparison will shed some light.

Morgaine.

PS. Here's SMSC's datasheet for the LAN9512 device for reference --- http://www.smsc.com/media/Downloads_Public/Data_Sheets/9512.pdf .  If you look at the EEPROM table on page 36, it seems to be configured to request Full-Speed [12Mbit/s] Polling Interval for Interrupt Endpoint (1ms) and Hi-Speed [480Mbit/s] Polling Interval for Interrupt Endpoint (4ms), so it seems there is a tradeoff between response latency and data bandwidth.

PS2. I can't make head or tail of what the BCM2835 SoC's USB controller capabilities are.  See page 200 of their ARM Peripherals doc -- http://www.raspberrypi.org/wp-content/uploads/2012/02/BCM2835-ARM-Peripherals.pdf .  The linked Synopsys site is too tied up in logins rubbish for me to bother, maybe someone can BugMeNot it.

PS3.  How sad, I have 3 dead-tree books on the Linux kernel on my shelf, and they all predate the appearance of USB.   There are at least two free online books in this area though --- http://stid.googlecode.com/files/Linux.Kernel.Development.3rd.Edition.pdf and http://lwn.net/Kernel/LDD3/ -- having a browse of the USB sections now.

I don't think it can be so John, because if it were, then all machines would be suffering the same USB polling overhead whenever they have downstream hubs and other devices attached.

As I understand it, all machines do suffer 8k interrupts/sec, if they use the specific USB driver macroblock IP that Broadcom used. But I bet most machines use a more sensible architecture. The USB microframe length is 128 usec so 8k/sec needed. http://en.wikipedia.org/wiki/Universal_Serial_Bus

 

(My knowledge on this is pretty limited. We bought in a block of hardware and got with it the dwc_otg linux driver. No one at Broadcom knows in detail how it works).

Yes, the USB hardware is "dumb" and needs to be told what to do (by the core) every frame. More advanced USB hardware blocks would handle the simple responses in hardware, and so produce less load on the CPU (esp. when idle).

http://www.raspberrypi.org/phpBB3/viewtopic.php?f=28&t=7866&p=95796#p95082

jbeale wrote (quoting the Foundation's dom):

Yes, the USB hardware is "dumb" and needs to be told what to do (by the core) every frame. More advanced USB hardware blocks would handle the simple responses in hardware, and so produce less load on the CPU (esp. when idle).

Oh god, no.

I really cannot fathom how a designer can purchase a very low spec USB core from Synopsys, one with primitive features that will soak up 20% of your CPU power and with an obscure driver which cannot easily be improved through lack of open hardware API documentation, and somehow consider that they've done a good job of "design".  They haven't.  That's something that a recent graduate might think was reasonable, but an experienced designer would reject as extremely myopic design and certainly not of the calibre required for a widely used Broadcom product.

Morgaine.

Morgaine Dinova wrote:

... and certainly not of the calibre required for a widely used Broadcom product.

Well, from what I deduce from RasPi discussion (since there's not much in the way of official techical literature) the BCM2835 is a high-performance media processor designed for things like Roku and cell phones, with an inexpensive and reasonably capable general-purpose ARM11 processor primarily for control and communications.  For these applications, the expected ARM and USB requirements are modest and if it's inefficient, it doesn't matter as long as movies download fast enough.  RasPi is using the ARM11 processor as a GNU/Linux engine and when applications use all CPU cycles the inefficiencies become apparent.

I'll be interested in hearing about any performance comparisons you do with your BeagleBone.  At least its AM335x has plenty of USB documentation :-)

Agreed.  The BCM2835 was the wrong SoC to use in this application.  It was never designed to be the heart of a general purpose computer, as is now coming to light.  It's a media processor with a tiny little chunk of silicon dedicated to ARM over in the corner.

But the Pi designers knew that, so why would they have chosen it?  I don't know Broadcom's ARM product line, but one possible answer is that they simply have no directly suitable device, because they're not in the business of manufacturing SoCs for general computer use.

Re performance, yes, I'm looking forward to obtaining real numbers on this.  Gut feel is not engineering.

Agreed.  The BCM2835 was the wrong SoC to use in this application.  It was never designed to be the heart of a general purpose computer, as is now coming to light.  It's a media processor with a tiny little chunk of silicon dedicated to ARM over in the corner.

But the Pi designers knew that, so why would they have chosen it?  I don't know Broadcom's ARM product line, but one possible answer is that they simply have no directly suitable device, because they're not in the business of manufacturing SoCs for general computer use.

Re performance, yes, I'm looking forward to obtaining real numbers on this.  Gut feel is not engineering.

I think it's all a matter of perspective. This Pi  has been cooking for quite a while... back in 2006 it looked quite different: it used a 22.1 MHz ATmega644 and a 512K SRAM for data and framebuffer storage:  http://www.raspberrypi.org/archives/264

So when they scrapped that idea and went to the Broadcom SoC it was quite a step up in functionality in just about every way.  I know people want to do all kinds of things with it that the designers didn't imagine for their educational platform- which is fine- but the foundation designed to their price point and I still think they got a pretty good bang for the buck.

I certainly agree with that!

I think what many of the discussions are considering at the moment is how to overcome the downsides that came with having sought and achieved "a pretty good bang for the buck", because there are quite a few of those.  But given the "million eyeballs" in the community, I'm quite optimistic that most of the difficulties can be overcome.

In think what's upsetting is the realization that a $35 computer need not be

sluggish.  The usb interface does not need to waste 20% of the cpu time.

The hardware floating-point unit does not need to be wasted by compiling

for the Arm v5 architecture.  The L2 cache does not need to be wasted by

attaching it to the GPU instead of the CPU.   The GPU does not need to be

wasted when running X11.   The SD card interface does not need to be slow.

The GPU/CPU ram split does not need to be fixed at boot time, but can be

dynamically allocated where most needed.  512MB ram chips are in mass

production from multiple vendors, and users are willing to pay for them. 

Arm V7 cpus are readily available and users are willing to pay for them.

eventually somebody will come up with a working solution.

Yep.  And the current magic moment for Pi won't last long unless some effort is made to remedy the known deficiencies, while at the same time working on the next version to raise the bar to the next level.  That's how the world of technology advances.  Pi is no exception to the process.