Timeout Waiting for Hardware Interrupt on mmc0

I took a look at the Verified Peripherals wiki ( http://elinux.org/RPi_VerifiedPeripherals#SD_cards ) and saw Patriot 8GB Class 10 cards both on the Working and Problem lists (with your "mmc0: timeout waiting for hardware interrupt" error).  So you're not alone.

You might check the fine print on the cards to see if they're all the same exact model number, but I think it's more likely that RasPi is clocking the SD card faster than two of them can handle.  That could be as simple as a rounding error when RasPi calculates the SD card clock rate.  I don't know much about SD card classes and how a processor goes about deciding how fast to clock an SD card.  It could be that your PC is more conservative than RasPi about SD card timing, so it's fine but RasPi is a bit too optimistic about SD performance or has a rounding error in its clock rate calculation.  This is a SWAG based on ignorance, so add salt.

Question: does a misbehaving SD card fail in many RasPis or just one?  If it's the latter, it could be a subtle timing problem like clocking the SD card on the wrong edge, but I think that's unlikely or we would have heard much more keening and gnashing of teeth.

Coming from experience I know that any retail cards are a crap shoot as to what they really have inside of them, no matter what the outside label or marking is. Selling to OEMs there are tight constraints as to who is on the AVL but for the consumer retail market it's fair game as long as the card is the correct format (and even that's debatable!).

It ended up that 11/28 just wouldn't boot and would throw these errors on the RPi. The systems are already being assembled so I only have access to my one RPi.

I just returned them all to Fry's and ended up exchangin them for even scarier brand called Unirex (whom I've never heard of). The first one imaged fine and booted up good but I've 10 more to do before I breath easy.

I should have kept one to see if I could image it for the Pandaboard and Beagleboard and see if they had any problems but I didn't. I had placed a call into tech support of Patriot and asked them to call me back but of course they haven't.

I'm assumming that the RPi is using SPI mode to access the SD Card so they don't have to pay the SD licences fee? Is that the clock you're referring to?

George Ioakimedes wrote:

 

I'm assumming that the RPi is using SPI mode to access the SD Card so they don't have to pay the SD licences fee? Is that the clock you're referring to?

There's a clock signal no matter which mode you're using.

Yes but if it's using SPI that's about as common a bus signal as there can be in an embedded system (next to I2C) and with only 1 data pin (as opposed to the 4 bits) I would think the timing wouldn't be as critical.

So USB doesn't really work, and now SD Card access also doesn't really work, and I've read that there are problems with ethernet as well. I guess I'm getting what I paid for, eh?!

But seriously, are there really these issues and people haven't fixed them yet? I'm beginning to wonder more and more about the BCM2835 chip itself.

It's not using SPI, it's using all of the capabilities available at 3.3v. There's no way to change the interface voltage to 1.8v, so some modes are not available.

SD Card stuff really is a game of chance unfortunately and the Pi has had no end of problems in that respect. Most have now been fixed, but had you been trying this back in April you'd have been having a lot more problems.

Essentially which mode and clock speed is used is based on a descriptor that's read from the card using the slowest most basic mode that all cards must support. From the descriptor the system can then work out which version of card you have (i.e. SDHC, SDXC etc) and what speed modes it's capable of supporting. Buried in the drivers there's also some attempts at retraining at lower speeds and fallback to more basic modes. That doesn't always work, especially if the card isn't genuine and the descriptor lies.

For some values of "every other system has no problems" that may be due to other factors. For example, the onboard sdcard reader in my laptop only uses SDSC speeds. That makes it much slower, but also means it works with more cards.

Sadly, sd-card issues are not so easy to fix. Would you care to test the driver with every possible card out there, including counterfeit and simply faulty cards ?  The other option could be that the RPF castrate the driver in the 'official' images to only use the slowest possible modes, but I suspect that would bring more screams of annoyance from a different set of users.

Remember here that the original images had a very much worse sdcard driver and that the RPF weren't much interested at the time. It took some outside developers to fix the worst of the problems.

General advice from some months of watching the problem and testing some cards - avoid class 10 cards if possible, the faster modes are more prone to issues. Class 10 is also often optimised for a very different use case (large sequential writes) and a good class 6 card is often better in the Pi. YMMV of course depending on the card.

selsinork wrote:

 

It's not using SPI, it's using all of the capabilities available at 3.3v. There's no way to change the interface voltage to 1.8v, so some modes are not available.

By "all of the capabilities" do you mean it is using 4-bit mode? My understanding is that 1.8V is only for lower power consumption and that the mode of operation or available commands does not change. Of course 1.8V is only possible for SDHC/SDXC cards.

selsinork wrote:

 

Essentially which mode and clock speed is used is based on a descriptor that's read from the card using the slowest most basic mode that all cards must support. From the descriptor the system can then work out which version of card you have (i.e. SDHC, SDXC etc) and what speed modes it's capable of supporting. Buried in the drivers there's also some attempts at retraining at lower speeds and fallback to more basic modes. That doesn't always work, especially if the card isn't genuine and the descriptor lies.

I know nothing about their SD card driver but at initial boot up I would expect the driver to read the descriptor (CSD) and that is defined to happen at 400kHz. If the driver is written to use 4-bit mode then based on the descriptor read the clock speed will be determined. Do you think it is at this point in the boot process that these errors are getting thrown? It just seems odd to me that at this point it totally looses communication with the SD Card.

From my understanding the class rating of the cards just defines how fast the card can be written to and with Class 10 they need to be able to maintain a 10MB/sec write speed. I can understand during normal operation problems happening because not all cards may really be able to maintain certain speeds and if the card is being clocked at a rate that it says it can support but in reality it doesn't then problems can happen.

selsinork wrote:

 

Sadly, sd-card issues are not so easy to fix. Would you care to test the driver with every possible card out there, including counterfeit and simply faulty cards ?

My findings have nothing to do with counterfeit or faulty cards and I don't believe that a driver should be written to support those cards. I don't know the real reason why those 11 cards didn't work but failing so soon in the boot process makes me believe that the driver couldn't communicate with the card. To me that seems to be a major problem in the driver. Wouldn't you as an embedded engineer write a driver that reads the CSD, sets the system parameters (block size, speed, etc.) and then try to read the card, if the read fails try a slower speed? To totally crap out on a perfectly fine card seems pretty bad to me.

Now I would expect that USB card readers have paid the SD license and operate cards in 4-bit mode but maybe I'm wrong and they only operate in SPI mode so those other bits are never excercised and for all I know they could have opens on those lines but 11/28 cards have manufacturing defects?

I always try to learn something from problems and I just don't see what I learned here other than that nice little $35 board is fine for schools but not for all applications. I just don't get that with so many units sold that these types of problems are still happening though....

By "all of the capabilities" do you mean it is using 4-bit mode? My understanding is that 1.8V is only for lower power consumption and that the mode of operation or available commands does not change. Of course 1.8V is only possible for SDHC/SDXC cards.

Yes, err.. maybe, sometimes..

Lets take a step back for a moment, there are several distinct phases involved.

1. the minimal on-SoC GPU firmware tries to initialise the SD card and load bootcode.bin
2. bootcode.bin pulls in config.txt and sets up stuff defined there, then likely has a better look at the sdcard and tries to load start.elf, again into the GPU
3. start.elf loads cmdline.txt and finally kernel.img then boots linux on the arm

Only after all these things happen does the linux sd-card driver get a look in. At this point there's probably been three different drivers used already and the linux driver is the fourth.

So it may be that the first step uses a slow SPI-only mode and bit-bangs it, then as bigger and more intelligent bits of code get loaded the clock gets increased, 4bit mode gets turned on and DMA gets used. What's the point where it breaks ?  Can the previous driver leave the h/w in some screwed up state ?  Is there any communication up the layers to tell the new driver there's a problem etc..

1.8v does enable addidional modes - particularly for class 10 cards, here's a small snippet from the linux driver: https://github.com/bootc/linux/commit/ac1dba37f1e0ba1acdc19e12a70199c420101459  the point here is that you can have a card that supports a faster mode, a SoC that does, and a driver that does, but the particular hardware implementation on the R-Pi omits any way to switch voltage.

One of the early problems was that the default mmc clock set by one of the early boot steps was very non-optimal, resulting in the clock divisors forceing a very low clock speed on cards that were capable of much more.  So all the early bits would work fine and the linux kernel would load, slowly...  Then the problems would start.

Look at the comments to Grigori's fixes in early June for some context here https://github.com/grigorig/rpi-linux/commits/sdhci-perf

I know nothing about their SD card driver but at initial boot up I would expect the driver to read the descriptor (CSD) and that is defined to happen at 400kHz. If the driver is written to use 4-bit mode then based on the descriptor read the clock speed will be determined. Do you think it is at this point in the boot process that these errors are getting thrown? It just seems odd to me that at this point it totally looses communication with the SD Card.

Yeah, but which driver in which step uses what bit of the CSD ?  Some are probably making assumptions and hard-coding slow/safe speeds.. Later when the higher speeds get enabled is when you lose communication. Could that be a bug in the new driver ? Sure.  But it's hard to seperate the driver from the hardware and from the unknown quality card and work out where the problem lies.

From my understanding the class rating of the cards just defines how fast the card can be written to and with Class 10 they need to be able to maintain a 10MB/sec write speed. I can understand during normal operation problems happening because not all cards may really be able to maintain certain speeds and if the card is being clocked at a rate that it says it can support but in reality it doesn't then problems can happen.

I suspect there's some invalid assumptions on the part of both card manufacturers and driver writers. What happens when someone assumes that if you support mode 5 then you also must support 1-4, but in reality the thing only supports mode 1 and 5 ?  Any fallback to mode 4 when mode 5 fails is likely going to have problems too.

My findings have nothing to do with counterfeit or faulty cards and I don't believe that a driver should be written to support those cards.

Ok, but exactly how do you (or the driver) tell if the card is counterfeit or faulty ?  Read the details from the card, try to use them and it doesn't work. Now what ?  You don't know if the card is broken, lying to you or what..

I always try to learn something from problems and I just don't see what I learned here other than that nice little $35 board is fine for schools but not for all applications. I just don't get that with so many units sold that these types of problems are still happening though....

What I take from it is that I don't like SD cards   Thing is that for lots of people the sd card issues on the Pi have been fixed, maybe you were just incredibly unlucky and got a bad batch of cards, maybe your Pi is faulty......   While I believe the Pi is fine for lots of things, you do need to be aware that it has it's problems. Some fixable, some not. In light of that you have to weigh up the risks and decide what you're comfortable with. Maybe a BeagleBone or an OlinuXino is a better choice due to the issues you've had with the Pi, only you can decide that though.

George Ioakimedes wrote:

 

So USB doesn't really work, and now SD Card access also doesn't really work, and I've read that there are problems with ethernet as well. I guess I'm getting what I paid for, eh?!

Although "you get what you pay for" is a commonly heard line, it's not actually all that accurate in our context.  There is no shortage of examples of extremely cheap electronics components that work exactly to spec, even very complex SoCs.  With each passing year, the mountain of cheap but perfectly working electronics devices just grows and grows.

A more relevant observation (I think) is that using a device outside of its intended design space is risky.  The BCM2835 was not designed to be the primary SoC in a general purpose host computer.  Its small ARM core's primary function seems to be chip housekeeping such as servicing the minimalist hardware USB core, which decreased overall device cost.  This made some sense in the intended role, but maybe not in others.

We now know officially  that the BCM2835's USB design is inadequate for providing full USB host functionality, and it's an open question whether this can ever be remedied in the driver since Linux is really poor at realtime response even when the realtime kernel patches are applied.  Success remains to be proven.

So in summary, I'm not sure that you should blame the low cost of Pi for the faults you have encountered.  In terms of engineering, a more accurate diagnosis would appear to be that the wrong chip was chosen for the job at Pi design time.  We'll have to see how this issue pans out, but that will take a while.

Morgaine.

Not so much that you get what you pay for amounting to the low cost of the RPi but as to the "engineering" take on the problems. We're all somewhat assumming that some of these problems we've encountered can be attributed to the BCM2835 but if so what's the big deal for the Foundation to fess up and say hey the chip works great but these are the known problems and we're stuck with them.

I'm also somewhat surprised that I find very little actual engineering analysis put into some of these problems. Take for instance the SD card issue I've experienced. Are there any scope plots anywhere that show what speeds the RPi uses during the different phases of boot up, I didn't see any. That really seems odd to me especially since there are so many people working with these boards with all different levels of experience.

The SD card fails at such an early stage in the boot process and it is 100% repeatable so isn't that one of the easiest engineering problems to fix? I think you're probably right and I'm starting to believe that the BCM2835 has some known bugs and since Broadcom is so secrative they won't admit them and anyone who tries to tell them that is told to be quite...

George Ioakimedes wrote:

 

The SD card fails at such an early stage in the boot process and it is 100% repeatable so isn't that one of the easiest engineering problems to fix? I think you're probably right and I'm starting to believe that the BCM2835 has some known bugs and since Broadcom is so secrative they won't admit them and anyone who tries to tell them that is told to be [quiet]...

While there are some RasPi problems that seem to be ignored, the SD card problem is not one of them.  The SD card problem is well known and RasPi is trying to solve it.  Do take a look at this RasPi forum thread started by dom: Request for Failing sdcards.  He may be interested in testing your failing cards.  Good engineers love reproducible bugs because they're already half solved.