element 14 for the win

Element14 and Farnell certainly did very much better than their commercial competitor, but to be fair, this launch was one that the Raspberry Pi Foundation will want to forget as soon as possible.  I wrote up a review of the day's painful events on G+ here:

https://plus.google.com/117463006296185212823/posts/RHZMDDKs3V5

I think what's important now is that the Foundation recognizes that things didn't go as planned, learns from its mistakes, and moves rapidly into Phase 2 to put this new-found industrial muscle to work.  Many hundreds of thousands of devices are needed even now in this initial hobbiest / development phase, and millions more once Rpi hits the educational bigtime.  There is much work to be done by all three partners.

And from our community end there is much work to be done as well, turning the Rpi ecosystem into something vibrant, innovative, educational, and inspiring.  But for that we need boards. :-)

Morgaine.

"important now is that the Foundation recognizes that things didn't go as planned, learns from its mistakes" Well the only significant mistake the foundation made was to believe Farnell (E14) and to a lesser extent RS when they said there servers could cope with the rush!

Now that the sourcing of components, manufacturing, distributuion and finance have largely been off loaded they can concentrate much better on the fundamental aims of the charity.

If the Foundation had realised say six months ago the massive interest and demand, yes they could have done things better, but the project gradually snowballed and has become something of a monster to handle.

I now predict sales of 1,000,000 this year

Remember all* the members of the foundation are doing everything in their spare time.

*If I'm correct Liz may now be an employee, but she has been putting in 16 hour days 7 days a week, well above and behond the call of duty. I'm glad to see she has taken a break from twitter and the forums, for her sake and ours in the longer term.

If the Foundation had realised say six months ago the massive interest and demand, yes they could have done things better, but the project gradually snowballed and has become something of a monster to handle.

Oh I don't think that the Foundation misjudged the level of interest at all.  Not only did they realize it, but were totally terrified of it.  That's why they turned off all their Internet services except for their single static release page, in order not to contribute towards a collapse at their end.

Unfortunately they made two huge mistakes:

• They single-handedly created the conditions for a huge instantaneous spike in traffic, by naming the exact time of the announcement and telling people to set their alarm clocks.  If they had said "Availability and pre-ordering details will be announced sometime during the 24 hours of 29th February, at least a couple of days before the units arrive in stock" then people would have gone to work or to sleep normally, and the traffic would have been very gentle and spread out through the day.
• They shut down communications with the community entirely. Twitter did nothing but increase the sense of panic among bewildered fans, because everyone was witnessing everyone else having the same trouble as they, and any "official" tweets were rapidly swamped by the desperate masses --- it's simply the wrong mechanism for company presence.  Element 14 was a bit more useful, but unfortunately not many people came here to the Raspberry Pi group, and there was no official Foundation presence here acknowledging that the two commercial sites had collapsed and offering advice on what to do.  Perhaps they found it difficult to accept what was happening, or the sheer magnitude of the disaster.

The Foundation can't be held to blame for website inadequacies at Farnell/Newark and at RS though.  The companies were warned, and it's no secret that there were ~100k highly expectant people on the RasPi mailing list ready to pounce at 0600 GMT.  The fact that the webbies employed by two major industrial-strength distributors have no clue how to design resilient and scalable websites is sure to be causing waves inside the companies concerned, but that cannot be held against the Foundation.

There is NEVER an excuse for a well designed web service to crash or to hang through excess requests, particularly when advance warning is given so that more machines can be added to the load pools.  When back-end databases or front-end webservers serving dynamic content exceed their maximum rated throughput, the front end should simply return a static page explaining the problem and throttle requests by IP address a little, in proportion to the problem.  It's not rocket science.  It's just normal negative feedback, and without it a website is not a well engineered system.  It's a system designed to collapse, and it did.

Anyway, it's spilt milk.  On the positive side, one only learns by making mistakes, rarely from success.  There are a couple of things for people to learn here, and then focus on the future of RasPi, which I think will be glorious.

Morgaine.

In defence of Farnell/RS techies and their web servers, there may not have been a lot they could have done

Their systems and servers are probably designed to handle normal traffic levels for them plus a bit more for redundancy/growth. I am sure they do have a plan to scale up as they grow, but even with some warning, it can take time to implement this - servers have to be purchased, delivered, space found for them, racked, os + software installed, tested and finally put live

Also, the traffic they got on Tuesday was probably hundreds of times their usual traffic level. Even if they expected this and managed to get the (tens or hundreds) of new servers in on time to handle this load, what should they do with these servers once traffic goes back to normal levels. Most companies cant afford to have servers sitting around doing nothing.

Adding code to return static page under load is possible. But it takes development time to implement to write and test. If you have enough hardware to handle say 5 times the maximum expected load which you arent expected to reach for several years (by which the time the hardware will have been replaced and so you could handle more load), would you really want to to spend time doing this should never happen

Finally this is assuming their systems can scale infinitely. There are always decisions made that may stop this being the reality

For example

Their website normally takes x number of requests

This can be handled by 2 web servers

They have 4 web servers for resiliance

They have a choice of load balancers they can use - one that can scale up to 10 web servers that costs £1000, or one that can scale up to 100 web servers that costs £10000

Any sensible management would go for the £1000 load balancer as it provides enough for 5 times the current traffic level which wont be reached for several years, rather than go for the £10000 one that provides enough for 1 days traffic for a new product that nobody had heard of and they werent expecting to stock when this decision was made

In reality, there are probably only a handful of companies that are set up in the UK that could have handled the amount of traffic that was generated on the 29th (amazon, johnlewis.com, tesco.com etc) - none of which are an obvious place to sell raspberry pis or have the global logistics to ship internationally.

Anthony Brown wrote:

 

In reality, there are probably only a handful of companies that are set up in the UK that could have handled the amount of traffic that was generated on the 29th (amazon, johnlewis.com, tesco.com etc) - none of which are an obvious place to sell raspberry pis or have the global logistics to ship internationally.

TicketMaster and See Tickets could have handled it. Either or both of these could have sold an individual voucher that would have been matched to your own ID (credit card). These vouchers would then have been redeemed in exchange for a PI.

The problem though, is that these companies are not known for their benevolence and would have wanted to make a charge on top of the Pi cost for their service to each voucher. Personally I would have paid an extra fiver without a problem, but that does go against the Raspberry Pi ethos.

Anthony Brown wrote:

 

In defence of Farnell/RS techies and their web servers, there may not have been a lot they could have done

Yes I agree fully and tbh neither are a consumer orientated site and so their aren't used to having to handle heavy load at sale time. I've seen quite a few posts saying "just allocate the web servers more resources" or "just put in a few more web servers" its just not that simple.

Anthony Brown wrote:

 

In reality, there are probably only a handful of companies that are set up in the UK that could have handled the amount of traffic that was generated on the 29th (amazon, johnlewis.com, tesco.com etc) - none of which are an obvious place to sell raspberry pis or have the global logistics to ship internationally.

Yes and we've seen various retailers sites taken down when there have been the odd mad buying sessions such as the HP touchpad sell off last year. Its a shame that ebay couldn't have been used in some way as I'm sure they could take the load and have options for charities to sell without incurring much expense.

Anthony Brown wrote:

 

In defence of Farnell/RS techies and their web servers, there may not have been a lot they could have done

Yes I agree fully and tbh neither are a consumer orientated site and so their aren't used to having to handle heavy load at sale time. I've seen quite a few posts saying "just allocate the web servers more resources" or "just put in a few more web servers" its just not that simple.

Anthony Brown wrote:

 

In reality, there are probably only a handful of companies that are set up in the UK that could have handled the amount of traffic that was generated on the 29th (amazon, johnlewis.com, tesco.com etc) - none of which are an obvious place to sell raspberry pis or have the global logistics to ship internationally.

Yes and we've seen various retailers sites taken down when there have been the odd mad buying sessions such as the HP touchpad sell off last year. Its a shame that ebay couldn't have been used in some way as I'm sure they could take the load and have options for charities to sell without incurring much expense.