Raspberry Pi server clusters

Hooray,  objective measurements are starting to appear as more people receive their Pi boards:

• http://candgsoc.host56.com/2012/04/quality-time-with-a-raspberry-pi/

It's very important to match what the board can offer against the specific requirements of your application, otherwise it can lead to disappointment.  Also, it's crucial not to make a one-dimensional analysis when multiple factors come into play, as cand's article highlights --->  the Pi's Ethernet performance can approach the full bandwidth of the line, but only at a huge CPU cost.

First indications then suggest that networked applications of Pi are likely to be most successful when the network use is occasional and when not much else has to be executed concurrently with the communications.

This will need a lot more careful analysis so that we know exactly where the bottlenecks are.  Some of them are quite likely to be remedied by kernel config improvements or with better drivers, but one has to know the detailed cause of a problem before one can tackle it.

Morgaine.

The following is about HPC which isn't my focus, but interesting nevertheless.  From Slashdot today:

"Phoronix constructed a low-cost, low-power 12-core ARM cluster running Ubuntu 12.04 LTS and made out of six PandaBoard ES OMAP4460 dual-core ARMv7 Cortex A9 chips. Their results show the ARM hardware is able to outperform Intel Atom and AMD Fusion processors in performance-per-Watt, except it sharply loses out to the latest-generation Intel Ivy Bridge processors."

More at:  http://linux.slashdot.org/story/12/06/16/1510257/12-core-arm-cluster-beats-intel-atom-amd-fusion

Morgaine.

Probably not quite what you're thinking of but the Design Spark forum (the competition) has an article on someone using Pis to make a VAX cluster.

Colin

OMG, DECnet!  http://www.designspark.com/content/raspberry-pi-vax-cluster

Thanks for the headsup, Colin.

There is an interesting article benchmarking a 12-core ARMv7 cluster

against some PC's.

http://www.phoronix.com/scan.php?page=article&item=phoronix_effimass_cluster&num=1

Yeah, saw that.  It's very underwhelming though.  I'm a bit surprised that ARM (the company) isn't working harder in that area and dedicating more research to multicore clustering.  A reference board from them bearing say 16 dual-core Cortex-A9 MPcore chips would really fan the flames wonderfully.

Intel can still warrant its total complacency about server-side ARM because a modern Atom core is nearly an order of magnitude faster than the best ARM core, so it doesn't have to push very strongly.  Well ARM can't do anything about the speed difference yet, but because it has the power advantage over Atom it really ought to use that to promote high core counts in clusters and start closing the gap with sheer core numbers.  It's not doing that, and I'm not too sure why not.

Morgaine.

There must be better ARM chips in the pipeline. At least Dell & HP are said to have arm based servers in the works and I can't see them being the only ones. Better performance per watt for arm only goes so far. I can see it being a tough sell if you need 10x the number of servers to meet some performance point.

Morgaine Dinova wrote:

 

I'm a bit surprised that ARM (the company) isn't working harder in that area and dedicating more research to multicore clustering.  A reference board from them bearing say 16 dual-core Cortex-A9 MPcore chips would really fan the flames wonderfully.

I would hazard a guess that ARM feels that the licensees are doing a fine job on their own, e.g.,

Calxeda EnergyCore:

Calxeda's so-called EnergyCore will come in versions with two and four Cortex A9 cores running at 1.1 to 1.4 GHz and sharing 4 Mbytes L2 cache. The chip includes an 80 Gbit/s fabric switch capable of supporting 4,096 nodes.

 

The EnergyCore supports up to five 10 Gbit/s ports. The SoC includes up to three 10 Gbit/s Ethernet MACs, four PCI Express Gen 2 links and five 3 Gbit/s serial ATA interfaces. It also includes an ARM M3 core to run server management software and supervise power management tasks.

 

The Calxeda device delivers about two-thirds to two-fifths the performance of a Westmere-class Intel Xeon 5620 four-core server processor, depending on the targeted application, said Karl Freund, vice president of marketing for Calxeda.

The article includes a photo of a nice-looking four-chip server card.

In other news from last year, AMCC is working on a 64-bit ARMv8 server chip: AMCC demos 64-bit ARM server chip.

Morgaine Dinova wrote:

 

I'm a bit surprised that ARM (the company) isn't working harder in that area and dedicating more research to multicore clustering.  A reference board from them bearing say 16 dual-core Cortex-A9 MPcore chips would really fan the flames wonderfully.

I would hazard a guess that ARM feels that the licensees are doing a fine job on their own, e.g.,

Calxeda EnergyCore:

Calxeda's so-called EnergyCore will come in versions with two and four Cortex A9 cores running at 1.1 to 1.4 GHz and sharing 4 Mbytes L2 cache. The chip includes an 80 Gbit/s fabric switch capable of supporting 4,096 nodes.

 

The EnergyCore supports up to five 10 Gbit/s ports. The SoC includes up to three 10 Gbit/s Ethernet MACs, four PCI Express Gen 2 links and five 3 Gbit/s serial ATA interfaces. It also includes an ARM M3 core to run server management software and supervise power management tasks.

 

The Calxeda device delivers about two-thirds to two-fifths the performance of a Westmere-class Intel Xeon 5620 four-core server processor, depending on the targeted application, said Karl Freund, vice president of marketing for Calxeda.

The article includes a photo of a nice-looking four-chip server card.

In other news from last year, AMCC is working on a 64-bit ARMv8 server chip: AMCC demos 64-bit ARM server chip.

That Calxeda chip looks nice, John.  But I still think that ARM should prime the pump with a reference design, it's not enough to leave it to a few specialist licensees.  In addition to raising the level of activity in ARM clustering, it would establish some sort of standard for licensees to follow.  Without that you can more or less guarantee that every specialist's clustering technology will be different --- a nightmare for everyone, including for Linux.

Morgaine.

Morgaine Dinova wrote:

 

That Calxeda chip looks nice...

More news about the Calxeda chip:

ARM Server Ships From Boston Limited (pcworld.com):

Boston Limited on Monday said it was manufacturing and distributing a low-power server with ARM-based chips, becoming one of the few companies to make such a server commercially available.

The Viridis server has the EnergyCore chip from Calxeda...

pcworld.com wrote:

 

 

The Boston Viridis server has up to 48 Calxeda chips -- 192 ARM cores in a 2U enclosure -- with integrated networking and storage units. Each Calxeda chip consumes as little as 5 watts per chip, U.K.-based Boston said in a statement.

Interesting, for the server farm, but I'd be happy with a much more down-to-earth 16 ARM cores of Cortex-A vintage in a 1U form factor, with gigabit Ethernet and SATA.  Pretty low spec really.

I'd be happy with a much more down-to-earth

It appears that at least the first generations are going to target datacenters, blades, and windows.

Microsoft mandating that 'secure boot' can't be disabled on Arm servers has already been widely reported elsewhere and could make it much more difficult to use alternative operating systems in the future.

Thankfully that doesn't appear to be the case with the Boston server as reported, hopefully they continue to produce systems along those lines.

Looking at it, there appears to be 12 processor cards in that system. Presuming it's modular enough, a single card would give you 16 cores and could be integrated into a smaller chassis ?

selsinork wrote:

 

Looking at it, there appears to be 12 processor cards in that system. Presuming it's modular enough, a single card would give you 16 cores and could be integrated into a smaller chassis ?

In principle, perhaps.  But in practice a company that is targetting data centers isn't likely to be interested in the other end of the market.  The breakthrough, if it happens, will come from an ARM licensee that isn't afraid to sell ARM application processors for $5 in volume, like TI did with the AM3358/9.