Raspberry Pi server clusters

Hello!

I've been interested in building a cluster of computers ever since I read "How to Build a Beowulf" many years ago, but had neither time nor resources to do so until now.

The thinking in that book was that as Personal Computers became more powerful, so less powerful ones should drop in price. However, that didn't happen - they just disappeared from the market. So unless you

were lucky to find an organisation that was about to dump dozens of PCs, it was difficult to collect enough machines to get started. And then there were considerations about maintainability, power consumption

and waste heat removal...

However, now that Raspberry Pi's are readily available from Farnell and others, I've built myself a 4-node Beowulf, following on from the starting instructions given by Professor Cox of Southampton University

( http://www.southampton.ac.uk/~sjc/raspberrypi/ ). I'm now able to compile and run sample programs in C and Fortran, using MPI, and have started to write my own programs.

I aware of the reasons for a RPi Beowulf never being able to be a true "supercomputer", but my objectives at the moment are simply to learn about parallel processing, so performance is not a key issue for me at the moment.

I plan to add an Rpi node at the rate of one or two a month as my budget permits. Soon I'll have to address the hardware engineering issues as my litle Beowulf simply lives in a plastic Tesco storage box at the moment!

I've shared my learning so far on the raspberrypi.org forum at

http://www.raspberrypi.org/phpBB3/viewtopic.php?f=41&t=548 , but interest there seems to have died out.

I hope it's not going to do the same here, although the discussion about clusters seems to have moved away lately from RPi's to more exotic architectures.

Alan.   

Alan,

  What kind of linpack numbers are you getting on your 4-node cluster?

I think the initial interest in RPi clusters died out when people realized

that the CPU was slow, the GPU was inaccessible, the network is high

latency, layered on USB 2.0, network booting is unavailable, and

connectors come out from all sides.

  Why aren't you using a quad-core PC for learning mpi?

coder27 wrote:

 

  What kind of linpack numbers are you getting on your 4-node cluster?

 

I've not run linpack yet. My timing tests have been limited to using WallClock and the "time" parmeter on the command line interface.

However, I'll be happy to get some numbers once my Beowulf is up and running again. (I've just cannibalised the master node to make a media player - replacement RPi due this week!)

I think the initial interest in RPi clusters died out when people realized

that the CPU was slow, the GPU was inaccessible, the network is high

latency, layered on USB 2.0, network booting is unavailable, and

connectors come out from all sides.

Well, as I said, I'm not too concerned about performance at the moment.

Aren't there moves afoot to make the GPU accessible? In any case, I don't want to work at that level. My main interest is in scientific programming. Anything useful that I develop I would want to to be transferable to a "real" supercomputer.

The connectors are not yet an issue, since everything lives in a plastic storage box just now. However, they will need some thought when I have to engineer the system properly.

  Why aren't you using a quad-core PC for learning mpi?

Well, I don't have such a machine and probably couldn't afford one. However, I have a handful of RPis. Also, I want to get away from Wintel and into Linux.

And I can't cannabalise a core to use as a media centre!

Alan.

> Well, as I said, I'm not too concerned about performance at the moment.

That's fine.  But you noted that others had lost interest in clustering RPi's,

and I think their initial interest was due in large part to a mistaken assumption

that adding enough nodes would be a cost-effective way to get decent performance.

> Aren't there moves afoot to make the GPU accessible?

No, there are not.  This is another widely held, but mistaken assumption.

The GPU instruction set is highly proprietary to Broadcom, and there is no

general-purpose interface, like OpenCL, nor any plans for such an interface.

Coder 27, the main reason for doing a RPI cluster is to get your organisation on the front of what ever media you are intending.

For real work the idea sucks ....looks well cool though

Coder 27, the main reason for doing a RPI cluster is to get your organisation on the front of what ever media you are intending.

For real work the idea sucks ....looks well cool though