Doing more with less: Raspberry Pi & Software Engineering

Excellent post, Morgaine.  The only part I disagree with is your assertion that there isn't a solution at this time.  Sure, it's a hard sell since so many people believe that "more features" means better, and "more apps" means better, and "more complex" means better.  But I do read articles occasionally about "good enough computing" and how most people can easily get by with RasPi-like machine capabilities.

I like to make an analogy with the RISC versus CISC argument of the 1980s.  (One of my favorite article titles of the time is from Electronics (5 May 1986): “RISC: Is it a Good Idea, or just another Hype?”)  In the 1980s transistors became so cheap you could afford to design computers with every instruction you always wanted.  However, those instructions were useless if the compiler couldn't take advantage of them, and their presence slowed down the performance of the instructions you really needed.

I say software is similar: useless features clog the software with complexity, making the features you really need harder to use and learn.  I have a longer essay on the subject at http://code.google.com/p/xxicc/wiki/RSCwiki if you're interested.

I liked your linked article a lot, John.  I can't find fault with any of it, and since I started my Unix days on a PDP-11/34, I both agree with your premises and also found your writeup very nostalgic.

With regard to solutions for the Software Crisis, I guess it comes down to where one draws the finish line.  You've pointed out some directions that help reduce the speed of headlong rush towards the precipice, and I agree with them, but in the end the velocity vector is still towards it, just a little less quickly.

And sadly there is no existing solution that could sway the current high priests of the software fraternity, who totally believe that they are on the true path of enlightment already.  To make matters even worse, there is no shortage of programmers who do what they do because it's fun and COOL, rather than for engineering reasons.  You can't argue against COOL, engineering disaster be damned.  It's one of the biggest problems of computing today --- programming is fun and creative, and as a result, serious engineering is almost entirely ignored.

Morgaine.

Morgaine,

you wrote:

"256MB is such a huge amount of memory that if any significant part of it is used to hold the program code of a monolithic application then that application is in crisis, whether it's admitted or not."

this is true, but beside the point.  Most linux applications aren't monolithic, and none that I know of are anywhere near 256MB of code, or even 10% of that.  But a lot of commonly used applications do often consume 256MB of memory as data.  For example, a web browser viewing a large page, or with several tabs open, or a word processor editing a large book, or a compiler compiling a large program.   It doesn't indicate that the application is needlessly monolithic, or bloated with needless features, or in any sort of crisis. 

Modern applications are doing exactly what you want them to do, which is to keep their data in fast memory rather than paging it to slow disk.  With memory prices as low as they are, at only a few dollars per GB, and with just about every PC sold in the last 5 years having a 64-bit address space,  it makes no sense for applications to try to minimize memory usage.

As you say, engineering is about minimalism, but the flip side of the coin is that engineering is about meeting requirements, and with adequate margins.  The minimum memory requirements for Debian, Fedora, Ubuntu, Android, etc., are all around 1GB.  Below that you will bump into the limits, often for reasons beyond your control, such as trying to view a large web page.

@coder27: I wasn't referring to amount of memory used to hold data, as that doesn't impact on program complexity.  After all, a one-line program can be designed to use arbitrary amounts of data, so there is no direct correlation between data memory used and program complexity.

I'm certainly not suggesting that Pi's memory should be restricted --- in fact, in another thread I argued expressly against holding to 256MB as some people were proposing.  Here I'm referring only to program design and the software engineering of applications, so it can't be used as an argument for restricting data memory.

While it's true that applications tend to be less monolithic in *nix than in Windows because of the Unix Tools legacy, that way of compositing larger applications out of many smaller and independent programs is almost completely unused today except in minor scripting.  The malaise of monolithic programming has crept into all operating systems, and it is now the completely standard way of working.

I once tried to gain a programming community's interest in non-monolithic design for a 3D graphics client -- see this wiki page of mine: https://wiki.secondlife.com/wiki/Multi-Process_Client_VAG_--_draft .  It got nowhere, not because anyone argued against it, but because it's just not the standard way of doing things so there was no mental buy-in.  When the entire software industry is designing monolithic horrors and thinking that they're cool, you have to provide a lot more than just a conceptual framework to gain any interest at all.  For some reason that I find difficult to understand, people know that their huge monolithic programs are utterly bug-ridden and almost impossible to maintain, yet are not willing to do anything about the root cause of the problem.

It must have something to do with the almost total lack of engineering mentality in the software community today, because engineers don't create things that way outside of software, no matter how complex they are.  If the space shuttle had been built as a monolithic unit, it would never have got off the ground because of complexity meltdown .

Systems of that complexity are built out of innumerable smaller parts, each of which is completely independent and presents only a very narrow interface to external systems.  It goes far beyond mere object orientation and unit testing at development time:  hardware "units" continue to be independent even when put together into a whole, they don't lose their independence and observability when they are linked together.  Software does not really have the equivalent of that yet, although I know of some systems that come close.  The key difficulty isn't that the problem is hard, but that the problem has no recognition.

You mentioned web browsers.  There could be no worse horror to illustrate what I'm saying. :-)

@Drew Fustini (admin): I'm conscious that we've accidentally strayed outside of the topic of Pete's thread.  It's probably not too bad yet, but if this discussion about software engineering continues too long, you might want to consider doing another split as you did for the FPGA topic.  Perhaps something like "Raspberry Pi and Software Engineering" might be an appropriate title.

I guess it's Pete's call.  I hope he's not too annoyed with us yet,

I can branch if that would be helpful.  What do you think would be the best place as which to do so?

Probably starting at my #112 because coder27's preceding post was still very much on topic about expanding Pi memory.  It was me who went off at a tangent about software engineering.  (Sorry Pete. )

But I think the new topic may have died anyway, so might as well leave it for now unless more replies follow.

Morgaine - your a mind reader (was travelling today and following the conversation and thought the same thing) - interesting discussion - but as you say we have gone a bit off topic and therefore a shame its buried in something that is titled differently.

I would like to suggest that we try to keep this thread to things that people have found wrong with the Pi rather that the philosophy of programming. What the Pi could have been, rather than what actually would also be better in another thread but these work so well together I would leave them together if that is OK?  So I would cut earlier and title something like "Lack of Memory on the Pi?" and it would hopefully encourage people to read and comment.

Never annoyed - it's a community thread - it is Drew's and the Admin's job to keep us in some sort of order. 

Off to eat some humble Raspberry Pi whilst I fix up the daft mistakes on the schematics, good job they are not on the board - V1.1 soon.

Pete

I went ahead and branched after seeing what Pete wrote, too.  I've manually set the language setting again to English to avoid that visibility problem which happened last time I branched a thread.  But the issue with the subject of individual replies not changing hasn't been fixed yet.

@Pete: Haha, don't kick yourself, things of this complexity are improved iteratively, they're not born magically perfect.  Well at least mine aren't. 

@Drew: Many thanks for doing the deed.  Of course, now that it's split, the discussion is guaranteed to dry up.