software engineering;  was: Is this official blog article intended as a joke?

Actually, the 'blog makes a useful point:

Most computer science students get to learn high-level programming languages and application development but they don’t really understand computers.

JMO/YMMV:

Now, while interfacing to the real world is fun, RasPi doesn't really address the high-level application problem.  It's still a GNU/Linux box, running a mainframe operating system with dozens of processes.  It's darn cheap for a mainframe, but it's still a mainframe.

If you really want to learn what computers do IMO you really need to learn assembly language and run it on a real mini- or micro-computer with no operating system present to protect you from stomping all over your code and data so that your program goes off to la-la land.  That's when you really see that all that's in computer memory is bit-strings, and the computer will interpret them as instructions or data depending on where you told it to execute -- whether you meant to or not.  Once you've mastered the machine-language level, then you can understand why your C program is misbehaving in that extraordinary fashion.  IMO this mastery is necessary for someone to be called a computer scientist, just as you expect an EE to know how to use Kirchhoff's Laws and a mathematician to be able to do Epsilon-Delta proofs.

> IMO this mastery is necessary for someone to be called a computer scientist,

Of course, computer science is a big field, with many specialties, and many layers

of abstraction, and it's entirely possible for hardware specialists to know very little

about software, and vice versa.  And theoretical computer scientists often know very

little about either software or hardware, focusing their attention on topics like

algorithmic efficiencies that don't provide any benefit on realisitic sized data sets.

Dijkstra famously said "computer science is no more about computers than

astronomy is about telescopes."

Of course, it's still important to know how to read an abend dump when you have to.

Someone five levels up the abstraction tree makes an innocuous decision because he doesn't think he needs to care ...

Yes, sometimes abstractions leak, but hopefully it's the exception rather than the rule.

As you probably know, there is a software engineering notion of "information hiding"

that is often misunderstood.  The goal is to separate interface from implementation,

with the implementation details hidden from users of the abstraction so that they can't

unwittingly rely on them, which would endanger the system if the details are changed.

Some people think information hiding is bad in that it prevents them from maximizing

efficiency by taking advantage of implementation details that they don't expect will

actually change, such as multiplying an integer by 2 by shifting it left one bit, on the

assumption that it's represented in base 2 and always will be.

Dudley Nelson wrote:

 

excellent article once written by Orson Scott Card. This is a jaded example, but it definitely points at some of the key problems in software development - how it is handled by non-software types, and especially by management. The world has changed in many ways since this article was written, but in many ways, it remains the same. A large number of problems are caused by external people not understanding either how software works, or how the people who work on software work.

Although poor management is common in all fields, the core problem in software isn't lack of understanding of software by non-software-types, but lack of understanding of software by everybody involved in the process, Alpha Programmers included.  In fact, the finger of blame should mostly point at the "geniuses" of the field because they should be informed enough to understand the root cause of the problem, and able to communicate it to others, including management and beancounters.  They aren't, and they don't.

After all, management deals with huge projects in many other fields of engineering and still succeeds in delivering bridges that don't fall down continually, complex modern cars that are much more reliable than their drivers, and space rovers that continue to explore Mars some 10 years after the end of a nominal 90-sol mission.  It is not beyond the capabilities of engineers and management to make extremely complex systems that work to spec.

Unfortunately, the vast bulk of software development is not an engineering discipline at all, but a rather ad hoc craft with very little in the way of engineering methodology, and frequently no measurement or analysis done at all.  Although Computer Science is a formal discipline, it underpins the creation of software only in principle, rarely in practice to any great extent.  Well-chosen scalable algorithms or the use of formal language grammars is often as close as CompSci gets to helping a software project achieve its goals.  The situation is very bad indeed.  The Software Crisis that was so often described in the 80's is still present and shows no signs of abating.  It's an extreme embarassment to those of us who would like to consider themselves engineers through long experience in the discipline, but in all honesty cannot.

Orson Scott Card's little article is certainly funny.  Unfortunately it also perpetuates the myth that  Alpha Programmers are on top of things, and that if only they were left alone, coders and their coffee and pizzas will solve everything.  If only it were so.

Morgaine.

Morgaine Dinova wrote:

 

Although poor management is common in all fields, the core problem in software isn't lack of understanding of software by non-software-types, but lack of understanding of software by everybody involved in the process, Alpha Programmers included.  In fact, the finger of blame should mostly point at the "geniuses" of the field because they should be informed enough to understand the root cause of the problem, and able to communicate it to others, including management and beancounters.  They aren't, and they don't.

My opinion:  I think a lot of great programmers got that way precisely because they've always been bad at communicating with humans.  They were ostracized as children, and bonded early to computers as an alternative.  To get stuff through management, it's not just communcication -- you have to sell yourself and your ideas.  Well, not everybody is a good salesman.  Those who are go into sales and make a whole lot more money than techies.  JMO/YMMV

coder27 wrote:

 

As you probably know, there is a software engineering notion of "information hiding"

that is often misunderstood.  The goal is to separate interface from implementation,

with the implementation details hidden from users of the abstraction so that they can't

unwittingly rely on them, which would endanger the system if the details are changed.

I think the key is designing excellent interfaces between layers and writing excellent documentation to define those interfaces so that people can develop separate pieces and have them work together properly when integrated.  This goes along with information hiding, since if the interfaces are designed and documented well enough there is rarely a need to peek beyond the interface.

coder27 wrote:

 

As you probably know, there is a software engineering notion of "information hiding"

I'm aware of the idea, and that most current programmers have fallen for it.

You can't seriously think that if the interface specifies that you're getting an integer binary value that it's ok for the implementation to change it to a trinary integer (or anything else for that matter - opaque value maybe).  To do that obviously breaks the agreement the interface provides, the consumer no longer knows how to store the value it's been given never mind what to do with it.

Indeed, whether we like it, agree or not, the consumer is inextricably and unavoidably reliant on the implementation - even if it's not allowed to know any of the hidden details.

I was reminded about this discussion while reading the stuff on avoid_safe_mode=1 in the other thread earlier - everything hidden behind an interface that the user likely doesn't even know exists. Closed source too, so no way to know the details. You're presented with something that's not right, not expected, and with no explanation. Not helpful.  What's worse is that he's not only dependant on the implementation, he needs to know all those hidden details to work out how to fix it.

Morgaine's right, the whole thing is a mess. I don't have a better alternative, or have a clue how to fix it, but I know what we're doing today is the wrong thing and we're unlikely to solve it while we keep digging.

John Beetem wrote:

 

Well, not everybody is a good salesman.

Very true. Engineers also often have a way of seeing the facts quite clearly, whether they like what they see is a different question.  Salesmen don't care about the facts and that tends to put salesmen and engineers at odds - the engineer can smell the bullshit the salesman is shoveling regardless of how good the salesman makes it appear.

I had the misfortune of having a salesman in the desk over the partition from me for a while, not a pleasant experience and if hadn't been for the partition it would have been much worse

Morgaine Dinova wrote:

 

In fact, the finger of blame should mostly point at the "geniuses" of the field because they should be informed enough to understand the root cause of the problem, and able to communicate it to others, including management and beancounters.  They aren't, and they don't.

Bit of a catch22 there though, I tend to find that even when you do understand a problem it's difficult if not impossible to communicate it to non technical people because fundamentally we don't think in the same terms anymore.

selsinork wrote:

 

Bit of a catch22 there though, I tend to find that even when you do understand a problem it's difficult if not impossible to communicate it to non technical people because fundamentally we don't think in the same terms anymore.

Yes, communication can be very difficult.  It becomes even more difficult when engineers see the problem but don't yet have a full solution to offer, because then the communication can be perceived as destruction of an ongoing process without purpose rather than an earnest warning that a cliff lies ahead.

The software crisis lies in that category,  the problem being blatantly obvious to anyone with even half an eye open, yet no full solution being available even in the halls of academia.  Fortunately many partial solutions exist, or at least aids that help avoid the most elementary sources of failure, such as defensive programming practices and test-driven development.

Rather remarkably though, and very unfortunately, advocating even for such simple and relatively painless strategies often falls on deaf ears, for one really terrible reason --- we're human, and deep down it seems that we don't really want to face our own fallibility, even those who recognize the problem.  The "coolness" of programming exacerbates this further, as perhaps a majority of programmers are so busy having fun that they have no inclination to look further than the tips of their fingers.

The problem isn't going to be solved for a long while yet.  There are too many factors reinforcing it, and they're not just technical.

Morgaine.

Hey all, I have moved this discussion to our Raspberry Pi group from Feedback and Support.

Dave,

Hey all, I have moved this discussion to our Raspberry Pi group from Feedback and Support.

The article really is intended for Feedback and Support.  We're just keeping the discussion

going so it stays visible.

Dave,

Hey all, I have moved this discussion to our Raspberry Pi group from Feedback and Support.

The article really is intended for Feedback and Support.  We're just keeping the discussion

going so it stays visible.