software engineering;  was: Is this official blog article intended as a joke?

Actually, the 'blog makes a useful point:

Most computer science students get to learn high-level programming languages and application development but they don’t really understand computers.

JMO/YMMV:

Now, while interfacing to the real world is fun, RasPi doesn't really address the high-level application problem.  It's still a GNU/Linux box, running a mainframe operating system with dozens of processes.  It's darn cheap for a mainframe, but it's still a mainframe.

If you really want to learn what computers do IMO you really need to learn assembly language and run it on a real mini- or micro-computer with no operating system present to protect you from stomping all over your code and data so that your program goes off to la-la land.  That's when you really see that all that's in computer memory is bit-strings, and the computer will interpret them as instructions or data depending on where you told it to execute -- whether you meant to or not.  Once you've mastered the machine-language level, then you can understand why your C program is misbehaving in that extraordinary fashion.  IMO this mastery is necessary for someone to be called a computer scientist, just as you expect an EE to know how to use Kirchhoff's Laws and a mathematician to be able to do Epsilon-Delta proofs.

> IMO this mastery is necessary for someone to be called a computer scientist,

Of course, computer science is a big field, with many specialties, and many layers

of abstraction, and it's entirely possible for hardware specialists to know very little

about software, and vice versa.  And theoretical computer scientists often know very

little about either software or hardware, focusing their attention on topics like

algorithmic efficiencies that don't provide any benefit on realisitic sized data sets.

Dijkstra famously said "computer science is no more about computers than

astronomy is about telescopes."

Of course, it's still important to know how to read an abend dump when you have to.

Remember Bertrand Meyer's very insightful work on Eiffel, in which preconditions, postconditions, and invariants played such an important part?   He embedded semantic constraints into software components, although his focus was on simple design contracts at a level far below anything related to what people might call "commonsense".  Unfortunately, static constraints are fundamentally at odds with flexibility and reuse, so we see them only very rarely in practical software beyond basic assertions for defensive programming.  We are as far away as ever from the model of companies or nations working in software.

Yes, I'm somewhat familiar with Eiffel's semantic constraints, but I've never heard that "static constraints

are fundamentally at odds with flexibility and reuse", so I'd be interested in further elaboration or references.

I am aware of various issues such as whether constraints should be statically checked at compile time vs.

run-time checked.  It seems both are needed.  I'm also aware of some confusion when inheriting constraints,

whether you should inherit the union of the constraints of all your ancestors, and how to override any constraints

you don't want to inherit.  But I assume that constraints are relatively rarely used in practice due to their availability

in mostly non-mainstream languages.

In any case, I think abstraction as a solution for complexity doesn't necessarily depend on the power of

available constraint mechanisms.  It mostly depends on the suitability of divide and conquer, which allows

for a hierarchical decomposition of the problem.  A CEO can run a company because the company can be

divided into various departments, each with a vice president, and with limited interaction between departments.

coder27 wrote:

 

In any case, I think abstraction as a solution for complexity doesn't necessarily depend on the power of

available constraint mechanisms.  It mostly depends on the suitability of divide and conquer, which allows

for a hierarchical decomposition of the problem.  A CEO can run a company because the company can be

divided into various departments, each with a vice president, and with limited interaction between departments.

In 1984, there was a USA TV comedy series called Empire about boardroom shenanigans at an unnamed USA corporation.  The hero is a young man from R&D who was made VP because of competence, and you get to watch him try to swim among the sharks.  It only ran 6 episodes -- I think it struck a nerve with the sponsors.  Not yet on DVD, alas.

coder27 wrote:

 

Yes, I'm somewhat familiar with Eiffel's semantic constraints, but I've never heard that "static constraints are fundamentally at odds with flexibility and reuse", so I'd be interested in further elaboration or references.

Oh it's nothing particularly deep, just an inherent consequence of adding constraints to any code.

In functions for example, without additional constraints, the domain of a particular function is given by the types of its declared parameters (or the types of its supplied arguments if function parameters in the language are untyped), and so those input types determine the extent of the function's input space.  The range of the function (ie. its set of possible outputs or the extent of its output space) then follows naturally from the set of possible inputs constrained only by the definition of those input types.

If you add additional constraints within the function to narrow down the allowable function domain to less than the full input space given by its input types then the output range will frequently be more limited as well, ie. the extent of the output space will be smaller.  Typically this is done to bring the semantics of the function more into line with the desired behavior of the application, for example to narrow its domain and range for the purpose of defensive programming.  The function will then have a more limited set of possible uses than would its original version that is constrained only by its input types, which is why adding semantic constraints is fundamentally at odds with flexibility and reuse.

The function will then have a more limited set of possible uses than would its original version that is constrained only by its input types ...

OK, I understand.  I think normally this is no great loss, because the region of the domain

that is trimmed is usually erroneous or uninteresting.  For example, adding a constraint to ensure

that a parameter is nonzero, because you intend to use it as a divisor.  This allows pushing the

divide-by-zero check to the caller rather than the callee, which is a good thing, because the

compiler may be able to remove it based on information known at the call site.

John Beetem wrote:

 

I have met way too many soi-disant software engineers who hate writing software (not to mention documentation), so they mash together inappropriate modules from Edsger Dijkstra knows where.  I have also met way too many engineering managers who want something quick-and-dirty now instead of doing it right -- and then re-doing it righter after learning from the first go-around.

My experience is that a large proportion of software engineers actually like writing software, or at least did at some point.  But anyone who's done a couple of projects will have seen the pattern of the clueless engineering managers who just won't listen and will have quickly become jaded to the whole process.  They know they're probably going to have to do at least a couple of ground-up re-writes due to whatever daft managerial decisions get imposed.

If they learn anything the first time round it should be to never listen to the manager!

selsinork wrote:

 

My experience is that a large proportion of software engineers actually like writing software, or at least did at some point.  But anyone who's done a couple of projects will have seen the pattern of the clueless engineering managers who just won't listen and will have quickly become jaded to the whole process.  They know they're probably going to have to do at least a couple of ground-up re-writes due to whatever daft managerial decisions get imposed.

 

If they learn anything the first time round it should be to never listen to the manager!

This is an interesting discussion that I've been lurking on with some interest. I think everyone makes valid points, and I think that any language is the wrong language to use when it is used wrong. Java (my language of choice), which many touted as being the answer to many people's problems, still has a lot of problems inherited from C and C++, Ultimately, it's quite possible to write bad code in any language. The more powerful the language, the easier it is to paint yourself into a corner, and find yourself with a sticky problem that you have to fudge your way out of.

Speaking from my own experience, deadlines are necessary, you need something to focus the mind and finish up. But as the deadline looms, you have to make short term compromises that tell yourself you will come back to at a later date, and then don't because after the deadline has passed you have new work to work on. Those compromises, or the the code that is built on those compromises, cause problems further down the line. Good commenting or peer review can deal with this, but too often, neither happen.

The real problems arise when the people setting the deadlines are focussing too much on the short term. Sure, it makes good business sense to minimise your spend, but all too often PMs decide on the quickest or cheapest solution offered to them, without caring about the effects this causes. "Act in haste, repent at leisure". Quick and dirty solutions often cost a lot more in the long term than the slower, better solution, but some engineers have a terrible time trying to persuade a PM of why they should get 8 weeks to do a solution when a Q&D one can be done in 2. A PM in a previous job once said to me "Do the quick and dirty solution, but do it without writing the bugs". These days I show people the iron triangle.

Now, that all said, it's fair to say that all software ships with bugs; it is part of the nature of software, and it happens for all the reasons everyone said. Bad coders, wrong choice of language, poor management decisions, quick & dirty solutions, sloppy testing, the list goes on and on. The trick is to accept that there will be defects, and making sure that none of them break functionality or cause a terrible user experience. A good iterative QA cycle is essential for good software.

However, what got me out of lurking on this post and actually posting on this discussion was the last post by Selsinork, which reminded me of an excellent article once written by Orson Scott Card. This is a jaded example, but it definitely points at some of the key problems in software development - how it is handled by non-software types, and especially by management. The world has changed in many ways since this article was written, but in many ways, it remains the same. A large number of problems are caused by external people not understanding either how software works, or how the people who work on software work.

I think it's as important to teach young coders how to deal with bad managers as it is to teach coders how to code.

coder27 wrote:

 

The solution to complexity is abstraction.  A single person can run a large company

or run a large nation, but not by doing all the jobs that a sole proprietor does.  Instead,

they operate at much higher levels of abstraction.

Not sure I really agree with that.  I accept that abstraction is necessary, but I'd argue that it's a necessary evil.

The problem with abstraction is that by definition it leads to the problem Morgaine brought up that no one person has a clue how it all works, or even how it's supposed to work. It also leads to the problem that once it becomes obvious that somethings broken it's often impossible to fix in any meaningful way.

You mention large companies or nations. I think they're often a perfect example - you look at the individual, abstracted, parts and they all look perfectly reasonable, but put them together and the result is incoherent and often fails even the most basic common sense analysis.

I accept that abstraction is necessary, but I'd argue that it's a necessary evil.

Yes, no doubt that large companies, nations, etc. are run with huge inefficiencies.

Sometimes that is intentional.

Similarly, there are a lot of examples where inefficiency is inherent. 

For example, airliner load factors.  It's tempting to see an airliner flying

with empty seats, and say that's inefficient because the seats are

going to waste and any revenue that could be obtained by filling them

would go directly to the bottom line profit. 

But an airline intentionally flys more seats on any particular route than

it expects to fill, in order to accommodate last-minute bookings from

high-paying business travelers.  It's just as important to avoid the waste

of turning away last-minute bookings because your plane is full than it is

to avoid the waste of flying empty seats.

The idea of abstraction is that low-level details are often not of critical

importance at higher levels of design.  For example in coding, an

assembly language programmer might care a lot about which register

holds which variable, in order to maybe keep two variables in adjacent

registers so they can be moved at the same time in a single double-word

move instruction.  But that's of no concern to the project's chief architect.

Every low-level detail needs to be worried about by someone, but

no single person needs to worry about every low-level detail.

Great post, Nelson.  I really like the Orson Scott Card essay -- hilarious and insightful.

Dudley Nelson wrote:

 

Ultimately, it's quite possible to write bad code in any language. The more powerful the language, the easier it is to paint yourself into a corner, and find yourself with a sticky problem that you have to fudge your way out of...

Reminds me of some classic C versus C++ jokes (from memory):

1.  C gives you enough rope to hang yourself.  C++ gives you enough rope to tie up everybody in your neighborhood, rig a small ship, and still have enough left over to hang yourself from the yard-arm.

2.  With C, it's easy to shoot yourself in the foot.  With C++ it's harder, but when you do you blow your whole leg off.

Dudley Nelson also wrote:

 

Now, that all said, it's fair to say that all software ships with bugs; it is part of the nature of software, and it happens for all the reasons everyone said.

All software that performs a non-trivial function has bugs.  There are just way too many moving parts, and when you try to fix the code the bugs scurry away to other parts of the program, just like in the song:

99 little bugs in the code,
99 bugs in the code...
Take one out, compile again,
100 little bugs in the code.
[Repeat until bug count goes to zero.]

As long as we're sharing humorous bits about software engineering, here's one of my favorites: Software Engineering: Now with Cats.

As long as we're sharing humorous bits about software engineering, here's one of my favorites: Software Engineering: Now with Cats.

John Beetem wrote:

 

As long as we're sharing humorous bits about software engineering, here's one of my favorites: Software Engineering: Now with Cats.

I like it