software engineering;  was: Is this official blog article intended as a joke?

Actually, the 'blog makes a useful point:

Most computer science students get to learn high-level programming languages and application development but they don’t really understand computers.

JMO/YMMV:

Now, while interfacing to the real world is fun, RasPi doesn't really address the high-level application problem.  It's still a GNU/Linux box, running a mainframe operating system with dozens of processes.  It's darn cheap for a mainframe, but it's still a mainframe.

If you really want to learn what computers do IMO you really need to learn assembly language and run it on a real mini- or micro-computer with no operating system present to protect you from stomping all over your code and data so that your program goes off to la-la land.  That's when you really see that all that's in computer memory is bit-strings, and the computer will interpret them as instructions or data depending on where you told it to execute -- whether you meant to or not.  Once you've mastered the machine-language level, then you can understand why your C program is misbehaving in that extraordinary fashion.  IMO this mastery is necessary for someone to be called a computer scientist, just as you expect an EE to know how to use Kirchhoff's Laws and a mathematician to be able to do Epsilon-Delta proofs.

> IMO this mastery is necessary for someone to be called a computer scientist,

Of course, computer science is a big field, with many specialties, and many layers

of abstraction, and it's entirely possible for hardware specialists to know very little

about software, and vice versa.  And theoretical computer scientists often know very

little about either software or hardware, focusing their attention on topics like

algorithmic efficiencies that don't provide any benefit on realisitic sized data sets.

Dijkstra famously said "computer science is no more about computers than

astronomy is about telescopes."

Of course, it's still important to know how to read an abend dump when you have to.

I'll go a bit out on a limb and say I don't think the equivalence of instructions

and data is necessarily fundamental to computer science.  I agree with you

that when debugging C, you will often see the machine fault when it tries to

mistakenly execute data.  But that's to some extent an artifact of C's broken

abstraction model.  You don't often see that happen say in Java.

C for example uses pointers in places where it's completely unnecessary,

such as passing a parameter by reference.  Pascal and similar languages

treat reference parameters at a higher level of abstraction, eliminating the

need for pointer syntax.  And stray pointers are probably the most frequent cause

of machine faults.  I think it's been said that pointers are to data what gotos

are to code. 

Computer scientists should understand the difference between value semantics

and reference semantics, but shouldn't fundamentally need to understand about

machine faults due to mistakenly trying to execute data.

coder27 wrote:

 

... I agree with you that when debugging C, you will often see the machine fault when it tries to

mistakenly execute data.  But that's to some extent an artifact of C's broken
abstraction model...

Nothing wrong with C's abstraction model.  IMO you just need to regard C as a portable assembly language that uses high-level notations.  When people think of it as a high-level language, that's when trouble begins.

C for example uses pointers in places where it's completely unnecessary,

such as passing a parameter by reference.  Pascal and similar languages

treat reference parameters at a higher level of abstraction, eliminating the

need for pointer syntax.  And stray pointers are probably the most frequent cause

of machine faults.  I think it's been said that pointers are to data what gotos

are to code. 

Well, when processors give me higher-abstraction instructions I'll think about dropping pointers and gotos.  As long as CPUs let me use registers as memory addresses and provide jump instructions I'll keep using them, because they work well if you use them properly.  You do want to master the basics before using them, but IMO it's silly to leave training wheels on a bicycle forever.  JMO/YMMV

> but IMO it's silly to leave training wheels on a bicycle forever.

It all depends on what the cost is.  If protecting yourself from

machine faults is nearly free, for example because your higher

level language enforces its abstraction model, then why not.

Parameter passing and array indexing are inherently high-level

abstractions that although they will be implemented with machine-level

code, they need not use machine-level pointer syntax.

The equivalence of code and data was a factor early in the history

of computer science, when it was believed that self-modifying code

was important, especially in AI where machine learning was thought

to require "rewiring" the brain. And LISP was ideal for self-modifying code. 

Since then, self-modifying code has fallen into disfavor, especially

with viruses taking advantage of executable stack pages in certain OS's. 

So I think the trend in CS is to as much as possible keep code and data

separate.  They can each have their own memory pages, with code pages

being read only, and data pages being non-executable. They can even have

their own separate caches in a harvard architecure.

John,

Nothing wrong with C's abstraction model.  IMO you just need to regard C as a portable assembly language that uses high-level notations.  When people think of it as a high-level language, that's when trouble begins.

You are entirely correct that if C is regarded as a portable assembly language, then it's abstraction model isn't broken.

But it's tempting for people to assume that C's variables and other abstractions are just like any other language's,

and as you say, that's what causes trouble.  I'm sure none of the following is news to you, but others may find it interesting.

I remember my first visit to a pediatrician where he did a reflex test by tapping a rubber hammer

on my knee.  I thought something's not right here.   My model of how the knee was supposed to work

was that the brain should control the knee.  I tried real hard when he tapped the other knee to kick

him with the wrong leg.

Then I was over at a friend's house and his dad was working on their car, with the engine idling

and the hood up.  My dad wasn't a mechanic and so I had very little understanding of how things

worked under the hood.  My friend's dad grabbed the throttle linkage attached to the carburetor

and revved the engine, much to my surprise.   I thought something's not right here.  There's nobody

sitting in the driver's seat, but the engine is only supposed to rev up if the driver presses the gas pedal.

Then I learned an early version of Fortran, and needed to convert a character from upper to lower case.

I found out that you could do it by declaring an integer array after the character variable, and index the array

with a negative index to obtain the integer representation of the character, and add 32 to convert to

lower case.  Something didn't seem quite right about that either.

Then I learned Pascal, where this trick was neither possible nor necessary, because array indexing

was checked, and there were intrinsic functions for converting characters to integers and vice versa.

This seemed like an improvement, although there was still a trick available of using a variant record to

subvert the type system.

In most high-level programming languages that have variables, a variable has associated with it a

value, and the fundamental rule of the abstraction model is that the value doesn't change except by

assignment to that variable, or by assignment to a parameter that is an alias of the variable.  In C,

on the other hand, a variable has associated with it an address, and the value stored at that address

can be changed by just about any code in the program, because it's trivially easy and common to

get hold of addresses and do arithmetic on them.   This makes it difficult to reason about

the behavior of a C program, whether the reasoning is done by a human or by an optimizer trying

to improve the code, or by some sort of static analysis tool trying to prove properties of the code.

coder27 wrote:

 

Then I learned an early version of Fortran, and needed to convert a character from upper to lower case.

I found out that you could do it by declaring an integer array after the character variable, and index the array

with a negative index to obtain the integer representation of the character, and add 32 to convert to

lower case.  Something didn't seem quite right about that either.

Very dangerous, since a different compiler or version might allocate the variables differently.  Much better to use EQUIVALENCE to ensure that arrays and variables are allocated on top of each other the correct way   Back in the 1970s I used EQUIVALENCE to create an arrays that were numbered from 0 instead of from 1.  I was writing an FFT and the algorithms were much simpler with arrays numbered from 0.  Fortran was actually a pretty good language for FFT at the time -- it was the only available language that had COMPLEX as a built-in type with built-in operators.

Then I learned Pascal, where this trick was neither possible nor necessary, because array indexing

was checked, and there were intrinsic functions for converting characters to integers and vice versa.

This seemed like an improvement, although there was still a trick available of using a variant record to

subvert the type system.

Variant records are a fine example of why a practical language should not be too restrictive.  You sometimes do need to bend the rules, and it's better to provide a well-defined way to do it that makes clear to someone reading the code that's you're bending the rules.

At one time I used to do a lot of programming in Pascal/VS, which was an excellent IBM mainframe extension of Pascal that allowed separate compilation of source files.  Unfortunately, it did not have a mechanism for function pointers, so when I needed to write a logic simulator I had to use C... K&R, before the ANSI improvements like function prototypes.  Awful C implementation, with 100 messages per syntax error

In most high-level programming languages that have variables, a variable has associated with it a

value, and the fundamental rule of the abstraction model is that the value doesn't change except by

assignment to that variable, or by assignment to a parameter that is an alias of the variable.  In C,

on the other hand, a variable has associated with it an address, and the value stored at that address

can be changed by just about any code in the program, because it's trivially easy and common to

get hold of addresses and do arithmetic on them.   This makes it difficult to reason about

the behavior of a C program, whether the reasoning is done by a human or by an optimizer trying

to improve the code, or by some sort of static analysis tool trying to prove properties of the code.

Modern C compilers do an excellent job of optimization.  As I understand it, if you use the address of a variable then the compiler can no longer assign that variable to a register (since registers don't have addresses in the most common computer architectures) and your performance will suffer.  If you willy-nilly increment and decrement pointers to access nearby variables, that's not defined by the semantics of the language so it's not necessary for the compiler to figure out what's the right thing to do.  Static analysis tools will give warnings about things like that, as well as warnings about many other things -- I've heard they produce lots of false warnings, and each static analysis tool generates different warnings for the same source code.

My general PG-13 rated comment about C and Asm versus "safe" programming languages like Basic is that it's a question of how many layers of condoms you want when programming.  If you don't use "safe" languages, the language won't protect you from unexpected negative consequences, and you'll need to be careful in other ways.  But it can be a lot more fun to program in "unsafe" languages, and a lot more satisfying

Very dangerous, since a different compiler or version might allocate the variables differently.  Much better to use EQUIVALENCE

Yes, that was my point, that languages that allow you to violate their abstraction models

are dangerous.

Awful C implementation, with 100 messages per syntax error

I've been noticing a lot of such cascading error messages in the Arduino forums when

for example there is a spurious semicolon before the opening '{' of a function.

http://www.element14.com/community/thread/10083?start=1138&tstart=0

Modern C compilers do an excellent job of optimization.

But the optimization suffers due to the availability of untyped memory operations

such as memcpy and memmove:

http://gcc.gnu.org/onlinedocs/gccint/Memory-model.html#Memory-model

and the difficulties of type-based alias analysis in such cases:

http://gcc.gnu.org/onlinedocs/gccint/Alias-analysis.html#Alias-analysis

But it can be a lot more fun to program in "unsafe" languages, and a lot more satisfying

you can also waste a lot of time debugging that way, because errors that would

otherwise be caught at compile time or in a run-time array bounds check, 

can turn into mysterious run-time behavior where evidence of the error may

not show up until long after the program has gotten off track.

coder27 wrote:

 

But it can be a lot more fun to program in "unsafe" languages, and a lot more satisfying

 

you can also waste a lot of time debugging that way, because errors that would

otherwise be caught at compile time or in a run-time array bounds check, 

can turn into mysterious run-time behavior where evidence of the error may

not show up until long after the program has gotten off track.

In my own programming, I rarely have bugs that could have been caught by run-time array bounds checking.  It's usually logic errors such as forgetting to handle an unusual case and the occasional dangling pointer, which is part of the price of writing code that uses (and reuses) linked data structures.  My solution is to try to keep things as simple as possible, and it works for me.  One way to keep things simple is to use a language that's well-adapted to the problem, so you don't have to encrypt the problem to fit the programming language.

My general opinion is that if you have sharp programmers, you can give them sharp tools.  If you don't, then you either need to give them "safe" tools or replace them with sharper programmers.

John Beetem wrote:

 

My general opinion is that if you have sharp programmers, you can give them sharp tools.  If you don't, then you either need to give them "safe" tools or replace them with sharper programmers.

Some programmers are sharper than others, but there are no human programmers sharp enough to fully comprehend algorithmic and state complexity in any non-trivial software system.  The disparity between programs and human capability to understand complexity and track dynamic state is vast, many orders of magnitide.  We would need IQs in the thousands (maybe millions) to program without introducing errors, no matter how many decades we've spent mastering a language.

This is why the bridges of software engineering collapse around the world millions of times each day.  It's not that the vast majority of programmers are amateurs or that the education system has failed us or that programming is a cool craft instead of an engineering discipline.  Those things could be fixed.  The underlying problem of human capability mismatch cannot be fixed, as long as we continue along the present track.

And there is little sign of anyone looking for alternative tracks, at least in practical computing.

Morgaine Dinova wrote:

 

Some programmers are sharper than others, but there are no human programmers sharp enough to fully comprehend algorithmic and state complexity in any non-trivial software system.  The disparity between programs and human capability to understand complexity and track dynamic state is vast, many orders of magnitide.  We would need IQs in the thousands (maybe millions) to program without introducing errors, no matter how many decades we've spent mastering a language...

Excellent comments, Morgaine.

You've heard me talk about this before, so the following is for others following this thread.  I believe we have complex systems because it's easier to slap together a complex system out of parts that "ought to work together" than to  think carefully through a system -- and document it -- before getting so far into the implementation that it's impossible to make fundamental changes.  I have met way too many soi-disant software engineers who hate writing software (not to mention documentation), so they mash together inappropriate modules from Edsger Dijkstra knows where.  I have also met way too many engineering managers who want something quick-and-dirty now instead of doing it right -- and then re-doing it righter after learning from the first go-around.

C.A.R. Hoare says it best, IMO:

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.

Actually, IMO humans are quite good at working with complex things once they have come up with good notations -- textual and graphical -- to describe them.  However, the development of those notations usually takes decades or centuries.

The solution to complexity is abstraction.  A single person can run a large company

or run a large nation, but not by doing all the jobs that a sole proprietor does.  Instead,

they operate at much higher levels of abstraction.

I don't think the analogy of "sharp tools" to "low level programming languages" works too well.

Certainly low-level features get removed from higher level languages, for example, gotos have

been removed from recent languages.  And if you focus on the loss, it may seem like your language

has gotten duller.  But that's usually not true, because new higher-level abstractions are added

as lower-level ones are removed.

In some cases, the benefits are not obvious to the programmer, but they are to the compiler

writer, because keeping lower-level abstractions around allows them to be used to subvert

the higher-level abstractions.

For example, the typical compiler intermediate language these days is based on SSA,

and that requires a control-flow graph in the form of a DAG (no cycles), which in turn requires

that you don't have arbitrary gotos, such as between branches of an if/then/else construct.

So by taking away the low-level goto, and replacing it with higher-level if/then/else and looping

constructs, you end up with a language that is no less capable, but much more suitable for

SSA-based optimization, code generation, and other analysis.

The solution to complexity is abstraction.  A single person can run a large company

or run a large nation, but not by doing all the jobs that a sole proprietor does.  Instead,

they operate at much higher levels of abstraction.

I don't think the analogy of "sharp tools" to "low level programming languages" works too well.

Certainly low-level features get removed from higher level languages, for example, gotos have

been removed from recent languages.  And if you focus on the loss, it may seem like your language

has gotten duller.  But that's usually not true, because new higher-level abstractions are added

as lower-level ones are removed.

In some cases, the benefits are not obvious to the programmer, but they are to the compiler

writer, because keeping lower-level abstractions around allows them to be used to subvert

the higher-level abstractions.

For example, the typical compiler intermediate language these days is based on SSA,

and that requires a control-flow graph in the form of a DAG (no cycles), which in turn requires

that you don't have arbitrary gotos, such as between branches of an if/then/else construct.

So by taking away the low-level goto, and replacing it with higher-level if/then/else and looping

constructs, you end up with a language that is no less capable, but much more suitable for

SSA-based optimization, code generation, and other analysis.

coder27 wrote:

 

The solution to complexity is abstraction.  A single person can run a large company

or run a large nation, but not by doing all the jobs that a sole proprietor does.  Instead,

they operate at much higher levels of abstraction.

Abstraction is part of the solution, but it's not sufficient by itself.  With abstraction alone, computers will continue to do exactly what we tell them to do, and will inevitably end up doing the wrong thing because it is entirely beyond our mental ability to fully understand (let alone specify) the behavior of a non-trivial system.  Removing unnecessary detail and bringing the language of specification closer to the natural language of the application domain through DSL abstraction clearly helps, but it is not enough.  The desired complex behavior still has to be defined, somewhere, and that's the problem.

Your example of the large company or nation is a very good one, but not as an illustration of the power of abstraction.  What it illustrates instead is that complex systems can work well when there is semantic commonsense built into the components of the system, so that daft commands from above don't cause an immediate collapse.  And that's what is lacking almost completely in software today.

Remember Bertrand Meyer's very insightful work on Eiffel, in which preconditions, postconditions, and invariants played such an important part?   He embedded semantic constraints into software components, although his focus was on simple design contracts at a level far below anything related to what people might call "commonsense".  Unfortunately, static constraints are fundamentally at odds with flexibility and reuse, so we see them only very rarely in practical software beyond basic assertions for defensive programming.  We are as far away as ever from the model of companies or nations working in software.

Morgaine Dinova wrote:

 

coder27 wrote:

 

The solution to complexity is abstraction.  A single person can run a large company

or run a large nation, but not by doing all the jobs that a sole proprietor does.  Instead,

they operate at much higher levels of abstraction.

 

Abstraction is part of the solution, but it's not sufficient by itself.  With abstraction alone, computers will continue to do exactly what we tell them to do, and will inevitably end up doing the wrong thing because it is entirely beyond our mental ability to fully understand (let alone specify) the behavior of a non-trivial system.  Removing unnecessary detail and bringing the language of specification closer to the natural language of the application domain through DSL abstraction clearly helps, but it is not enough.  The desired complex behavior still has to be defined, somewhere, and that's the problem.

Another problem is that of leaky abstraction layers.  My favorite example of this is C.  As I said earlier in this discussion, I think of C as a portable assembly language which uses high-level notations, and if you think of C that way it works very sharply indeed.  However, if you think of C as a high-level language, you will be disappointed at run time when machine-level behavior damages your nice high-level model.

Just adding abstraction layers can be problematic, if only because the abstraction layer may hide terrible performance problems that you'll only uncover when the system is fully deployed.

Also, IMO opinion many large (and even small) companies and countries are very poorly run whether or not their leaders are capable of abstract thinking or common sense.  I'm sure you can think of examples.  I believe the principal cause of this is the Law of Large Organizations:

In any Large Organization, Loyalty will always be rewarded over Competence.

Come to think of it, many large software systems suffer from this as well

Remember Bertrand Meyer's very insightful work on Eiffel, in which preconditions, postconditions, and invariants played such an important part?   He embedded semantic constraints into software components, although his focus was on simple design contracts at a level far below anything related to what people might call "commonsense".  Unfortunately, static constraints are fundamentally at odds with flexibility and reuse, so we see them only very rarely in practical software beyond basic assertions for defensive programming.  We are as far away as ever from the model of companies or nations working in software.

Yes, I'm somewhat familiar with Eiffel's semantic constraints, but I've never heard that "static constraints

are fundamentally at odds with flexibility and reuse", so I'd be interested in further elaboration or references.

I am aware of various issues such as whether constraints should be statically checked at compile time vs.

run-time checked.  It seems both are needed.  I'm also aware of some confusion when inheriting constraints,

whether you should inherit the union of the constraints of all your ancestors, and how to override any constraints

you don't want to inherit.  But I assume that constraints are relatively rarely used in practice due to their availability

in mostly non-mainstream languages.

In any case, I think abstraction as a solution for complexity doesn't necessarily depend on the power of

available constraint mechanisms.  It mostly depends on the suitability of divide and conquer, which allows

for a hierarchical decomposition of the problem.  A CEO can run a company because the company can be

divided into various departments, each with a vice president, and with limited interaction between departments.

coder27 wrote:

 

In any case, I think abstraction as a solution for complexity doesn't necessarily depend on the power of

available constraint mechanisms.  It mostly depends on the suitability of divide and conquer, which allows

for a hierarchical decomposition of the problem.  A CEO can run a company because the company can be

divided into various departments, each with a vice president, and with limited interaction between departments.

In 1984, there was a USA TV comedy series called Empire about boardroom shenanigans at an unnamed USA corporation.  The hero is a young man from R&D who was made VP because of competence, and you get to watch him try to swim among the sharks.  It only ran 6 episodes -- I think it struck a nerve with the sponsors.  Not yet on DVD, alas.

coder27 wrote:

 

Yes, I'm somewhat familiar with Eiffel's semantic constraints, but I've never heard that "static constraints are fundamentally at odds with flexibility and reuse", so I'd be interested in further elaboration or references.

Oh it's nothing particularly deep, just an inherent consequence of adding constraints to any code.

In functions for example, without additional constraints, the domain of a particular function is given by the types of its declared parameters (or the types of its supplied arguments if function parameters in the language are untyped), and so those input types determine the extent of the function's input space.  The range of the function (ie. its set of possible outputs or the extent of its output space) then follows naturally from the set of possible inputs constrained only by the definition of those input types.

If you add additional constraints within the function to narrow down the allowable function domain to less than the full input space given by its input types then the output range will frequently be more limited as well, ie. the extent of the output space will be smaller.  Typically this is done to bring the semantics of the function more into line with the desired behavior of the application, for example to narrow its domain and range for the purpose of defensive programming.  The function will then have a more limited set of possible uses than would its original version that is constrained only by its input types, which is why adding semantic constraints is fundamentally at odds with flexibility and reuse.

The function will then have a more limited set of possible uses than would its original version that is constrained only by its input types ...

OK, I understand.  I think normally this is no great loss, because the region of the domain

that is trimmed is usually erroneous or uninteresting.  For example, adding a constraint to ensure

that a parameter is nonzero, because you intend to use it as a divisor.  This allows pushing the

divide-by-zero check to the caller rather than the callee, which is a good thing, because the

compiler may be able to remove it based on information known at the call site.

coder27 wrote:

 

The solution to complexity is abstraction.  A single person can run a large company

or run a large nation, but not by doing all the jobs that a sole proprietor does.  Instead,

they operate at much higher levels of abstraction.

Not sure I really agree with that.  I accept that abstraction is necessary, but I'd argue that it's a necessary evil.

The problem with abstraction is that by definition it leads to the problem Morgaine brought up that no one person has a clue how it all works, or even how it's supposed to work. It also leads to the problem that once it becomes obvious that somethings broken it's often impossible to fix in any meaningful way.

You mention large companies or nations. I think they're often a perfect example - you look at the individual, abstracted, parts and they all look perfectly reasonable, but put them together and the result is incoherent and often fails even the most basic common sense analysis.

I accept that abstraction is necessary, but I'd argue that it's a necessary evil.

Yes, no doubt that large companies, nations, etc. are run with huge inefficiencies.

Sometimes that is intentional.

Similarly, there are a lot of examples where inefficiency is inherent. 

For example, airliner load factors.  It's tempting to see an airliner flying

with empty seats, and say that's inefficient because the seats are

going to waste and any revenue that could be obtained by filling them

would go directly to the bottom line profit. 

But an airline intentionally flys more seats on any particular route than

it expects to fill, in order to accommodate last-minute bookings from

high-paying business travelers.  It's just as important to avoid the waste

of turning away last-minute bookings because your plane is full than it is

to avoid the waste of flying empty seats.

The idea of abstraction is that low-level details are often not of critical

importance at higher levels of design.  For example in coding, an

assembly language programmer might care a lot about which register

holds which variable, in order to maybe keep two variables in adjacent

registers so they can be moved at the same time in a single double-word

move instruction.  But that's of no concern to the project's chief architect.

Every low-level detail needs to be worried about by someone, but

no single person needs to worry about every low-level detail.

coder27 wrote:

 

Every low-level detail needs to be worried about by someone, but

no single person needs to worry about every low-level detail.

I couldn't agree more.. Unfortunately it's also exactly the problem.  Someone five levels up the abstraction tree makes an innocuous decision because he doesn't think he needs to care about the person trying to keep those two values in the same register, but by the time the decision makes it all the way down, that assembly language programmer can no longer do what's required without forcing a re-write at every other level. Resulting bodged solution unsurprisingly has issues because a very frequently executed action goes from a single cycle to something that takes very many orders of magnitude longer.

I know of at least one example where this sort of thing forced changes in hardware to compensate for software design that was unchangeable due to this way of thinking.

Someone five levels up the abstraction tree makes an innocuous decision because he doesn't think he needs to care ...

Yes, sometimes abstractions leak, but hopefully it's the exception rather than the rule.

As you probably know, there is a software engineering notion of "information hiding"

that is often misunderstood.  The goal is to separate interface from implementation,

with the implementation details hidden from users of the abstraction so that they can't

unwittingly rely on them, which would endanger the system if the details are changed.

Some people think information hiding is bad in that it prevents them from maximizing

efficiency by taking advantage of implementation details that they don't expect will

actually change, such as multiplying an integer by 2 by shifting it left one bit, on the

assumption that it's represented in base 2 and always will be.