Open Source ARM Userland

I've just seen this (thanks John for the Slashdot link, interesting comments and responses there), and on the whole I welcome it as a move in the right direction.  Something being open sourced is better than nothing being open sourced.

However, it's most definitely not what the PR on the blog describes, because RPF's diagram of Raspberry Pi Software Architecture clearly shows OpenGL ES as being open sourced under the BSD license but that is not the case.  OpenGL ES runs on the VideoCore, so it has not been open sourced.  Too much is being claimed, and the claims are not accurate.

It's easy to give such PR statements an acid test.  Does this open sourcing mean that OpenGL ES can be hacked and modified and improved and bugs fixed by the community?  No it doesn't, because only the API or interface to it has been open sourced (what people are calling the "shim"), and having open source access to this shim does not enable you to modify OpenGL ES itself.  The acid test shows that claim to be false.

In typical fashion, RPF is overstating the merits of what it has done.  Nevertheless, I welcome the shims being open sourced.  I have been calling for at least the APIs to the closed hardware to be opened and documented for a long time, so this is a start.  Next comes the documentation phase.

Unfortunately, developers who thought that the news meant that the GPU is now directly programmable or that OpenGL ES is now open source will be very disappointed.

Morgaine.

Morgaine Dinova wrote:

 

I've just seen this (thanks John for the Slashdot link, interesting comments and responses there), and on the whole I welcome it as a move in the right direction.  Something being open sourced is better than nothing being open sourced.

 

However, it's most definitely not what the PR on the blog describes, because RPF's diagram of Raspberry Pi Software Architecture clearly shows OpenGL ES as being open sourced under the BSD license but that is not the case.  OpenGL ES runs on the VideoCore, so it has not been open sourced.  Too much is being claimed, and the claims are not accurate.

 

It's easy to give such PR statements an acid test.  Does this open sourcing mean that OpenGL ES can be hacked and modified and improved and bugs fixed by the community?  No it doesn't, because only the API or interface to it has been open sourced (what people are calling the "shim"), and having open source access to this shim does not enable you to modify OpenGL ES itself.  The acid test shows that claim to be false.

 

In typical fashion, RPF is overstating the merits of what it has done.  Nevertheless, I welcome the shims being open sourced.  I have been calling for at least the APIs to the closed hardware to be opened and documented for a long time, so this is a start.  Next comes the documentation phase.

 

Unfortunately, developers who thought that the news meant that the GPU is now directly programmable or that OpenGL ES is now open source will be very disappointed.

 

Morgaine.

Once again a deliberately misleading post from Ms Divona.

I've read the release, and checked out the diagram carefully, they have now released the source to the OGLES library running on the Arm (and all the other Arm side libraries). This appears to be a shim/wrappper layer to convert GLES calls to GPU api calls, and therefor is correctly refered to as a driver. This means that anyone can no recompile/modify those libraries easily and use OpenGLES calls from their baremetal code, or their homegrown OS etc. At no point have they claimed any code on the GPU iteself has been open sourced, that would indeed be incorrect, and in fact checking slashdot, the Eben bloke has been quite specific in what has been released.

Why is it that ANY good news from these people is immediately slagged here? Do you people want them to fail? Are you really that vindictive as to want a charity to fail, one set up to improve the teaching of children? Sometimes you lot are unbelievable.

Bill

Stick to facts please, fanboyism doesn't get you anywhere on a site where engineering facts are considerably more important than PR.

RPF's own diagram shows the Pi's OpenGL ES running on ARM and licensed under BSD.  The Pi's OpenGL ES does not run on the ARM and is not licensed under BSD nor under any other open source license.  It is implemented as firmware on the VideoCore, so the diagram is completely misleading, and has led to non-technical people misunderstanding what was open-sourced.  Only the shim was open-sourced, which contains nothing of OpenGL ES's implementation that people could usefully fix or improve.

Your reply is doubly misplaced because I welcomed even just the shim being opened.  But of course you wouldn't care about the facts here.

Morgaine Dinova wrote:

 

Stick to facts please, fanboyism doesn't get you anywhere on a site where engineering facts are considerably more important than PR.

 

RPF's own diagram shows the Pi's OpenGL ES running on ARM and licensed under BSD.  The Pi's OpenGL ES does not run on the ARM and is not licensed under BSD nor under any other open source license.  It is implemented as firmware on the VideoCore, so the diagram is completely misleading, and has led to non-technical people misunderstanding what was open-sourced.  Only the shim was open-sourced, which contains nothing of OpenGL ES's implementation that people could usefully fix or improve.

 

Your reply is doubly misplaced because I welcomed even just the shim being opened.  But of course you wouldn't care about the facts here.

It appears I know more facts that you do, but then I spend a lot of time in the real world doing a real job with real engineers. The diagram shows the Linux libraries for EGL, OGES and VG now as OSS, along with the GPL VCHIQ kernel driver (which IIRC has always been GPL'd). These are all on the Arm side, as the diagram states. These are the libraries that they have been talking about, and at no point that I can see have the Raspberry Pi foundation ever stated the GPU code was being made OSS. The fact that these libraries are shims to translate the LInux standard API's to whatever the GPU does is irrelevent, since they are, to all intents and purposes the implementation of the API.

And there is an interesting post on the Raspberry Pi forums which says something about the lines of code running on the GPU - over 5 million! If you think any person unfamilar with the code woudl be able to make improvements or fix bugs without spending a hell of a lot of time on, then your engineering skills are not of this world. And that coming from someone who has experience in a similar area. I'll leave the bug fixing to the experts thanks mate!

As an aside, anyone found any bugs on the GPU yet? Or anyone think they could make the implementation better than the team that designed the hardware and wrote the software? Or have a go at writing their own 5m lines of code?

You appear to be willfully or accidentally disregarding the words shown on the diagram, so I'll enumerate the newly BSD-licensed blocks for you.  For reference, the diagram is here .  Please read carefully the words on each of the BSD-licensed blocks.  To help you avoid confusing implementations with interfaces, I'll provide the Wikipedia references as well so that you can see what the block labelled with each of these words is supposed to do.

• OpenMAX --- http://en.wikipedia.org/wiki/OpenMAX
• OpenGL ES  --- http://en.wikipedia.org/wiki/OpenGL_ES
• OpenVG --- http://en.wikipedia.org/wiki/OpenVG
• EGL --- http://en.wikipedia.org/wiki/EGL_%28OpenGL%29

Now that you know what each block is supposed to do, let's see what the evidence is that OpenGL ES (one of the blocks in question) runs on the ARM in any shape or form, in libraries or in daemons or in any other form of executing entity that you prefer:

• Eben Upton has very clearly stated (link referenced earlier in this thread) that OpenGL ES runs on the VideoCore and presents a high-level messaging interface to the ARM side.  OpenGL ES itself does not run on the ARM in any shape or form.  Since you appear to disregard technical facts, perhaps giving you Eben's description here first will convince you even if you ignore the facts that follow.
• The RPF blog announcement thread about this open sourcing contains code listings showing that the userland code that has been open sourced is merely a shim passing messages to the VideoCore.  Very clear to see (probably even to non-programmers) is that the released code does nothing at all except pass arguments through to VideoCore using RPC messages.  It is clearly nothing but an API, almost worthless to any open source developer wishing to fix or improve OpenGL ES, as actually open sourcing ARM-based OpenGL ES would have allowed.
• The diagram does not show OpenGL ES implemented on the VideoCore at all, so people who don't know the details of BCM2835 architecture could be forgiven for thinking that the orange block labelled "OpenGL ES" is the implementation of OpenGL ES for the Pi.  I doubt that it was deliberate, but it's a very unfortunate omission which has led many non-technical people into thinking that OpenGL ES resides in the block called "OpenGL ES", and has been magnanimously open-sourced.  It does not, and it has not.
• Developers of the open-source Lima driver for ARM's MALI-400 GPU and of *BSD have explained clearly that OpenGL ES has not been open sourced, and that only some small shims interfacing to the OpenGL ES on the VideoCore have been released under the BSD license..

So, I don't know how you can argue that the diagram which clearly show OpenGL ES as BSD-licensed is accurate.  It's not.  Only small interface libraries (the API which hides the messaging to OpenGL ES which runs on the VideoCore) have been open sourced, not OpenGL ES itself, neither in the form of libraries nor in any other form.

If this had been stated clearly on the blog, and especially if the diagram had been labelled correctly on the blog (because non-technical people just look at the diagram), then much of the confusion suffered by non-programmers would have been avoided, and so many non-programmers would not be thinking that Broadcom is the first ARM company to have made OpenGL ES open source.  Broadcom has not done this, despite many people now thinking that they did.

Morgaine.

Play nicely, people.

At least, play nicer than this thread over on the Foundation's original post:

http://www.raspberrypi.org/archives/2221#comment-34981

I think the main problem people are having is that the press release and original headline are a bit misleading as to what has been released. i.e. people have mistakenly taken it as 'Everything has been open sourced, hurrah!' whereas there's plenty of smaller print saying 'But not this bit - way too difficult to release at the moment'.

I think we should be grateful for what we've got and see what we can do with it.

@Billy,

If you post here you are a part of 'you lot'

It might be more helpful is some posters could cut back a bit on the emoting but we all get carried aways sometimes, best ignore the 'tone', which is often as much a matter of style as intent.

Although the forensic examination of everything that happens here may be a bit painful it does often get to the heart of things - so now I understand much better what is and is not on offer than I did before I read this E14 thread.

It's just the same at any kind of design review - you turn up with your lovely new code/circuit and everyone picks holes in it, which will almost always improve it. I have had pointless design reviews where everyone was too polite/uninterested/shy to comment and of course went out of them no wiser than I went in.

Michael Kellett

> I think we should be grateful for what we've got and see what we can do with it.

I think if you realized how little was opened up, compared to what was advertised:

"Today we have some really big news, which is going to mean a lot to many programmers"

you might agree with the experts who are showing their extreme disgust.

Liz said at the link you give: "There's some microcode in the videocore."

Abishur similarly said "there is still just a small amount of GPU stuff in the GPU that is closed".

http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=20900&start=28

But it turns out that essentially *all* the functionality is in the closed videocore,

which JamesH says is more than 5M lines of code, not some miniscule amount.

This is not even a step on the way to opening up the rest.

As was explained, the shim was opened up because it didn't

contain the crown jewels, not as a first step in opening up the

crown jewels.

It may in fact be possible for some people to get some benefit from

the shim, but the OpenBSD guys who supposedly were going to benefit

the most, were the most vocal against it.

http://www.mail-archive.com/misc@openbsd.org/msg116656.html

http://www.mail-archive.com/misc@openbsd.org/msg116657.html

Michael Kellett wrote:

 

Although the forensic examination of everything that happens here may be a bit painful it does often get to the heart of things ...

 

It's just the same at any kind of design review ...

Those two sentences make a remarkably good single paragraph, despite not having been written together as shown.  It's accurate not only as an analogy but also as a proper reflection of the engineering process.

Engineering analysis of problems and solution candidates is a sort of forensic examination, very precisely dissecting the evidence and making logical conclusions based on what is observed.  And if the coroner's autopsy shows us that our best friend or relative died because she was a closet alcoholic, we will be distressed, but the facts stand.

And so it is when we analyse the Pi hardware and software, the project goals, the Foundation, the RPF forum admins and moderation process, the Pi community, open sourcing, and every other aspect of this interesting technical and social event.  It doesn't always make happy reading when one's beloved Emperor is shown to have no clothes, but winter is coming and a hint to put on the woolies may be useful.

~If~ the announcement had been something like "GPU interfacing SDK released - documentation to come (subject to manpower)" then it would have saved a lot of traffic.Using a word as emotive as "open" might have been a touch foolhardy. Oh well.

Eben was pretty straightforward about it all in the Slashdot comments that John Beetem linked to. Perhaps he should have written the blog post in the first place...

Anyway, free (as in beer) stuff never hurts - I'm sure that it will find a use with folks writing their own applications, or minimalist embedded types. Someone may even figure out how to migrate the USB traffic into the GPU, or use it to number crunch a discrete Fourier transform etc, but I don't think it's anything for "ordinary" users to get too excited about just yet.

"Pi blog headline writer to my office immediately..."

Does anybody knows what compiler and compression algoritm they use at broadcom?

5M lines of code...

Is it?

loader.bin 269KB

start.elf    2401KB

bootcode.bin 17KB

the start.elf for the 240MB memory split is even only 600 - 700KB.

I know Dom throw some stuff out, not sure if it's the gpu code.

Isn't it amazing? 2 lines of code produce one byte in the final binary.

Maybe employees are payed based upon the number of codelines they produce?

Does anybody knows what compiler and compression algoritm they use at broadcom?

5M lines of code...

Is it?

loader.bin 269KB

start.elf    2401KB

bootcode.bin 17KB

the start.elf for the 240MB memory split is even only 600 - 700KB.

I know Dom throw some stuff out, not sure if it's the gpu code.

Isn't it amazing? 2 lines of code produce one byte in the final binary.

Maybe employees are payed based upon the number of codelines they produce?

Luc Cool wrote:

 

...Maybe employees are payed based upon the number of codelines they produce?

I think that only happens at Microsoft! [/shooting fish in a barrel mode]

Luc Cool wrote:

 

Does anybody knows what compiler and compression algorithm they use at Broadcom?

 

5M lines of code...

Is it?

loader.bin 269KB

start.elf    2401KB

bootcode.bin 17KB

 

the start.elf for the 240MB memory split is even only 600 - 700KB.

I know Dom throw some stuff out, not sure if it's the gpu code.

 

Isn't it amazing? 2 lines of code produce one byte in the final binary.

I've never seen any of VideoCore's 5 MLOC* and given jamesh's various comments chez RasPi I don't think I want see it, but here are some general comments about how 5 MLOC gets down to 2 MB of binary, or whatever it is.

First, compression: general-purpose compression algorithms can usually get you a factor of 2 for typical binaries.  If the binary is sparse, like an FPGA image or VLIW instructions you can get 3-4.

Second, SLOC usually includes comments.  Well-documented code will have higher KLOC per KB of binary.  This is particularly true if it's clever code that uses tricks to save instructions.  They say that every time you remove a line of code by doing something clever, you should add 5-10 comment lines to explain what you did so the code can be maintained.  Performance-sensitive video/graphics code should fall into this category.

Third, VideoCore may have a standard coding style which increases the number of lines of code.  For example, one place I worked forbid having single-line conditionals and loops.  Instead of "if (condition) consequent else alternative" you had to write:

if (condition)

{

   consequent

}

else

{

   alternative

}

There's a factor of 6 increase in SLOC right there.

Fourth, there may be lots of conditional compilation so that the same code supports different VideoCore versions and configurations.  Code that gets omitted by conditional compilation adds to the SLOC count but don't add to the binary.

Fifth, we don't know what's encompassed by jamesh's 5 MLOC.  It might be the code that goes into the standard RasPi binary blob, or it might include other VideoCore code such as diagnostics and codecs.  I would hope the basic video and graphics processing is pretty clean and simple and is 10's of KLOC and not MLOC.  When I hear of 5 MLOC complexity, I hear C.A.R. Hoare saying:

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.  The first method is far more difficult.

OTOH, jamesh's comments on VideoCore suggest that they did it the easy way.

*Glossary:

SLOC/KLOC/MLOC: Source/Kilo/Mega Lines of Code.

VLIW: Very Long Instruction Word.

John Beetem wrote:

 

Luc Cool wrote:

 

Does anybody knows what compiler and compression algorithm they use at Broadcom?

 

5M lines of code...

Is it?

loader.bin 269KB

start.elf    2401KB

bootcode.bin 17KB

 

the start.elf for the 240MB memory split is even only 600 - 700KB.

I know Dom throw some stuff out, not sure if it's the gpu code.

 

Isn't it amazing? 2 lines of code produce one byte in the final binary.

I've never seen any of VideoCore's 5 MLOC* and given jamesh's various comments chez RasPi I don't think I want see it, but here are some general comments about how 5 MLOC gets down to 2 MB of binary, or whatever it is.

 

First, compression: general-purpose compression algorithms can usually get you a factor of 2 for typical binaries.  If the binary is sparse, like an FPGA image or VLIW instructions you can get 3-4.

 

Second, SLOC usually includes comments.  Well-documented code will have higher KLOC per KB of binary.  This is particularly true if it's clever code that uses tricks to save instructions.  They say that every time you remove a line of code by doing something clever, you should add 5-10 comment lines to explain what you did so the code can be maintained.  Performance-sensitive video/graphics code should fall into this category.

 

Third, VideoCore may have a standard coding style which increases the number of lines of code.  For example, one place I worked forbid having single-line conditionals and loops.  Instead of "if (condition) consequent else alternative" you had to write:

 

if (condition)

{

   consequent

}

else

{

   alternative

}

 

There's a factor of 6 increase in SLOC right there.

 

Fourth, there may be lots of conditional compilation so that the same code supports different VideoCore versions and configurations.  Code that gets omitted by conditional compilation adds to the SLOC count but don't add to the binary.

 

Fifth, we don't know what's encompassed by jamesh's 5 MLOC.  It might be the code that goes into the standard RasPi binary blob, or it might include other VideoCore code such as diagnostics and codecs.  I would hope the basic video and graphics processing is pretty clean and simple and is 10's of KLOC and not MLOC.  When I hear of 5 MLOC complexity, I hear C.A.R. Hoare saying:

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.  The first method is far more difficult.

OTOH, jamesh's comments on VideoCore suggest that they did it the easy way.

 

*Glossary:

SLOC/KLOC/MLOC: Source/Kilo/Mega Lines of Code.

VLIW: Very Long Instruction Word.

What a good post. Not vitriolic, pretty sensible assumptions, no whinging. Are you sure youa re in the right place? Only thing I'd say is the GPU's are now very very complicated, and need a lot of code to make them work - if I rememebr right this has 3D units, 2D, video stuff in many differetn formats, a camera interface, LCD control, HDMI, i2c, i2s and probably other stuff not made public. And even an implementation of OpenGLES is a *** load of code - try a line compare with the MESA librearies.

As to the compiler, a quick google find you its a custom job, so presumably the CPU is a custom design.

Billy Thornton wrote:

 

What a good post. Not vitriolic, pretty sensible assumptions, no whinging.

What a pity you couldn't have resisted the temptation... Give it a rest, Billy.

Billy Thornton wrote:

Only thing I'd say is the GPU's are now very very complicated, and need a lot of code to make them work - if I rememebr right this has 3D units, 2D, video stuff in many differetn formats, a camera interface, LCD control, HDMI, i2c, i2s and probably other stuff not made public.

Billy, as far as I know, the code for the  CSI and DSI interfaces is not yet distributed with the firmware images, as it hasn't been written yet.

Argh, been away on business, so had my rest, now back for more fun.

Roger Wolff wrote:

 

Billy Thornton wrote:

Only thing I'd say is the GPU's are now very very complicated, and need a lot of code to make them work - if I rememebr right this has 3D units, 2D, video stuff in many differetn formats, a camera interface, LCD control, HDMI, i2c, i2s and probably other stuff not made public.

Billy, as far as I know, the code for the  CSI and DSI interfaces is not yet distributed with the firmware images, as it hasn't been written yet.

So, bearing in mind the GPU in the Pi is the same as that in the latest Nokia Symbian phones, you know, the ones with the best camera phones in the world, and with LCD panels on the front, what make you think the software hasnt been written yet? I reckon all the GPU software is there, it's the Linux shim that needs doing, and that stuffs a PITA. Since I rememebr seeing some images from a camera attached to the board, I guess it takes pictures already.

Bill

Yeah, in theory the code is "somewhere", but in practise the guys at broadcom have to configure their source tree to make a "start.elf" that runs on the 'pi. And there might be code to write to make the "can take a picture" code talk to Linux, not just the linux code to take advantage of it, but also the code on the GPU that talks to Linux.

On the other hand, even if there is a Nokia phone with broadcom support, maybe nokia said: "We've heard that Nikon makes good video processing software".... -- "but that'll add $3 to the price of the SOC!"... -- "It's important to us to have the best video processing available, NOW, so we'll pay for it".

Roger Wolff wrote:

 

Yeah, in theory the code is "somewhere", but in practise the guys at broadcom have to configure their source tree to make a "start.elf" that runs on the 'pi. And there might be code to write to make the "can take a picture" code talk to Linux, not just the linux code to take advantage of it, but also the code on the GPU that talks to Linux.

 

On the other hand, even if there is a Nokia phone with broadcom support, maybe nokia said: "We've heard that Nikon makes good video processing software".... -- "but that'll add $3 to the price of the SOC!"... -- "It's important to us to have the best video processing available, NOW, so we'll pay for it".

I read on the Pi website that they were writing the Linux side code to communicate with the camera driver, although was already available using something called OpenMAX.

From what I've read around, the broadcomm GPU does all the camera work in the Nokia smartphones, even the 808.