SBC CPU Throughput

Well, this has been interesting. (not really)

Here is what I liked about the data.

1) It cost me nothing

2) It confirmed what I expected

If the data had said that given the problem in question the BBB took twice as long to compute the answer as the RasPi, it may have given me pause and perhaps prompted me to investigate further using other more appropriate benchmarking techniques.

So while at first blush it may come across as "Move along, nothing to see here" data, I find that type of data can be useful as well. Especially when it is free.

Mike

Michael Conners wrote:

 

So while at first blush it may come across as "Move along, nothing to see here" data, I find that type of data can be useful as well. Especially when it is free.

Exactly.  Perfectly good data, perfectly usable in the correct context, and the usual cautions given to discourage people from deriving inappropriate conclusions.

Why anyone would want to vent their ire over this simple gathering of useful data is hard to see, but it clearly hasn't been aimed at being helpful to the forum.

Guys/Gals

I learnt a lot from this .....um ....discussion.

Not from the bickering about benchmarks, but from coders detailed explanation regarding how some of the inner workings applied.

I note that in the tests for various computers in magazines, they have a number of 'real use' benchmarks, and the results vary, so I've always regarded benchmarks with some concern.

While I can understand the passion about the subject, I think the short single responses were far less threatening,( for us in the observation deck), than a long 'attack' on what seemed like each line.

I would also like to point out, that not everyone understands the inner workings, nor do they need to, to have it actually do something.

You guys all drive cars (I presume) but how many of you know the inner workings of the engine, down to the nuts and bolts and their interaction to make it go.???

For most there some pedals, that have to operated in some form of order, and a thing in the middle you shift around, and fuel that is always running out.

Morgaine

I wonder if a slightly different worded caution about the figures (for the less knowledgable ones that might read it.) might statisfy the masses.

something from Dabs comment maybe.

True performance is always dependant upon the compiler, application structure, operating system, I/O drivers and many other details.

 

Raw power is seldom the only answer needed for a good system design.

What is the next  ... discussion ...

Mark

Mark,

   I'm happy to see the discussion was useful!

   With regard to a differently worded caution, I think your suggestion misses the point.

This benchmark is a decent benchmark, not just a "poor-man's benchmark" as Morgaine

originally described it, and it produced results which demonstrated a clear winner

in price/performance between BBB and RPi, within its domain of applicability,

which is integer compute bound.  The warning should warn against extrapolating

the results beyond the domain of applicability, but not much more than that.

Prior to the benchmark results, there were legitimate concerns raised about

things like memory bandwidth on a 16-bit bus, but these concerns have been

laid to rest.   Similarly, there were concerns raised about the supposed

triviality of the test, and the domain of the test, but I think those concerns

have been shown to be false.

In any benchmarking activity, there will always be claims that the results might

have turned out differently if a different compiler design or application structure

or operating system or I/O drivers, etc., had been used.  But in a result this

decisive, those concerns ring hollow.

Replying to Mark and coder27 together, as you both commented on phraseology::

Mark, you're quite right that depending on the audience, a more extensive and explanatory discussion about appropriate use of benchmarks could be very useful.  My comment was the bare minimum intended for an audience of engineers and techies, who typically know very well already that measuring A doesn't generally allow you to conclude anything about B.  It wasn't intended as an explanation, but merely a reminder not to do anything silly with the numbers, and written in humourous style so that no expert would be affronted by teaching grandma to suck eggs.

That said, your suggested comment or wording addresses a very different topic to mine, namely "performance", and "raw power".  I purposely said nothing about such things, because you can easily slip into the realm of inappropriate conclusions by assuming that a particular benchmark is a good metric of specific types of performance unless you have checked that it directly measures or correlates properly with them.  As a result, your suggestion is much bolder than mine, but also more likely to be wrong without further study.  I just advised caution, which is always safe to do.

coder27, you're entirely right that calling this a "poor-man's benchmark" might be underselling the utility of this measurement.  That was not intended though, since the measurement was useful to me as I indicated, and I assumed that it would be useful to others as well.

I used the phrase only to mean easy, lightweight, simple, fast, built-in, no package to buy or compile, etc etc, and not implying any criticism whatsoever.  After all, if I had been critical of the benchmark or of the results then I wouldn't have gathered and displayed these measurements in the first place!  Nevertheless, it's possible that the phrase conveyed the wrong message to some readers, so a more neutral description might have been better, I do agree.  The word lightweight seems especially suitable.

coder27 wrote:

 

Prior to the benchmark results, there were legitimate concerns raised about things like memory bandwidth on a 16-bit bus, but these concerns have been laid to rest.

I've seen that matter raised a number of times myself, so it is indeed very useful to possess numeric data that addresses the issue objectively.

Morgaine

Addendum.  After considering the impact of "poor man's" still further, I'm now convinced that your observations require it to be changed to avoid derailing new visitors to the thread.  I've changed "poor man's" to "lightweight" in the opening article.  Many thanks!

Replying to Mark and coder27 together, as you both commented on phraseology::

Mark, you're quite right that depending on the audience, a more extensive and explanatory discussion about appropriate use of benchmarks could be very useful.  My comment was the bare minimum intended for an audience of engineers and techies, who typically know very well already that measuring A doesn't generally allow you to conclude anything about B.  It wasn't intended as an explanation, but merely a reminder not to do anything silly with the numbers, and written in humourous style so that no expert would be affronted by teaching grandma to suck eggs.

That said, your suggested comment or wording addresses a very different topic to mine, namely "performance", and "raw power".  I purposely said nothing about such things, because you can easily slip into the realm of inappropriate conclusions by assuming that a particular benchmark is a good metric of specific types of performance unless you have checked that it directly measures or correlates properly with them.  As a result, your suggestion is much bolder than mine, but also more likely to be wrong without further study.  I just advised caution, which is always safe to do.

coder27, you're entirely right that calling this a "poor-man's benchmark" might be underselling the utility of this measurement.  That was not intended though, since the measurement was useful to me as I indicated, and I assumed that it would be useful to others as well.

I used the phrase only to mean easy, lightweight, simple, fast, built-in, no package to buy or compile, etc etc, and not implying any criticism whatsoever.  After all, if I had been critical of the benchmark or of the results then I wouldn't have gathered and displayed these measurements in the first place!  Nevertheless, it's possible that the phrase conveyed the wrong message to some readers, so a more neutral description might have been better, I do agree.  The word lightweight seems especially suitable.

coder27 wrote:

 

Prior to the benchmark results, there were legitimate concerns raised about things like memory bandwidth on a 16-bit bus, but these concerns have been laid to rest.

I've seen that matter raised a number of times myself, so it is indeed very useful to possess numeric data that addresses the issue objectively.

Morgaine

Addendum.  After considering the impact of "poor man's" still further, I'm now convinced that your observations require it to be changed to avoid derailing new visitors to the thread.  I've changed "poor man's" to "lightweight" in the opening article.  Many thanks!