SBC CPU Throughput

Reading that thread is depressing. The general lack of comprehension of cpu speed governors - yep, it's doing nothing, so slow down the clock turn things off etc to save power. That's why it shows 300 bogomips, nowt to do with what connector you supply power through.

Would we really like our smart phones battery to last 5 mins instead of (maybe) a day ?  Or is this technology a good thing ?  

(I prefer the phones battery to last a week, no smartphone for me!)

Anyway, I claim victory on the benchmark

I'm sure it'd be faster if it'd run long enough to bring the processor out of idle and off the slowest clock speed too.

On a more serious note, it's about 12.8 secs on a sabre-lite with debian armhf, so the BBB angstrom build seems to have been better optimised than the generic debian.

On the BBB with angstrom I get 12.090 with the ondemand governor and 11.86 with the performance governor, showing that the governor has some latency, but nothing that anyone will care about.

Bogomips on the A9 core of the sabre-lite is 1988 per core vs 990 on the BBB's A8 showing the futility of artificial benchmarks.

I did my share of chuckling at some of the beginners' posts there too while looking for usable numbers, but I think the interaction with CPU speed governers eventually dawned even on the beginners in the thread (well aided by people like dom), so it occurred to me while browsing that the discussion there was having a useful educational effect.

They were even advising caution about assigning inappropriate meaning to BogoMips.  Win! \o/

PS. I added your Sabre-lite and BBB findings to the table as additional datapoints.

PPS. Cortex-A8 looks set to having a long life, despite many newcomers buzzing around its heels.  It's also the ARM core in my original Samsung Galaxy Tab 7" tablet, and has no trouble being snappy enough for that application.

Data is always good, and sharing it is also good

Only if it is good data. This is so obvious it almost hurts to have to write it.

As usual, take benchmarks with a truckload of salt, and evaluate with a suitable mixture of suspicion, snoring, and mirth.

The warnings are to help people avoid unwarranted conclusions.

How do these warnings convey any level of confidence at all in the results of the benchmark ? How can any conclusion (warranted or otherwise) be gained using a benchmark with all those

disclaimers tacked onto it ? The warnings you provide do not seem to keep you from attempting to provide conclusions about the usefulness of the "benchmark" though.

As to the benchmark itself, no single line program line can be considered as a valuable synthetic benchmark of anything. In my 35 years in the profession I have never seen (other) professionals

claim that it could.

And when used properly, synthetic and other artificial benchmarks can be very valuable

These are normally suites of benchmark programs not a single line program which I'm assume you are already aware of. And even the simplest of the programs in the synthetic benchmark

suites normally consist of many lines of code. A single line program can hardly be called proper use of synthetic benchmarking.

I did my share of chuckling at some of the beginners' posts there too while looking for usable numbers, but I think the interaction with CPU speed governers eventually dawned even on the beginners in the thread

The level of competence of the participants in that thread or how well they understand CPU speed governors has nothing to do with the benchmark results you published other than that's were the "benchmark"

came from. I find it hard to understand why you think defending a benchmark from a thread with all those "chucklehead beginners" (paraphrasing you of course) in it is a good idea anyway.

I wrote:

> Data is always good, and sharing it is also good

 

Gary Stewart wrote:

> Only if it is good data. This is so obvious it almost hurts to have to write it.

I have no reason to believe that any of this data is bad data.  You have not pointed to any datum in the list and suggested that it is bad, nor have you given any reasons why we should believe that one or more of the numbers captured are erroneous.  It sounds like you're raising FUD just on general principles, as you seem prone to do in this forum.  In any event, if you disbelieve any of the data that the members of the RPF forum have released, take it up with them first.

The only data in the table provided by members of this E14 forum is selsinork's.  If you believe that any of his data is bad, please take it up with him first, and give solid engineering reasons because he's not likely to take any nonsense from you.  Bad data is bad data and I dislike it as much as anybody, but all indications are that all these numbers are genuine and accurate.  Unless we have evidence to the contrary, this is entirely good data.

The interpretation that people give to good data is an entirely different matter, and my cautions were the usual advice about taking perfectly good numbers and making wholly incorrect conclusions about them.  Even you have agreed with that, so you're really just looking for a fight as usual.

There is absolutely nothing at all to argue about here, we all agree about the merits or otherwise of benchmarks.  AFAWK the numbers are totally accurate.  Use them appropriately and they can be useful, but use them incorrectly to tell you how real-world applications will perform and you may well conclude something erroneous.  News at 11 !!!

 * morgaine sighs

The interpretation that people give to good data is an entirely different matter, and my cautions were the usual advice about taking perfectly good numbers and making wholly incorrect conclusions about them.

How can you provide a useful interpretation of data that in your own words you should "take (benchmarks) with a truckload of salt, and evaluate with a suitable mixture of suspicion, snoring, and mirth.". I mean really,

if this is your idea of normal cautions for data I'd like to see your idea a real bona-fide warning.

AFAWK the numbers are totally accurate.

But not really useful for the real world (see previous responses) which is the point you keep dancing around.

Even you have agreed with that,

I agree with what you said, but not that the data in this "benchmark" is good in the sense that a useful interpretation of much of anything is possible using it. Which is

again the point you keep dancing around. It is far too simplistic a benchmark to provide that.

so you're really just looking for a fight as usual.

Something I've seen you do on numerous occasions yourself.

> As to the benchmark itself, no single line program line can be considered as a valuable synthetic benchmark of anything.

...

> I was totally unaware that triviality in any form was considered a good trait for a benchmark. Doesn't make sense to me though.

...

> It is far too simplistic a benchmark to provide that.

You obviously have no clue what this benchmark does.  It isn't a "single line program" at all.

Invoking the program takes a single line, and that is a good thing.

There's a very important difference between the command to invoke a program

and the program itself. 

Gary Stewart wrote:

 

How can you provide a useful interpretation of data that in your own words you should "take (benchmarks) with a truckload of salt, and evaluate with a suitable mixture of suspicion, snoring, and mirth.".

A rational person can provide a useful interpretation by using ordinary engineering knowledge and commonsense.

I refer you to coder27's post about the data confirming that execution times are proportional to clock rates for a given architecture, and likewise confirming that there is a significant improvement from ARM11 to Cortex-A8 for a given clock rate.  These are helpful observations in that they confirm what is expected, and if the numbers had indicated something entirely different then we would have some very serious issues to investigate.

> I wrote:
> AFAWK the numbers are totally accurate.

 

But not really useful for the real world (see previous responses) which is the point you keep dancing around.

See previous section.  It is you who chose to dance around and ignore the useful and helpful interpretations of this data explained well in coder27's post, and instead dived in here directly at me without provocation nor valid reason.

I have consistently stated that good data is useful when used appropriately, but unhelpful when used inappropriately.  I clearly said "Results roughly as expected" in my opening post which points to the data being useful, and then I gave the usual cautions about using benchmark data wrongly to reach inappropriate conclusions.

Nobody here has made any inappropriate conclusions from this data, so whatever are you arguing about?

Morgaine.

Gary Stewart wrote:

 

How can you provide a useful interpretation of data that in your own words you should "take (benchmarks) with a truckload of salt, and evaluate with a suitable mixture of suspicion, snoring, and mirth.".

A rational person can provide a useful interpretation by using ordinary engineering knowledge and commonsense.

I refer you to coder27's post about the data confirming that execution times are proportional to clock rates for a given architecture, and likewise confirming that there is a significant improvement from ARM11 to Cortex-A8 for a given clock rate.  These are helpful observations in that they confirm what is expected, and if the numbers had indicated something entirely different then we would have some very serious issues to investigate.

> I wrote:
> AFAWK the numbers are totally accurate.

 

But not really useful for the real world (see previous responses) which is the point you keep dancing around.

See previous section.  It is you who chose to dance around and ignore the useful and helpful interpretations of this data explained well in coder27's post, and instead dived in here directly at me without provocation nor valid reason.

I have consistently stated that good data is useful when used appropriately, but unhelpful when used inappropriately.  I clearly said "Results roughly as expected" in my opening post which points to the data being useful, and then I gave the usual cautions about using benchmark data wrongly to reach inappropriate conclusions.

Nobody here has made any inappropriate conclusions from this data, so whatever are you arguing about?

Morgaine.

Nobody here has made any inappropriate conclusions from this data

Never said they did, I just said that useful conclusions can't be derived from the data.

so whatever are you arguing about?

DAB says it best:

In my architecture analysis days we used the term " Lies, damn lies and benchmarks!"

Most computer architectures are too varied to assess with simple benchmarks.  You really need to look at your proposed application and look into the system architecture to get a good feel about how well one processor will perform over another.

Unless you work with each processor in assembly language, you will seldom collect anything but interesting data.

True performance is always dependant upon the compiler, application structure, operating system, I/O drivers and many other details.

Raw power is seldom the only answer needed for a good system design.

I wrote:

> Nobody here has made any inappropriate conclusions from this data

 

Gary Stewart wrote:

> Never said they did, I just said that useful conclusions can't be derived from the data.

Which coder27 disproved very specifically by making useful engineering conclusions from the data, and which I had also noted as being useful in my opening post by saying that the results were as expected.  Indeed, they are as expected, by any rational engineer with basic knowledge of architecture and types of ARM core.  You've now ignored these useful conclusions twice over in a hopeless attempt to salvage your completely empty and nonsensical set of posts.

If you dispute the data, go argue with those who captured it.  If you dispute that it is useful when applied sensibly in the way that we have applied it, that suggests lack of familiarity with basic CPU architecture.  In fact, you have not disputed anything at all, neither the data nor the reasoning, and you even agree with the cautioning about inappropriate use of benchmarks.

Yet somehow you still find a way to argue beligerantly here without denying the given data, without providing new data, without denying the given reasoning, without providing new reasoning, and in fact without contributing anything at all.  Your posts on this topic have significantly raised the bar on the concept of empty.

This thread was created to informatively and usefully summarize the simple benchmarking started in the RPF forum.  If you can't get your head around how this data can be interpreted usefully in the limited way that makes sense, fine, nobody is forcing you.  But please don't disrupt without clear reasoning an objective engineering thread in which data is considered valuable and in which the people present know how to use this data safely and sensibly.

Morgaine.

Which coder27 disproved very specifically by making useful engineering conclusions from the data, and which I had also noted as being useful in my opening post by saying that the results were as expected.

The only thing I have trouble wrapping my head around is why stating the obvious is a considered a useful engineering conclusion.

Which coder27 disproved very specifically by making useful engineering conclusions from the data,

coder 27:

They confirm that when the clock rate increases from 700MHz to 1GHz on

the same architecture, that the benchmark timings have a corresponding

improvement.

Fairly sure this has been well known since the dawn of of non one of a kind computers and doesn't really need confirmation.

They confirm that when the architecture improves from ARMv6 to the superscalar ARMv7, but the clock rate is held constant,

there is acorresponding improvement.

Again well known since the ARMv7 has been out for quite a while now. I guess it doesn't hurt to confirm it one more time.

They confirm that when both the architecture and clock rate are both held constant, such as between the cubieboard and BBB, that the

benchmark timings are relatively constant.

Again simply stating the obvious. At least it disproves the negative right ?

They confirm that between the RPi and BBB there is a substantial performance difference, for integer compute bound, much greater than

the difference in price.

As long as this narrowly defined benchmark is the only performance difference that is important to you, great. One simple benchmark

and done.

Back to you:

If you dispute that it is useful when applied sensibly in the way that we have applied it, that suggests lack of familiarity with basic CPU architecture.

No, I have been around almost all of the microprocessor based CPU architectures (and a couple that used discrete TTL and real magnetic core memory

as well) since before the Z80, so there's no lack of familiarity there.

In fact, you have not disputed anything at all, neither the data nor the reasoning, and you even agree with the cautioning about inappropriate use of benchmarks.

Never disputed the accuracy data, I did dispute the reasoning used to "prove" that it provided much in the way of useful engineering information with a

reason in every reply to you. Of course I agree with cautioning about the inappropriate use of a benchmark (different from benchmarks as only one

benchmark was run here and that one of the reasons I used that you "lost") but yours went well beyond any semblance of an appropriate caution

and showed a distinct lack of faith in the usefulness of the benchmark data (not the data itself) which is exactly what I said.

Yet somehow you still find a way to argue beligerantly here without denying the given data

Never questioned the accuracy of the data just the usefulness of the interpretation of a single narrowly focused benchmark so why do you keep bringing

that up ?

without denying the given reasoning, without providing new reasoning

I have provided both in my posts and have done so again in this one. That you continue to "miss" it is hardly surprising. You have a knack for

never responding to any "given" or "new" reasoning at all or with anything other than repeating what you said before in effect doing exactly

what you say I am doing. Or just accusing someone of not using reasoning at all or at least your interpretation of it again exactly as you

have done here.

Your posts on this topic have significantly raised the bar on the concept of empty.

An area you have also ventured into yourself on occasion including this very thread. And several personal attacks from you to boot !

I thought you didn't approve of such things. Well again, not surprising.

This thread was created to informatively and usefully summarize the simple benchmarking started in the RPF forum

And the simplicity of this single benchmark and the fact that it is a single benchmark have been the center of the reasoning about the usefulness

of its data since the beginning. Any conclusions about the performance of a complex system based on one simple test is more likely to misinform

than inform, or at best just prove what you already know. I guess that can be considered useful to some.

If you can't get your head around how this data can be interpreted usefully in the limited way that makes sense, fine, nobody is forcing you.

If you can't get your head around what is actually useful fine nobody is forcing you.

But please don't disrupt without clear reasoning an objective engineering thread

Sorry, I don't consider you to be the final arbitrator on what clear reasoning or objective engineering is although I know you have

projected yourself as such several times in the past often rejecting clear reasoning and objective engineering when doing so.

in which data is considered valuable and in which the people present know how to use this data safely and sensibly.

Yes it is valuable, the more data (from benchmarks, note the plural) the better, especially when you need to use it sensibly.

I'm not sure were using benchmark data safely comes from and you forgot responsibly.