SBC CPU Throughput

Nice, thanks for posting.

Mike

As usual, take benchmarks with a truckload of salt, and evaluate with a suitable mixture of suspicion, snoring, and mirth.

You're absolutely right ! As an indicator or real world application performance this "benchmark" is worthless. Thanks for sharing it.

Data is always good, and sharing it is also good.  The warnings are  to help people avoid unwarranted conclusions.

And when used properly, synthetic and other artificial benchmarks can be very valuable, for example as a way of checking that an upgrade hasn't altered your compiler optimization defaults.  As part of regression testing, they're a very useful engineering tool.  You just have to be conscious of their limits, appropriate use versus inappropriate use.

Reading that thread is depressing. The general lack of comprehension of cpu speed governors - yep, it's doing nothing, so slow down the clock turn things off etc to save power. That's why it shows 300 bogomips, nowt to do with what connector you supply power through.

Would we really like our smart phones battery to last 5 mins instead of (maybe) a day ?  Or is this technology a good thing ?  

(I prefer the phones battery to last a week, no smartphone for me!)

Anyway, I claim victory on the benchmark

I'm sure it'd be faster if it'd run long enough to bring the processor out of idle and off the slowest clock speed too.

On a more serious note, it's about 12.8 secs on a sabre-lite with debian armhf, so the BBB angstrom build seems to have been better optimised than the generic debian.

On the BBB with angstrom I get 12.090 with the ondemand governor and 11.86 with the performance governor, showing that the governor has some latency, but nothing that anyone will care about.

Bogomips on the A9 core of the sabre-lite is 1988 per core vs 990 on the BBB's A8 showing the futility of artificial benchmarks.

I did my share of chuckling at some of the beginners' posts there too while looking for usable numbers, but I think the interaction with CPU speed governers eventually dawned even on the beginners in the thread (well aided by people like dom), so it occurred to me while browsing that the discussion there was having a useful educational effect.

They were even advising caution about assigning inappropriate meaning to BogoMips.  Win! \o/

PS. I added your Sabre-lite and BBB findings to the table as additional datapoints.

PPS. Cortex-A8 looks set to having a long life, despite many newcomers buzzing around its heels.  It's also the ARM core in my original Samsung Galaxy Tab 7" tablet, and has no trouble being snappy enough for that application.

I think this benchmark is actually pretty decent in a lot of ways.

1) it is trivially easy to run, nothing to download, nothing to compile,

    so you can easily get results from lots of people, allowing you to

    see how consistent the results are, and they seem to be pretty consistent.

2)  it is not subject to personal differences in what compiler was used

    to compile it, or what optimization levels or other compiler switches

    were used, although it will exhibit such differences between distros.

3)  it doesn't rely on computing the same value over and over in a loop.

     Benchmarks that do that can be overly sensitive to compiler loop

     optimizations, and to just-in-time code-generation techniques.

4)  It has a pretty-well understood area of application, integer compute bound.

    Obviously you wouldn't use it to measure floating-point performance, or

    gpu performance, or I/O performance, etc.

5)  It uses data that is large enough to show the benefit of large data caches,

     similar to typical user applications.

6)  It takes about the right amount of time to run--not so short that the time to

     load the benchmark matters, or that the accuracy of the clock matters,

     and not so long that you can't easily run it several times to see that the

     results are consistent.

Data is always good, and sharing it is also good

Only if it is good data. This is so obvious it almost hurts to have to write it.

As usual, take benchmarks with a truckload of salt, and evaluate with a suitable mixture of suspicion, snoring, and mirth.

The warnings are to help people avoid unwarranted conclusions.

How do these warnings convey any level of confidence at all in the results of the benchmark ? How can any conclusion (warranted or otherwise) be gained using a benchmark with all those

disclaimers tacked onto it ? The warnings you provide do not seem to keep you from attempting to provide conclusions about the usefulness of the "benchmark" though.

As to the benchmark itself, no single line program line can be considered as a valuable synthetic benchmark of anything. In my 35 years in the profession I have never seen (other) professionals

claim that it could.

And when used properly, synthetic and other artificial benchmarks can be very valuable

These are normally suites of benchmark programs not a single line program which I'm assume you are already aware of. And even the simplest of the programs in the synthetic benchmark

suites normally consist of many lines of code. A single line program can hardly be called proper use of synthetic benchmarking.

I did my share of chuckling at some of the beginners' posts there too while looking for usable numbers, but I think the interaction with CPU speed governers eventually dawned even on the beginners in the thread

The level of competence of the participants in that thread or how well they understand CPU speed governors has nothing to do with the benchmark results you published other than that's were the "benchmark"

came from. I find it hard to understand why you think defending a benchmark from a thread with all those "chucklehead beginners" (paraphrasing you of course) in it is a good idea anyway.

In my architecture analysis days we used the term " Lies, damn lies and benchmarks!"

Most computer architectures are too varied to assess with simple benchmarks.  You really need to look at your proposed application and look into the system architecture to get a good feel about how well one processor will perform over another.

Unless you work with each processor in assembly language, you will seldom collect anything but interesting data.

True performance is always dependant upon the compiler, application structure, operating system, I/O drivers and many other details.

Raw power is seldom the only answer needed for a good system design.

Just my opinion,

DAB

I wrote:

> Data is always good, and sharing it is also good

 

Gary Stewart wrote:

> Only if it is good data. This is so obvious it almost hurts to have to write it.

I have no reason to believe that any of this data is bad data.  You have not pointed to any datum in the list and suggested that it is bad, nor have you given any reasons why we should believe that one or more of the numbers captured are erroneous.  It sounds like you're raising FUD just on general principles, as you seem prone to do in this forum.  In any event, if you disbelieve any of the data that the members of the RPF forum have released, take it up with them first.

The only data in the table provided by members of this E14 forum is selsinork's.  If you believe that any of his data is bad, please take it up with him first, and give solid engineering reasons because he's not likely to take any nonsense from you.  Bad data is bad data and I dislike it as much as anybody, but all indications are that all these numbers are genuine and accurate.  Unless we have evidence to the contrary, this is entirely good data.

The interpretation that people give to good data is an entirely different matter, and my cautions were the usual advice about taking perfectly good numbers and making wholly incorrect conclusions about them.  Even you have agreed with that, so you're really just looking for a fight as usual.

There is absolutely nothing at all to argue about here, we all agree about the merits or otherwise of benchmarks.  AFAWK the numbers are totally accurate.  Use them appropriately and they can be useful, but use them incorrectly to tell you how real-world applications will perform and you may well conclude something erroneous.  News at 11 !!!

 * morgaine sighs

> " Lies, damn lies and benchmarks!"

While it is certainly possible to lie with benchmarks, it is also possible

to tell the truth, and in this case the benchmarks appear to be telling the truth

within their scope of application (integer compute bound), because they

confirm what you would expect based on clock rate and architecture.

They confirm that when the clock rate increases from 700MHz to 1GHz on

the same architecture, that the benchmark timings have a corresponding

improvement.

They confirm that when the architecture improves from ARMv6 to the

superscalar ARMv7, but the clock rate is held constant, there is a

corresponding improvement.

They confirm that when both the architecture and clock rate are both held

constant, such as between the cubieboard and BBB, that the benchmark

timings are relatively constant.

They confirm that between the RPi and BBB there is a substantial

performance difference, for integer compute bound, much greater than

the difference in price.

Do you really think that a different integer compute bound benchmark would be

likely to show that the RPi actually has the better cost/performance ratio?