| ||||||||||
Our good friend biglesp put the Raspberry Pi 2 to the test the last time around, though I feel that he missed a good number of possible benchmarks to run on the hardware. With that in mind, I'm going to test the main boards of the Raspberry Pi that I can get my hands on and not only run the main benchmarks Les ran, but also a good few more thanks to Roy Longbottom's benchmark collection. I also have some approximate information on idle power usage, which everyone appears to be interested in.
One technical capability that I would have loved to put through its paces with Roy's benchmarks is the OpenGL driver that the Pi Foundation have made available. Unfortunately the Roy benchmarks are currently not compatible with it, so I guess it has just been testing against the binary blob for now. Regardless, the VideoCore is one part of the ARM processor that practically has not changed from the early Model B to the most recent version of the Raspberry Pi.
Modelling for the Camera
It does not hurt to go over the versions of Raspberry Pi again, and here are the models that I tested:
| Component / Board | Raspberry Pi 3 Model BRaspberry Pi 3 Model B | Raspberry Pi 2 Model BRaspberry Pi 2 Model B | Raspberry Pi Model B+Raspberry Pi Model B+ |
|---|---|---|---|
| Processor Chipset | BCM 2837 64bit ARMv8 Cortex A53 Quad Core | BCM 2836 32bit ARMv7 Cortex A7 Quad Core | BCM2835 32bit ARMv11 Single Core |
| Processor Speed | 1.2Ghz per core | 900Mhz per core | 700Mhz core |
| Max Power Draw | 2.5A | 1.8A | 1.8A |
If you want more information on the specifications you can check the comparison chart linked in the Raspberry Pi space. What we're mainly concerned with is the processor, power usage and somewhat also RAM, however the RAM runs at 400Mhz for each board so that may make little difference.
What does make somewhat of a difference is that the Raspberry Pi 3 is an ARMv8 Processor with 64bit support, however, Raspbian at the time of writing does not have 64bit support. In part this is so that it is backwards compatible with the previous Raspberry Pi. There is yet to be a 64bit Repo. Debian has been building up its 64bit support over a period of years, and potentially someone could compile together a kernel and binaries for a 64bit Jessie Debian for the Raspberry Pi 3. Go on, accept the challenge!
Taste Testing the Pi
Before I get into the software used for the benchmarking, I decided to run a few commands to get the processor information and supported instructions:
| cpuinfo - Raspberry Pi 3 Model B | cpuinfo - Raspberry Pi 2 Model B | cpuinfo - Raspberry Pi 1 Model B+ | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CPU architecture: 7
<... and so on for each core...>
|
CPU architecture: 7
<... and so on for each core...>
|
CPU architecture: 7
|
What is pretty clear with cpuinfo is that it identifies the Raspberry Pi 3 processor as an ARMv7, this appears to be a failure in the software for now but this may change with a kernel/firmware update to ARMv8. What is interesting is that we have a new feature: crc32. I also never realised that "java" was actually a feature on the Model B+ processor chip and that's one that has been lost over time (some would say that is no love lost).
| lscpu - Raspberry Pi 3 Model B | lscpu - Raspberry Pi 2 Model B | lscpu - Raspberry Pi 1 Model B+ |
|---|---|---|
Architecture: armv7l Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 Model name: ARMv7 Processor rev 4 (v7l) CPU max MHz: 1200.0000 CPU min MHz: 600.0000 | Architecture: armv7l Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 Model name: ARMv7 Processor rev 5 (v7l) CPU max MHz: 900.0000 CPU min MHz: 600.0000 | Architecture: armv6l Byte Order: Little Endian CPU(s): 1 On-line CPU(s) list: 0 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 1 Model name: ARMv6-compatible processor rev 7 (v6l) CPU max MHz: 700.0000 CPU min MHz: 700.0000 |
With the lscpu command we can see that the Pi 3 processor is perhaps still not being identified properly, but at least that it, like the Pi 2 before it, can perform cpufreq scaling (which is also shown when issuing the lshw command). We can also clearly see that we have a confirmed, rather gladly, four core processor with the Pi 2 and Pi 3. Phew!
If you want to run these commands yourself on your Raspberry Pi with Raspbian then you can make sure you have them installed thusly:
sudo apt-get update
sudo apt-get install lscpu cpuinfo lshw
From a terminal window either within your desktop environment or from pressing CTRL-ALT-F1 (to F7, typically).
Power Usage
You can find some strange little devices that will plug in line with your USB hardware and it will tell you the most useful of things, such as how much power is being drawn! So I thought "sweet! Let's see how much power these draw while idle!"
| Raspberry Pi 3 Model B | Raspberry Pi 2 Model B | Raspberry Pi 1 Model B+ | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
Let me give these values some context, each Raspberry Pi was set to boot to the terminal, so that the X windows environment was not running. The only devices connected were a HDMI to DVI adapter to a 19" Widescreen monitor, a Dell USB keyboard, a 16gByte Class 10 microSD card and the power supply, which was providing 5 Volts, 2 Amps. There was no ethernet cable plugged in (though I can note that when it was, the power usage went up in all cases). We can see from the charts that the Pi 3 is pulling more power at idle than its predecessors, it is likely worth noting that the on board WiFi adapter of the Pi 3 was active, though it was not associated with an access point. The Bluetooth 4.1 adapter status was not intentionally active, as there was no driver loaded or software to use it. So it is likely the increase in power usage is due to the WiFi chip.
SysBench
Now this software has been around since 2004, it was originally intended for input/output (io) file operations and database benchmarking. Thanks to being open source it gradually developed into an almost all-round system benchmark which also includes aspects of processor testing as well as IO and databases.
SysBench's processor tests verify prime numbers by going through all possible divisions and only being satisfied when the result is zero. This does mean that it does not test all features of the processor, except for raw number crunching. SysBench was ran with the following parameters:
sysbench --num-threads=1 --test=cpu --cpu-max-prime=20000 --validate run
sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 --validate run
Here is a breakdown of the command line parameters:
sysbench - The name of the software to run
--num-threads - This is the number of processes to run, in the tests we run 1 thread and then 4 threads, this means that it will create 1 or 4 processes and run one process per core. Since the Raspberry Pi 1 Model B+ has one core it made sense to run a one core test on each model of Raspberry Pi for fairness alongside running 4 threads.
--test=cpu - This parameter ensures we are only testing the processor, as mentioned previously SysBench can perform other tests, too
--cpu-max-prime - This is the maximum prime number value we want to calculate up to.
--validate - This ensures that the results we have returned are valid
run - The software can emulate or test rather than actually perform the requested benchmark, so we want to tell it to actually run it.
| sysbench with 1 thread - Raspberry Pi 3 Model B | sysbench with 1 thread - Raspberry Pi 2 Model B | sysbench with 1 thread - Raspberry Pi 1 Model B+ | |||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Total time 477.0617 s
diff between min and max 2.22 ms | Total time 768.6476 s
diff between min and max 5.73 ms | Total time 1318.933 s
diff between min and max 168.64 ms |
With the Pi 3 versus the Pi 1, the results show about a 94% total speed difference, compared to the Pi 2 there is a 47% speed difference on the total time taken. What I find most interesting about these results, though, is the difference between the minimum and maximum time taken. The Pi 3 shows a 194% speed improvement over the B+ and an 88% speed improvement over the Pi 2. So while the overall time may not feel that big of a deal, per request it is performing them a lot faster. There's also evidence that it is overall significantly faster than the B+.
| sysbench with 4 threads - Raspberry Pi 3 Model B | sysbench with 4 threads - Raspberry Pi 3 Model B | sysbench with 3 threads - Raspberry Pi 1 Model B+ | |||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Total time 119.4716
diff between min and max 11.35 s | Total time 191.8972 s
diff between min and max 25.07 ms | Total time 1321.493 s
diff between min and max 160.06 ms |
This test is an unfair comparison with the Pi 1 B+ since it only has one processor core, so it is not surprising that it takes significantly longer than the Pi 2 and 3. The Pi 2 closes the gap a bit closer when working with all four cores here than with the single core test though the Pi 3 still outperforms by 75% with the difference between the min and max time it takes for the requests, overall there is still only a 47% speed difference which is almost half.
Overall the Pi 3 comes out on top as faster, though it would be interesting to see how the Pi 2 compares if it was over clocked. Still, something to remember is that computing prime numbers is just one function of what this processor is capable of.
nBench
I thought it would be good to include this for retro' computing sake, especially considering that the Raspberry Pi is intended to be going back to the roots of learning how to code and programme with a capable hands-on computer platform. nBench has been around for so long that you can compare these benchmark results against older processors such as the 386 and even 486 based processors right up to the Intel Core i7. It can even run on Android.
The software is intended to test three main components of the processor, the capabilities of the CPU (Central Processing Unit), FPU (Floating Point Unit) and memory system. You can find more information on this site. Once you have compiled nBench then it is simply ran with a single command of what the program is called. In the results, the higher the number, the better, as it is the number of iterations it can perform per second, summarised.
| nbench - Raspberry Pi 3 Model B | nbench - Raspberry Pi 2 Model B | nbench - Raspberry Pi 1 Model B+ | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
Since I threw this benchmark in for fun, here are some other processors that have taken the same benchmark, to put these in perspective:
| nbench - AMD K7 Thunderbird | nbench - Pentium 3 900Mhz | nbench - LG Optimus GT540 | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
It is totally not fair to compare the Raspberry Pi processor to these chips, except for the LG Optimus, that is using an ARM processor, however I found it interesting that the Broadcom chip has its own capabilities and areas where it excels in comparison to these older processors, for example the integer capability on the Pi 3 excels against the old AMD K7 processor and both the Pi 3 and Pi 2 processors give the Pentium 3 a run for its money, though Intel are typically known for strong Floating Point arithmetic, it is typically known that GPU (Graphical Processing Units) are better for this these days and using the VideoCore in the Broadcom chip would likely perform better.
That and these latter processors were intended for desktop computers. There are are a lot of numbers which the software outputs, like the SysBench benchmark I have attached the spreadsheet containing all of the values to this blog post that you can find at the bottom.
MemTester
This software is mainly intended to diagnose or test your system RAM (Random Access Memory). You can read more about it in its man page on linux. In this test I have put a limit on it, it is to only test 256mByte of RAM. This helps to make it a fair test across the different models of Raspberry Pi. If you are not aware, the Raspberry Pi shares its system RAM with the VideoCore processor, and it is not really recommended to deny the VideoCore processor from using any of the RAM available. So that means we cannot test the entire 512mByte or 1gByte of RAM available on the Pi 1 or Pi 2 and 3.
Memtester by itself does not time how long it takes to check the amount of memory we specify. However, we can set how many times it does it. There is also a command in Linux called 'time' which, when used in conjunction with a command, tells us how long it has taken for the command to run. Using this simple command we can check how long it has taken to test 256mByte of RAM on each Raspberry Pi:
sudo time memtester 256M 1
| Raspberry Pi Board | Time Taken |
|---|---|
| 1 Model B+ | 76 minutes 23.296 seconds |
| 2 Model B | 23 minutes 39.07 seconds |
| 3 Model B | 8 minutes 37.078 seconds |
The numbers speak for themselves, even though the RAM speed is pretty much the same being at least 400Mhz, evidently the limit is largely with the processor and we are seeing significant speed increases. In this case the Pi 3 is more than 50% faster than the Pi 2 at allocating and accessing RAM.
Roy's Benchmark Collection
There are a lot of tests. I mean, a lot. They range from whestone, ARMv6 specific tests, to ARMv7 and NEON tests. They also cover OpenGL and also memory tests deeper than by the ones previously in this blog post. So rather than go through every single test, I have compressed them together and attached them to this blog post.
I cannot cop out entirely though, can I? Well, I could, but I am not going to. Here are a few highlight comparison benchmarks from the text files that you can see as worthwhile to read:
Memory Reading Speed Test 32bit Version 4 (memSpeedPiA6)
| Memory KBytes Used | Double MB/S - Raspberry Pi 3 Model B | Double MB/S - Raspberry Pi 2 Model B | Double MB/S - Raspberry Pi 1 Model B+ |
|---|---|---|---|
8 16 32 64 128 256 512 1024 2048 4096 8192 | 1523 1641 1523 1524 1524 1525 1409 1094 1075 1023 1071 | 1015 1015 1016 930 853 853 682 393 310 301 307 | 602 538 292 262 176 142 132 134 134 136 134 |
Now there is no doubt here that the Pi 3 is performing really well, what I am most interested in is the scaling from where it starts to drop off in performance. On both the Pi 2 and the Pi 1 we can clearly see that performance reduces at around the 32/16Kbyte used mark, but with the Pi 3 we do not start seeing degradation in speed until we hit 512kByte, though 16kByte is obviously a sweet spot even when we hit 1024kByte and above the drop is not as consistently severe as it is with the Pi 2. So there is obviously a fundamental change in how the hardware handles memory or pushes its numbers around.
NEON Speed Test v1.0
| Raspberry Pi 3 Model B | Raspberry Pi 2 Model B |
|---|---|
 |
|
NEON technology was added with the Raspberry Pi 2, so it is not possible to test it on the Pi 1 Model B+, however I had not seen any benchmarks that had actually put it to the test until now. This increase in speed result should mean that the Pi 3 is faster at handling calculations relating to video, vector graphics rendering (so gaming and 3D) and potentially audio processing. Again it is interesting to see the point at which the performance starts to fall off.
There was also another NEON related test, which was the Linpack Single Precision Benchmark. This related the speed to MFLOPS, the Pi 2 came in at 299.93 MFLOPS and the Pi 3 at 462.07 MFLOPS.
Worth it?
The Raspberry Pi 3 Model B is paving the way for not only a faster processor, but hopefully an inclination as to the future of the Raspberry Pi platform as a whole. The move over to supporting 64bit with the ARMv8 processor should allow the chip to be better supported by operating systems from now on. The addition of WiFi and Bluetooth on the board is welcomed by many, and this is one feature that I did not have time to benchmark. What I would love to do is compare the WiPi to the Broadcom on board WiFi to see what the throughput is like and to see if how the chip is now connected has improved the speeds at all. Though we may still be limited to the speed of connected storage (SDCard or even USB).
It would also have been nice to properly benchmark the OpenGL functionality of the VideoCore with the new open source driver and to compare that with the past iteration(s) of Raspberry Pi. Though it is likely there is negligable difference as shown by the comparisons between the benchmarks using Roy's OpenGL program (see attachment to this blog post), and it is nigh impossible to test is on the Raspberry Pi 1 range due to the low amount of supported RAM on the board.
Aside from the repeated call for "faster processor!" that people keep calling out, in some hope that a small board can replace their desktop computer, the Raspberry Pi 3 is still a gradual step in the right direction and the price of it is amazing. I find it highly commendable that the Pi Foundation have made everything backwards compatible so far, but this cannot last forever and the support for the systems will have to fracture somewhere as packages move over to 64bit, even if it does mean supporting them side by side. Time to educate the masses!
So, is it worth it?
Yes. Yes it is.
Bonus Coverage: Temperature
This was not by me, turns out the Raspberry Pi 3 runs hotter than previous models, enough that it should require a heatsink, and ventilation if contained. I discovered this was the case with the Raspberry Pi 2 overclocked to 1.1Ghz, so it makes sense. Though you will encounter people that claim the Raspberry Pi does not need cooling, I think we may see a change.
If I had a thermal camera, I would do a shot of every Raspberry Pi. That would be awesome. Oh, SOMEONE DID!
Top Comments