RoadTest the Raspberry Pi 4 Model B (2GB) - Review

Table of contents

RoadTest: RoadTest the Raspberry Pi 4 Model B (2GB)

Author: ashutosh_pandey

Creation date: 6 Nov 2019

Evaluation Type: Independent Products

Did you receive all parts the manufacturer stated would be included in the package?: True

What other parts do you consider comparable to this product?: In my RoadTest below, I have compared the performance of the Raspberry Pi 4B against its predecessor, the Raspberry Pi 3B+, both boards are comparable for usage as an everyday system. There are boards like Nvidia Jetson Nano which offer great performance but are thrice as expensive and lack networking options. But at the price to performance ratio, there's nothing quite like the Raspberry Pi simply because of the community behind it which makes it go way beyond the hardware in it.

What were the biggest problems encountered?: For a newbie the difference between Linux for ARM machines and Linux for x86 machines is not very apparent. A whole lot of benchmarks and software suites are developed keeping the x86 counterparts in mind and there is a lot of manual installing needed to get things to work sometimes. The Raspberry Pi also does not have an internal current sensor - it is difficult to measure the power draw without a dedicated USB wattmeter. Yet another problem was replacing the HDMI port with Micro HDMI and the Type C power port. While these are necessary keeping the boards footprint and power needs in mind,it also means that new connectors are needed and connecting old VGA displays just got harder.

Detailed Review:

Being a technology enthusiast, I regularly read product reviews and RoadTests. Most of them can be grouped into one of the three categories:

1.) General use case scenarios.

2.) Benchmarking.

3.) Project building and prototyping.

In this RoadTest I have tried to cover all the three scenarios.

When I was selected for this RoadTest, I had made up my mind that I'd not only test the Raspberry Pi 4B, I'd compare it thoroughly to its predecessor. Because there is no point of looking at technical specifications if they cannot be compared to a standard. So effectively, this is a RoadTest of both the Raspberry Pi 4B and the 3B+.

1.) Setting it up

Large monitors, power hungry RGB keyboards, inadequate power supplies among other components can skew the testing process. To make sure that my results can be easily replicated, I chose to go with only the official components provided by Element14. These components are easy to set up, and have built in support - I did not have to do any debugging for the peripherals. The components used to test the Raspberry Pi 4B were as follows:

1.) Official 7 inch touchscreen display - Raspberry Pi 7” Touchscreen Display

2.) Official Keyboard -  https://www.raspberrypi.org/products/raspberry-pi-keyboard-and-hub/?variant=raspberry-pi-kyb-uk-rw

3.) Official Mouse - https://www.raspberrypi.org/products/raspberry-pi-mouse/?variant=raspberry-pi-mouse-rw

4.) Official PSU - https://www.raspberrypi.org/products/type-c-power-supply/

5.) Sandisk Ultra 16GB Memory card - https://www.amazon.in/SanDisk-Ultra-MicroSDHC-Memory-SDSQUNB-016G-GN3MN/dp/B010NE3U2M

The Operating system used was Raspbian Buster 2019 - 09 -26 which I flashed onto the MicroSD cards using BalenaEtcher v1.5.59. I had two MicroSD cards with me, The Sandisk U1 A1 98 MBPS Ultra and the Sandisk Edge A1 U1 Class 10. Since both cards are rated Class 10, I did a quick comparison along with the time needed to flash the card with Raspbian (3.83 GB File).

The Sandisk Ultra was rated for a higher read and write speed, and also flashed faster, so it was chosen.

2.) Setup Comparison - Boot and Update times

The first boot and subsequent boot times were compared across both boards. The Raspberry Pi 4B is marginally faster in both.

The first thing one does after booting into Raspbian is invoke:

sudo apt-get update && sudo apt-get upgrade

The The time taken to update and upgrade was noted (this includes the download time). I observed that for the exact same Wifi connection, The RPi 4B seemed to download files faster than the 3B+, I tested this multiple times, with the boards side by side. The 4B+ matched my Laptops Wifi speed (Around 35 Mbps as measured by fast.com) while the 3B+ struggled to hit even 25 Mbps, consistently dipping below 20 Mbps. Upon doing some further research, I found that this was due to the updated CYW43455 Cypress chipset that the 4B was using which is a step up from the 3B+ ( You can read more about it: https://www.cypress.com/documentation/product-overviews/cyw43455-wiced-ieee-80211ac-wifi-bluetooth-41-connectivity-solut…  ).

Image: The Raspberry Pi 3B+ and my laptop. The Rpi 3B+ is noticeably slower than the 4B on the WiFi front.

3.) USB 3.0 Real World implications

One thing that stopped the Rpi 3B+ from being a truly great NAS (Network Attached Storage) device was that the device had only USB 2.0 ports. Sure, you could connect external hard drives and USB sticks to the USB ports, and even boot from a pendrive via USB but USB 2.0 is still slow. To find out just how much of a difference it makes in a real world scenario, I tested the Raspberry Pi 4B's USB 3.0 and 2.0 ports and the Raspberry Pi 3B+ by copying a 1.6 GB file from a Hard drive to the Pi, as well as from the Pi to the Hard drive. The results were pretty evident even though the Hard drive I used was a 5400 RPM ADATA 650 USB HDD, so I was pretty far away from maxing out the USB 3.0 spec. The results are compiled below:

The Raspberry Pi 4B connected to a USB powered HDD

4.) USB Bandwidth and Modifications

Providing two USB 3.0 ports was not the only thing was changed on the 4B, the underlying controller was changed as well. The Raspberry Pi's earlier than 4B used a LAN7500 Series chip, which shared a single USB connection on the SoC between multiple USB ports and the Ethernet controller. Because of this, the Ethernet controller on the 3B+ was bottlenecked. True Gigabit ethernet on the Pi has been a demand for quite a while now, and the LAN7500 chip meant that the speeds never got over 480 Mbps. Using a tool called USBtop, it was easy to visualise the difference between the two boards. USBtop can be installed with the following commands:

$sudo apt install usbtop
$sudo modprobe usbmon
sudo usbtop

The first Image is of the 4B where there are 4 Bus ID's (for the 4 USB ports) while the Rpi 3B+ has 2 Bus ID's with 7 device IDs (one of them being the ethernet port, no doubt). This has very important Implications for the usability of the Pi for applications that require both external storage and a fast Ethernet connection. Now, one will not bottleneck the other. The Bandwidth while doing various tasks (such as typing on a keyboard, using a mouse, transferring files to and from a hard drive) can be seen in the image gallery below:

Please note that the values of the bandwidth fluctuate greatly over time, so the screenshots I took above at certain instants in time do not represent the average.

5.) Voltages and Power Supply

The Raspberry Pi 3B+ came with a 5.1V, 2.5A power supply while the Raspberry Pi 4B changed that to a 15.3 W power supply (5.1V, 3A) . Increasing the current draw is what prompted the Raspberry Pi foundation to switch to a type C connector, which is a good thing as finding microUSB power supplies that can output over 3A can be a problem. They had to break backwards compatibility with the older charger, but at $8 the charger is pretty affordable. The only issue seems to be that the single shared resistor on the Pi means that 'smart' chargers for macbooks might not work with the Rpi 4B, but Eben Upton has already clarified that this will be soon fixed.

For testing, the voltage across the 5V and ground pins was noted as different peripherals were connected, even with a 7 inch screen, keyboard, mouse and hard drive all attached, the voltage drop was only a little over 0.1V, which proves that the new power supply is more than up to the task.

Unfortunately, I did not have access to a USB ammeter or a wall mounted wattmeter so I could not measure the current draw. I tried splicing the charger to connect the multimeter in series, but the wire is wrapped around the core and it wasn't possible to do, I didn't want to risk bricking a charger. I will update the roadtest once I get access to a USB ammeter.

6.) Setting up the Phoronix Test Suite

Real world use case scenarios are great, but they are often prone to differing slightly or greatly depending upon the testing conditions and quality of peripherals used. This is where synthetic benchmarks shine, they provide a standardised way to test different components of the hardware. I ran several benchmarks that put the Raspberry Pi 4B through its paces - Its memory, Processor, graphics and its usability for machine learning were some of the things I tested.

That being said, the Raspberry Pi is not an easy device to benchmark. Many of the popular benchmarks such as Cinebench, CrystalDiskmark etc are written for Windows only machines, and wherever Linux alternatives exist, they are often written for the x86 counterparts and not ARM based processors. That being said, I found the Phoronix test suite (link: https://www.phoronix-test-suite.com/ ) to be amazing. It had an easy installation process, an intuitive CLI interface and the ability to graph the benchmarks automatically and stack them to execute one after the other - this was important as several of the benchmarks took very long to execute! It was here that the difference between the Raspberry Pi 4B and its predecessors becomes the most apparent, the Rpi 4B absolutely blows the other boards out of the water! All of the benchmarks took aroun ~5 hours to complete on the Raspberry Pi 4B, while they took over 15 hours to run on the 3B+. Setting up the Phoronix Test Suite was done as follows:

// Download the Debian version from the Phoronix test suite website.
//Click on the .deb file in the downloads folder (or wherever the file is downloaded)
// The GNU Multiple Precision Library and PHP is required for the test suite to work.
$sudo apt-get install php
//after downloading the .tar.xz file from the GNU MP website, to install we do the following:
tar xzf file.tar.gz 
./configure
make
sudo make install

Please do not make the mistake of running the universal test suite - this will make the software attempt to download packages for ALL of its 250+ tests, and you will run out of space on your MicroSD card, I learnt this the hard way. Generally avoid running suites as well, most of them take really long and can be redundant. It is far better to run individual benchmarks.

7.) Benchmarking RAM

Just about everyone knows that the Raspberry Pi 4B has more memory than its predecessors, there are 2GB and 4GB versions. A lesser stated but equally important fact is that the memory has been upgraded from LPDDR2 to LPDDR4. This has very important implications for the usability of the Pi, and even the 1GB version should perform better memory wise than its predecessors. During my testing I found the difference really apparent in applications which cause the RAM to spike, such as opening Chromium tabs. Almost everything felt smoother.

The RAMspeed SMP Benchmark ( Github: https://github.com/cruvolo/ramspeed-smp ) which is described as:

This command line utility measures effective bandwidth of both cache and memory subsystems. It has been written entirely in C for portability purposes, though benchmark routines are also available in several assembly languages for performance reasons.

The Raspberry Pi 4B's numbers for Integer Add and Integer Copy are given in the first image, they are almost 2x -3x of the figures of the 3B+ given in the second image. This is due to the LPDDR4 memory.

Similarly, the results for Integer-Scale and Floating-Point Add for the 4B is given in the First image, it is almost 2x of the 3B+ given in the second Image.

After running RAMspeed SMP, I ran Tinymembench (Github: https://github.com/ssvb/tinymembench  ) which is described as:

This is a simple memory benchmark program, which tries to measure the peak bandwidth of sequential memory accesses and the latency of random memory accesses. Bandwidth is measured by running different assembly code for the aligned memory blocks and attempting different prefetch strategies.

The results of Tinymembench for 4B and 3B+ are given as follows:

Again, the gulf in performance in the 4B and 3B is pretty pronounced, with the 4B being 2X of the 3B+

8.) Coremark

The Raspberry Pi 4B is the first Pi to incorporate a 28nm process node. The Processor has changed to a high performance Cortex A72 from A53 on the 3B+. There is a 100 Mhz bump in speed as well.

To test the CPU, I used Coremark (Link: https://www.eembc.org/coremark/ ). The description of Coremark on their website is:

EEMBC’s CoreMark® is a benchmark that measures the performance of microcontrollers (MCUs) and central processing units (CPUs) used in embedded systems. Replacing the antiquated Dhrystone benchmark, Coremark contains implementations of the following algorithms: list processing (find and sort), matrix manipulation (common matrix operations), state machine (determine if an input stream contains valid numbers), and CRC (cyclic redundancy check). It is designed to run on devices from 8-bit microcontrollers to 64-bit microprocessors.

The CRC algorithm serves a dual function; it provides a workload commonly seen in embedded applications and ensures correct operation of the CoreMark benchmark, essentially providing a self-checking mechanism. Specifically, to verify correct operation, a 16-bit CRC is performed on the data contained in elements of the linked-list.

To ensure compilers cannot pre-compute the results at compile time, every operation in the benchmark derives a value that is not available at compile time. Furthermore, all code used within the timed portion of the benchmark is part of the benchmark itself (no library calls).

Once again, the difference is pretty apparent. Although there was a rather large deviation in the 4B compared to 3B+ (I wonder why?) nevertheless even the lowest scoring iteration scored more than the 3B+'s highest.

9.) Sysbench Memory and CPU

From Sysbench (github: https://github.com/akopytov/sysbench )

sysbench is a scriptable multi-threaded benchmark tool based on LuaJIT. It is most frequently used for database benchmarks, but can also be used to create arbitrarily complex workloads that do not involve a database server.

10.) H.264 Video Encoding Test

The Pi 3B+ has hardware based encoding and decoding for H.264 video, this was dropped in favor of H.265 support on the 4B. Sadly the benchmark I did didn't seem to take advantage of the hardware on either, and I could not get the H.265 test to run on the 4B for some reason. Even then, the Pi 4B was twice as fast as the 3B+. The 4B had an average of 4.72 frames per second, compared to the 2.14 frames per second of the 3B+.

11.) GLMark2 Graphics

The Graphics chipset was updated on the Raspberry Pi 4B to Videocore VI clocked at 500 MHz. The 3B+ had Videocore IV. I ran the GLMark2 (Github Link: https://github.com/glmark2/glmark2 ). The test renders animations on the screen and while both the 4B and 3B+ failed to complete the test (Error LLVM: Ran out of memory), the 4B rendered animations faster, more smoothly and almost completed the test. I am sure that if I had the 4GB version of the board, the test would be completed. I have attached a few screenshots of the rendering:

12.) Thermal Characteristics and throttling management

The heat generated by the 4B has been a talking point. It is said to run hotter than its predecessors and for many throttling was a real concern. The Raspberry Pi's now come with a heat spreader, and many people go the extra mile and install a heatsink and fan to further reduce the temperatures. To test out if these are actually useful, or if the Pi throttles under heavy load, I used the following commands:

//To measure internal CPU temperature
/opt/vc/bin/vcgencmd measure_temp
//To measure CPU Clock speed
/opt/vc/bin/vcgencmd measure_clock arm
//To measure CPU Voltage
/opt/vc/bin/vcgencmd measure_volts
//To measure CPU Throttle, if value is not zero something is wrong
/opt/vc/bin/vcgencmd get_throttled

While running CPU Intensive Benchmarks such as Tensorflow, The following temperatures and core utilisation % was observed:

82 degrees Celsius is no Joke, the values of clock speed, voltage and throttle indicated that there was some throttling happening when the temperatures went over 80 degrees.

I wanted to see the effectiveness of the heat spreader, I decided to rig up a digital thermometer by connecting a BMP180 temperature-pressure sensor to my Arduino Mega and using the serial monitor to measure temperatures outside the heatsink. I realize that this might not be the most scientific way of doing things, but it worked well enough.

I decided to check if a small 5V GPIO fan could reduce CPU temperatures on the Pi, and boy did it make a difference. There was a 12 degree celsius difference (it went from 81 to 69 C) which is very important as the Pi starts to throttle to 600 Mhz mode over 80 degree Celsius.

From the above figures, we can infer that the 4B does indeed run hotter than its predecessor. Although the 4B handles high temperatures very well. The Tensorflow benchmarks took a long while to complete and the Pi ran without a hitch at 81 degree Celsius without a hitch (Other than the overheating icon warning).

13.) Machine Learning Frameworks - Tensorflow, NumPy Benchmark and Scikit-learn

There is an increasing interest in 'Edge' and 'Fog' computing as an alternative to purely cloud based computing, and why not? using the raspberry pi to read data from sensors and just pushing it to the cloud is not taking full advantage of the processing power it offers. One particular area of interest is using the Pi to run machine learning models - to take data from sensors and process it locally. I've worked on a few projects like this, and its great. To test out how good the Rpi 4B is for ML compared to 3B+, I ran three popular ML based benchmarks - Tensorflow, Numpy Benchmark and Scikit-learn. Even before I could see the results, it was very obvious that the 4B is much faster than the 3B+, running the whole battery of benchmarks took 15 hours on the 3B+, while the 4B took less than 5 hours. The results of the benchmarks are compiled below:

Tensorflow, Raspberry Pi 4B vs 3B+

From wikipedia:

TensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks.

The Raspberry Pi 4B is about twice as fast as the 3B+ on this benchmark. Lower scores are better and the 4B scores 685 compared to 1314 for the 3B+. There were more deviations in the 4B score than 3B+.

NumPy Benchmark, Raspberry Pi 4B vs 3B+

From Wikipedia:

NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.

Again, the difference between the two boards is pretty apparent. Lesser time is better, and the 4B's time is less than half of the 3B+

Scikit-learn, Raspberry Pi 4B vs 3B+

From Wikipedia:

Scikit-learn (formerly scikits.learn and also known as sklearn) is a free software machine learning library for the Python programming language. It features various classification, regression and clustering algorithms including support vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy.

Here, the gulf is very wide. The Pi 4B takes only a quarter of the time as 3B+.

14.) Common Software install times and browsing

To test out real world use case scenarios, I installed a couple of commonly used software and noted down the dowload and install times. I then uninstalled the software, and proceeded to redo the process. The times are given in the graph below:

To check the general browsing experience on both, I opened 6 chrome tabs on both and noted down the RAM usage. Its difficult to quantify but the 4B felt much smoother at switching between tabs, there was less jitter and lag and in general, it made for a more pleasant browsing experience. The free -h command can be used to check the amount of RAM at any given time.

15.) Powering the Raspberry Pi 4B - Powerbank and PiJuice testing and Battery drain test

Due to the increased current draw and overall greater power consumption, I had my doubts over the portability of the RPi 4B. I usually depend on generic off the shelf power banks to run my setups in a portable manner, and my 20,000 mAh Mi Power bank was only rated for 2.4A maximum at 5V. This is noticeably lesser than the 3A at 5.1V supplied by the official power supply. I needed to check the system at full load to know if there was a chance of the Pi abruptly shutting down incase the current was insufficient, but I faced no problems running it from the power bank at even 100% CPU load. This shows that the RPi 4B can operate in a 12W envelope.

I also tested a popular HAT, the PiJuice with a really small 1800 mAh battery. Although the battery drained rather quickly with everything attached (about an hour), the Pi had absolutely no problems running off it. I also had a 5 inch WaveShare LCD display with me, which worked great with the 4B! I used the Pi Power app on the google play store to estimate the run time.

16.) Building a project with the Raspberry Pi - Portable Health Monitoring system

I regularly attend hackathons, and I decided to take the Rpi 4B out for a spin. I built a portable health monitoring system where the Pi was connected to 2 arduinos which had various sensors - ECG, EMG, KY039 Pulse sensors, MAX30102, BMP180, DS18B20 and a ILI9341 display. There was also a 5 inch display to see the plotted data live.

I ran a few ML models with data sets from Kaggle too. Overall, the Pi 4B handled everything with ease. Usually I have to switch between using my PC to browse chrome and using the Pi to install software directly, but the 4B did not feel sluggish even with multiple tabs being open and updates being run in the background. The Pi was also sending serial data to an app. It handled all these with ease.The documentation for the project can be found on the following Github Links:

1.) https://github.com/amogh-jrules/Y-tality

2.) https://github.com/AmanL04/health-app-hashcode

In the image above, you can see the live rectified ECG and EMG graphs on the 5 inch LCD, while the smaller ILI9341 displays other data. It was a fun project, and the 4B made everything a lot quicker.

17.) Conclusions

I'd like to thank rscasny and danzima for being patient with me for this RoadTest. I loved the experience, and there was definitely a learning curve involved with benchmarking and I learnt a lot along the way. The Raspberry Pi is a great piece of hardware, with the only downsides being the need to get a new PSU and a new MicroHDMI cable. I hope my delayed submission does not affect my chances for future RoadTests, but I was really short on time due to college and other commitments.

Thank you to the element14 community for encouraging me, your comments and kind words on my Summer Code Club challenge posts are what motivated me to work on this Roadtest .