How Long Would It Take for You to RoadTest the Raspberry Pi 4 Model B (prep, testing and writing)?

Once upon a time, before a Raspberry Pi release we would get hold of the products early, and they would be passed to me to put through their paces. While I didn't necessarily test them to their fullest because we didn't have all of the datasheets, they were typically early engineering models with pre-release operating systems so not all functionality was there, I was able to compile linux software and run it through some benchmarks.

That would take most of the time, and sometimes a bit of back and forth with people internally and externally if I encountered any problems.

Since that'd take 4-8 hours of the day depending on the speed of the hardware and having to reformat microSD cards, that could take a couple of days to get the data, and then a bit longer to format the results into a reasonable report. So 16 to more than 40 hours can be easily realistic considering there's even more to test, and that's the electrical parts, connectivity, functionality, protocol responses, and the fact that it can run practically as a computer as well as connecting hardware to it.

There can be a lot to test and it needs a lot of scope definition - do you simply benchmark the processor and how many flops it can do? But what about the GPU and does it support the full OpenGL specification? What about the speed of SPI and utilising the clocks? Now what about the pinmux? Oh wait, there's been a firmware update, does that affect anything? Well now it can network boot and I could do with a PXELinux setup to network boot it.

Expansive and capable of a full RoadTest.

Once upon a time, before a Raspberry Pi release we would get hold of the products early, and they would be passed to me to put through their paces. While I didn't necessarily test them to their fullest because we didn't have all of the datasheets, they were typically early engineering models with pre-release operating systems so not all functionality was there, I was able to compile linux software and run it through some benchmarks.

That would take most of the time, and sometimes a bit of back and forth with people internally and externally if I encountered any problems.

Since that'd take 4-8 hours of the day depending on the speed of the hardware and having to reformat microSD cards, that could take a couple of days to get the data, and then a bit longer to format the results into a reasonable report. So 16 to more than 40 hours can be easily realistic considering there's even more to test, and that's the electrical parts, connectivity, functionality, protocol responses, and the fact that it can run practically as a computer as well as connecting hardware to it.

There can be a lot to test and it needs a lot of scope definition - do you simply benchmark the processor and how many flops it can do? But what about the GPU and does it support the full OpenGL specification? What about the speed of SPI and utilising the clocks? Now what about the pinmux? Oh wait, there's been a firmware update, does that affect anything? Well now it can network boot and I could do with a PXELinux setup to network boot it.

Expansive and capable of a full RoadTest.

Agreed, and if anything happens to not go perfectly - oops wrong software version, oops wrong pin, etc etc - then it’s easy to waste several hours just trying to figure out what is going wrong.

A big mistake people tend to make when estimating time is that they only estimate things going right, not factoring in the time required for figuring out when things go wrong.

Agree. Often times testing means intentionally trying things that may cause problems to make sure they dont. I am a champion of breaking software at work, even when I'm not trying to. :-) 

I anticipate that a product is fairly well tested before requesting a road test, unless it is clearly stated the product is an alpha or early beta release. 

Most road tests seem to be Charlie releases, and that's why a supplier benefits from the experience diversity of the E14 community and some unique project ideas to try. Those unique ideas do take time to troubleshoot and blossom especially if they exercise functions that may not be tested well enough yet.

And even "oops I was tired and I transferred a 500mb file instead of a 1tb file, so that's why the numbers are off for the file transfer benchmark!" so you have to re-do the whole test again, for science.

How many wasted hours does the term "oops" cover? As in I took a job where the company folded after 6 months - oops  

You sure are expensive :-)

You’re bad luck, Doug 

It didn't happen to me - I'm just wondering if that is still an oops...

that could be a wasted learning curve, but with a decent severance package and then employment insurance, the extra paid time off might just make up for it