Should a reviewer keep a $20K Oscilloscope?

As a person whose name was bound to turn up a few times in this thread, I’ve resisted the temptation to jump in and offer my opinion right off the bat scottiebabe , as it’s not always been welcomed. But after seeing the multitude of viewpoints that have already arisen, I think it’s only fair that I chime in and give my perspective, as someone who reviews things on this site as a RoadTester, and as a person who also reviews items by direct contract – both for keeps and under loan agreements.

As usual, I’m probably not going to be able to keep this reply brief, so I’m going to have to split it up into parts.

Why the sponsor might want you to keep the item?

• Many sponsors know that people like to do as they see – if you’ve used X product in Y application and show it online, people will probably gravitate to buying X product if they’re interested in doing Y.
• Many products are not “complete” at the outset – new features are released over time with firmware updates and unlock codes and some companies are nice enough to keep in touch after a RoadTest and offer you these features for you to do an “update” or “refresh” of your review.
• Many sponsors also know that perceived market share may translate into sales – if you see many projects with scope screenshots that say “Brand G”, then you probably will think Brand G products are better.
• It may be less work for them to give items away as part of a marketing budget than it is for the internal accounting required to move an item across borders and loan it out.
• If you report a bug and they take the time to fix it, it's best if you still have the equipment and the set-up to replicate the bug to ensure that it is truly fixed.

Why you might want to keep the item?

• It’s a good product and you’ve invested time to learn it – so losing it means throwing away a lot of knowledge that you have spent time to build.
• It’s a good reward for the hours invested – my RoadTest reflection from 2020 shows that my RoadTest procedures requires anywhere from 27-71 hours of “attended” time, with up to 500 hours of “unattended” time (automation) to obtain the level of results I get. For that, I have the pleasure of sleeping with fans blaring, paying for the energy bills and mentally being invested in the product. I also spend time communicating with the manufacturer, seeking support etc. You may feel the value of the product (nominally) exceeds what your time is worth, but such a product is likely to be a high-margin product of which a sale would be significantly profitable for the company. If you do many RoadTests of different types, it averages out (statistically speaking).
• You want to truly review it – by that, I mean attempt to break it open for a teardown, or do risky things which could damage the equipment. You don’t have this freedom especially with a loaner.
• You don’t want the legal responsibility for a loaner – I’ve signed a few loan agreements and they’re not simple. There are penalties for damage and penalties for late return that you are personally responsible for as the signatory. You are also responsible for transport damages as well. I’ve had equipment on loan that arrived and failed within hours – it was a bit of a hassle, in spite of all the verbal agreements, to iron it out and ensure that I was not responsible for damages (as I had not actually caused the failure).
• The hassles of transportation costs and import/export taxes can make it uneconomic to send the item around and you may gain more benefit from keeping it. For example, I had to send the Keithley 2450 SMU back to Singapore for them to on-ship it to USA and back to me because of a manufacturing defect. The cost to me, out of pocket, was about AU$334 outbound and AU$110 back in to the country for a warranty repair. Imagine it’s for the full cost of the item where I don’t get a duty waiver on return (as it was coded as repair return) and it’s likely that anywhere from 20-50% of the item value can be eaten up by shipping and taxes. The risk of damage is also present, along with delays.

Why the sponsor might not want you to keep the item?

• Sometimes items are very expensive, so giving them away is not going to do their bottom line any good. But if they’re offering to give it away – I think they’ve already decided that the benefit outweighs the costs.
• They may not have many items on hand and want to give others a chance to evaluate for purchase or for reviews – I’ve had some items promised to me for loan that never eventuated because other “higher-ranking” potential customers kept jumping the queue.
• Keeping an item for longer may give you more opportunities to uncover deficiencies – but those who are confident in their product will probably not mind.

Why you might not want to keep the item?

• It might not be very useful to you and someone else could benefit from it more – I’ve certainly given away my fair share of equipment. So far, the tally of equipment donated away rests at two oscilloscopes, three power supplies, five digital multimeters, an insulation resistance tester and a thermocouple reader. Quite a few of these were RoadTest products or equipment purchased for my PhD which had specifications below other products which I have since attained. As I felt I wouldn’t benefit from keeping more than I can use, they have been sent away to forum members, colleagues and educational institutions overseas who are less fortunate (through colleagues willing to handle the paperwork and pay for freight).
• It could result in income tax implications.
• Wife acceptance factor (where applicable) is a problem. Or just generally, there’s not enough space.
• Guilt that you might not have the right experience to make the most of it – I think this is imposter syndrome and I get this from time to time. But over time, I find that having more capability is usually better than having less, so perhaps this is “space for you to grow.”
• It could be an early-revision device, complete with bugs, patches and no warranty in case things go wrong.

The competition for big-ticket RoadTest items is usually fierce. While the numbers may not be the same as the old days, I feel that the quality of applications has only grown over time and the quality of test methodologies has similarly become more comprehensive. One doesn’t expect to win without a relatively all-encompassing proposal, so potential applicants are probably wisening up to the fact that this is not a game-of-chance, so they're saving their effort for things that are more likely to result in a win. Also, keep in mind, last-minute applications are common - I used to submit my application only within the last three days ...

But I guess the crux of the thread really is the problem of uneven RoadTest compensation – many smaller RoadTests are giving out boards of $100 or less and suddenly a whopper of a $20k RoadTest comes along. To that end, I feel there is no simple solution – while you may feel that the $20k RoadTest report is not 200 times more work, or 200 times more valuable, I guess that’s a judgement you are free to make and I wouldn’t necessarily dispute. But what I will say is that the marketing departments have made their decision on value and you can do the same by refraining from RoadTesting items that you don’t feel are worthwhile in either fashion. Giving away different items in exchange for reviews comes with the pitfalls of losing your investment in terms of time/knowledge but also potentially can be seen as a paid-review (rather than in-kind).

I guess this is where the "Experimenting With Design Challenges" take the approach of awarding a desirable prize for using a more "pedestrian" device. It's something that I've participated and benefited from and I don't think it's a bad model but could be a little problematic for review-centred content which is supposed to be impartial.

I don’t think loaning out items is as simple of a fix as it seems. Logistically, it’s expensive just in shipping costs and duty. The legal responsibility for the loan will also be an issue – I’ve signed a few in my time and that’s not been fun when things break. The shackles are also on you as to what you’re allowed to do – no breaking it or taking risks. Some companies I’ve spoken to have taken months to get loaners sorted because the internal accounting and logistics is more complex.

So, my advice scottiebabe  is to apply for the RoadTest unashamedly and guilt-free. Know that the best candidate will be picked. As a full-disclosure, I will not be applying for the aforementioned RoadTest, although I will be reviewing the unit sooner-or-later. May luck be on your side.

- Gough

As a person whose name was bound to turn up a few times in this thread, I’ve resisted the temptation to jump in and offer my opinion right off the bat scottiebabe , as it’s not always been welcomed. But after seeing the multitude of viewpoints that have already arisen, I think it’s only fair that I chime in and give my perspective, as someone who reviews things on this site as a RoadTester, and as a person who also reviews items by direct contract – both for keeps and under loan agreements.

As usual, I’m probably not going to be able to keep this reply brief, so I’m going to have to split it up into parts.

Why the sponsor might want you to keep the item?

• Many sponsors know that people like to do as they see – if you’ve used X product in Y application and show it online, people will probably gravitate to buying X product if they’re interested in doing Y.
• Many products are not “complete” at the outset – new features are released over time with firmware updates and unlock codes and some companies are nice enough to keep in touch after a RoadTest and offer you these features for you to do an “update” or “refresh” of your review.
• Many sponsors also know that perceived market share may translate into sales – if you see many projects with scope screenshots that say “Brand G”, then you probably will think Brand G products are better.
• It may be less work for them to give items away as part of a marketing budget than it is for the internal accounting required to move an item across borders and loan it out.
• If you report a bug and they take the time to fix it, it's best if you still have the equipment and the set-up to replicate the bug to ensure that it is truly fixed.

Why you might want to keep the item?

• It’s a good product and you’ve invested time to learn it – so losing it means throwing away a lot of knowledge that you have spent time to build.
• It’s a good reward for the hours invested – my RoadTest reflection from 2020 shows that my RoadTest procedures requires anywhere from 27-71 hours of “attended” time, with up to 500 hours of “unattended” time (automation) to obtain the level of results I get. For that, I have the pleasure of sleeping with fans blaring, paying for the energy bills and mentally being invested in the product. I also spend time communicating with the manufacturer, seeking support etc. You may feel the value of the product (nominally) exceeds what your time is worth, but such a product is likely to be a high-margin product of which a sale would be significantly profitable for the company. If you do many RoadTests of different types, it averages out (statistically speaking).
• You want to truly review it – by that, I mean attempt to break it open for a teardown, or do risky things which could damage the equipment. You don’t have this freedom especially with a loaner.
• You don’t want the legal responsibility for a loaner – I’ve signed a few loan agreements and they’re not simple. There are penalties for damage and penalties for late return that you are personally responsible for as the signatory. You are also responsible for transport damages as well. I’ve had equipment on loan that arrived and failed within hours – it was a bit of a hassle, in spite of all the verbal agreements, to iron it out and ensure that I was not responsible for damages (as I had not actually caused the failure).
• The hassles of transportation costs and import/export taxes can make it uneconomic to send the item around and you may gain more benefit from keeping it. For example, I had to send the Keithley 2450 SMU back to Singapore for them to on-ship it to USA and back to me because of a manufacturing defect. The cost to me, out of pocket, was about AU$334 outbound and AU$110 back in to the country for a warranty repair. Imagine it’s for the full cost of the item where I don’t get a duty waiver on return (as it was coded as repair return) and it’s likely that anywhere from 20-50% of the item value can be eaten up by shipping and taxes. The risk of damage is also present, along with delays.

Why the sponsor might not want you to keep the item?

• Sometimes items are very expensive, so giving them away is not going to do their bottom line any good. But if they’re offering to give it away – I think they’ve already decided that the benefit outweighs the costs.
• They may not have many items on hand and want to give others a chance to evaluate for purchase or for reviews – I’ve had some items promised to me for loan that never eventuated because other “higher-ranking” potential customers kept jumping the queue.
• Keeping an item for longer may give you more opportunities to uncover deficiencies – but those who are confident in their product will probably not mind.

Why you might not want to keep the item?

• It might not be very useful to you and someone else could benefit from it more – I’ve certainly given away my fair share of equipment. So far, the tally of equipment donated away rests at two oscilloscopes, three power supplies, five digital multimeters, an insulation resistance tester and a thermocouple reader. Quite a few of these were RoadTest products or equipment purchased for my PhD which had specifications below other products which I have since attained. As I felt I wouldn’t benefit from keeping more than I can use, they have been sent away to forum members, colleagues and educational institutions overseas who are less fortunate (through colleagues willing to handle the paperwork and pay for freight).
• It could result in income tax implications.
• Wife acceptance factor (where applicable) is a problem. Or just generally, there’s not enough space.
• Guilt that you might not have the right experience to make the most of it – I think this is imposter syndrome and I get this from time to time. But over time, I find that having more capability is usually better than having less, so perhaps this is “space for you to grow.”
• It could be an early-revision device, complete with bugs, patches and no warranty in case things go wrong.

The competition for big-ticket RoadTest items is usually fierce. While the numbers may not be the same as the old days, I feel that the quality of applications has only grown over time and the quality of test methodologies has similarly become more comprehensive. One doesn’t expect to win without a relatively all-encompassing proposal, so potential applicants are probably wisening up to the fact that this is not a game-of-chance, so they're saving their effort for things that are more likely to result in a win. Also, keep in mind, last-minute applications are common - I used to submit my application only within the last three days ...

But I guess the crux of the thread really is the problem of uneven RoadTest compensation – many smaller RoadTests are giving out boards of $100 or less and suddenly a whopper of a $20k RoadTest comes along. To that end, I feel there is no simple solution – while you may feel that the $20k RoadTest report is not 200 times more work, or 200 times more valuable, I guess that’s a judgement you are free to make and I wouldn’t necessarily dispute. But what I will say is that the marketing departments have made their decision on value and you can do the same by refraining from RoadTesting items that you don’t feel are worthwhile in either fashion. Giving away different items in exchange for reviews comes with the pitfalls of losing your investment in terms of time/knowledge but also potentially can be seen as a paid-review (rather than in-kind).

I guess this is where the "Experimenting With Design Challenges" take the approach of awarding a desirable prize for using a more "pedestrian" device. It's something that I've participated and benefited from and I don't think it's a bad model but could be a little problematic for review-centred content which is supposed to be impartial.

I don’t think loaning out items is as simple of a fix as it seems. Logistically, it’s expensive just in shipping costs and duty. The legal responsibility for the loan will also be an issue – I’ve signed a few in my time and that’s not been fun when things break. The shackles are also on you as to what you’re allowed to do – no breaking it or taking risks. Some companies I’ve spoken to have taken months to get loaners sorted because the internal accounting and logistics is more complex.

So, my advice scottiebabe  is to apply for the RoadTest unashamedly and guilt-free. Know that the best candidate will be picked. As a full-disclosure, I will not be applying for the aforementioned RoadTest, although I will be reviewing the unit sooner-or-later. May luck be on your side.

- Gough

Excellent insight, thank you. The wife factor made me laugh. 

The discussions have come off the rails a bit, altho good points are raised, a $20k Road Test seems an outlier. I've seen $2k Road Tests, those aren't the norm either. 

The Road Tests I've done are usually less than $100, the most expensive being less than $300.  I do know that the positive road test reviews have resulted in additional sales, word of mouth does get around, and having a applied product on hand to show off is golden. 

This particular $20k Road Tests specifically stated the Testing Goals
"Conduct a comprehensive set of performance tests in order to review its significant capabilities and features. The tests conducted should be of a highly technical nature that an experienced oscilloscope user would value. "

This requirement is one reason I expect the applicant count is fewer than normal for a really nice instrument.  I disclose I didn't apply because I didn't think I was a good candidate, and a strong candidate for this goal is special.