Software roadtests?

I recently got a 30 day trial license of Keysight's "PathWave RF Synthesis". The tool is pretty amazing and I'll probably miss it once the trial ends. This got me thinking that it would be nice if software could also be roadtested. I suspect some of these tools would get as many applications as test equipment (i.e, Who wouldn't like to own a license of Altium Designer?).

Just realized Pathwave RF is the new name for Genesys.. fantastic software, but very few people would be able to review it in a way to do any justice. The users of the software won't necessarily be people who can or are prepared to write it up, otherwise there would be plenty of material on it. I don't want to sound negative, because I too would love the opportunity to review and own such software, but it's setting up the manufacturer, and the reviewers, for disappointment if expectations are very different from reality once they see the review or the software respectively, where there's a mismatch between what is required and what occurs during the process.

Just realized Pathwave RF is the new name for Genesys.. fantastic software, but very few people would be able to review it in a way to do any justice. The users of the software won't necessarily be people who can or are prepared to write it up, otherwise there would be plenty of material on it. I don't want to sound negative, because I too would love the opportunity to review and own such software, but it's setting up the manufacturer, and the reviewers, for disappointment if expectations are very different from reality once they see the review or the software respectively, where there's a mismatch between what is required and what occurs during the process.

Bad reviews are always a risk for companies, but how much of a risk is it really? Lets take RF Synthesis as example. In the worst case, if the review has no value to them, unless the software ends up being sold to someone who otherwise would have payed for the software, the company's only loss was the time one of their employees had to spend to get the software into the program.

On the other side, software usually doesn't get as much exposure as hardware, so it eventually can have higher impact than hardware roadtests. If you check eevblog for instance, zillions of posts can be found on hardware, but not so much on software. Also many times the only place where you can find info on software is in the manufacturer's page, a little bit more exposure eventually has a good impact.

I think this is a low bar, if the reviews are poor or miss the point. Taking your example, what if someone who reviews the software, spends time on the unboxing and UI, and then complains that it is useless compared to Altium Designer? That's a disappointment for the reviewer, the manufacturer, and everyone reading it, and who benefits? Just because the RF Synthesis software can allow you to draw circuit traces, it's a means to an end, to get the job RF task done, not to replace PCB CAD software. If you can find the right reviewers and find them the training to use the software (this is not a UI issue, beyond a superficial one - the UI merely needs to be adequate - it still won't replace training) then this is great.

With more popular tools such as PCB CAD as you mention, there have been RoadTest reviews in the past - I definitely recall an EAGLE CAD roadtest, and I think it was popular, and generated useful content. I just don't think it's the same with a lot of the Keysight software, that definitely needs training or finding the types of engineers who not only know how to use it, but are also willing to write up a report. I think there was a Matlab RoadTest a while back too, and that has slightly more mass appeal too, and most people should be able to use a Maths related package to do useful work.