Example of a piece o’ crap, useless review


I posted this originally in the Analog section as the review is of a turntable, but the point extends more generally and thought more people would get to see it here.

I’ve harped on how crappy and useless many “professional” reviews are because they lack rigor and omit critical information.  This one is from TAS that is a main offender of pumping out shallow/unsupported reviews, but most of the Euro mags among others are guilty of this too IME.  One key giveaway that a review is crap is that after reading it you still have little/no real understanding of what the piece under review actually sounds like or if it’s something you’d like to consider further.  I mean, if a review can’t accomplish those basic elements what use is it?  This review is so shallow it reads like it could’ve been written by someone who never even listened to the review sample and just made it up outta thin air.  In addition to failing on this broad level, here are some other major problems with the review:

- There is no info regarding any shortcomings of this “budget” turntable — everything is positive.  Sounds like it was perfect, ehem.

- There are no comparisons to another product in the same general price category or anything else.

- The reviewer doesn’t even share what equipment is in his reference system so we can at least infer what he may have based his impressions on.

In short, in addition to this review being so bad/useless for all the reasons stated it actually reads more like advertisement for the product than an actual unbiased review.  I can think of nothing worse to say about a review, and sadly many reviews out there are similarly awful for the same reasons.  Sorry for the rant, but especially as a former reviewer this piece of garbage pushed all my buttons and really ticked me off.  What say you?

https://www.theabsolutesound.com/articles/sota-quasar-turntable-and-pyxi-phonostage/

soix

So in the end it is a business for the professional reviewers. If they were to give critical reviews about specific products other manufacturers will avoid them as it could stop the business for a small manufacturer. So in mutual dependency we only get great reviews. Other comment in the past from Stereophile was that they would not review bad sounding products in the first place. But how would you know beforehand?

@tom2015 I’m pasting a post from another thread that relates to your comment and that may shed a little more light on the issue…

 

Over 17 years I only wrote one negative review, and I was not opposed to writing more if warranted. The thing is, for a product to make it to the level of getting a review among the thousands of products out there it needs to have some positive and usually even some special buzz about it. So almost every component that gets reviewed has been well vetted by many users and others so bad sounding equipment just never tends to even get reviewed. In short, the system self selects very good sounding gear right from the start. Also, the major manufacturers know what they’re doing, and it’s almost non existent that something they’d release to the public sounds bad. So this is really why you rarely read a bad review.

That said, that’s where the “art” of reading between the lines in reviews becomes important. Unlike the crappy “review” I mentioned in this thread, most good reviews will include a paragraph or two near the end just before the conclusion (that’s where I always put it) where a reviewer will elaborate on any shortcomings/limitations or qualities that he may have alluded to during the review that seem a bit off or things the reader may want to hone in on while listening or comparing to other equipment. THIS IS CRITICAL INFO SO READ IT CAREFULLY! So, while the product overall may be very good, these would be the potential “negatives” that the reviewer feels are important to highlight but they’re not nearly enough to trash the product and throw the baby out with the bath water. This again is why product comparisons are so important because these potentially aberrant qualities can be analyzed on a relative basis versus another known product and provide a very useful perspective on any potential issues. Also, reviewers aren’t all-knowing oracles and what we may think is a shortcoming for our system/tastes may well not be the case for many other potential customers, so better to just point out what we heard objectively and let the reader decide if that’s something they’d still like to explore further. Last, most audio manufacturers are not rich and do what they do for the love of it and many others might be relatively new, and a bad review could literally sink a company. Now, if the product outright sucks so be it although it’s highly unlikely that product would’ve made it to review anyway as mentioned above, but if a product isn’t perfect yet shows some promise as a reviewer you’ve gotta weigh the extent of the shortcomings and if they’re bad enough to potentially put the company out of business. So there are many things to consider as a reviewer. I wasn’t on the business end so can’t speak to manufacturers threatening to not send any more gear if they get a bad review, but I never heard of anything like that although it may happen. Anyway, I hope this sheds a little light on why there are so few negative reviews. It’s really not a conspiracy in my experience.

If a reviewer likes something I own and also like, they're brilliant. I like old Herb also because he has a fun style, but he doesn't use MY ears which means there is no general "baseline." There's no "baseline" to any artistic endeavor by the way...I love jazz, and most people don't. Amazing to me...I like a lot (not all) of abstract art, and most others don't and actually get upset by it. I don't care.

Very important post ! Thanks...

It is why i always recommended reading between lines and statistical analysis of each one of the acoustic factors in all official reviewers but also all users reviews...

 

I will give another reason which is not in your post about why there is not often negative review of a product apart this selective sifting process by selected users and the designer himself .

Any reviewer with experience i imagine know as i know that the gear design quality matter but the synergy and especially the mechanical, electrical and the acoustical working dimensions and their embeddings controls make a world of difference as well as each user specific needs...Then why bashing a product because of his design trade-off choices if it is not a disastrous design to begin with ?

 

 

 

 

But there is exception in this selective process sometimes a very not so well product can get acclaimed for other reason than audio....😊

i will not give my example ...It will be too inflammatory...😁

Anyway your post is very important read for all ....

 

 

So in the end it is a business for the professional reviewers. If they were to give critical reviews about specific products other manufacturers will avoid them as it could stop the business for a small manufacturer. So in mutual dependency we only get great reviews. Other comment in the past from Stereophile was that they would not review bad sounding products in the first place. But how would you know beforehand?

@tom2015 I’m pasting a post from another thread that relates to your comment and that may shed a little more light on the issue…

 

Over 17 years I only wrote one negative review, and I was not opposed to writing more if warranted. The thing is, for a product to make it to the level of getting a review among the thousands of products out there it needs to have some positive and usually even some special buzz about it. So almost every component that gets reviewed has been well vetted by many users and others so bad sounding equipment just never tends to even get reviewed. In short, the system self selects very good sounding gear right from the start. Also, the major manufacturers know what they’re doing, and it’s almost non existent that something they’d release to the public sounds bad. So this is really why you rarely read a bad review.

That said, that’s where the “art” of reading between the lines in reviews becomes important. Unlike the crappy “review” I mentioned in this thread, most good reviews will include a paragraph or two near the end just before the conclusion (that’s where I always put it) where a reviewer will elaborate on any shortcomings/limitations or qualities that he may have alluded to during the review that seem a bit off or things the reader may want to hone in on while listening or comparing to other equipment. THIS IS CRITICAL INFO SO READ IT CAREFULLY! So, while the product overall may be very good, these would be the potential “negatives” that the reviewer feels are important to highlight but they’re not nearly enough to trash the product and throw the baby out with the bath water. This again is why product comparisons are so important because these potentially aberrant qualities can be analyzed on a relative basis versus another known product and provide a very useful perspective on any potential issues. Also, reviewers aren’t all-knowing oracles and what we may think is a shortcoming for our system/tastes may well not be the case for many other potential customers, so better to just point out what we heard objectively and let the reader decide if that’s something they’d still like to explore further. Last, most audio manufacturers are not rich and do what they do for the love of it and many others might be relatively new, and a bad review could literally sink a company. Now, if the product outright sucks so be it although it’s highly unlikely that product would’ve made it to review anyway as mentioned above, but if a product isn’t perfect yet shows some promise as a reviewer you’ve gotta weigh the extent of the shortcomings and if they’re bad enough to potentially put the company out of business. So there are many things to consider as a reviewer. I wasn’t on the business end so can’t speak to manufacturers threatening to not send any more gear if they get a bad review, but I never heard of anything like that although it may happen. Anyway, I hope this sheds a little light on why there are so few negative reviews. It’s really not a conspiracy in my experience.

 

I read (skim through) reviews to see what music they are using to evaluate a system. Found some good stuff that way. Not much use for them otherwise.