do blind tests have any practical significance


do the results of a blind or double tests have any usefulness ?

a blind ab test does not prove anything. it only states facts, namely that a certain percentage of listeners were able to distinguish the sound of one component from another.

there are many issues as to the validity of these tests.

the question, "can a listener detect the sonic difference(s) between components", is never answered definitely by one test.

suppose a blind test is administered 10 or more times to the same group of listeners. assuming the first test is replicated 9 more times, there is a chance that there may be an inconsistency between the outcomes of the test.

suppose a statistically significant percentage, say less than 60 percent, for a large sample of listeners, identifies differences between 2 components, what does that mean ?

what is the application of such a result for a prospective buyer of a component ?
mrtennis

Showing 8 responses by pawlowski6132

Blind tests are absolutely useful. I wish all reviewers would NOT know ahead of time the brand, cost, etc. of the component they're reviewing. I think marketing, reputation, price sets expectations that influence a review.

Double blind tests are even more useful. I would love to know if expensive components really sound better than inexpensive ones. Relative comparisons can be very useful. 80% of threads on this board start like this, "What's better X or Y?" Again a blind comparison would provide VERY objective opinions.
Edartford, if the evaluation is completely subjective and therefore useless, why bother reading any reviews at all??
Eldarford; Although I agree with your approach, I think most people gather the specifications of products from the manufacturers websites and read reviews to find out, simply put, if the gear sounds any good since dealers aren't always available.

I just happened upon this quote this morning in the February 2007 issue of "The Absolute Sound". It's a review of the Mark Levinson No. 436 Power Amp by Sue Kraft:

"So it was with much anticipation that I awaited the arrival of the Mark Levinson reviews. I was also quite excited at the prospect of hearing how these $12,500, 350Wpc brutes would mate with the stellar imaging and accuracy of a loudspeaker like the B&W 800D. Would my decade long wait to finally hear a pair of Levinson amps in my system meet my expectations? At the risk of ruining the ending...I'm afraid I knew the answer to that question nearly straight out of the box, and have heard nothing in the months to follow that would change my mind."

Come on. How can anyone argue that blindfolding Sue and telling her to review this amp would not result in a more objective review????? Wouldn't that be more helpful to the above mentioned "typical" reader I mentioned????
Mrtennis; are you serious?

Because, I think a high quality blindfold (preferably cotton) be administered properly to ensure the integrity of the test.

What do you think?
"Either something sounds good, or it doesn't..."

Correct, however, referring to my example, I will never know if these amps really sound good or not because SK is NOT giving an objective observation. Her predjudices and impartiality are clouding the review. Period. How can anyone dispute this????? She even comes right out and tells you.

I'm not questioning her integrity. She probably has lots. Or maybe none. I don't know and I don't purport to.

I'm commenting on the process which most reviewers are subject to.
Nrchy; You're missing the point. Sue can continue to review. Just don't tell her what's she's listening to. That's not absurd at all. Just say, Sue, you'll be listening to a solid state amp in your system. Tell us what you think.

I'm not sure how we keep missing each other on this. Doesn't this make sense?
Nrchy; point taken. It may not be practical. But, I hope you agree with me in theory at least.