Why Do So Many Audiophiles Reject Blind Testing Of Audio Components?


Because it was scientifically proven to be useless more than 60 years ago.

A speech scientist by the name of Irwin Pollack have conducted an experiment in the early 1950s. In a blind ABX listening test, he asked people to distinguish minimal pairs of consonants (like “r” and “l”, or “t” and “p”).

He found out that listeners had no problem telling these consonants apart when they were played back immediately one after the other. But as he increased the pause between the playbacks, the listener’s ability to distinguish between them diminished. Once the time separating the sounds exceeded 10-15 milliseconds (approximately 1/100th of a second), people had a really hard time telling obviously different sounds apart. Their answers became statistically no better than a random guess.

If you are interested in the science of these things, here’s a nice summary:

Categorical and noncategorical modes of speech perception along the voicing continuum

Since then, the experiment was repeated many times (last major update in 2000, Reliability of a dichotic consonant-vowel pairs task using an ABX procedure.)

So reliably recognizing the difference between similar sounds in an ABX environment is impossible. 15ms playback gap, and the listener’s guess becomes no better than random. This happens because humans don't have any meaningful waveform memory. We cannot exactly recall the sound itself, and rely on various mental models for comparison. It takes time and effort to develop these models, thus making us really bad at playing "spot the sonic difference right now and here" game.

Also, please note that the experimenters were using the sounds of speech. Human ears have significantly better resolution and discrimination in the speech spectrum. If a comparison method is not working well with speech, it would not work at all with music.

So the “double blind testing” crowd is worshiping an ABX protocol that was scientifically proven more than 60 years ago to be completely unsuitable for telling similar sounds apart. And they insist all the other methods are “unscientific.”

The irony seems to be lost on them.

Why do so many audiophiles reject blind testing of audio components? - Quora
128x128artemus_5

Showing 12 responses by edgewound

Harman Int'l uses blind testing quite frequently to develop cost effective products that the market will consume.

Audiophiles reject blind testing out of fear. Fear of what? It's pretty obvious. The Oz syndrome.
I’ve had a few customers...musicians...that are actually blind, unsighted.

The interesting thing about such people...especially musicians...is they actually listen with their ears. I’ve made repairs and adjustments for them based on whet they can hear, not what they can see. It’s rarely the most expensive or esoteric thing either. Simplicity works very well in this regard.
When I followed Andrew Jones’...an actual physicist by training/education... career from TAD/Pioneer to ELAC, I last heard his masterpiece TAD Reference One before he went to ELAC...where he showcased the diminutive Navis ARB-51... I was expecting diminutive sound from such relatively small drivers. It ended up like Spud Web against Shaq. Simply huge performance from such a small footprint. Was it the same visceral experience as the commanding presence of the $85,000/pair TADs to the $2,200/pair internally triamped ELAC ARB-51? No. Was it a difference worth the $81,000? Oh hell no. That would be a fun blind test demo to listen to, where you no nothing of what’s coming through the blind screen until the reveal, including pricing. Even Andrew was surprised at what he’s accomplished.
Wow...and actual engineer's posts removed because they don't agree with the narrative.

That's pretty juvenile, and petty. It's actually pretty disgusting.
jjss492,655 posts05-03-2021 9:27pmskypunk under new username

so little time, so much bile...

unfortunate
If you're referring to me, you would be so wrong. 

Probably not the first time.

peguinpower
29 posts
05-03-2021 11:54am
the biggest irony with these testing nerds is that they are usually the ones with the least exposure to a wide and varied spectrum of equipment.
Its not even a question of affordability, but in most cases, its a dogmatic view of the hobby and refusal to listen.
that being the case, they fail the number criteria in scientific methodology. they fail to observe.
If you listen to a lot of music, and over the years have experience a good assortment of equipment, trust me, trust your ears. audio reproduction is no different from food or wine. Its is consumption that stimulates the senses. Its basis is science, but satisfaction and fulfillment comes down to giving the listener an experience.

if you want plain sustenance, Mcdonalds is right there for you. Its got everything you need and it measures well :)

Actually...McDonald's "food" is not good you, and will lead you to an early grave. This has been measured. 

Making such snobbish comments on those of us that don't buy into the all the audiophile nonsense, have actually listened to this varied gear and reach a conclusion that the prices don't justify the performance...by actually listening, and not falling for the psychologically subjective things they are told are there. Refusal to listen? That's an incredibly ignorant assumption and statement to make. You can snicker all you want at those that don't join your elitist club...the same reason's I never joined a fraternity. It's not what you know...it's who you know. And it's pretty fake.

peguinpower
33 posts
05-03-2021 9:04pm

Yep. I work in tech. Have more data that I can use. I have a data scientist working for me. The first thing you learn is to make decision whether you have enough data or not, and the second reject data that is no use.

I use data more than most. I use data to make decisions at work overseeing close to two dozen people. Not an intellectual masturbator who found Google and thinks he has the key to the universe.
More revelation. Thanks.
Do you look for accuracy in a recording, or what pleases you, regardless of what the artist/producer/engineer is trying convey?

peguinpower
32 posts
05-03-2021 7:33pm
Elapid,

Data can be manipulated. You could be measuring the wrong thing.
Your so sure of yourself. Dont know from Adam.

I know joy when I feel it. Thats all that matters.

You can take your data. Its no good to me.
So much revealed in one comment. Well done.
So...it sounds like you're anti-engineering to get a baseline of what driver parameters are suitable for a specific design.

Ears are different. So are driver parameters, and you gotta start somewhere.

The reason why audio is so interesting, especially with speaker systems, is because multiple systems can measure similarly, yet sound quite a bit different. That's why measurement systems have become so sophisticated. It's not just a single line anymore.

audition__audio
798 posts
05-05-2021 10:20am
Passion driven undertakings are no more or less prone to real answers than purely intellectual undertakings. You are making a meaningless comparison because no "best" exists when it comes to most of the pursuits driven by our senses. There are no real answers, just opinions as there is no standard of correct/real. Further, measurements get you part of the way, but certain devices have attributes that cant be measured but sound better in many instances. Vacuum tubes would be a perfect example or tube amps if you like. Take it one step further and determine if it actually is some of the things that make it measure poorly that actually make it sound better.
There is a reason vacuum tubes sound better in many instances. One has to do with superior linearity in a well designed hifi amp/preamp. Same could be said for powering subwoofers with a solid state amp of good design.

So....you being the arbiter of "meaningless comparisons" is simply absurd.

Guidelines in engineering standards by an organization like AES means that a manufacturer just can't make up stuff that can't be proven through tests.

pauly
534 posts
05-19-2021 11:33am
I don’t know whether Audiogon can block IP addresses from accessing the site, maybe they cannot, but knowing this dude is tech savvy, I am pretty sure he gets around that limitation too.


Using a VPN doesn't make anyone tech savvy. My dad used one and he didn't know the difference been a laptop and a typewriter. I suspect his computer knowledge is about the same as his electronics knowledge ; none.


Please keep posting. It's very enlightening.