Why Do So Many Audiophiles Reject Blind Testing Of Audio Components?
Because it was scientifically proven to be useless more than 60 years ago.
A speech scientist by the name of Irwin Pollack have conducted an experiment in the early 1950s. In a blind ABX listening test, he asked people to distinguish minimal pairs of consonants (like “r” and “l”, or “t” and “p”).
He found out that listeners had no problem telling these consonants apart when they were played back immediately one after the other. But as he increased the pause between the playbacks, the listener’s ability to distinguish between them diminished. Once the time separating the sounds exceeded 10-15 milliseconds (approximately 1/100th of a second), people had a really hard time telling obviously different sounds apart. Their answers became statistically no better than a random guess.
If you are interested in the science of these things, here’s a nice summary:
Categorical and noncategorical modes of speech perception along the voicing continuum
Since then, the experiment was repeated many times (last major update in 2000, Reliability of a dichotic consonant-vowel pairs task using an ABX procedure.)
So reliably recognizing the difference between similar sounds in an ABX environment is impossible. 15ms playback gap, and the listener’s guess becomes no better than random. This happens because humans don't have any meaningful waveform memory. We cannot exactly recall the sound itself, and rely on various mental models for comparison. It takes time and effort to develop these models, thus making us really bad at playing "spot the sonic difference right now and here" game.
Also, please note that the experimenters were using the sounds of speech. Human ears have significantly better resolution and discrimination in the speech spectrum. If a comparison method is not working well with speech, it would not work at all with music.
So the “double blind testing” crowd is worshiping an ABX protocol that was scientifically proven more than 60 years ago to be completely unsuitable for telling similar sounds apart. And they insist all the other methods are “unscientific.”
The irony seems to be lost on them.
Why do so many audiophiles reject blind testing of audio components? - Quora| Post removed |
It's really a depressing question. Why do so many people reject/fear science?So who rejects science? Flat Earth people? Who fears science? Why would anyone be afraid of science? Science or equipment tests can tell us a lot of things but it is completely unable to show how it sounds compared to a live performance. Have you looked at how complex sounds look? Even a miniscule change can make a massive difference, yet it is impossible to detect with scientific equipment. In that regard it is entirely appropriate to reject science and instead actually listen, and don't believe reviewers who provide glowing reports just to stay in business and get the next piece of equipment to test. |
Short attention span theater intermission. The subject was the premise double blind testing is inappropriate and meaningless. The supporting evidence is a study involving speech recognition. The flaw in the reasoning, no evidence is given that speech recognition and component evaluations involve similar brain functions. Let alone skills sets. So the OP premise is fatally flawed. We now return you to our regularly scheduled blather. |
Everyone sites this and that and my ears and brain are better than yours, etc., etc., but frankly none of this matters. All that matters is the simple question, “does music move you emotionally and does the system you have deliver that emotion to you in a way that maximizes your enjoyment?”. Like a long discussion in another recent thread where @millercarbon hit the nail on the head, this hobby isn’t just about your ears. You feel the music all over your body as it literally washes over you with your ears being a primary, but not only component. I look at all of the performance graphs I can take in my FIbre Channel Data Centre switch related career “day” job. I’m frankly not interested in it at all when it comes to music and my system. I recognize only emotional, visceral enjoyment is what has kept me enamored with music and high-end audio constantly for my entire life. I’ll continue to exercise the other half of my brain more when it comes to choosing the combination of gear for my other main hobby of considerable expense - Astrophotography :). Then again, I’m also much less comfortable with Astrophotography since it is a new hobby for me. Maybe all of this graph obsession some have with audio is a manifestation of insecurity, not yet feeling totally grounded in the hobby like I certainly feel currently with telescopes, Astro-cameras, myriad of hardware accessories, photo stacking/post processing software, etc. I don’t know. We all have our own paths to take in life. |
There you go again cakyol with the under breath slighted remarks that spew nothing but JEALOUSY. Not because you can’t hear the difference, but because you WON’T.. you would make a horrible mechanic. I’d give you 3 months before they’d give you the boot for being too darn thick to learn HOW to listen.. The apprenticeship program I went through taught you HOW to learn too.. 101... how to learn. FIRST things FIRST. THEN to listen, and then ask questions... Not show what you haven’t learned. You ask question until you have the proof you need, not doubt, what you obviously can’t seem to grasp... Tough being an army of one... No roxy54 it’s not a word ( ordinary’ness). , BUT we can LEARN to think it is.. :-) |
@artemus_5 "He found out that listeners had no problem telling these consonants apart when they were played back immediately one after the other. But as he increased the pause between the playbacks, the listener’s ability to distinguish between them diminished. Once the time separating the sounds exceeded 10-15 milliseconds" OK Mr Artemus explain this: Miller, say, (applies to many many others) has a listening session. Then he replaces, say, the footers under his loudspeakers with springs (boing, boing!). The replacement takes say 15 minutes; they are big heavy speakers. Then he listens again. He pronounces the SQ massively improved. As he would. But the experiment said he would not be able to tell the difference reliably after 15 milleseconds and Miller waited a quarter of an hour. |
Every audiophile friend of mine (and myself) are totally OK with blind testing. It is fun, it is a good exercise, and it is also a learning tool to perfect our skills of perception. We do it once in a while, for fun. Speaking of myself, I welcome blind tests, (being a research scientist that's not at all surprising), but I cannot suffer people who force it to the nth degree. Blind test fanatics cannot trust their own honesty to not to attach bias to what they see, to be independent of their own previous judgements. What's the point of audio then, if one is so utterly insecure of what they hear, and so caught up with ego, pride, that it gets in the way of objectivity? To me, the sort of attitude that "mine has to be better" or having stakes at proving something with a blind test is utter rubbish. I am an audio pilgrim, welcome and embrace new experiences, even when they strengthen other's positions. Especially then, because that's an opportunity for me to learn. |
| Post removed |
@djones"Why do so many people reject/fear science?" There's a whole area of study to this called psychological rigidity, but in simplest terms it boils down to some people's problem of equating being wrong with feeling stupid. So they never consider that they are wrong. And they never consider any evidence or procedure that could prove they're wrong. That way they get to stay comfortable. When I was younger I used to suffer from this. Fortunately I worked my way out of it. It's mostly about whether you look at being wrong as feeling stupid or as a learning experience. |
True jss. Beyond that though most of the ones on here that deny blind testing is because it disputes their personal preferences or beliefs. Either towards a product in general such as cables, or a specific model that either wins or loses in such a comparison that goes against their thoughts on it. They will many times try to pick apart the testing procedure and explain why it was done wrong, no matter how it was done. There are fast earthers in our hobby just as there are in general. Most times when someone tells me they hear something as better and I do not, it is not because I do not hear a difference, it’s because of my personal preference in what I hear. When I don’t hear anything ( rarely, but it happens) I assume either it’s hearing degradation due to age, or my personal inability to distinguish what others are hearing. I don’t assume large numbers of people are making it up and fooling themselves about it as some will try to say. I have had a few tell me in private that they didn’t want to admit they hear a difference for exactly the reason you’d stated, or for a few, because they could t afford to play at a high level and didn’t want the embarrassment of saying so. Those ones just need to realize that we all have budgets, and so the best we can at what we can afford. Very few can actually afford the best of the best, and as we all know, the returns for the dollars get smaller as you climb up the ladder. |
Meanwhile, as our friends debate in the hall, I will retire to the lounge, where I will listen to some wonderful music on a system that has been optimized by comparisons of entire sets of cables. Imagine that, what a concept! No, I'm not really interested in engaging with skeptics on this. Been there, done that. Waste of life, unless they are willing to do comparisons of sets of cables. Otherwise, not worth my time. :) |
This makes one wonder how various aural skills might affect the study in the OP. In particular, I wonder how rigorous conservatory ear training might affect results (when you have to have aural memory foundations, like harmonic pitch intervals, and different chord inversions and types), and speaking multiple languages, would affect the results. One would assume people with these skills would perform better on the test. Musically illiterate and monolingual people are the norm today, at least in the English-speaking enclaves of North America and England, so this is probably indicative of these areas. |
1-Most of us have pieces of music we've listened to hundreds of times if not more. An example for me would be Abbey Road. A test of systems (or components in a system not my own) playing it to me would still be useless as my reference is my own system and without a direct comparison to that, I would have virtually no frame of reference to opine on what may or may not be different. But even worse, playing random bits of anything one is not used to under any circumstances is just pissing in the wind. That said, blind switching of cables, components, etc in my system playing that album would give me useful information. 2-What exactly is better? Again in reference to what we are used to in our own systems, different might not be defined as an improvement by everyone or anyone. Various posters here have mentioned that different cables have made an audible difference to them when heard in their systems. An interesting test would be to see if that would also be the case when played in a system they were not familiar with. 3-The often mentioned blind Pepsi vs Coke discussion has no relevence here. One soda has more carbonation than the other and one is far sweeter. However there are people who crave salt that taste no difference at all. |
@millercarbon The subject was the premise double blind testing is inappropriate and meaningless. The supporting evidence is a study involving speech recognition. The flaw in the reasoning, no evidence is given that speech recognition and component evaluations involve similar brain functions. Let alone skills sets. So the OP premise is fatally flawed.OP Also, please note that the experimenters were using the sounds of speech. Human ears have significantly better resolution and discrimination in the speech spectrum. If a comparison method is not working well with speech, it would not work at all with music The pivotal paragraph is the one above IMO. The question is the veracity of the last sentence... If a comparison method is not working well with speech, it would not work at all with music We know that correlation doesn't necessarily mean causation. But there is a correlation and sometimes they DO overlap to the point where they are the same. Which begs the question, if we cannot discern speech while blindfolded, can we discern music any better? I've lived long enough to know that there are those in this world that no amount of evidence will prove anything to them. They have a closed mind which cannot learn. So I don't try to convince but like Fox, I report, you decide. FWIW, this was written by another, not me. But I have plenty more which will be disclosed later |
@douglas_schroeder I agree with your perspective. The reason I post this is because we have a lot of new people who have been putting forth this scientismist POV. There are also new people here who really want the proper info. They don't know. Hopefully they can come to an understanding, much like I did from some of the more knowledgeable posters from the past & present |
@orgillian192
A test of systems (or components in a system not my own) playing it to me would still be useless as my reference is my own system and without a direct comparison to that, I would have virtually no frame of reference to opine on what may or may not be different.
Indeed a very good point and one which Paul McGowan (PS Audio) & Hans Beekhuyzen have made as well. Harman Int. trains all their people who do testing, on how to listen. @mikelavigne above has given many problems which he faces in the many DBX he has been involved. |
Absolutely no one asked me to so I must therefore opine. Double blind studies are for product development, not for making a choice in buying. A double blind study on the value of a feature will help me tailor my product and marketing. It won't help me, not even a little bit, chose among products I'll purchase. |
might be good to remember that science is the art of logic based investigation of incomplete unknowns. The statement does not say that science itself or logic or the art of investigation is in any way complete, nor does it suppose that dogmatism is a thing. Arguing with human emotion as a deep colorant and filter in one’s own formation of logic... is also an act of partial and limited depth analysis with the logic bits being all askew. Since we don’t know everything, this part is inevitable. to be on guard for it with every single thought form of any kind. at no point in any of the given motions of working through the considerations, should anyone assume that any of the bits in place are perfected or immutable. that’s the creep of the deep mindset of humans in ’dogmatism’, coming into the fray again. coloring. owning. projecting. deciding and living in the emotional coloring and skewing of logic. really bad news for science. which the idea of science is specifically designed to sidestep and take into account, and never issue any facts of any kind. to never be dogmatic, to never have facts, to revert all to mutable, changeable theory. otherwise dogmatism rises in the mind and colors facts into existence - when there are none. Most times these sort of threads are rife with limited depth projections built out of dogma and factualization. A fundamental and core human issue. if you want to have a discussion that is to be based in the idea and form/scope of science.... then shed the dogma and facts, in everything. Then..maybe we'll start to get somewhere useful. |
@ dletch2, jdane, djones, jjassmith and all the other Scientismists on board. You boys are good at casting aspersions on those of us who do not agree with your religion of Scientism. But do you have any REAL science to back up your assertions? You aren't calling on science. I've debated too many small minded people like you who when backed in a corner, always cry out that my question was not answerable by science. So on one hand, you infer that science has all the answers and then when backed into a corner you contradict yourself and say it doesn't. So I know who you are, what you think you know and your next move. As I use to tell my son, "I know what you are going to do before you do it". Why? Because I have been you. But I grew up. maybe you will too. But its not guaranteed. |
unreceivedogma368 posts04-30-2021 11:36amAnd why exactly you feel the urge for OTHER PEOPLE to prove it to you that they hear a difference? Kudos to you for using the blind testing for making your own choices (assuming you indeed do so), but to impose to other people who do not is a bit of stretch. By that logic, nobody should share any experiences with equipment, gear, cables, etc. unless they can "prove" it with blind tests certified by a panel of third party "specialists"? |
@
speedbump6 : Part of what you describe is one of humans' most powerful instincts, the pursuit of status. And when your livelihood or acceptance into a desired group is dependent on a belief, it's apt to be even more deeply rooted and challenges to it avoided.
Can you imagine how devastating it would feel to spend $130,000 on a pair of D'Agostino monoblocks and find out you just blew at least $127,000? Why take the risk? And that can be scaled down by income. Imagine the normal Joe spending $5,000 on a DAC then blind testing $140 one and you couldn't tell the difference. That would be like going to the casino, putting down $4,860 on one hand of Blackjack - and losing! Not a good feeling. Well, maybe you could sell the D'Agostinos for $80,000 and the DAC for $3,000 and only lose ~$50,000 and $2,000 respectively on that one hand of Blackjack, but still not a good feeling. One to be avoided at all cost. If you never take the risk, you never have to confront your mistakes. |
| Post removed |
@thyname Ditto Seems as though they want to force their thinking & ways on me. I could care less if they want to use electric fence or dynamite wire in their system. If they hear no difference, fine with me. Just don't try to force me to use electric fence, dynamite or barbed wire. But your analogy goes even further. if we can't hear any difference in wire, can we hear it in capacitors, resistors? I wouldn't think so. Then are all amps the same? What difference would it make with an amp? Their argument gets pretty ridiculous IMO |
Blind testing is quite useful for determining if something is plainly audible to most people. It can also be used to determine if some difference that’s been shown to be audible is preferable or not to most people. Proving that something is inaudible under all conditions to all listeners is trying to prove a negative and that’s always a tough thing. A naysayer can naysay forever. I watched a flat earther group reject the visible sinking of a marker on a boat below the horizon line even though it was being demonstrated directly for them. It’s just a heat effect, they said. So which side does the burden of proof land on? If 999 out of 1000 people reliably detect a difference in a blind test someone who felt strongly from their personal experience that it wasn’t true could argue that they were all just lucky or there must have been something wrong with the test and that people really generally can’t hear the difference. If 999 out of 1000 people can’t reliably detect a difference during the test, someone who felt strongly from their personal experience that it was audible could say that the test was somehow fundamentally different than real listening situations so most people who care to pay attention can really hear that difference. I have listening experiences that seem to line up really well with the technical and scientific consensus that I read about everywhere except some audiophile literature. If double blind tests indicate that something is inaudible, it is inaudible to me. If I try to achieve the measurements that are specified, I find I really, really like the results. The closer I get, the better I like it. High end systems that don’t meet those specs don’t sound as good to me, and ones that exceed the specs in certain areas - like extended frequency response or ultra low distortion don’t sound any better to me than those that just meet the specs. So I have to admit here that my personal experience is what motivates me to go with the science and currently known measurement criteria. It doesn’t contradict my subjective experience and it gets me where I want to go. Those who find measurements and testing don’t line up with their perceptions have a different journey. I have noticed that many of those who feel there is an audible difference beyond what the scientific literature has established seem to have little interest in coming to a better, more scientific understanding of it so that can be reliably and perhaps affordably reproduced. Maybe they feel it’s just too complicated. Why argue and get all technical when they know they perceive better sound and all they have to do is buy it and set it up. |
I totally believe in blind testing. Seeing a label on a piece of equipment known to be prestigious does not mean it sounds better. Let your ears be the judge. For example I purchased a BlueSound Node 2i and the dealer talked me into adding a DAC. I played it with and without the DAC and felt it sounded better without the DAC. The DAC made the bass sound thin. So I got it home and still found it sounded better without the DAC. I ended up selling the DAC on Audiogon and learned a lesson. Just because you add something doesn’t mean it automatically sounds better. |
Paul at PS Aduio's video posted above is right on the money. Keep everything the same, blind folded A vs. B for a minute or as long as you need to get a feel for it and then play the same thing on the B option. If you can't hear a difference, then who cares? Just make sure someone else sets it up for you so you are not biased in any way. It's the only way to justify an expenditure. I can't believe this is even debated. What else are you going to go by, some review whose magazine has paid advertisers or is friends with the designer or manufacturer or is incentivized by them? |
| Post removed |
@ dletch2 Because people like to hear themselves talk, they do it about things they don't have any real knowledge of,93 posts in this thread and you, have 18 of them or nearly 20%. That's about the most accurate thing I've heard you say. Psychological projection - Wikipedia Projection tends to come to the fore in normal people at times of personal or political crisis but is more commonly found in narcissistic personality disorder or borderline personality disorder. |
| Post removed |
Irwin Pollack, Professor Emeritus of Psychology and Research Scientist Emeritus in the Mental Health Research Institute at the University of Michigan, passed away January 23, 2021 at the age of ninety-five. Professor Pollack received his B.S. degree in 1945 from the University of Florida and his M.S. and Ph.D. degrees in 1946 and 1949, respectively, from Harvard University. Following his doctoral work, Professor Pollack was a research psychologist at the U.S. Navy Electronics Laboratory. He then served as director and senior research psychologist at the Air Force Operational Applications Laboratory prior to coming to the University of Michigan. In 1963, he received a joint appointment at Michigan as professor of psychology and research scientist in the Mental Health Research Institute. He retired as Emeritus Professor of Psychology in 1995. Professor Pollack was an active member of the Department of Psychology throughout his retirement years, attending research talks, faculty meetings, and department social events. He was generous in his praise and support for our faculty and students, and a valued colleague and friend. Professor Pollack is internationally known for his novel approaches to the study of the auditory system. Over the span of his career, he was interested in the nervous system interpretation of auditory information. He recently focused on an experimental approach that distinguished between two general modes of processing sensory information: "within-signal" and ''between signal" sound comparison, which are necessary for auditory discrimination tasks. The long-term goal of his research was to improve our understanding of the human binaural system. Such understanding provided a foundation for new prostheses and signal processing procedures to aid localization of hard-of-hearing patients. Professor Pollack a Fellow of the Acoustical Society of America and of the American Psychological Association. |
Projection is the process of displacing one’s feelings onto a different person, animal, or object. The term is most commonly used to describe defensive projection—attributing one’s own unacceptable urges to another. For example, if someone continuously bullies and ridicules a peer about his insecurities, the bully might be projecting his own struggle with self-esteem onto the other person. The concept emerged from Sigmund Freud’s work on defense mechanisms and was further refined by his daughter, Anna Freud, and other prominent figures in psychology. Unconscious discomfort can lead people to attribute unacceptable feelings or impulses to someone else to avoid confronting them. Projection allows the difficult trait to be addressed without the individual fully recognizing it in themselves. Easy read: "The Ego and Mechanisms of Defense," Anna Freud. |
