Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

I once wasted my time watching a video “review” of some audiophile switches. It was half an hour of blah blah blah about how his measurements demonstrated without a shadow of a doubt that there was absolutely no measurable difference between X Y and Z switches therefore it was impossible to hear any difference so it was all in your small “audiophile” brain, his mantra so to speak. Then he went on to briefly demo the switches he had just “measured” in his (joke of a) system. And guess what: he could clearly hear a difference. His wife could hear a difference. According to him the audiophile switches sounded significantly and audibly worse.
 

Without realising he had just contradicted and refuted his half hour case for “it’s in your head”. If he accepts that his experience contradicts his measurements, for better or worse, then his measurements are flawed. For me Amir became irrelevant after that video.

@reven6e And guess what: he could clearly hear a difference. His wife could hear a difference. 

Please supply a link to this video review.  I'd enjoy watching it, not least because I've not witnessed Amir bringing his wife in for a hearing experiment - was it an A/B blind test?

Amir appears believe we all suffer from expectation bias.  One  problem how does that explain when the expectation is for things to sound no better or worse but they don't?

Blind testing works for components that have a wider spread of differentiation. It works for "preferences" as in which spaghetti sauce do you like. You can tell that immediately.

The trouble w extended listening is audio memory is the shortest memory we have and differences are more nuanced. There, "preferential comparison" (flavor of presentation over weeks or months) might be harder to assess except to say "I never noticed that before" in terms of sound articulation.

Even this cannot be written in stone as everyone’s brain is different.

What we hear is often compelled by desire of what we want to hear (expectation bias).

There is an aspect that adds further confusion to the process...chance.  What would be an interesting experiment is if the aspect of "gamble" or "skin in the game" or "taking a chance" was eliminated because spending larger amounts of money on higher end stuff creates conflict of interest in my mind.

I cannot tell you the amount of times people coming into our salons, agonized over a decision when A/B (or C and D etc,) comparison differences were not as stand out between what they were listening to. Even taking things home for try out did not completely eliminate this conflict between money spent and objective/subjective auditory acquisition as the listener could not escape that big money was about to be spent at some point.

For the well healed, this aspect did not loom as large.

 

Earlier on this thread I mentioned a couple of the issues that I have with those who base their judgments almost exclusively on measurements, and what I consider to be some limitations of A/B testing as it is typically performed.

But as to Amir and ASR, I think that in many cases his work is both valuable and practical. As a current example, on Friday he tested a "Hi-Res" download recording from PS Audio. He tested both DSD and PCM. What he found, and clearly demonstrated, is that the recording is flawed, and that there were dubious decisions made during the process.

As one who is not well-versed in the intricacies of streaming, I learned some interesting things from his brief video. Beyond that, and this is typically the case, I learned even more by reading the many comments, as they included insights from very sharp, highly experienced people, some of whom are electrical engineers.

Take a look at the thread, and, if you’re in the same boat as me, you will be surprised at what you may learn (e.g. that DSD files are of dubious value!).

PS Audio DSD review