To Blind Test, or Not To Blind Test, That Is The Question


 

Circa 1988 I held an issue of Sterophile in my extra-large mittens for the first time.  I have subscribed from that day till nowIn the beginning I was particularly impressed with the thoughts and words of one J. Gordon Holt (1930- 2009).  If I recall correctly J. Gordon was positively death on the idea of blind testing.  Many of J. Gordon’s thought soon became mine also.  But now after considering the ideas of Amir, via his website and U Tube channel, both called Audio Science Review, I have reconsidered my position.  Amir’s logic seems irrefutable.

 

Fred has a significant Hifi.  While in Ace Hardware he spots a power cable that seems identical to the one that is attached to his Mark Levinson amp.  Fred reasons that that power cord is limiting his whole system.  He unleashes a well-planned research effort and fixates on a cable that sells for 900 sovereigns.  A local brick and mortar guy offers him a 2 week return policy, and “poof” it’s home and hocked up.  Fred lets the cable “cook” for 48 hours and then begins “the test.”  He warms up everything, and them dawns a sleep mask.  A loved one, or friend plays a piece of music, of Fred’s choosing, 10 times randomly switching the cables such that each cable is played a total of 5 times.  After each playing the assistant takes note of which cable was used and what Fred’s reaction was.  If Fred discerns correctly 8 or 9 times and clearly prefers the new cable, it is established that the new cable has proven itself and is well worth the price.  But if Fred is able to discern the difference between the cables correctly only 5 times, that is meaningless, because he could have done that by guessing.  If Fred wants to increase his confidence in his decision, he could repeat the process, with a different piece of music in few days later.

 

So, using your ears, in your listing space, with your system, and playing your music, you are unable to document an improvement in perceived sound quality, created by a new gizmo, why would you want it?

 

That’s what I think, what do you think?

 

alwynlarryv

Sure. You go ahead and do that. Whatever floats your boat. I approve. And I hope everyone here will approve it for you

i have this tool useful for a/b blind testing... not the only evaluation needed towards a full and proper assessment of a system change, but a/b testing can be elucidating...

https://avahifi.com/products/abx-switch-comparator

Long term evaluation is still the gold standard. Lots of hours with lots of different music. Short term testing or A/B blind testing is prone to error. You can't evaluate a car by just pulling out of the driveway, you need to speed some time with it. 

I've not done much with blind testing partly because my interest (including time or budget) isn't at a level where I feel the need to seek upgrades that are not easily discernible.

I did a lot of A/B testing (not blind) with a set of Nordost Sort Kones under my SACD player in years past and thought that there may be some improvement, but never to a point that I considered that I've know if they were in or out in any type of blind test.  My system is more resolving now, but none of my equipment or current cabinet was a good fit for the Sort Kones, so I finally sold them.

I've dabbled with some affordable cable upgrades (most expensive was about $250ish) and felt that the improvement was immediate and obvious when I heard a difference.  I heard a clear improvement with two interconnect upgrades, but can really only say that there was an improvement from power cords on one of three components.  I consider that the upgraded power cords should be better than what they replaced even though I didn't hear a difference, so I kept them on the outside chance that they might benefit from burn-in.

I've found that differences are often small when compared A/B, but with some extended listening there's a different level of enjoyment.  My first "hifi" gear was an Integra receiver that I still have, so can compare to my current Pathos amplifier.  In an A/B there's a clear difference, but the receiver does not sound bad to my ears, but if I use it for background listening can start to grate on my nerves over time.

 

No, don't blind test, but ask yourself repeatedly these two questions:

  • Am I hearing something better or different?
  • If it's better, is it worth the asking price better?  How much am I willing to pay for this difference in sound quality?

To often too many of us will fork out any amount of money for the slightest of differences, to the point where more audiophiles than I can mention have ended up with horrible sounding systems because they kept chasing "different."

Wire has become a religion. You’ve got to believe it to hear it! All of this is, like religion, faith-based. The wire believers read the spiels from ad copy men masquerading as physicists! Expectation bias leads to the ever more costly cable as somehow being more "musical". Add mythical concepts like " burn-in" and "directionality" and the cable hucksters are eyeing dollar signs! I welcome a blind shootout between some cheap and some expensive power cords.

I welcome a blind shootout between some cheap and some expensive power cords.

First: please demonstrate the results of YOUR blind tests. Obviously, certified by a panel of third party independent commissioners 😉

 

Judging by your repeated constant spiels of cable hatred here, I often wonder what kind of trauma you have experienced in your early life, cable induced. 

Long term evaluation is still the gold standard.

One thing that long term evaluation does is allow one to get used to -- to form expectations and standards around -- a new piece of gear. One gets used to it, and that can mean (a) one learns to overlook the deficiencies AND (b) one learns to appreciate the character of the sound.

It would seem that after getting used to the gear, long term, one might use the too@jjjsss invested in to compare, A/B two things that one is used to.

The question I ask myself about blind testing is not, "Is this the gold standard?" but "why not include it as part of evaluation"? I see no reason it is not a valuable part.

@thyname : if you lived nearby we could do a comparison of your expensive power cord vs. my cheap cord using my modded Quad 405/Quad 57's. Not high quality enough for you, eh?

Some of these differences can be pretty small, and can take a long time to get used to and to recognize. IMHO, it’s better to experience the smaller changes over a significant period of time, allowing for all the variables involved on a daily basis to balance out.

jasonbourne52

2,028 posts

 

@thyname : if you lived nearby we could do a comparison of your expensive power cord vs. my cheap cord using my modded Quad 405/Quad 57's. Not high quality enough for you, eh?

And this would serve which purpose for me? These are your equipment and speakers. Not mine. What will you be attempting to “prove” by such experiment?

Blind testing isn’t so much about the differences in the equipment as much as it is a test about the listener.  Moreover, just because a listener is able to perceive a difference between Product A and Product B, you can only talk about the listener’s preference for a particular product.  If you want to take the variable of the listener out of the equation, then your listener panel needs training.

Blind Audio Testing: The Scientific Truth Behind the Myth

Claims for objectivity in this, talk about misplaced confidence. So many subjective vagaries in this, performance anxiety being only one of many.

@jasonbourne52 - There's not much point in using your system (no matter how good it is) for an A/B since you believe that it won't make a difference.  It would make more sense for an A/B to be done on a system where the owner believes that there's a clear difference.  This would give you an opportunity to hear (or not to hear) the difference.

You're very adamant that cables can't make a difference, I'm genuinely curious what systems you've done some comparisons on to form that opinion.

Great, another one who loves parlor tricks. What's next? Balloon animals?