Take it on faith: A cease-and-desist letter to those who only believe in measurements

Anybody who tune his system room takes hundred of simple informal  blind tests(Not double blind test )...

No one with a brain can be against double blind test in acoustics...

But in acoustics unlike ASR they takes subjective hearing seriously because they study it...

In ASR they use the formal complicated test to eliminate subjective hearing as the main player and to promote a short set of established measures as  the gold standard for sound quality...

It measures well : it must sound good...

It is marketing ploy or pure ignorance ...

It was well debunked by Milind Kunchur...

I never understand the resistance to true A/B blind testing.  Sure, it is not the be all and end all, but it is an important data point in comparing competing equipment.  I remember doing an imperfect AB test (not blind) on a digital and optical cable and it was really only because we could switch between them that we were able to detect subtle but existing differences.  OP makes an interesting point as well about people’s varying ability to detect differences.  My wife has the nose of a GMO bloodhound; she can smell whether our cat took a dump in the Litter Robot upstairs and must run up and clean it because it smells so badly.  On the other hand, I can’t smell it even standing next to the litter robot.  Does that mean that out doesn’t smell, or can’t be detected.  No.  It just means that I can’t detect the minute differences in the air, but she can.  Perhaps the same sort of analogy applies to audio equipment.  Some people have very sharp and discriminating hearing, or even the same sharp hearing but respond to one set of frequencies more than another.  For example, I don’t like sharp treble -- it sounds "bright."  Other people might hear the same sound that I dislike as "bright" and believe that it has amazing "realism."  So of course, that leaves us pretty much where we started.  Measurements are an objective data point, AB testing is a datapoint, but ultimately, bad measurements or not, the question is how something sounds to you.  Just because it sounds different or better does not mean that it is the result of a placebo effect or confirmation bias.  To my wife, a piece of cat poop stinks up our entire two story house.  To me, I can’t smell it at all.  Doesn’t mean that its not there!

I have faith that science is real and faith was my former girlfriend 

The human entity is one, science(mind) and faith (heart and will) are one. Dividing them artificially  is the recipe for failure.

I have faith that science is real and faith was my former girlfriend

@tcutter  interesting initial post, thanks for starting the thread.

@newton_john is this the post you wanted to reply to?

”Bringing this back to audio reproduction, some of the biggest unsettled arguments remain how different cable materials and configurations could change the sound we hear in our rooms, or how two amplifiers or DACs that measure nearly identically could sound different in our listening rooms, or how an amplifier or DAC that measure worse on the bench could actually sound better when experienced in our listening rooms.  And don’t even get me started on soundstage…

As you noted, the reductionist method works very well when the systems under consideration are uncomplicated and conducive to solving with simple experimental design. When we add complexity, controlling variables becomes much more difficult. In the case of evaluating hifi equipment, those variables include but are not limited to the quality of the supplied power and any power conditioning, the assemblage of equipment in the test, the configuration of the listening room, its materials and the placement of the speakers, and the seating position and listening skills of the person evaluating it.  

As commonly discussed here and elsewhere, the results of an evaluation of stereo equipment A versus B are only really valid for that particular “system”.  To get any statistically valid results requires careful ABX blind testing procedures and multiple listening “subjects” for both statistical power and to cover the potential variance in listening skills and hearing acuity. I have yet to see a really well designed “experiment” to test this because it is time consuming, and requires a large number of subjects tested one at a time in the sweet spot. This is still a reductionist approach, it is just a very complicated, and logistically and cost prohibitive in service of a point that has little transferable utility.

Based on all of this, the appeal among potential hifi shoppers to lean on bench test results derived with proven measurement equipment and protocols testing gear in complete isolation and calling that “Audio Science Review” is apparently immense. The alternatives are to trust the claims of manufacturers, the collective “wisdom” and experience on audio forums like this one, and/or trial and error testing at home in your listening room. Given the time and expense of the latter, there is ample motivation to disregard whole segments of products out of hand as voodoo, and take measurements as an insurance policy that an investment is well advised.

High end audio is for people with time and money. For many with limited time and a basic understanding of classical physics and electrical engineering and who just want to listen to music, a spec sheet is apparently plenty good.”

kn

@moto_man 

+1