”Bringing this back to audio reproduction, some of the biggest unsettled arguments remain how different cable materials and configurations could change the sound we hear in our rooms, or how two amplifiers or DACs that measure nearly identically could sound different in our listening rooms, or how an amplifier or DAC that measure worse on the bench could actually sound better when experienced in our listening rooms. And don’t even get me started on soundstage…
As you noted, the reductionist method works very well when the systems under consideration are uncomplicated and conducive to solving with simple experimental design. When we add complexity, controlling variables becomes much more difficult. In the case of evaluating hifi equipment, those variables include but are not limited to the quality of the supplied power and any power conditioning, the assemblage of equipment in the test, the configuration of the listening room, its materials and the placement of the speakers, and the seating position and listening skills of the person evaluating it.
As commonly discussed here and elsewhere, the results of an evaluation of stereo equipment A versus B are only really valid for that particular “system”. To get any statistically valid results requires careful ABX blind testing procedures and multiple listening “subjects” for both statistical power and to cover the potential variance in listening skills and hearing acuity. I have yet to see a really well designed “experiment” to test this because it is time consuming, and requires a large number of subjects tested one at a time in the sweet spot. This is still a reductionist approach, it is just a very complicated, and logistically and cost prohibitive in service of a point that has little transferable utility.
Based on all of this, the appeal among potential hifi shoppers to lean on bench test results derived with proven measurement equipment and protocols testing gear in complete isolation and calling that “Audio Science Review” is apparently immense. The alternatives are to trust the claims of manufacturers, the collective “wisdom” and experience on audio forums like this one, and/or trial and error testing at home in your listening room. Given the time and expense of the latter, there is ample motivation to disregard whole segments of products out of hand as voodoo, and take measurements as an insurance policy that an investment is well advised.
High end audio is for people with time and money. For many with limited time and a basic understanding of classical physics and electrical engineering and who just want to listen to music, a spec sheet is apparently plenty good.”
Well put. An excellent post. Just one small point I take issue with. Audio relies on more than just classical Physics. Without quantum mechanics, most electronic components wouldn’t exist.
Even in the world of classical Physics, Maxwell’s equations indicate that things are a great deal more complex than the ASR lobby admit. Perhaps we need a whole system-based approach. Inspired by Morin’s work on complexity, I attempted to create a model of audiophilia. It seemed to me that it could be thought of as a complex system consisting of three interacting sub-systems, Music, Gear and Listener in an Environment.
This is not to deny the importance of the individual components of each sub-system and the environment, but to stress that the whole may be greater than the sum of the parts. There are more components than I have listed. The ones shown are just for illustration.
Audio is multi-disciplinary. Hopefully, the model illustrates why the more focussed approaches taken on hifi forums don’t capture the entire picture. It should be obvious why the qualitative and quantitative are both important. They should not be characterised as being subjective and objective. As I said before, there’s no good reason why the quantitative should be privileged over the qualitative. There's plenty of qualitiative data in Physics.
