In Defense of Audiophiles, Bose, Pass, Toole and Science

I don’t know why I look at Audio Science Reviews equipment reviews, they usually make me bang my head against my desk. The claims they make of being scientific is pretty half-baked. Don’t get me wrong, I appreciate measurements, and the time it takes to conduct them, along with insights into the causes, but judging all electronics based on 40+ year old measurements which have not really become closer to explaining human perception and enjoyment, they claim to be objective scientists. They are not. Let me tell you some of the people who are:

  1. Bose
  2. Harman
  3. Nelson Pass
  4. Floyd Toole

This may look like a weird list, but here is what all these have in common: They strive to link together human perception and enjoyment of a product to measurements. Each have taken a decidedly different, but very successful approach. They’ve each asked the question differently. I don’t always agree with the resulting products, but I can’t deny that their approach is market based and scientific.

Floyd Toole’s writing on room tuning, frequency response and EQ combines exact measurements with human perception, and as big a scientist as he is he remains skeptical of measurements, and with good reasons.

The process Nelson Pass uses is exactly right. His hypothesis is that a certain type of distortion, along with other important qualities, are what make for a great sounding amp, and lets face it, the process, and his effectiveness cannot be denied as not being scientific or financially successful. Far more scientific than designing or buying an amp based on THD% at 1 watt alone.

Bose is also very very scientific, but they come at the problem differently. Their question is: What is the least expensive to manufacture product we can make given what most consumers actually want to hear?" Does it work? They have 8,000 employees and approximately $4B in sales per Forbes:

Honestly, I don’t know how your average Bose product would measure, but you don’t get to these numbers without science. Assuming they measure poorly, doesn’t that mean measurements are all wrong?

The work Harman has done in getting listening panels together, and trying out different prototypes while adhering to previous science is also noteworthy. Most notably and recently with their testing of speaker dispersion which has resulted in the tweeter wave guides in the latest Revel speakers. They move science forward with each experiment, and then put that out into their products.

Regardless of the camp you fall into, crusty old measurements, perception measurements or individual iconoclast, we also must account for person to person variability. It’s been shown for instance that most people have poor sensitivity to phase shifts in speakers (like me), but if you are THAT person who has severe sensitivity to it, then all those studies don’t mean a thing.

My point is, let’s not define science as being purely in the domain of an oscilloscope. Science is defined by those who push the boundaries forward, and add to our understanding of human perception as well as electron behavior through a semi-conductor and air pressure in a room. If it’s frozen in 40 year old measurements, it’s not science, it's the worship of a dead icon.



My impression of Toole is he’s basically your garden variety old school gum flapper. 
My impression of Toole is he’s basically your garden variety old school gum flapper.

Toole is a giant and our ability to buy and enjoy the best in audio reproduction rests on the shoulders of men like him.  I don't agree with everything he says, but his engineering skills are top notch and his willingness to pass that to others is priceless.
I am active on ASR but I am trying to be less active. It’s hard because it’s one of the nicer communities that exist in this hobby so I keep coming back. I disagree with you about their measurements and claim that they are unscientific. They are more scientific then Nelson Pass. I won’t make any claims about Bose or Toole but I think Toole is highly respected (and often misquoted) on ASR as well. A lot of the points you make would actually be agreed upon by many ASR members.

Seperate from all of this is my observation that the forum consists largely of sycophants. Amir is too defensive when it comes to criticism of his measurement process, of which there has been plenty. He generally listens to feedback, but not without throwing a fit.

That being said, ASR is one of the best websites that exist online. His measurements are extremely helpful and beneficial for newcomers to the hobby like me.
To me there is a chasm between 'science' and technology applied in creating/making great hifi gear, and 'measurement' as a tool to evaluate gear and assess its quality.  Clearly the former is essential in pushing the state of art forward, both in terms of absolute performance, and in providing more consumers with improved performance at a given price point.  

ASR is an interesting site, like all sites it is seeking attention, membership and participation,  Like all reviews (from JA at Stereophile from way back posting measurements after the qualitative reviews) they need to be taken with a grain of salt, but it still can be informative at some general level, and more importantly, entertaining -- if you are into such things.  

For me, in my travels through this hobby, I just feel that there is only mediocre correlation between gear that sounds great and measures great.  Single ended tube amps driving horns can be a joy, but will measure very poorly.  In the end, it is about the music, I trust my ears to tell me what is better for me, above all else.  
Hey d2girls always a pleasure.

I am active on ASR but I am trying to be less active. It’s hard because it’s one of the nicer communities that exist in this hobby

Seems like that to me!!

I disagree with you about their measurements and claim that they are unscientific. They are more scientific then Nelson Pass.

I think you missed my whole point. Here's the thing, if you define "scientific" as "measurement driven" then you are right, ASR is "scientific."  However, science does not end at the oscilloscope, nor should oscilloscope measurements ascribe value or desirability. That's kind of my point. 

Pass is involved very much in research and development.  He has a target sound, behavior and distortion profile. His approach is very much to experiment, listen and measure. He's doing real life science. It may not be to achieve the targets the folks at ASR think he should, but he's actually pushing the state of the art in matching electronic devices to human experience.  That's science.

If I measure things defined 40 years ago, and give those measurements power to define how I spend money, well, unless I'm buying paint, or lubricant, that is not really scientific.