Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

@1extreme 

I used to follow their rankings because I thought well maybe at least they would identify a component that is a disaster but no longer. Why, because a few months ago I started listening again to my MHDT Orchid Tube DAC which as you probably all know is an R2R ladder DAC and I was amazed at the quality of music it was producing over my delta sigma DAC’s, especially with acoustical music like traditional jazz which I mostly listen too. Out of curiosity I looked on the ASR site and Amir had ranked the MHDT Pagoda DAC, which is the same as the Orchid DAC but with XLR outputs as the WORST DAC THEY HAVE EVER MEASURED. They have the Pagoda DAC at the farthest right in the red scale with the lowest ranking. Never even bothered listening to it.

Here is the problem you have with your argument: anyone with measurement gear can verify my findings.  No one can do that with your subjective claim.  Maybe you are right, but we don't know.  We don't know because you didn't follow any protocol to make sure you are only evaluating the sound of the device and nothing else.

We need to know for example that the knowledge of a DAC being R2R didn't influence your perception.  We need to know if  you match levels when evaluating audio gear. We need to know if you repeated your testing enough to arrive at reliable results.  We all know that in casual listening tests like yours, any and all perceptions can exist. I can listen to said R2R DAC and convince myself that it sounds like you say without said controls.  Indeed I routinely perceive differences that are not there.  We know they are not there because measurements show evidence of that, and blind controlled listening confirms the same.

This is what separates us and why ASR has Science in the middle of it.  Audio science with zero ambiguity says your evaluation of audio is faulty and without value.  When folks with those claims are tested formally, they cannot repeat their outcomes.  It is for this reason that claims like yours cannot be part of any paper for example submitted to Audio Engineering Society, ASA, etc.

As with other aspects of life, you can be a science denier and live your life happily.  Just don't put that opinion forward as an argument in mixed company.  And certainly don't use it in a thread aiming to create FUD against the other camp.  

Post removed 

Is there a moderator in the house? @tammyholt- These continued childish personal insults @mastering92, which are unfounded, have no place here, or anywhere frankly. You do a disservice to this forum and to others here by acting this way. I will remind you that you though that blind testing involved using a blind fold?  You also didn't understand the basic reason behind the number of test trials.

You are in no position to be taking on this attitude let alone acting superior to someone obviously far far more qualified.

 

Nice graphs. Looks like something from a high school powerpoint presentation in statistics class.

I’m sure that trained experts can do a better job here.

I’m sure Bill Nye the Science Guy could go head to head with you; and emerge as the champion.

 

anyone with measurement gear can verify my findings

Agreed. And anyone with ears can become audiophiles. Wait!! All human being have ears. But why do the non-audiophile brains not perceive the sound the way audiophiles do? I hope you will agree that ears receive the sound waves which are processed by the brain? Isn't that a question what the scientific community currently has no answers and is still researching - why each of us hear differently?

Is there a moderator in the house?

Gentle reminder that this is not ASR where you cry "mommy" and the moderator comes out and bans the poster.

someone obviously far far more qualified

A mechanic at a BMW service center also has scientific gears. He is qualified to service the car and tell what problems the car might have. But only an experienced driver can understand how a car handles and how it can perform on a race track or even on normal roads. Owning a piece of measurement gear and experiencing a drive are two separate things. Same applies to audio, which obviously you will not agree. But yes, there are some mechanics that have passion for driving.

This is common sense folks... what @milpai wrote.

Is there a moderator in the house?

Gentle reminder that this is not ASR where you cry "mommy" and the moderator comes out and bans the poster.

Exactly! One of my discussions has some back-and-forth, posters started chatting about food! Am I going to get upset and reach out to a moderator...asking them to stay on topic? No. Because I'm not the boss of anyone on this website. I can accept that some posters will add value to what I've written, while others may not. And that's perfectly fine with me. I don't mind...so long as we're having a good time.