Audio Science Review = "The better the measurement, the better the sound" philosophy

In which a (well-meaning, I'm sure) poster is so angered by ASR guys that he mistakes John Darko for one. Lol. If that's what you got out of that video, I don't know what to tell you buddy 😂

@fleschler he doesn't say that in the video. darko has been quite critical of the 'measurements over everything' crowd. in fact, the "audiophiles are snobs (with money to burn" title is in scare quotes because he spends most of that video directly criticizing their closed-mindedness - he calls the dogmatic measurement people inverse snobs. i don't think you actually watched it 😂

was over there reading up on something earlier today and i will agree with whoever mentioned it that it is absolutely hilarious to watch the couple guys falling all over themselves to kiss amir's ring on every review. no self respect. he's like their dad 😂

all told, i think the site is fine. whatever. we've all been around the block. some people think they have all the answers and they look kind of silly...but many of us do the same, perhaps in other areas of life. what can you do 

@jtgofish banning someone from posting on your forum is not a human rights violation. lmao

@jtgofish it's a privately owned site. they can set whatever rules they want. if i had a site i could set a rule that says if you criticize phil collins "face value" album in any way whatsoever, you are permanently banned. and no amount of whining or "but my free speech" could change it. that's just how it goes. i'm not infringing on anyones right to criticize phil collins albums, you are perfectly free to do that anywhere else online, but on my site it's not gonna fly. this is all those dudes are doing. the mods there are just weenies who think they know everything, they aren't violating anybody's rights. this whole thread proves it

measuring a power cable that’s not plugged into anything and then claiming it has no impact, redlining amps and transducers way past manufacturer recommended spec and pointing out that they distort under these conditions, "listening" at active war zone levels in near field or on headphones, evaluating stereo imaging in mono, claiming nobody else with an AP analyzer is using theirs correctly, and so on. just another day in the life of the internet’s premier audio expert 😂

doxxing somebody over their opinions about stereo equipment is one of the saddest things i've ever heard. unreal

@fleschler "Are these music genres adequate to evaluate audio equipment?"

yes. the music you like to listen to is the music you should "use" to evaluate equipment. 

@fleschler re: music choices, we are definitely in agreement then.

anecdotally, i have owned a few of of the highly praised chinese dacs on the ASR list, including the TOTL topping from a couple years ago, and it simply didn't do as much for me as my little $100 schiit dac (topping and smsl sound were both thinner, more "computer-y" to my ears). so i got rid of them.

i don't play in the "audiogon budget" arena due more important financial obligations - my gear is all "cheap" by this site's standards - but i have messed around with a lot of sub 1k high performance (or "high performance" rather) gear over the last few years. and most of it has been a waste of money. for me, for me.

i actually love sites and channels that:

1. cover budget friendly gear on occasion 

2. acknowledge that how it sounds (rather than how it measures) is the most important criterion at the end of the day

3. doesn't pretend that spending more on hifi is always a waste of money, but likewise doesn't pretend that you need 25k to really get good sound

4. keeps the obvious truth at the center of things: at the end of the day, it's all about music and enjoyment 

so for these reasons i like michael lavorgna's site, darko's youtube, handful of others. ASR only meets (1) of the above 

worth noting that you’ve got several other fairly prominent measurement focused reviewers out there (atkinson, erin, goldensound) and ASR members have argued, downplayed, & criticized all of them. and who here has a problem with any of these reviewers? i think they do great work.

and i personally don’t think it’s really about objectivity or "science" to some of those asr guys, and i don't think most so-called subjectivists have a problem with an engineering-based critique of a piece of equipment. it's really just about not being a jerk and pretending you have all the answers about hi-fi because you can perform a test tone sweep. 

@quiet do mine

you fellas are ridiculous. "this is war" lmao. has anyone called ASR "snake oil" yet? can't believe how bad this thread got in such a short time 😂

@nonoise superb read, appreciate you posting that 

@amir_asr cool of you to show up. i have a question - would you say that "if

[insert audiophile claim] can't be measured by my tests, it doesn't exist" is a fair description of your view? why or why not? 

 

Let me first tell you that there is far more to conclusions we draw than measurements. Myself and many members of the forum are engineers and understand how your audio devices operate. We then combine this with careful measurements. And then look at what audio research (published) tells us. If all three arrows point the same way across testing multiple categories of products, then we have very high confidence in our conclusions as to efficacy of such claims.

As an example of above, we know how power supplies work in audio products. So when someone says this power cable "filters" noise that then does the same in your audio output, we can analyze this on all fronts. We know that there are multiple filters working far more effectively in your audio gear than anything a power cable (or conditioner) can do. We then combine that knowledge by showing that said power cable provided no filtering. And even the company itself showed no such evidence. We then go further and produce highly distorted AC waveform and show that the audio gear did its job and nothing changed in its output. After testing a number of such products with the same outcome, we then have a very high confidence answer with strong data to back it.

Please note that this is VERY different than what other objectivists do. I put in tons and tons of effort in testing these audiophile claims. I have tested more interconnects and power conditioners than I can keep track of. And when a new one is offered to me, I test it again in the thought that it may be the one that shows a difference. This should show you the openness I bring to this field. There is nothing "cult-like" about what we do.

Note that there situations where measurements provide part of the answer but not all. Speaker and headphone measurements are very powerful in their predictive power but not sufficient. We don’t for example fully know the effect of radiation patter for a speaker in different rooms and for different people. Measurements do however rule out the poor designs and do so with authority. Maybe some of those are still good but there are so many good choices with good engineering so why take a chance?

I recently recommended an IEM. A bunch of people purchased it. About 70% love it and can’t imagine how great this $50 IEM is. 10% to 15% say it sounds good but better with EQ. 10 to 15% say it is not for them. This shows how powerful imperfect or incomplete measurements can still be.

my question @amir_asr was whether "if it cannot be measured then it doesn’t exist" is an accurate description of your views.

all of the above is perfectly fine and i don’t take issue with any of it. and i appreciate you answering at length.

but i am genuinely curious whether you believe there are real-life auditory experiences with recorded sound that simply fall outside the purview of your approach to evaluating gear. in other words, do you think that everyone who hears a difference that isn’t reflected in your tests (to say nothing of the tests of other measurement-focused reviewers like goldensound and erin - guys whom i’m given to understand have been banned from your forum for mysterious reasons) is simply delusional? or is it possible that this phenomenon is just an obvious example of the incomplete/imperfect measurements you refer to in your last paragraph?

i find the "he’s just doing this for money" critique ridiculous. way to offer up a perfect mirror image of the bad faith argument you can see on ASR every single day. "we must conclude that by making poor (according to my arbitrary tests) design choices, these manufacturers are clearly only out to deceive gullible audiophiles" "that’s just snake oil" etc : "amir is only in this for money". it’s the same stupid argument from both camps. better to judge peoples opinions on their merits instead of reading malice into every little thing you disagree with

@crymeanaudioriver let the record show that never once have i referred to myself as an intellectual 😂

seems like this whole thing is really just a left-brain vs right-brain debate. anecdotally, all the best sounding systems ive heard have been put together by people who cant be bothered with such a binary view of things. instead of either/or it’s both/and

there's a pass int-150 for 3k on tmr right now...very tempted to do my own "testing" 

in my experience, mood and environment and company and context affect enjoyment at least as much the quality (sub- or objective, take your pick) of the playback system. some of the best experiences with recorded music you can have might involve mono, 78rpm, poor/decayed/scratched/noisy/generally subpar recordings, crappy bar speakers, harsh PA's, etc. in other words if you need everything to sound like Aja to really get into it that's fine. i don't.

would also note that if your goal in building a home system is to get the stuff that's best-engineered according to one website's measurements, that's a little different from setting the goal to maximize your enjoyment of music. you very well might love your topping stack and that's great. more power to you! but you also might not. and what then?

from my perspective there's no way around the fact that you have to listen to know whether something is going to work for you. this isn't an argument against measuring equipment, but it is an argument against relying on "the science" instead of your ears

relevant to the topic at hand (as well as the initial confusion between darko and ASR outlooks) - here john darko interviews paul barton of psb, who explains the importance of measurements and listening tests (following toole’s methodology) to his process, as well as the subjective choices he has to make as he finalizes his designs. fascinating stuff, confirms much of what both "camps" are saying:

@prof appreciate the thoughtful reply. to your point about studies re: general preferences, there is a massive, ongoing replication crisis in the soft sciences to contend with. this doesn’t mean that all studies are bad - far from it - only that "there is research which shows that..." is not a silver bullet. i hasten to add that (much of) physics does not appear to have this problem. but there are a great many studies which have been proven impossible to reproduce, so i approach this sort of "studies show that more people prefer strawberries to peaches" reasoning with caution. human motivation is enormously complex and the factors which lead person x to prefer A over B can (and very often does) vary at the individual level.

the assumption that there is a fundamentally rational, measurable basis for emotional responses to an experience (like listening to music) is flawed. no one denies that there is ample research and data to back up assertions about engineering, and everyone is aware that there are results from controlled listening tests that indicate the preferences of the majority (of test participants anyway). my view is that figuring out "what sounds good" is something different. life is not a controlled listening experiment, and our individual experiences with music are shaped by circumstances which cannot be generalized, let alone replicated. 

the act of measuring gear is not a threat or a problem for anybody. i agree with amir that more information about a product is always a good thing. where i disagree is when people insist that i shouldn't trust my senses when i am determining whether a piece of equipment is good for my system.  reviewers with a subjective bent don't bother me at all, for the same reason - seems blandly obvious that what sounds good in john darko's room might sound like junk in mine. how do i figure it out? i buy the thing and return (or sell it) if it sucks!

i've done this with topping gear which measures great on ASR. some people love topping - i don't say they're wrong or have bad hearing or only care about specs. my $100 schiit dac simply sounded better and the topping gear was ultimately not worth the expense. why? i don't know! and at the end of the day, i don't care. will i spend another thousand or two bucks trying to better the schiit? probably. i like trying new things and seeing if i can hear the differences - it's fun

@djones51 i appreciate that. 

i became interested in this general topic due to ASR guys on reddit telling me that i had no idea what i was talking about when all i said was that i generally prefer vinyl to digital and that it sounds better to me. "you are an idiot, digital is objectively better" etc. like, ok buddy. that's your opinion. have had many other similar exchanges over the past couple years in various forums. i simply don't agree that better spec (however defined) = better sound in all cases, and i think that there are limits to what testing and measurements can actually tell us, for the reasons i offer in my previous post. 

@crymeanaudioriver i am not talking about physical ability (or limitations). i am referring to the unique circumstances in our lives that inform our personal, emotional responses to experiences. 

@axo1989 very thoughtful post

@prof i think we basically agree - when push comes to shove, the one view i've expressed in this thread that i will 100% 'go to the mat' for is that listening to music is the only way to really know whether a piece of equipment is going to work for you. and i do think there are aspects of the experience of listening which vary by individual and affect enjoyment. testing is fine, and extremely valuable for a designer, but it does not tell you how something is going to affect you on an emotional level. and from my perspective, the whole point of this hobby is to deepen our love and appreciation of music - the emotional response. how we get there - by graphs and charts and spec sheets, by trial and error, even by wasting a ton of money following some reviewer's hype - is ultimately a pretty low stakes conversation. as long as music and enjoyment are front and center, that's what is important (to me)

ive devised a blind listening test of my own. i flip through a bin of records blindfolded and choose one at random. then i put it on, crack a beer, and enjoy the music. you guys should try this - VERY revealing, and unimpeachable on the scientific level.

when all is said and done, if i need to know how something sounds, i'm simply going to go listen to it and make my own judgements

again, the idea that a handful of measurements conducted by the members of one site somehow invalidates the actual musical listening experiences of generations of listeners just seems kind of absurd. you can learn a lot from measuring gear, but how that translates into "what sounds good" is where things get murky. i can go to ASR right now and say "rogue sphinx is a killer little amp for the money. ive owned it and it rules. trust your ears" and people who have never even heard one will immediately score points off me because it measured "bad". like buddy, try it and then there might be a conversation. otherwise you’re just repeating someone else’s argument

also, axo1989 posted links to some contentious threads there and i recommend checking them out - despite how some of those fellas speak to others, fundamental issues like "what to measure" are far from settled