Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)
About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.
Yes, each meteor produced a sizzling sound, like the sound of a frying pan.
Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.
ASR would have agreed with this sound reasoning based in elementary science.
Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.
Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.
I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.
The fireball sounds were a "long standing mystery" according to the article I read. Not some source of mockery for those who claimed to hear them, although there always are some mockers out there of absolutely everything. The problem comes from people insisting that their explanation of the sound they are perceiving is adequate when it isn't. The explanation for this one has been hard to come up with, partly because it's hard to be there on the scene with test equipment when it's happening. But if you come up with a plausible explanation and then test that idea by generating your own bright light that's sequenced to play a tune, and that tune can be audibly heard and recorded from a black surface receiving the light, then you've done something that sheds some light on the subject.
I'm sure they could do a blind test easily with this. Just cover people's eyes and have the indicate when they hear sound emitting from the surface. If they accurately indicate the right times that correspond with the light emission then you know they are actually hearing the photo acoustic effect. Blind test - should be easy to pass. If it's not, you have to ask yourself why, and at least question whether or not it's really a sound wave that you're perceiving.
If anything this whole story validates the usefulness of plausible hypothesis and then measurements to verify perceptions that are surprising and unexpected. When it comes to high end audio not too many serious people are doubting that audiophiles are really perceiving differences in sound. The debate is about people's unverified hypotheses about why they're perceiving those differences.
Here is an example of the good work the folks at Audio science review group do for the community. About a year ago, a speaker company known as GR research perpetrated a hoax on the audio community and came out with a cheap little speaker which they called a Little giant killer. It was essentially a cheap wooden box with a single small cheap driver in ’em. The simplest speaker you could possibly have.
News got out and eventually ASR decided to get their hands on them and test them. The results were shockingly bad and the hoax had been exposed.
It is work like this which the audiophile community needs. ASR are the unsung heroes of the audio community.
We have to test the product to see how good it really is. Its not a question of enjoyment. Its a question of TRUTH. If you dont care about how accurately your system can reproduce the signals you put in then you are in the wrong hobby. A speaker should not be enjoyable it should be neutral. The enjoyment should come from the music itself not the equipment.
Accuracy is a fools game, you can never achieve it no matter how good the measurements are, it will never sound identical to the actual musicians playing so why not enjoyable.
Amir really lives rent free in a lot of heads here.
I don’t understand the animus or energy directed against him or his site.
If it’s as ludicrous as you think, dismiss it and ignore it.
If it’s credible, at least in part, give some credit where that is due and then explain why it needs to improve.
--
This is a good take. It’s easy to bitch about competing sites/opinions, but at the end of the day, this is all internet noise. Even if it’s bad info, in your opinion, maybe it’s a good thing that "Science" site is furthering this ridiculous hobby in a way.
If anything this whole story validates the usefulness of plausible hypothesis and then measurements to verify perceptions that are surprising and unexpected.
The debate is about people's unverified hypotheses about why they're perceiving those differences.
+1 @asctim, I think you hit the nail on the head. It’s also about keeping an open mind.
There is some focus on something sounding better because it measures better. A bigger question to me is if you have gear that sounds better or worse than another is there a set of measurements to support the difference you hear?
Some categorize equipment based on how it measures despite the odds that the inferior numbers may have little to do with anything a human may or may not hear as good or bad.
I would say differences in design may well be a bigger factor even in the case where the 'better design' may have slightly worse specs (but still very good) compared with the runner up.
The enjoyment should come from the music itself not the equipment.
Then why are you nowhere to be found ... I mean, if you were at least following the music threads and I’ll take a stab at that, we’d witness a post every "once" in a while.
@nyev it’s a shame to let facts get in the way of a good story, but meteors do not travel at the speed of light.
it also would have been nice if you were able to tell us the nature of the specific radiation from these meteors that was causing the sizzling sound that you say you heard. i’m sure it was mentioned if your source was a credible scientific one as you say it is. i’m curious about that. Thanks.
@noske, interesting interview, thank you for sharing!
@bruce19 , no one ever said meteors travel at the speed of light! All your answers to your questions are in the Earth & Sky link I pasted above in this thread.
I read a book, The Final Theory, where the author asserts that gravitational force does not exist. He believes in the "atomic expansion theory" where every atom in the universe is expanding at the same rate, thus our inability to observe the changes. Per the author, we are standing on a large expanding "beach ball" being inflated and expanding at around 30fps, and THAT is what we feel under our feet. He goes onto explain how airplanes fly, the weather patterns, and how large bodies attract each other.
It may all be BS, but the confirmation/rebuttal is clearly above my paygrade.
Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors.
@bruce19look again and read carefully; no one suggested the meteors are traveling at the speed of light. Is it the meteors that are travelling at the speed of light, or the radiation that is travelling that fast? What would make more sense to you? Hmmm.
Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light
Amir really lives rent free in a lot of heads here.
I don't understand the animus or energy directed against him or his site.
If it's as ludicrous as you think, dismiss it and ignore it.
If it's credible, at least in part, give some credit where that is due and then explain why it needs to improve.
I think this is the most relevant post in this thread @hilde45. I see some making up obvious lies and I wonder what their motivation is. Beyond the obvious libel, are they not at all concerned with how that will reflect on them, or are they so sure of the mob mentality they feel they will get away with it? I think the behavior disheartening. There is enough of that in politics.
@nyevI am a bit of an amateur astronomer, so the meteor thing was not new to me. What may be new to you is that the theory of radio waves being the cause is not in vogue. I understand they were not able to prove it. The latest theory is the effect is photo acoustic; light/heat, the predominant radiation heating objects near us. If your story/cause was incorrect, what conclusion should we draw about audio?
The electrophonic theory, that the sound is from RF exciting things near us, seems to be a torch mainly carried by amateur scientists. I think professional scientists have moved on. All meteors will produce RF, the levels are not high enough, and what is around us is not effective as a receiver.
In case that got lost in the last post, the RF theory, has not been proven. It remains mainly the purvey of amateur scientist. Professional scientists measure the RF, then try to replicate and see if there is any sound. That experimentation has failed. The current theory is that it is light / heat energy. They have been able to simulate using similar light/heat levels and create sound.
In a universe where, "... meteors, traveling at the speed of light ... " is real, all things are possible, including two devices sounding different after passing a null difference test. 😂
ASR is biblical Canon to some people. I find ASR is an interesting read and can help somewhat but they never really nail why the really mesmerizing components sound awesome but measure as nothing special.
We can go opposite way: if audio waves are unmeasurable and so sophisticated for current audio measuring equipment, how companies then produce the equipment then? Do they measure the results at all? Or maybe they just use alchemy, philosophic stone and any other magic when engineering the equipment to reproduce audio for out fragile ears :). Sure no need to measure, just price it $1,000 000 and it will instantly become the best “audiophile” product or the year, lol.
"This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings."
Since the final result was validated by measuring, doesn't this validate ASR's approach?
I have a few products that ASR have trashed and a few they most likely would and they all sound great.
The other night I wanted to go to a local restaurant.... my lady said " I heard that place is horrible" I said " who told you that, it's great " She said "Chris" I said " I thought Chris is a vegetarian?"
This hobby has far too many variables to make choices without listening to the gear you want to buy.
Buying based on ASR recommendations is kind of like having a vegetarian tell you that a steak house is bad.
@jssmith , arguably it does especially since after recording the RF radiation and sound at the same time they simulated the RF radiation and no sound was heard. A few tried and no sound was heard. Correlation was assumed to be causation and it turns out that was not the case. The current theory is it is a thermal effect from visible and IR radiation. Scientists have recreated the measured levels and have been able to generate an acoustic effect similar to reported incidents. I understand one issue with the RF theory was lack of an identified receiver that could generate an acoustic effect, especially true as the effect has been reported for centuries dating back to a time when very little was made of metal.
We can go opposite way: if audio waves are unmeasurable and so sophisticated for current audio measuring equipment, how companies then produce the equipment then? Do they measure the results at all? Or maybe they just use alchemy, philosophic stone and any other magic when engineering the equipment to reproduce audio for our fragile ears :). Sure no need to measure, just price it $1,000 000 and it will instantly become the best “audiophile” product or the year, lol.
That’s not how it works. Of course they measure the audio products they build - during and after the design process.The most important things are a waveform of the output stage (null test), measuring with an oscilloscope, and a select few measurements that go beyond just the standard 5 or so. SINAD is an outdated way to measure audio equipment. Yet it is used as a gold-standard on ASR.
Then you must also consider the cost of parts, paying their employees, advertising costs, office space etc. There is obviously a lot that goes in to running a business.
Common sense tells us that for a hundred bucks, we shouldn’t be able to get a DAC with superlative performance, but ASR (Audio Science Review) tells us of course we can!
The word "Science" in the website should hint at a hypthosesis for why audio gear meant for the same purpose sounds different; and should welcome 3rd party testing - like other real scientists. However, that is not allowed over there...just try to challenge the results - suggest further measurements. Open the device up. Take a picture of internals and indentify the parts used. Do a reliability test. None of those things are done....not to mention countless errors in testing.
We actually do have fragile ears - since once hearing loss is an avanced stage (bad genetics or excessive exposure to loud environments) we can’t get it back. Using a hearing aid is not fun. I have some family members who unfortunately lost their hearing in one or more ears...
Human hearing (in particular the sensitivity; to detect changes in tonality) is incredibly advanced. As such, we can hear small differences easily; providing our hearing is not compromised.
I have a few products that ASR have trashed and a few they most likely would and they all sound great.
Smart design choices. They built a machine that could provide great sound quality; and left out meaningless specs like 0.0000000000002 % THD and SINAD as a yardstick measurement...
Buying based on ASR recommendations is kind of like having a vegetarian tell you that a steak house is bad.
Golden statement...
Totally different impressions and MEASUREMENTS on Head-Fi for the same product. @amir_asr likes to suggest that his "instrumentation" is so much more accurate than what others are using. Well with that logic, upgrade every 3 months or whenever AP releases a new flagship audio analyzer. This means that every former product was substandard or less accurate in some way. Right? Check out my profile to see a photo as proof of this IEM review...
And the way he EQs headphones is painful to see. It makes me furious. He simply drags up/down a line on a log EQ so it inherently influences the frequencies around that octave as well; rather than fine-tune with proper notches in place and compensate with a preamp option in the software so the levels are not compromised. You’re welcome @amir_asr
Then he has the NERVE to message me on here and tell me that I can’t hear a difference. Nobody is golden-eared - sure, however if we have good hearing and are trained listeners with knowledge, we can certainly discern all kinds of differences in audio equipment.
he says:
"And on topic of the above research showing how poor people like you in discerning differences in speakers let alone electronics:"
Also bad grammar. I am not poor, but he is suggesting that "people like me" do poorly at discerning differences in speakers and other electronics...people like me...lol
I'm all for objectifying audio if it is possible and correlates to what we hear. Speaker FR is one example of that.
May I ask what are your thoughts on listening as a way to evaluate a component? Do you ever listen to some of the components you measure and wonder why you hear a difference and why measurements don't explain it? Why something may measure good and sound bad?
This is not an attack on you. But internet may come across that way. I am Just curious.
Sensory experience is certainly imperfect; like all human senses. We are only human - not gods. Because we experience music from our audio systems with our senses, that sensory experience is all-important. All of it - sight, hearing, etc.
Hence the requirement for "blind" testing.
Blind listening for audio is a flawed practice. I don’t know anyone in pro audio that uses it. For example, at AIR studios in London, a power amplifier for their main control room (I believe) was chosen based on listening sessions. They bought a Class A/B power amp from a UK-based company called ATC.
Well except for probably most professional speaker companies. We use blind testing quite regularly in cross-over development, passive and active. There are a lot of trade-offs around crossover points, and with speakers, artifacts are audible. We used to do more blind testing around the electronics themselves, but we have a good handle on that from a measurement standpoint. We measure, we measure a lot.
😂Those who are fond of conducting blind tests for audio believe that a certain number of successful trials is sufficient for proving whether we can hear a difference or not... how did we arrive at this number of successful trials?!
Basic, well understood statistical functions. The more tests you do, the higher the confidence.
😅 8/10 or even 10/10 successful trials could be riddled with guesses and inaccurate auditory memory recall. The test subject may not admit they were unsure, because they wanted to be correct and prove their ability to be golden-eared to their peers.
Much of what you wrote is the whole point. If you are unsure and effectively guessing, that will show up in the randomness of the result. If your auditory memory is not good enough for a basic test, explain the high confidence of listening days, weeks, months apart?
Wearing a blindfold also creates problems that make an objective listening test more difficult. Blindfolds may hamper with the frequency response characteristics of speakers and headphones.
Um, blind testing has nothing to do with wearing a blind fold.
Because blindfolds are made of soft fabric with padding or a sheet of fabric, placing them over the eyes creates a sound-absorbing pocket, whereby the sound waves from speakers would not disperse as evenly with it on.
Blind testing still has nothing to do with blindfolds.
A blindfold may interfere with achieving a proper seal with over-the-ear headphones and on-ear headphones.
No one to my knowledge blind tests headphones.The feel of the headphone would be too obvious and you would know which is which. Defeats the purpose.
Lastly (for now anyway), you must acknowledge at some point in your subconcious that a "blind test" which you believe is wholly unfallible is being conducted.
Well no. For one, many here are convinced they are totally fallible. Explain why they would fail a blind test? It is really simple. You are listing to something. You don’t know what it is.
They have so little regard at ASR for actually listening to components that they don’t even bother listening to them. Amir performs all these tests and I don’t believe he listens to a single one or even has a reference system to drop the component in to get an impression. I read someone post here asking why the OP doesn’t take his post over there to get their reaction. Just read some of the posts there to find out. They are all a bunch of evangelical engineers who come down like a gang in mass on any one who even implies "but I like the way it sounds" just like a fundamentalist church comes down on someone questioning Adam & Eve or Noah's Ark.
I used to follow their rankings because I thought well maybe at least they would identify a component that is a disaster but no longer. Why, because a few months ago I started listening again to my MHDT Orchid Tube DAC which as you probably all know is an R2R ladder DAC and I was amazed at the quality of music it was producing over my delta sigma DAC’s, especially with acoustical music like traditional jazz which I mostly listen too. Out of curiosity I looked on the ASR site and Amir had ranked the MHDT Pagoda DAC, which is the same as the Orchid DAC but with XLR outputs as the WORST DAC THEY HAVE EVER MEASURED. They have the Pagoda DAC at the farthest right in the red scale with the lowest ranking. Never even bothered listening to it.
What’s sad is a lot of people new to audio go there to get recommendations. I really don’t care if engineers who can’t hear differences in components use the ASR guide for purchases. But I do care that some newer audiophiles who may be able to hear the differences never get a chance until it’s too late. And worse, ASR does wield a lot of power and influence and can hurt a company producing good audio products. I notice now that MHDT is no longer sold domestically in the US through a US distributor as it used to be. You now have to order one directly from MHDT in Taiwan.
That’s not how it works. Of course they measure the audio products they build - during and after the design process.The most important things are a waveform of the output stage (null test), measuring with an oscilloscope, and a select few measurements that go beyond just the standard 5 or so. SINAD is an outdated way to measure audio equipment. Yet it is used as a gold-standard on ASR.
Can you tell what those 5 select other measurements? You appear to put yourself forth as an expert, so as opposed to a vague statement, should you not detail that information? I mainly go to ASR for speaker reviews. Professional interest. SINAD is not even part of those measurements, probably because speakers don't make noise. Because they are using a Klippel system, their measurement suite is far more extensive than say Stereophile. What is published is extensive and is similar to what we would measure internally. We have proprietary weighting functions for some measurements. I would not expect a review site to have that. I counted 11 graphs on a DAC review, in addition to SINAD. I picked a DAC since that seems to be most discussed here.
Common sense tells us that for a hundred bucks, we shouldn’t be able to get a DAC with superlative performance, but ASR (Audio Science Review) tells us of course we can!
Whose common sense? Asian manufacturing costs, Asian parts costs, a DAC a model or two down from the top, high volume, low margin business model. In our speakers with digital in, the DAC section, with enough performance to have no audible impact, is not expensive. How do we know there is no audible impact? Measurements and blind listening tests.
The word "Science" in the website should hint at a hypthosesis for why audio gear meant for the same purpose sounds different; and should welcome 3rd party testing - like other real scientists.
Have you noticed that ASR is publishing reviews including extensive measurements from people other than ones done by Amir. The science in the website would indicate that proving something sounds different would be a necessary first step.
However, that is not allowed over there...just try to challenge the results - suggest further measurements.
Here I will agree. Amir can be quite defensive and arrogant. He is not as open minded as he should be.
Open the device up. Take a picture of internals and indentify the parts used. Do a reliability test. None of those things are done....not to mention countless errors in testing.
Do reliability testing? Can you name even one review website that does reliability testing? Do you have any idea of the cost and time required? I saw on several reviews pictures of internals. Not all, but a lot. What percentage of audiophiles are able to accurately assess the internals of a product?
Most of the speaker tests do not have errors. Some of the more esoteric speakers I feel have errors in testing. Some of the claims errors I have seen are more sour grapes. Given the volume of testing, errors are to be expected. Do you think other test sites are perfect? Some of the explanations and tests I see done by others, especially with lesser equipment make me shake my head.
Totally different impressions and MEASUREMENTS on Head-Fi for the same product. @amir_asr likes to suggest that his "instrumentation" is so much more accurate than what others are using.
Have you noticed that numerous reviewers are popping up and using guess what, the exact same equipment as ASR. Imitation is the best form of flattery I guess. His equipment does appear superior to most traditional sites. Headphone testing is very hard to do repeatedly to address that specific issue.
Well with that logic, upgrade every 3 months or whenever AP releases a new flagship audio analyzer. This means that every former product was substandard or less accurate in some way.
This is not a logical statement. As the test gear currently is accurate enough to identify artifacts we may hear, further accuracy only plays to marketing specifications, not audible artifacts. What is required now, is better tools for interpreting measurements, not better measurements. For speakers, there is room for improvements for measuring distortion over emission angle, but that applies to everyone.
And the way he EQs headphones is painful to see. It makes me furious. He simply drags up/down a line on a log EQ so it inherently influences the frequencies around that octave as well; rather than fine-tune with proper notches in place and compensate with a preamp option in the software so the levels are not compromised. You’re welcome @amir_asr
I will leave it up to others to make their determination if your statement is true of not.
Interesting take. I’ve heard meteors as well (one over Tampa, FL around 1973 that was seen and heard by tens of thousands of people that night and I was at an outdoor concert - the meteor was LOUD).
But, yeah, “SCIENCE!” said…
NOW do a little online searching for validity of eyewitness accounts of accidents.
It’s really simple, right? Four people (count two of them - the drivers - as participant/observers too) witness a collision at a 4-way intersection.
Which story is correct? Police and claims adjusters and courtrooms are often employed to adjudicate these things because people’s memories are fungible things; what I see isn’t necessarily what you see based on your perspective and focus and abilities to recall what you saw (or think you did).
It’s been 60 years since JFK was assassinated, and eyewitnesses insisted on shots from other areas of Dealy Plaza and the grassy knoll and people are still arguing about it today.
Basic measurements are only a benchmark, an objective standard, but how something SOUNDS is purely subjective and has to take into account intangibles like combined elements in the system, the room acoustics, speaker placement, and the listener, right?
A lot of this boils down to simple opinion, and everyone has one of those.
Basic measurements are only a benchmark, an objective standard, but how something SOUNDS is purely subjective and has to take into account intangibles like combined elements in the system, the room acoustics, speaker placement, and the listener, right?
Speakers are my thing so let's talk speakers. A reviewer does a review of a speaker. That review will tell you every piece of audio equipment used in the speaker review, probably what they had for breakfast and if they had a bathroom visit that morning. Most of that information will be useless to determine how that speaker will work for others. Missing from the review will be room dimensions, what specific treatments are in the room, the exact speaker and toe in and listener location though the latter may have some vague description. Also missing is usually the reviewers preference around imaging and soundstage. There are trade offs. Basically most listener reviews lack all the information you need to understand how that speaker will behave in your chosen listening space.
Give me a full test set and I can provide far more useful information about how it will behave in your room, options for toe-in, problem areas for acoustic treatments, options for boundary reinforcement and treatment in small rooms, amplifier compatibility, realistically how loud it can be played.
They have so little regard at ASR for actually listening to components that they don’t even bother listening to them. Amir performs all these tests and I don’t believe he listens to a single one or even has a reference system to drop the component in to get an impression.
You need to put aside "belief" and substitute reality. I listen to huge amount of audio gear as part of my testing. Every speaker, headphone, headphone amp and even some audio tweaks such as power conditioners, cables, etc. have listening tests. This adds up to about 200 devices a year that get reviewed with listening tests.
I have a reference system as well where equipment is tested. For example, I used RME ADI-2 Pro ($2000) driving a $4000 Dan Clark headphone to test a headphone amp. Cables are tested the same way if they are interconnects and such. Ditto for power conditioners, etc. Here is an example from this week alone, a headphone amp:
"This is a review, listening tests and detailed measurements of the Eleven XIAudio Broadway balanced battery operated desktop headphone amplifier. "
Far field devices like speakers get tested in and against my nearly $100K main audio system. Power is provided by $20K in amplification for example. I list these prices not that they should matter, but I suspect matter a lot to you as to what makes a "reference system."
But no, I don't listen to everything. Measurements so powerfully describe the performance of such devices, showing impairments well below hearing for example, where it makes no sense for me to listen to them. And proceed to make up stuff like subjective reviewers do. Mind you, if measurements show a problem, I do listen even in that category. Here is an example of that, the PS Audio DirectStream DAC:
"I started the testing with my audiophile, audio-show, test tracks. You know, the very well recorded track with lucious detail and "black backgrounds." I immediately noticed lack of detail in PerfectWave DS DAC. It was as if someone just put a barrier between you and the source. Mind you, it was subtle but it was there. I repeated this a few times and while it was not always there with all music, I could spot it on some tracks.
Next I played some of my bass heaving tracks i use for headphone testing. Here, it was easy to notice that bass impact was softented. But also, highs were exaggerated due to higher distortion. Despite loss of high frequency hearing, I found that accentuation unpleasant. WIth tracks that had lisping issues with female vocals for example, the DS DAC made that a lot worse."
You could have figured all of this out by simply going and looking. But instead you just repeated incorrect talking points.
Exactly. The measurements and data have little value if not correlated with what we hear.
They absolutely are correlated with what you hear based on science of psychoacoustics which is entirely based on listening tests. Problem you all have is that you don't to hear, pun intended, the consequences of that. Instead you want to live in a fantasy world where differences that don't exist or are below audible, are so obvious that your wife can hear from the kitchen. That kind of made up effect is psychological and can never be caught or explained in a proper, science based, review of audio product.
I really don't think ASR has much power or influence at all. It's just a noisy group with grievances, which is very common today.
Well, I know more than one company that has completely retooled their entire product development and product strategy precisely because of the work we have been doing (think Schiit). So we do have influence, having nearly 3 million visitors a month which puts us at or at the top of all audio based sites on the Internet.
The main grievances, as evidenced by threads like this, are people who don't want to come along. They rather cling to "beliefs" than accept reality of objective and science based evaluation of audio. Why? Because in some cases it negates their praise for some product. Instead of learning from that, they choose to go after ASR. And for what? Us providing more data than you had before about your purchases? There is no better definition of asking, heck demanding, to put one's head in the sand.
By all means, do that but don't make up stuff about ASR. At least do some homework before creating FUD like this.
.... you want to live in a fantasy world where differences that don't exist or are below audible, are so obvious that your wife can hear from the kitchen ...
You live in a fantasy world. You don't know anything at all about me. But you have an active imagination, I'll give you that.
Basic measurements are only a benchmark, an objective standard, but how something SOUNDS is purely subjective and has to take into account intangibles like combined elements in the system, the room acoustics, speaker placement, and the listener, right?
Yes and no. In most cases, audio gear is designed to have its own performance or you would never be able to assemble any system. Or trust any reviewer whatsoever, right? For example, a source device such as a DAC has a low impedance of say 100 to 200 Ohm. The pre-amp then has an impedance of at least 10X that. This way you get full voltage transfer which is what we want.
On the other hand, there are tube amps with high output impedance which then interact with the frequency response of the speaker. This causes tonality to shift in the system not because of anything good, but because of poor design. A solid state amp will have well below 1 ohm impedance as to eliminate this effect. You could hear these effects using either listening tests (if the person is properly trained and difference large enough), or measurements.
In vast majority of cases though, the modular aspect of audio allows us to independently test and evaluate a component by itself. Measurements are much more powerful in this regard because audiophiles as a group are terrible at detecting non-linear artifacts. But even for things like speakers where distortions are apparent, we are a) mostly alike when it comes to preferences and b) non-professionally trained listeners including reviewers and dealers are terrible at providing consistent and proper feedback. Please see this formal study:
Indeed, the subjective data from audio reviewers is so bad that you need 10 times as many of them to equal properly and formally trained speaker listeners! That is the problem with subjective remarks from audiophiles or audiophile press. It is so unreliable that it is not worth paying attention to. The same study by the way shows that listener preference is similar among a dozen different listener classes:
See how the ranking of each speaker did not matter (different colors) regardless of who was listening to it (X axis). Green speaker for example was bad no matter who was evaluating it, in controlled tests that is.
If you want to see a more detailed explanation of that, I have done a video on it:
No matter what you believe or what side of the argument you are on, I am quite sure that if all involved stuck purely to facts and what is being discussed, the conversation would be more cordial. Unsubstantiated personal slights, unsubstantiated accusations, and generally nastiness that has nothing to do with audio are not going to resolve anything.
I used to follow their rankings because I thought well maybe at least they would identify a component that is a disaster but no longer. Why, because a few months ago I started listening again to my MHDT Orchid Tube DAC which as you probably all know is an R2R ladder DAC and I was amazed at the quality of music it was producing over my delta sigma DAC’s, especially with acoustical music like traditional jazz which I mostly listen too. Out of curiosity I looked on the ASR site and Amir had ranked the MHDT Pagoda DAC, which is the same as the Orchid DAC but with XLR outputs as the WORST DAC THEY HAVE EVER MEASURED. They have the Pagoda DAC at the farthest right in the red scale with the lowest ranking. Never even bothered listening to it.
Here is the problem you have with your argument: anyone with measurement gear can verify my findings. No one can do that with your subjective claim. Maybe you are right, but we don't know. We don't know because you didn't follow any protocol to make sure you are only evaluating the sound of the device and nothing else.
We need to know for example that the knowledge of a DAC being R2R didn't influence your perception. We need to know if you match levels when evaluating audio gear. We need to know if you repeated your testing enough to arrive at reliable results. We all know that in casual listening tests like yours, any and all perceptions can exist. I can listen to said R2R DAC and convince myself that it sounds like you say without said controls. Indeed I routinely perceive differences that are not there. We know they are not there because measurements show evidence of that, and blind controlled listening confirms the same.
This is what separates us and why ASR has Science in the middle of it. Audio science with zero ambiguity says your evaluation of audio is faulty and without value. When folks with those claims are tested formally, they cannot repeat their outcomes. It is for this reason that claims like yours cannot be part of any paper for example submitted to Audio Engineering Society, ASA, etc.
As with other aspects of life, you can be a science denier and live your life happily. Just don't put that opinion forward as an argument in mixed company. And certainly don't use it in a thread aiming to create FUD against the other camp.
Is there a moderator in the house? @tammyholt? - These continued childish personal insults @mastering92, which are unfounded, have no place here, or anywhere frankly. You do a disservice to this forum and to others here by acting this way. I will remind you that you though that blind testing involved using a blind fold? You also didn't understand the basic reason behind the number of test trials.
You are in no position to be taking on this attitude let alone acting superior to someone obviously far far more qualified.
Nice graphs. Looks like something from a high school powerpoint presentation in statistics class.
I’m sure that trained experts can do a better job here.
You must have a verified phone number and physical address in order to post in the Audiogon Forums. Please return to Audiogon.com and complete this step. If you have any questions please contact Support.
