I Am Tired of Bogus Measurements

You argue too much.

Is that a response to asking for your comments on the full AES article you linked to (that is behind a firewall)?   I find a topic called, "I Am Tired of Bogus Measurements" without substantiating that measurements are bogus as argumentative, but is not the whole point of being here to discuss?

That's what stereophile does but they don't hand out prizes or shame the product. 

That could be the result of one being 100% advertiser supported, and the other deriving 0% from advertisements.

Under the handling conditions, a change in the capacitance of the cable due to the signal causes a change in the sound quality.

I will assume almost no one here has an AES membership?  Can you post a summary and comment on the test methodology? The parameters in the testing do not appear at all relevant compared to our audio systems.

Wait, I just thought of a measurement that is relevant, the FR of the room.

Well that would be impossible since every speaker will be different in every different room, but if you understand what is being published from Klippel, and you understand the acoustics of your room you can make some very good conclusions about how that speaker will behave. Horizontal and vertical dispersion plots provide a wealth of information about toe-in, floor and ceiling reflections, side wall reflections, etc.  I see most reviewers are providing the standard in-room calculated measurement, as well as the standard early reflections calculations which is a good summary too. Great information on distortion will tell you how load you can play too, or where some stridency may show up in the mid-range, or bass becomes uncontrolled.

@carlsbad , Publishing a bunch of numbers that the reader doesn’t understand and that don’t really tell you how a component sounds is perfect for the internet age where everyone thinks they can be an expert on everything by reading google.

@carlsbad, which is the worst issue, the lack of understanding or the does not tell you how it sounds?  I think your post was one of the most salient. I am biased towards speakers. I think there is a knowledge gap between what measurements can communicate, especially with the more detailed measurements reviews are now presenting, and audiophiles ability to interpret them accurately.

I've seen claimed ratings by the manufacturer being 8 dB lower when actually measured. Did things get worse when they changed the measurement process at 1W/1 Meter compared to 2.83V/1 Meter?

There is no enforced standard for reporting sensitivity, so different companies will use different methods. On-Axis anechoic. On-Axis anechoic listening window. Typical room response would be the most common. I think most are using Volts, not watts, but I am sure there are some holdouts too further complicating things. Last may be how they interpret the frequency response to arrive at a single number. I remember someone showing that some Klipsch models were >10db higher in literature versus a measurement. Klipsch uses room response.

Also, I don't buy the parts tolerance argument. I'm a retired bench technician (Component level) and most audiophile grade equipment tolerances are very tight with many components within 1%, They are much more consistent than that especially when new!

I can't speak for audiophile speakers, but for professional speakers, most of our parts are +/- 3%, some +/- 5%, some were specified tighter, and some were specified looser.  A good designer/company will do a sensitivity analysis as part of their design and design validation. The architecture affects that sensitivity. A low order crossover will be more tolerant of component variation than a higher order crossover.

If I was designing for the potential use of tube amplifiers, I would improve some tolerances to improve impedance consistency.

How can a measurement, unless it is inaccurate, "cancel a brand"  ?  Can you point out some very specific examples of this and how the measurement was inaccurate and cancelled a brand?

"Talking down to customers" ?  Do you have a specific person in mind and example? Specifics matter. If you mean the general tone at Audio Science Review  to those that don't believe the same thing without stating it, I would not totally disagree, it can be toxic, but I would hope you can then also agree that the treatment is no different here to people that disagree, it is equally toxic.

What proof do you have?

What, do you think I am going to post bills of material?  How about doing some research on audiophile speaker component tolerances and comes back to us with your report. +/-5% total tolerance on a film capacitor for audio is pretty standard, but depending on the vendor, 95% or more of their distribution is +/- 3% or less. It comes down to who is going to pay for screening. Air core inductors are typically much better than +/- 3%.  Even cheap iron core inductors are +/-3%.  High tolerance high wattage resistors are surprisingly expensive, so you ensure your designs are more tolerant of variation. Fortunately, where resistors are used, that is often the case.

The experimental method has been described in detail, to enable researchers to repeat the tests in order to verify the conclusions. The results of this experiment may embarrass those cable sound deniers who have hindered the advance of hi-fi for the past 50 years, and hence may allow the quality of high-fidelity sound reproduction to advance.

@kota1 , I realize you are trying to post counterpoint, but like the AES article, it would be best to validate what you are posting for relevance and accuracy first. You may want to ask an EE for a review of this before putting your name to it by posting.

I want to know why some people claim you have to level match when listening to different speaker cables of the same Gauge, but different design. You shouldn't have to level match in this situation, after all they tell us that cables don't make a difference.

I would assume just good experimental practice, but there is no shortage of people who just regurgitate what they read on either side of any argument on the internet. I guess it is possible that there could be crazy high inductance or capacitance, or maybe someone could use something like aluminum instead of copper? Measuring the resistance to validate the claim of same gauge seems like a good practice.

So are after market power cords, vibration control devices and power conditioners bogus if they measure well and make the system sound better? Does it win a prize instead? 

https://nordost.com/downloads/NewApproachesToAudioMeasurement.pdf

That paper is brutal.

1x CD player, older, $4000 ---  I have an idea, how about tell us the model. Why is that a secret?   It was obviously a unit with troubles that conveniently measured poorly. I am making a leap there? You bet, but I have a reason. That reason?  A current, $250 Japanese CD player was better than their $4000 unit, quite a bit from the pictures shown. They needed to pick something really awful to show a difference. They gamed the test. Worse, they gamed it in another way I will mention later.

29 graphs, count them, 29, not one of them with an adequate scale being used, let alone always with a good description and most of them, with absolutely no scale at all.

They have some pretty big claims of timing errors, 20-40useconds, though how someone could replicate this with almost no data I have no idea. Of course, they are using what could be an ancient (and purposely chosen) CD player, and to make it as bad as possible, they are using a CD made on a computer CD writer, not even a pressed CD. At this point now very old CD player, well worn, and a computer made CD, with "sample sized", their words, timing errors. It sounds like the CD was misreading. I could see a mechanical platform improving that. Personally not played a CD in over a decade. Rip everything to storage.

Not all papers are brutal. Some are good, some are average, some are questionable, and some are really brutal. Both gaming the results with a really awful, but hidden source device, and a ton of graphs without scales, no pictures of the setup, limited details on the processing other than trust us, etc.  It is particularly bad.

@kota1 , if you want to be appear credible stop posting links to reports / white papers / etc. that have either been ripped apart, or very questionable in content. I would also not posts a corrected room response and indicate it is the real corrected room response when it is a calculated room response.

I would also suggest not deflecting and answering the questions posed, not the obvious deflection that has nothing to do with the content.

Your repeated bating that has nothing to do with the topic also lacks maturity, but I am sure you know that.

The thing is you can’t predict this based on how a speaker was measured or an amp was measured

Much more accurately, YOU cannot predict much about a speaker and how most people will feel about the sound when the room is known, and a full range of measurements are available. A lack of experience does not extend to all people.

However, even for other products your statement is still YOU, not everyone. Many amp vendors, D'Agostino, Pass, Macintosh even claim they tune their amps specifically for their sound (some have a few sounds) and that is based on a transfer function that is unique to their products or a group of their products. Given there is no way at all they can test their amplifiers with more than a small subset of speakers, but achieve reported consistent results, they have a good handle on the measured performance.  However, just a few topics over, you are advocating 100% for matched amplifiers to drivers for active speakers, which is 100% a measurement exercise. That position and your statement I quoted are at odds with each other.

That is 4 strikes @kota1 ,  Hans Beekhuyzen is not an expert, I suspect he is not an expert on anything. That video does nothing to negate the work published by Golden Sound (which I came across when researching MQA a while back). If this is the best rebuttal to a very extensive test, that indicates to me that Golden Sound is probably right. You don't negate technical arguments with words, you negate them with technical arguments.

As soon as Hans starts to call into question something as fundamental as Nyquist, you know he is highly unqualified to make any statement about the topic though he tries to recover by stating no one has proved him wrong.  Not content to make himself look bad trying to call into question, Nyquist, he then starts talking about distortion and filters, claiming all filters make distortion. Marginally true, but also linear distortion, which we are much less sensitive too. Good thing or every speaker every made would sound horrible. Try comparing the linear distortion of a speaker to a DAC. Oh boy, that would be an eye opener for Hans!  He then makes a statement that we should be using 192Khz, though no one seems to be even able to prove conclusively we need more than 44.1, but perhaps he is not aware of even the most basic things about DACs and ADCs that most of us working near the technology can pick up by osmosis. Maybe I am wrong, but I suspect Hans is not very technical.

But all that aside, he makes the claim that most people prefer MQA to not using MQA. Funny story, but researching that is how I became more aware of Audiogon. I searched the web, and the most frequent comparison was Qobuz and Tidal. It was not overwhelming in Qobuz's favor, but significantly more preferred Qobuz in my non scientific internet review.

ASR tried cancelling GR Research using bogus measurements so another reviewer showed up to check it out personally. What happened? What do you think:

I think Strike 5!

For one, that was probably the worse wine/audio analogy ever made. I always picked my wines purely on the alcohol content, color and grape type, and not the 25 other major chemical compounds that contribute to taste.

So we are on to two. Two two two .... I guarantee those two crossovers will not measure the same. Just by looking at it, I know the iron core inductor will be much lower resistance than the air core inductor that replaced it. Also, those enormous film capacitors will have different ESR at frequency also impacting the final response. Perhaps at a gross level they will be similar, but not in detail. The changes will be as substantial probably more than your worry about component tolerances.

How else do I know this is not done by what I would consider a truly experienced professional? Start with the flat wire inductor. They look so high tech don’t they? I would almost want to use one just because they look good. Their reason for existence is a claim of reduced skin effect, even up to 100KHz. First, why do I care about the woofer part of the circuit at 100KHz. They claim less self heating due to space factor and winding density, but each wire layer is heavily insulated and the total surface area is less than a standard wire wound inductor meaning that in practice, a standard air core is superior thermally. I could go on about highly variable winding diameter, etc. but I think I said enough. I will finish with the itty Miflex capacitor next to the enormous Miflex capacitor. They are both +/-5%. What exactly is that small one supposed to do? It looks like it would be about 1/100th the value. Is that supposed to be a high frequency bypass? You will not find that on the crossovers of high end speakers for a reason.

Interesting sidebar:  Do you know why audio film capacitors are always 250, 400, 630V even though audio signals are never anywhere near those levels?

If you need that device @kota1, maybe I should go a little easier on you, then again, making jokes about alcoholism, a serious disease for many, and genetically influenced is in pretty bad taste on your part.

@kota1. Now you have me really confused. Are you now saying that measurements are good as long as they validate your subjective experience?

I would reach that conclusion.

Conclusions
Performance here is not awful but clearly could be a lot better as sister group Denon has shown. $5,000 is a ton of money for an AV product so performance needs to be much more optimized than it is. .

Bogus, anyone buying equipment can skip both sets of measurements. One guy likes it, the other guy "meh". You need to audition at home.

The conclusion appears to be based on expecting a $5,000 AVR to have internal DAC performance maybe somewhere in the ballpark of a $100 DAC. That does not seem unreasonable, or are you saying that all DACs sound the same as long as they are half decent? If you play a lot with external digital volume control, it may be important to you to have DAC performance in your $5,000 AVR that is better than a $10 phone dongle but maybe it does not matter if they all sound the same. With that level of engineering detail, what else have they missed?

Excellent post @kota1, it completely makes the point I stated above. Here are some key statements in it.

In your most recent paper, you proposed a statistical model that predicts listeners’ preference ratings of headphones. How did you first come upon the concept of the model?

We now understand what the target response should be for achieving good sound.

The statistical model for predicting listeners’ preference ratings of headphones based on deviations in its frequency response was really an extension of a similar model I developed in 2004 for predicting listeners’ loudspeaker ratings. The only difference is the headphone ratings are based on a single curve whereas the loudspeaker’s radiation uses several curves to characterize its sound over a sphere.

@kota1

All of the videos I posted of reviewers being called out by other reviewers or the companies they tried to cancel are examples of this.

You batted 0 out of 5 on the videos you posted that I looked at. All were very flawed if not outright wrong. The videos you posted really highlight the need for competent technical sites to do the work they do as there are far too many people and companies publishing information that is misleading if not outright wrong.

@kota1

Repeatedly members on this site say that measurements don’t matter, and that sites like ASR have no influence in the audiophile community.

If that is the case, how could those sites possibly "cancel" a brand that caters to audiophiles.

I don’t think you have provided any good examples of a bogus measurement. The only one that comes close is in regards to a $5,000 AVR, were the performance was good enough, which was recognized on one graph, but the overall conclusion was that the high price was not justified given the high price. I have a hard to interpreting that as cancelling.

There is obviously the ongoing argument about what is audible, with the science sites taking a stance that they feel can be supported by science. I think it is a fair argument that in some cases system level issues such as system level noise, perhaps recognized, are overly glossed over. You could argue the conclusions are even too black and white for the average non technical reviewer. You are going to have far more success with these as arguments as opposed to something easily dismissed. If you care about the validity of measurements, why not address that with Amir as opposed to making assertions that are easily dismissed?

Saying bogus measurements repeatedly does not make it factual Using bogus links and bogus white papers to assert other things are bogus also does not make it factual. Also not proven is any evidence of a cancellation. Criticism is not cancellation especially if a relatively consistent basis is used for all criticism even if you do not like what that basis is.

@bigtwin how do you even like or dislike measurements? That is like saying I like or don't like 1+1=2.  I get your point though, I just find all the vitriol unfounded, but perhaps it is founded in lack of understanding or understanding, but being too invested in being right?

It feels like sour grapes to me @bigtwin 

You mean like this @texbychoice ,

https://www.audiosciencereview.com/forum/index.php?attachments/ansi-cta-2034-a-pdf.45978/

The important thing in any experiment is to document your process and measurement setup. Standards are important where safety is involved, interoperability or improved communication from a standard language.  It is not necessary to recreate test results.

If I posted 5 links / videos, and at least 2 of them are totally false/wrong, and the rest are at best highly questionable, then I would not be making disparaging remarks about others that have nothing to do what is being discussed.

Not in acoustics, concert halls, theaters, and recoding studios have been very consistent with standards. See THIS LINK.

The ASA standards are mainly around fundamental technology aspects of sound, as evidenced by the types of standards the publish:   https://asastandards.org/working-groups-portal/    The ANSI/CTA link I published for Consumer Speakers is more relevant for this discussion. The ANSI/CTA-2034 standard is 50 pages long, but still does not go into really deep detail on underlying measurement practices, but it is fair assumption the equipment is following ASA standards where applicable.

@kota1 I just like to keep you talking so you have enough time to change feet.

It is good that you have fooled yourself though. That is where most people start. Once you have fooled yourself, then you never need to learn anything new again and you feel quite justified, even proud of yourself for putting down others, especially those who know more and reveal the flaws in what you believe. It is why you resort almost 100% to insults of a personal nature and do not address the arguments presented.

Do you think @bigtwin , @yoyoyaya , @clearthinker and @westcoastaudiophile are impressed with you?

This paper on interconnect pathway measurements was published on AES

Strike 6. Can you point out the flaw in this paper?

It has been demonstrated that frequencies above the human threshold of hearing generate brain activity, so super tweeters with a decent source are indeed relevant.

This is cool, but I would not take this as a proof of much. From what I read, we cannot discriminate any ultrasonic frequencies, we can just detect presence of some. There have been a fair number of tests over the years of whether people could tell the difference of music recorded to 20KHz, and recorded to higher frequencies. No one can tell when the bandwidth is larger.

We don't have to agree on everything, or even anything, but keeping it respectful goes a long way. Thank you to those that are.

I've been saying it all along.  You have test it in time domain.  The frequency domain in steady state response will not tell you much.

No you do not have to test in the time domain, for almost all tests. If the transient response is the same every time you apply that transient, which is true in most cases, then frequency domain analysis is totally accurate.  Even where you may have periodic effects which you may think will impact the transient, such as AC line noise, frequency domain is sufficient. Even for speakers where we have power compression, which could be considered a transient effect, the speed of power compression, a thermal effect, is below our hearing range, that frequency analysis is good enough. For speakers, the transient response is almost always plotted in tests.