My Analogy Must Be Wrong


Here's a thought experiment.

Flip a quarter 20 times. It comes up heads 15 times. Hmmm. I was expecting heads 10 times.

Flip the same quarter 100 times. It comes up heads 65 times. Well, that's closer to what I expected.

Now flip the same quarter 500 times. It comes up heads 275 times. Well, closer still.

The lesson is that more data points are good. They allow trends to be seen and increase our confidence level.

Here's two situations to consider.

Scenario 1a: Have 1 person judge the goodness of 10 amps.
Scenario 1b: Have 10 people judge the goodness of 10 amps.
Scenario 1c: Have 100 people judge the goodness of 10 amps.

Of the 3 scenarios, which one would you place the most confidence in? Here's another situation to consider.

Scenario 2a: Have 1 person judge the goodness of 100 amps.
Scenario 2b: Have 10 people judge the goodness of 100 amps.
Scenario 2c: Have 100 people judge the goodness of 100 amps.

Of the 3 scenarios, which one would you place the most confidence in?

Based on the lesson with a quarter, Scenario 1c should warrant more confidence than 1a and 1b and Scenario 2c should warrant more confidence that 2a and 2b.

Should Scenario 2c warrant more confidence than 1c? Since they have the same number of data points - 100 people in both cases - I don't see how.

Well, what about person A having listened to 20 amps? How much confidence would you place in that person's judgement? It's a single data point so it's better than no data, but it effectively has no value what so ever. It's the equivalent of flipping a quarter one time, it coming up heads and concluding its a two headed coin.

Well, what about person B having listened to 200 amps? How much confidence would you place in that person's judgement? Again it's a single data point so it's better than no data, but it effectively has no value what so ever.

Does the judgement of person B with 10 times the listening experience warrant any more confidence from you than person A?
bob_reynolds

Showing 5 responses by dave_b

I have wrestled with this concept for many years, on and off, and it assumes that all of the data points are of equal value. Some of the data will be more insightfull and or reliable than the rest and therefore skew ones assumptions accordingly. In other words, sometimes one persons evaluation of a piece (s) of gear will have more value then anothers. Life is a bell curve event essentially. A few will be misleading, a few will be extremely valuable and most will get you more confused! Even so, I firmly believe that there are certain fudamental characteristics to well re-produced sound that should be important for all so called "Audiophiles" i.e...accurate tone compared to live acoustic instruments, frequency extension covering said instruments, soundstaging and clarity (transparency/focus), dynamics (micro and macro), and lack of compression/hardening of the soundfield. Most people can recognise live music when they hear it...the closer a Hi Fi system can come to capturing that efemeral quality, the better it is!
Kudos to you Douglas. My audiophile experience has taught me first hand how to read between the lines in most reviews. There are a handfull of reviewers around, mostly on the web, whom I respect and are consistent in their evaluations. Keep up the good work!
Forgot the obvious conundrum....no one piece of gear can be evaluated without hooking it up to other gear under variable conditions. This is not unlike quantum physiscs; once you observe the system, the system is altered. The best we can do is use all our resources, match them to our needs and then experience the choices under as many conditions as possible. Looking back, it was far more fun being an impulsive neophyte who chose by budget and gut and then was blissfully happy!!
Douglas has augmented my views in a complimentary fashion. PC cords can make or break a component/system as can the room you inhabit for listening. What bothers me about most reviewers however, is just how ill concieved and patched together many of their systems tend to be! One reviewer comes to mind that recently got up enough scratch to afford an MIT Magnum interconnect...sheeesh, and he was reviewing high end gear? Some reviewers residences leave quite a bit to be desired as well. The biggest problem for a reviewer seems to be their ability to tell the truth. Most main stream reviewers bend over backwards to accomodate manufacturers design flaws. In other words, a piece of crap is seldom called a piece of crap...it is presented as unique and for the few who can appreciate what it does do right. I am a would be reviewer (if I didn't have a full time job taking care of my 3 girls), but I understand that the first review I would submit that did not conform to the unwritten rule of not rockin the boat (not pissin off the advertisers), I would be out of work! What say you Doug?
Douglas, by "Crap" I mean an overpriced piece of gear that proclaims superiority over the mass marketed gear, but does not deliver on the basic set of parameters all gear should attempt to address. Albeit, most gear may measure up well enough to not be included in the crap category, when a stinker hits the reviewer, or the price of admission is above and beyond the norm vs it's capabilities...well, one should be man enough to throw up the window and say, "I'm mad as hell and I'm not gonna recommend you buy this gear!" Apologizing for a manufacturer does no one any good. By the way, someone reviewing gear should not be discovering or piecing together equipment for the first time. Time, gear and experience come first. I'd rather hear what someone with a well matched reference system has to say then the trivial anecdotal ramblings of an overly excited, unseasoned would be audiophile.