My Analogy Must Be Wrong


Here's a thought experiment.

Flip a quarter 20 times. It comes up heads 15 times. Hmmm. I was expecting heads 10 times.

Flip the same quarter 100 times. It comes up heads 65 times. Well, that's closer to what I expected.

Now flip the same quarter 500 times. It comes up heads 275 times. Well, closer still.

The lesson is that more data points are good. They allow trends to be seen and increase our confidence level.

Here's two situations to consider.

Scenario 1a: Have 1 person judge the goodness of 10 amps.
Scenario 1b: Have 10 people judge the goodness of 10 amps.
Scenario 1c: Have 100 people judge the goodness of 10 amps.

Of the 3 scenarios, which one would you place the most confidence in? Here's another situation to consider.

Scenario 2a: Have 1 person judge the goodness of 100 amps.
Scenario 2b: Have 10 people judge the goodness of 100 amps.
Scenario 2c: Have 100 people judge the goodness of 100 amps.

Of the 3 scenarios, which one would you place the most confidence in?

Based on the lesson with a quarter, Scenario 1c should warrant more confidence than 1a and 1b and Scenario 2c should warrant more confidence that 2a and 2b.

Should Scenario 2c warrant more confidence than 1c? Since they have the same number of data points - 100 people in both cases - I don't see how.

Well, what about person A having listened to 20 amps? How much confidence would you place in that person's judgement? It's a single data point so it's better than no data, but it effectively has no value what so ever. It's the equivalent of flipping a quarter one time, it coming up heads and concluding its a two headed coin.

Well, what about person B having listened to 200 amps? How much confidence would you place in that person's judgement? Again it's a single data point so it's better than no data, but it effectively has no value what so ever.

Does the judgement of person B with 10 times the listening experience warrant any more confidence from you than person A?
bob_reynolds

Showing 1 response by jjmali

Mr. Schroeder made some excellent points. He also made what I have found to be an error. I wouldn't depend on Consumer Reports for anything. Their method of reviewing puts far too much emphasis on the cost of items. I have read reviews of vacuums and lawnmowers as examples where they were so far off the mark it wasn't funny. In both cases I think they chose sears products. I went with my lawnboy and am still using the 21 year old over priced lawnmower (it still starts 1st pull).

Age and experience has taught me much in regards to audio. 1) One mans holy grail won't be everyones, 2) I don't care what anybody thinks about the way audio equipment sounds (except me), and finally 3) I will probably change my mind on the matter as my human components wear out some more. Have a great one week end guys.