My Analogy Must Be Wrong


Here's a thought experiment.

Flip a quarter 20 times. It comes up heads 15 times. Hmmm. I was expecting heads 10 times.

Flip the same quarter 100 times. It comes up heads 65 times. Well, that's closer to what I expected.

Now flip the same quarter 500 times. It comes up heads 275 times. Well, closer still.

The lesson is that more data points are good. They allow trends to be seen and increase our confidence level.

Here's two situations to consider.

Scenario 1a: Have 1 person judge the goodness of 10 amps.
Scenario 1b: Have 10 people judge the goodness of 10 amps.
Scenario 1c: Have 100 people judge the goodness of 10 amps.

Of the 3 scenarios, which one would you place the most confidence in? Here's another situation to consider.

Scenario 2a: Have 1 person judge the goodness of 100 amps.
Scenario 2b: Have 10 people judge the goodness of 100 amps.
Scenario 2c: Have 100 people judge the goodness of 100 amps.

Of the 3 scenarios, which one would you place the most confidence in?

Based on the lesson with a quarter, Scenario 1c should warrant more confidence than 1a and 1b and Scenario 2c should warrant more confidence that 2a and 2b.

Should Scenario 2c warrant more confidence than 1c? Since they have the same number of data points - 100 people in both cases - I don't see how.

Well, what about person A having listened to 20 amps? How much confidence would you place in that person's judgement? It's a single data point so it's better than no data, but it effectively has no value what so ever. It's the equivalent of flipping a quarter one time, it coming up heads and concluding its a two headed coin.

Well, what about person B having listened to 200 amps? How much confidence would you place in that person's judgement? Again it's a single data point so it's better than no data, but it effectively has no value what so ever.

Does the judgement of person B with 10 times the listening experience warrant any more confidence from you than person A?
bob_reynolds

Showing 1 response by mapman