I need help to select a music streamer

Here is the stereophile test I was referring to (It was linked in the head-fi compilation earlier in the thread) -  https://www.stereophile.com/features/113/index.html

It's one of the only tests in the head-fi compilation (or Archimago's many tests) that indicate an audible difference (in amps/cables/resolution), but, as you'll see if you read the footnotes and letters, that conclusion was far from justified, and the null hypothesis (no audible differences) remains unchallenged after properly controlling responses.

But it did suggest something interesting - an overall tendency to hear a difference between options, revealed by the results when the two options were identical. I've observed this in myself.

A good discussion of the flaw in the test design, and how it potentially revealed a bias towards hearing difference is in the letters on this page, particularly the second letter (also obliquely discussed in a footnote):

https://www.stereophile.com/content/blind-listening-letters-part-4

In the absence of a basic flaw in your experiment, the above statistical analysis suggests to me that Stereophile's amplifier test may have neatly pinpointed a key element in the high-end audio business: a propensity (is compulsion too strong a word?) on the part of aficionados to hear differences.

footnote 8 reveals that Larry Atkinson had also realized the problem, but the letter explains best the flaw in the data analysis that invalidates the (badly overstated) conclusion of the main article.

@geoffkait

Well, for one thing blind tests are used for a number of reasons, including testing a single component or audio device or tweak. So don’t hand me a whole load of horseman knew her.

Heh, interesting spelling variation there. 

Well, right, but every test I've seen has either been a) can you tell a difference with or without the device or tweak, or between the options or b) rank order several options, with a) being more prevalent.

Let’s go to the definition of false positive: " a test result which incorrectly indicates that a particular condition or attribute is present."  Or if you like, the type 1 error, the rejection of a true null hypothesis (components sound the same).

Happy to end the discussion, but "you just don’t get it" is the run of the mill insult when you can’t prove or even adequately support your claims.

You’re not paying attention. If there is something wrong with the component you throw the whole test out. It doesn’t mean anything. It’s simply one of many things that can go wrong with kind tests or any tests. That is why tests have to be repeatable and transferable. One test doesn’t mean anything.

Geoff, I explicitly referred to the possibility that a problem exists *of which we are unaware*.

Once again, this semantic discussion has little bearing on the evidence at hand.

Evidence is not proof. That’s precisely why one test proves nothing. Preponderance of the evidence requires multiple tests as I be already said at least twice. To be strong evidence a test must be repeatable and transferable.