Bob -- Thanks for calling AudioDiffMaker to our attention. I hadn't heard of it previously.
I looked through his 20 page slide presentation, though, and I'm a bit mystified as to how it is supposed to be able to reveal (or reveal the absence of) very subtle tweak-related differences, considering the time-varying inaccuracies that figure to be introduced in capturing the material into a computer.
For instance, on page 14 he lists, among what he calls "uninteresting differences," "small sample rate variations." On page 17, he indicates quantitatively how that can severely degrade the results, and indicates that "the best fix is to lock sample clocks." That would, I think, imply two sound cards doing simultaneous captures of the two sets of material, with their clocks locked together. Most computers are not set up that way, and depending on what is being tested both audio streams might not be available simultaneously.
Also, more generally, it seems to me that the zillion or so asynchronous things that continually happen in a computer, resulting in constantly changing noise conditions, are likely to result in some of that noise coupling into the sound card and its a/d converter, swamping the subtle differences being tested for, via sample rate jitter, signal-to-noise degradation, and other effects.
He does refer on page 18 to the desirability of running a dummy test comparing a sound file to another capture of itself. It would be interesting to know what kinds of results people have gotten doing that.
Thanks again for calling this to our attention.
Regards,
-- Al
