AB Testing Experiences


Anybody done much AB testing.  Hard to do at dealers with much variety, and in homes even tougher.   Anyone done much of this and able to share experience.
My kenwood tape deck from 1980 allowed this, tape play vs recorded sound.  
My curiously would be an AB test between avr vs stereo preamp,  tube vs ss, McIntosh vs audio research preamps, high end spkrs, amps, etc  I can do some AB testing at home between avr dac vs bluesound vs chord.  This shows lots of differences. Bluesound was quite bad. The 5014 Marantz avr dac for heos streaming wasn’t bad, very open and detailed. 


emergingsoul
I’ve tried it with a Sony MP3 player and a Marantz CD player. I could hear no practical difference and felt a little unwell afterwards. Everything I knew told me there should be a difference, but I didn’t hear one. End result was that I moved largely into the digital is digital camp.

There may be differences but they are from obvious.

I’ve also tried it with different masterings of the same tracks and albums.

Most of the initial differences tend to vanish once the levels are adjusted to match (Mp3Gain etc).

Sometimes though, it’s worth the effort as one track will demonstrate clear differences in dynamics and zest.

Recently I have been comparing different masterings of Adam Faith’s EMI recordings (orchestra/ strings) from that wonderful but often neglected period of musical history - after early Elvis and pre Beatles. I found some significant differences which suggest that slightly different generation tapes may have been used to compile different CDs.

I’ve noticed that, rather bizarrely, as well as volume levels, even differences in CD art can make you decide in favour of one over the other.

In fact it can be so disconcerting that it’s better to use the same art or just turn off the monitor.

So I usually let the tracks play randomly via Windows Media Player without knowing which is which. If there is a meaningful difference it will become soon become apparent.

About 12 years ago I tried it with different brands of CDRs against the original disc. I did find that some brands were a touch closer to the original (the ASDA/ Walmart seemed to be an exact copy) than others but it wasn’t an exhaustive test.

I did something similar with burn speeds and thought that the 4/8 times copy was superior to the 48x copy until I forgot which one I was playing and then realised once again, there was no practical difference.

I would have thought that the slower speed copies would have less errors and they would be audible, but without a way of checking, who knows which copy has less errors? Or at which burn speed.

Blind a/b would be great way to level the playing field if dealers would let you compare speakers, amps etc in this way. Might be embarrassing too.

Can you imagine if you ended up preferring a Rega Planar 3 over a Planar 10?
Fair amount of experience but almost impossible to do with speakers. If you can't match levels almost perfect it's meaningless. That is very difficult with speakers making true AB difficult. Even comparing DACs I would be surprised the levels are well matched unless you had a mechanism for it.
You guys are talking about methods of switching. But guess what? You can compare two things (A and B) even if it means disconnecting speakers, moving components around. 

Its a comparison, which includes volume and material. If the switching has to be instant, with levels matched and the same material, well then check out my recent post on listening skills. You're beginner Level One: volume. 

At this level I'm not surprised dealers don't want to waste their time. Just ask the guy to hook something up. Listen. Pay attention. You should be able to get through this level in 6 months to a year.