@clearthinker : (what a "relevant" username!):

He suggests that because confirmation bias is not present in wine tasting,

The result of course is rubbish. Confirmation bias exists in all fields

In fact the record shows that confirmation bias is present in wine tasting

legions of published accounts of confirmation bias

they tasted was the real thing - pure confirmation bias.

save to confirm confirmation bias is alive and well

This must have been the single post with most "confirmation bias" mentions. Which is real. But, it is real and true in your camp too, not just for people who report hearing some positive difference, have you ever thought about it? In other words, if you folks think everything sounds the same, they WILL sound the same. Even if you base your "listening tests" by reading stuff in the internet.

In the years I have been reading audio forums, I have come to believe that "confirmation bias" is simply a phrase naysayers throw in your face if you post something making a difference, anything making a difference. I.e. "this cannot possibly work, you are imagining things ---- confirmation bias!". Subjectively, the phrase / phenomenon has lost its true meaning to me. Unfortunately.

It just seems so subjective due to the human side of things. Perception is not just the sound coming to your ears but your personal experiences affecting how your brain interprets that sound. To me there is no right or wrong. I can't tell you how many times i listened to my system and loved it - came back a day later and was scratching my head on how the sound changed - came back another day later and loved it it again.  

Being an engineer I look at the science side but that is just a part of what I consider.  All choices are made to improve the sound I like. I have had people over that didn't like my setup and that is OK.

I have gone down many of the paths that are discussed here - some produced real improvements for me and some did not. I can't always explain why or why not.

The fun is the journey and everyone is on a different path.

The path I follow is -  read, learn, buy, try - then repeat the process.

Its been a wonderful journey that will never end!

OP, 

    Great post, Ive been using a similar analogy for a long time, Books and movies also very subjective...

@thyname    Thank you for your response.  I will take your comment on my username as a compliment.  I did choose it carefully.

I certainly did NOT say that in my camp everything always sounds the same.  This is not black and white.  It is not a case of you either hear differences or you don't.  You miss my point.  Of course I often hear differences when I make modifications to my system.  I am sure I suffer from confirmation bias too.  Certainly when drinking wine.

We have all found that sometimes our system sounds different on different days, even when warmed up to the same extent, at the same temperature and humidity and listening to the same source material.  Some people say their system sounds different in the dark to with the lights on.  These are all subjective impressions we get, resulting from our mood and all sorts of other transitory personal conditions, many of which we cannot know or identify, still less quantify. 

All I am saying is that I don't know to what extent the differences I hear and taste are differences that truly exist and to what extent they are manufactured in my head, entirely unbenown to me.  There is no way of knowing.  I am afraid this truth is incontrovertible and entirely well known to psychologists.

All I am saying is that I don't know to what extent the differences I hear and taste are differences that truly exist and to what extent they are manufactured in my head, entirely unbenown to me. 

So, how do you make your purchasing decisions, if you don't trust your ears & senses?