I lately wonder why I’m an Audiophile.

Listening to sound qualities of one's carefully curated system should be enjoyable, I don't feel the need to ignore it, I also very much enjoy the music. Perhaps its a learned skill to happily allow both listening modes to coexist. The only times I'm able to completely ignore sound quality is when listening to my mid fi work system and car systems. While I''m content listening to mediocre or lesser systems I derive much more pleasure from my audiophile system, the added bonus of fine sound quality adds much to my pleasure. 

I was exactly as the OP  but with no budget devoted to audiophilia save very low cost...

I learned acoustics and many "tweaks"...

i am lucky in a way, if i had money i will had bought a very costlier system, i would had learn nothing  and i will waiting with  my 50,000 bucks system to be upgraded by a 100,000 bucks one...

Now after learning i am in heaven with a low cost system (1,000 bucks) ...

 Acoustics knowledge rule audio  not prices...

Don’t overthink it. Just do it! 

With the endless music availability through streaming, there’s no shortage of well recorded music. Good stuff. 

Allow me to use Zen Buddhist notion to describe three stages of perception or understanding.

At first, seeing a mountain is just seeing a mountain; seeing water is just seeing water.
Later, seeing a mountain is no longer seeing a mountain; seeing water is no longer seeing water.
In the end, seeing a mountain is once again a mountain; seeing water is once again water

I think you are in the second stage and, depending on the individual, it may take you another year, three years, or even ten years to reach the final stage - or perhaps never.