What exactly is critical listening? Who does it?

Individuals like me are perfect to answer this type of question because I'm an analytical listener by nature. Even when I listen for enjoyment, I still listen critically without even intending to. I'm listening deep into the music but I'm not trying to listen to every single note.

For instance, when listening to vocal, I listen deeply into the naturalness, the richness of the vocal. The slightest shift of pitch and tone and volume in each singing word. The transparency, the smoothness. The angelic voice. It's incredible. 

The term "critical listening" means different things to different people. For me it just means at that time, listening is the primary thing I am doing. I'm not talking on the phone, reading a book, watching the TV, doing chores, etc. Just listening. I don't have to be "critical" about what I'm hearing, nor do I have to hear every small detail. But I am intent on connecting with the music.

You know you're a critical listener when you have about twenty copies of the same album over different formats, re-masters etc.

Yes, listening to every little tiny difference in timbre, tone etc. brings me joy.

Critical listening means optimizing the setup and listening closely to what the system is doing.  I use it when evaluating changes.  Otherwise, I just turn it on and listen more casually to the music.  

If critical listening used as a tool makes my system better, then yes, it does bring some enjoyment.  

Listening critically also gets easier with time / experience for a few reasons. It takes time to build the experience of what a realistic presentation sounds like - tonally, spatially, and effortlessly. And the better one’s source material, room, system, and hearing (or at least awareness of their personal hearing capabilities) are, the easier it is to understand the performance of any of the variables listed so that we may be able to critique what is contributing to what, or what is falling short.

Yes, it can be truly exhausting at times if you overdo it, but it’s a journey, not a destination, and one that, should you choose to accept its mission, you should traverse at your own pace. The key to success lies in motivation and the means to improve the variables for further fulfillment / enjoyment. Once either of those start to wane, it’s time to take a step back and stop taking it so seriously, and just try to enjoy the music. If you can’t do that, you’ll at least either have the motivation to find the variable that needs improvement so you can find more fulfillment, or you just need a time out from listening to music for a while.