Please define

I associate listener fatigue with distortion. It's obvious that if you're listening to a system that is relatively harsh, edgy or shrill, your ears soon will find it unpleasant. But frequency imbalances that are not nearly as noticeable in the short term can produce fatigue after a while. There may be shouty, boomy, or ringing parts of the musical spectrum that aren't really obvious but just get to you after a while. The louder you listen, the quicker these shortcomings get to you. The room, as well as the equipment, can produce listening fatigue.

Jayboard hit the nail on the head, in my opinion. This problem was apparent in my own system prior to the most recent upgrade. I discovered that it IS possible to have a highly accurate and resolving system that is still smooth and sweet without being veiled or syrupy, but finding the right speakers and components to accomplish such a thing was not a particularly easy task.

Often the systems being presented as "high end" sound either etched or veiled, and neither is good.
-Bill

I think it's something more subtle. A lot of people with both digital and analog (lp) front ends find that their listening sessions tend to be longer with records than with CDs and ascribe this to 'fatiguing sound'. It's definitely the case for me...