Trade offs?

As I have improved my system the quality of the CD recordings has become more and more obvious; unfortunately poor quality and harsh sounding discs seem to bother me more as the reproduction becomes clearer.
Having recently started using Ultrabit Platinum I find it sustantially improves the sound of better recordings but also reveals the harsness in poor recordings.
This all gets me wondering,on this quiet Sunday morning, if perhaps I'm reaching the end of the line on further upgrades to my Spectral/MIT based system?
For example will a better CD player simply reveal that the quality of the recordings are already the limiting factor in my enjoyment, better Cd players won't provide more enjoyment?