No you are not deaf. If they sound nearly the same, then go with the lowest cost/best value product. New equipment is not always the solutions. It your system sounds good to you -- enjoy it and stop worrying. However, if you insist upon obsessing, yes your amp is the "weak" link. However, any reasonable cost (under $1,500) change will not dramatically improve the sound of your system. The improvements will be subtle. Maybe they'll be musically important, maybe they won't. TRUST YOUR EARS.
Am I deaf?
I've got three CD outfits here on loan from my dealer. A Linn Ikemi, a Bel Canto DAC 1 running off a Pioneer 525 DVD player and a Classe CDP. They all sound almost EXACTLY the same. After hours of switching cables, power cords and anything else I could think of, I did form a slight preference. And that was for the Pioneer DVD running straight into my amp! This freaked me out so much that I subjected my wife to the same test. She also preferred the DVD player. What in the hell is going here? Associated Equipment: Classe CAP 100 integrated amp, Sennheiser HD600 headphones, Von Schewikert VR3 speakers, Cardas and Kimber cables.