When/why to upgrade?

I have a pair of Dunlavy SC-IV/A speakers that, as far as I can tell, work the same as when I bought them new in 2000. I would upgrade if I had confidence that replacement speakers would work better for me. But that would require considerable effort and expense, and is not something I would undertake just for the novelty effect. Over the years, I've upgraded amps and DACs, and always with positive results, but speakers seem different. How do you folks know when it's time to upgrade yours? Dunlavys, after all, have a great reputation for producing neutral sound, and I don't see compelling evidence that speaker design has advanced much, at least, within my price range.
That's a good question. How high in the amplification and cabling have you gone so far ? 
I also have 20 years old speakers that keep performing better with each and every upgrade in the chain, no matter how small it is.
Room acoustics are where I’d spend my money if you haven’t already.

I don’t think you’ll find better speakers per-se but there have been very trendy sounding fads in what high end speakers should sound like.

However... I would look around to see what crossover upgrades users have tried. Modern caps are really really good.
Great advice from inna & erik above. Those suggestions bring out the best in already great speakers. As does resonance control (feet, platforms, stands, etc.) & IMO power filtration for everything but the amps. 
@gary_moresky,Do YOU feel the need to upgrade/change the Dunlavys? If not, then why think?In my case, I wanted full range sound. My previous loudspeakers were very fine, but not full range. The upgrade price was 10X the price of my previous loudspeaker.

I have a bunch of acoustic panels strategically placed on the walls and ceilings, and use an AntiMode 2.0, which is quite an amazing device for bass correction. My DAC is a Bel Canto 3.7 and the amps are Bel Canto REF 500Ms, I wonder if some of the harshness I experience isn't from the BCD components. It would be far easier and cheaper to swap out the amps than the speakers, so I might start there. I might even try BCD's newer 600Ms. Each of their new new line of amps has been a major improvement over the previous, and I like the idea of cool-running amps I can just leave on.
I always advocate a view that if you already have good speakers and like their sound not to rush with replacing them, instead try to bring out everything or almost everything they are capable of. However, there is a limit, unless it is a masterpiece I probably would not drive $10k speakers with $50k amps. But I said "probably"..

+1 @inna 

@gary_moresky  Spend time listening to other audiophile systems. It's an easy way to gain perspective regarding your own and whether you want to start taking the steps to walk down other paths.
What cables, interconnects, power cords, and power conditioning are you using?