Speakers vs source/amp budget

I know many audiophiles advocate spending bulk of system budget on speakers with around 50 percent allocated to it. However I have noticed speakers suffer from law of diminishing return much quicker than either the amp or the source (dac, turntable). In fact I’ve noticed for bookshelf speakers especially after around $3k there is hardly much improvement except for the price tag. On the other hand the difference between $2k amp and $10k is night and day and same goes for turntables and dacs.
Am I just testing out the wrong speakers, has speaker technology come to the point there is only slight marginal improvement after certain point? The only rule for speakers seem choose the right size and sensitivity to match the amp and room size and spend the rest on quality amp and the source.
I came to this thought during my latest upgraditis run where I find myslef upgrading the source and the amp while being thoroughly satisfied with my elac vela bookshelves to the point my source and amp (naim 272/xps and 250) cost nearly 10 times the speakers! (I also own wilson audio sasha 2 for my reference system).