Beware advice from people with zero hands on experience.
Not only is it possible, its easy. Not only can you do it, there are whole companies built on it. So again beware advice from people with zero knowledge what they are talking about. Zee roh!
One of the biggest challenges designers face is building components to a price point. Every single item inside the box has its own cost. Little things like caps and diodes make a huge difference in sound quality. But the better ones cost a lot more.
Look, there are some very well known makers like Audio Research who do exactly this. The ARC PH3-SE phono stage is nothing but a PH3 with better caps. For that you pay a big premium. All they did is swap out the caps. Huge difference in sound quality.
Again, just appalling to be getting such awful advice. Well no editors on here you know! You take the bad with the good. Or even the truly inexcusably bad, as the case may be.
Where was I? Oh yeah, caps. They have values printed right on them. Pick pretty much any cap at random. Buy anything bigger/more expensive on-line. Swap em out. Guarantee you will hear an improvement, and in fact if you do a little more than random and actually look into it a bit and select really good caps you will be shocked how much better it sounds.
But wait! It gets even better! (And the awful advice above gets comparably even more egregiously shockingly awful when you think about it.) Because one of the more reliably beneficial places to mod is power supplies. Step One in a power supply is the crappy diodes. These little dudes take AC power and let it go only one way. Problem is 60 times every second the AC reverses polarity. Causing the diodes to switch on and off. Which they do. But not nice and clean like you want. They make this little ripple. Even with lots of great big caps like are used in the better power supplies a little of this ripple still gets through. We hear it as grain, etch and glare. Remove the ripple with some smoother faster HEXFRED diodes, you will be shocked to hear the noise floor drop, sound stage develop layers, images become more holographically 3D, and top end extension free of glare you never even knew was there. All from a few diodes, probably less than $20 all together.
In other words anyone tells you its hard to mod, or not worth it, you can be sure that person is so far off base you would probably do well to disregard anything and everything they say. Ever.