Which is the most important part of a stereo system?

My system consists of a pair of B&W 630's, an old Denon 50 watt reciever (DRA-550) from the mid 80's, a Marantz CD5004 cd player, and now a Pro-ject Debut Carbon DC Turntable.  I'm pleased with the speakers and the cd player and while the Denon sounds good it has some issues and I want to upgrade.

I'm planning on returning the Pro-ject TT and getting a higher end TT.  I'm also looking into getting a new amp pre amp with a internal DAC.

Is the source the most important? The speakers? 

Please Help!
 I definitely built a few sets of speakers, back in the day. Going to try single ended boutique amp kit next maybe with or from a Nelson pass design. 
I find that everything matters.

For me, the most important thing is the amp/speaker interface. Get it right and you are on your way to a system that can sing. Get this wrong and well, you may never get it right.
I own the 1210 GR...it’s a fit and forget audiophile TT.  It’s maintenance free and bulletproof.  Sound on mine with a Bronze Cartridge is incredible.  Buy it yesterday!
The weakest part of your system is the most important.
Which is the most important part of a stereo system?

 Your Wallet?