Which is the most important part of a stereo system?


My system consists of a pair of B&W 630's, an old Denon 50 watt reciever (DRA-550) from the mid 80's, a Marantz CD5004 cd player, and now a Pro-ject Debut Carbon DC Turntable.  I'm pleased with the speakers and the cd player and while the Denon sounds good it has some issues and I want to upgrade.

I'm planning on returning the Pro-ject TT and getting a higher end TT.  I'm also looking into getting a new amp pre amp with a internal DAC.

Is the source the most important? The speakers? 

Please Help!
klimt
Post removed 
Lots of good advice here. Millercarbon probably offers the best. For the last several decades I've been through many different audio setups and equipment have learned that one single component or cable doesn't make or break the sound that you want (unless that component is defective).
One example.....I once had a pair of speakers that I really liked but the amp driving them was old and sub par by many respects. So, I bought a really good and more costly amp. The sound wasn't nearly as good (to me) as that from the old amp. SYNERGY is the key.
One single upgrade may improve your sound but everything must work together as a "team". If you have the chance to audition a component in your system and in YOUR room that is the perfect scenario. But that can be a daunting task. There are dealers out there that offer that.
Good luck to you. Remember......good sound is what YOU think it is.

As a manufacturer of audio components, I was a firm believer of the source because I felt that all tone and sound stage came from there first.  You know garbage in and garbage out.  My partner and I recently built a hybrid 6SN7 power amp - think Counterpoint but way more advanced and two our items to a buyer.  He loved the DAC the preamp and power amp.  He had them there for a few weeks and he concluded that the amp was where he would start first.  So away we go!

Happy Listening.
@klimt for integrated amps with dacs

Modwright KWI 200 hybrid with dac is $6200 new, but used is under $5000

Hegel h390 class a/b with dac and streamer functionality is $6000 new 

NAD m33 class d with dac and streaming and bluetooth functionality is $5500 new

All admittedly over your budget but only marginally and imho worth a stretch