I am assuming we are talking about using the digital coax output from a transport. Some comments....
Avoid the Pioneer BDP-51. I had this unit and can attest that the digital output is horrible. It is bright and harsh -- very solid state sounding with no bass. I have no idea why, but this is what I experienced.
Avoid older Oppo players (like the 83 and before). They also have the same bright/harsh character from the digital outputs.
The recent generation Oppo players are actually pretty excellent as a transport. I'm talking about the 103/105 series. Adding an upgraded high-current power cord will make the sound even tighter and stronger. I have not tested the new 203 platform -- it has a completely different ASIC and digital board layout. The 203 power supply is also not as beefy as the 103 power supply (they may have engineered the 203 to use less current in the digital section as they have stripped out a lot of features (like Darby and internet streaming).
Looking at the internals of the Marantz CD5000, it appears to have a piss poor power supply for the digital board. I think the digital coax output is somewhat of an afterthought and I don't even see a good clock on the board for PCM output timing. Comparing this to the internals/power supply of the Sony 5400 is liking driving a semi-truck over a Honda civic. The Marantz probably had nice analog sound output because it used tubes (with their own separate power supply).
The current and recent cheap Sony bluray players actually make a decent transport (with regards to digital COAX output). These are players like the BDP-S590 or BDP-S790 or similar. The sound is natural, but it is definitely on the soft/laid-back side. Lack of good power cord option and lean power supply would contribute to the soft character, but if you have to spend under $100, this would be an excellent choice. Based on this, I think your audition of the Sony BDP-5000es might go very well.
On digital coax cables. I have tried several different types for COAX digital and, believe it or not, different cable material/design can drastically affect the sound. I have even got it where it is so fast and bright (great clarity), but it overdid the brightness and there was no bottom end weight. Your Silver Serpent could be doing weird things to the sound. I have found that the best digital coax is the Belden 1694A from Blue Jean Cables ($20 for six feet is a no brainer!). It actually performed better than a silver audioquest cable. The Belden sounded the most natural and warm in the end, but it still had excellent response.
If you had to use what you have, try putting the Silver Serpent between the transport and the DIP 24/96. Then put the better Monster cable between the DIP and the ARC DAC. That would allow you to use the Monster for the better-clocked signal.