What Sonically is the Difference between a $1,500 CD Player and a $10K-$25K One?

All the money saved by not chasing perfect electrons will also give you the opportunity of having two systems, one Solid State, the other Tube.

A mix of Digital and Analog source equipment.  Here again, the differences are easily audible, and comes down to a preference, perhaps even Solid State for this content, Tube for that.

It is advisable to avoid the esoteric when trying your first tube system, any decent tube system will be involving enough to reveal the difference from SS. 

Now LP. Any decent TT/cartridge properly set up (learning first) combined with any decent RIAA phono preamp, SS or Tube, will reveal the difference of LP to CD.

LP content, like CD content, varies greatly by the recording/production skills as much as the artists skills.

LP into SS or Tube, differences will also be easily heard.

I advise learning about Analog rather than chasing tweaks.

After that, it's on to tweaks, god help you to stay sane.

The cheapest CD player, given a great recording, can produce an immersive listening experience in nearly any decent sound system. 

The place to invest your time and then invest your money is speakers. Time to learn, time to listen in showrooms, audio shows, time to understand your listening space. Critically, time to understand it is best to avoid going for 'too much' (usually too much bass). Time to learn that if speakers are well chosen for your space, you won't need to plaster sound absorbing panels hither and thither.

Without a doubt, money makes a huge difference with speakers. Far beyond any source component or cable or tweak.

After the right speaker choice for your space, what will undeniably sound better in your system is all the new source material you can acquire because you didn't spend money on imperceptible, or barely perceptible differences. The joy of discovering new artists, new genres, is far far better than chasing perfect electrons. 

As it happens, ....

I am revising my garage/shop 2 channel system, and will add a CD/SACD player.

So, use 'new' one upstairs, put old one downstairs IF better.

Bought used Oppo BDP-83SE (special edition ... better audio)

https://www.theabsolutesound.com/articles/oppo-bdp-83se-special-edition-universal-blu-ray-player-the...

I started with this Big Onkyo DX-7500, dual matched burr brown ...

http://www.hifi-classic.net/review/onkyo-dx-7500-430.html

Loved it, until it's drawer got quirky, progressively worse.

Snatched a small Sony 5 disc Carousel my niece was tossing because she lost the remote. I had a Sony remote that made it do the basics,

https://www.amazon.com/Sony-DVP-NC80V-SACD-Changer-Black/dp/B00080FPSO

My deaf audiophile friends and I compared the Onkyo and Sony, surprisingly/shockingly the dinky Sony POC sounded as good. Got rid of the annoying Onkyo.

Now, system more revealing that ever, let's see if this old dog Oppo can beat the Sony POC.