Suggested upgrade paths

Used Harbeth Super HL5+ XD. Be careful on model number easy to mess up for they has updated over the years. $5k plus or minus. Audiogon, The Music Room and US Audio Mart has them.  I bought a used pair of Rosewood and I’m very happy with them.

They are called “Bookshelf” speakers. I do not no of any bookshelf that is large enough for the Harbeths to fit.

Note: The Ton Trager Stands are not California certified. Just joking but they will not hold up to some earthquakes. 

Buy a better pair of speakers! Go to ASR (Audio Science Review) and check out the reviews.

You will always eventually need a better DAC. IMHO all internal DACs are a compromise in lower end components.  Bite the bullet and check out a Denafrips Venus 15th. I am sure it will smooth things out. Start at your source and work your way out. @audioman58 sells them for a discount.  Run the A6 into it. Get a good quality USB cable too. Next consider speaker upgrade down the road. Good luck.

I'd be looking at the Mofi 888 or maybe the Kef R7 Meta. 

I am not familiar with these speakers, and I am sure they are part of the problem. However, there is a good chance that updating the amplification, in particular the preamp, will remove what might be distortion/noise at higher volumes, and give you  improved bass definition. I went through this with my NAD integrated, which luckily allowed me to bypass the pre, and try an outboard pre. It made a big difference.