What makes a speaker sound great at low volume?

A tube preamp will help.

I am thinking of getting the Luxman 509x for my office and one of the big reasons for it is the Loudness switch. It would help big time when I am working in my office at 3AM.

While some if not most of the above comments certainly have merit, I think one over looked crucial factor is the speaker placement itself.
Many audiophiles change things before really getting the most out of what they have in the first place and speakers are certainly one of if not the most important piece of equipment.

My speaker Dynaudio Special 25,s certainly aren't the most sensitive' I found that once I learned to REALLY set up speakers the low level listening details became more aparent with low level listening much more rewarding. Low level listening for me is early mornings when I'm preparing for class, but when I get home my listening volume goes up to high 70db,s to mid 80,s and the louder you play the more crucial speaker placement becomes. 

Another thing that really helps is an amplifier with a properly designed, "stiff" power supply".

There are many variables, starting with cables .
since I upgraded to the Verastarr cable line I feel that using far less dielectric material ,as well as metal purity and Cryo treatment 
all add up to a cleaner presentation .
that is just one example , the amplifier, preamplifier quality 
are seperate factors  ,that being said yourcable s carry your signal.
your sourse record, or digital is where the Whole ball game starts 
once that signal leaves it cannot bemadeupdown stream .
sice by far your Loudspeaker gives the most distortions in a 
factor in % , 1-2-3 or more vs.001 for a amplifier on average a good one from 20hzto 20,000khz. Quality counts even more so 
And complexity of the Xover , speakers sensitivity , and the ability 
ofthe Amplifierto drive properly and have dynamic head room leftover for when you hit dynamic peaks they are not compressed 
on peaks demands can call on 10x thst then on a soft paggage 
allways buy bigger Quality power then you need  to ensure 
ultimate fidelity. There are many other factors I didnot even get to 
such as keeping all connections ,including power cordsclean.
the recordings the self will dictate what you hear.Room sound
absorption,isolation underequipment ,including a qualitystand. The room can have a lot to do with what you hear.
proper speaker setup. On the Cardas cable website they have a good setup guide line. Myself having owned a Audio store in Europe learned a lot about how to get many systems to sound far better then systems 3x the cost just by following all these guide lines.as thesaying goes ,yoursystem isonly as strong as its weakest link  !!  Enjoy Your music !!