I found that low level listening was best done with super quiet gear. I had a lot of success with Benchmark AHB2 and LA4 preamp for that type of listening. Music details at low volume are not obscured by noisy gear.
Now if you want to skip the low level requirement but in the same physical space. Get a RAAL SR1a earphone and have 2 channel sound at any volume level in your space. I listen to the SR1a or CA-1a (not 2-channel sound) late at night, such as right now, when everyone is sleeping. Listening to Led Zeppelin now and it is as loud as I want it. I am using a Schitt Aegir amp for this ($800).