The Best Sounding Systems can Play Loudly with Low Distortion

I think the OP is directionally correct, though there are lots of good points added about having a system perform well at low volume.

I have two systems. My cabin system is in a much larger volume room that is better constructed and vastly more acoustically inert. I intend to build room treatments to improve things further, but it is already much better than the 2nd system even though the components are of similar quality. 

Though I usually listen at moderate volumes in the cabin system, I will periodically find the volume higher than expected when someone wants to talk and I have to turn down the volume to speak with them. There is an effortlessness and ease at any volume that the 2nd system lacks.

The 2nd system resides in a living room I am unable (unwilling) to treat. While the components can play much louder without stress, I cannot listen at anywhere near that volume as the room becomes too active. It’s adds audible distortion. As a “system” (room included), it’s much lower performing than the cabin system. Indeed, I’m considering selling the entire system because it is so much more satisfying to listen to the other one. I spend more time there, too.

So I would generally agree that being able to reproduce dynamics fully is indicative of a better system. It’s certainly not the only requirement as fidelity to signal, full range, imaging, and the ability to have the speakers “disappear” are even more important to me. But a system that can’t scale dynamics effortlessly is surely indicative.

For reference, I primarily listen to jazz, classical and acoustic folk though I do put Pink Floyd and the like in rotation.

I think that no distortion at high volume is a sign of a good system. So, I think the Title of this post is accurate. 
It could sound equally as good at low volume.
Define high vs low?
Any time I hear music played at super high volume, to give you that concert feeling, I find that the sound quality at concerts ( especially rock ) is terrible. Sound quality from recorded music in the studio sounds better to me than live, unless the concert hall has great acoustics. I have heard some great music live, at Hughes Room, in Toronto, Al Stewart for example. Not played very loud, abnormal for a live performance in my experience. Most 'live' performances crank it up..

Doug Schroeder welcome back brother. If you have enough power for your speakers you can play your system louder without any distortion.Even on low volume it will sound good, assuming your system is  already musically good.

Volume vs. distortion to determine your systems value is like saying the best cars drive the fastest.  There’s a lot more nuance to driving than speed.  In fact it’s very easy to make a car go fast.  How it feels and handles the road is more important to me.

OK, if we are talking symphonies using an orchestra as the question then this puts a very different slant on the question. I have had season tickets to the symphony for over ten years and have attended hundreds of concerts. By and large symphonic concerts are not loud. Most of the time are interludes between brief very loud crescendo. So, you are asking about the rare crescendo. A great system must be able to handle them. But most of time you will be listening to music and much lower levels.

I mentioned recently, my seats were 7th row center and I would go home and set my system to reproduce the whisper from the silence of some of the great openings to the crescendo to be what I heard at the concert. I use this setting to start with when listening to classical music... or other where I want to duplicate "live in the concert hall". Obviously there is variation in recording. 

But still... 95% not loud. 

When I first saw the question... I was thinking typical mixtures of jazz, rock, world....etc.