Natural sweet spot in your system.

Probably mostly the speakers. Soundstage has hundreds of speaker measurements showing large deviations from linearity even on SOTA speakers.

It is amazing how good the system sounds at certain volumes and how it turns to utter crap 15- 20 dB later. I have heard that many volume controls are at fault for this as well. 

I think it’s a combination of room and speakers. There’s a point in sound pressure/volume where the room begins to be heard and often a small range above this that is useful. Below this range, soundstage is small, and above it, standing waves and other anomalies begin to become apparent.

Several contributing factors here. Volume control non-linearities occur frequently, but seldom account for the severity of the problem you describe. But, it IS a matter of non-linearity, but more generalized throughout your entire system(s). Although some amps may have a bit more difficulty playing at, say, low listening levels, especially when paired with some speakers, I'll assume this is not really the main cause here. I'll also assume that your speakers and amps have been well enough matched to each other impedance-wise and power-wise...that your wiring is of the appropriate gauge and the your amp's watt rating is either right around or slightly above your speakers rating. 

If all that checks out, then I would say that you are actually dealing with a power treatment issue. Too much audible hash from electrical noise (everybody has it, whether they're aware of it or not) will cause exactly your result.

You can try the cheap stuff from Furman, IsoTek, Panamax, Monster and the like, or you can try the more expensive stuff that will not introduce negative impacts on the sound, like Bybee Devices or Alan Maher Designs. But, any of those should have some impact and the closer you get to an effective solution for it, the bigger the sweet spot will become. It is possible to take things as far as winding up with a Completely linear system (an impressive animal, to be sure), but it may entail considerable expense to actually get all the way there.

Hope that helps.

Oh, and room reflections can be a contributing factor too, if your room is particularly live sounding.

So here is my system. Zu Audio Def, Allnic 300b, Bryston Bda 2, Laptop. Zu speaker cable, Audioquest Water RCA. Core power 1200 on Dac only

EquiCore 1200 or Monster Core 1200?

It may also have to do with the level at which the recording is 'cut'-- not everything comes into full bloom at the same marking on the dial. 
I agree, though, that there seems to be a 'natural' level where everything gels-- get beyond it, and the system sounds like it is 'playing at you' and below it, you don't get all the goods. (Obviously, some Fletcher Munson stuff or whatever it is now called, plus room, plus gear, plus?)

Many folks are sitting right smack dab in the middle of a standing wave and don’t even know it. Besides, each corner produces sound pressure peaks three or four times higher than the average sound pressure level in the room. That’s equivalent to having eight additional speakers in the room. Hey-loo! And no, it’s not like surroundsound.

+1 for whart.

Fletcher-Munson curves/effects for some reason often get forgotten in this equipment-centric world. Add the room, add the gear, add hairstyle (almost joking, but not quite).