Ok, I cheated.
I use neutral amps/pre with speakers which I built from scratch. But more importantly the amps allowed me go with active crossovers for these 3-ways, so I was able to 'dial out' crossover related roughness and nonlinearities and naturally I had a say in driver quality matching to begin with and driver levels and delay were easily set and so on. But another major factor for me was finding a great power treatment than manged not to screw up the audio band at all.
So the upper-mid/treble playing field leveled considerably...the number of choices of possible driver candidates expanded greatly in terms of tonal response smoothness and inner warmth vs detail improvement. The number of tweeters, for example, that might otherwise have been deemed by me as too harsh, all but fell off a cliff. Things have worked out well here, no matter what the volume.
There must be any number of possible influences and variables here in your scenario, but the worst ones I was able to work around. But, as far as among existing speakers, that may be be another matter. But, like some others here, I tend to think that there's no real, direct relationship between neutral and detailed. I tend I guess to think of the relationship you describe as not necessarily fundamental and I'm thinking may only superficially exist as a combination of various things like noise and the vagaries or errors of crossover/speaker design in general in the market today. But, I mean, if you go and unravel that onion, and I have, the underlying terms of neutral and detail appear to be rather unrelated.