Jay - You're right, there are trade-offs, as with all audio gear, I guess. I've been listening this evening as we've been grilling and eating, and I'm reminded that the Harbeths do seem very oriented toward having a "sweet spot"--if I'm in the right place, they sound really good, but if I'm wandering around, the tonal balance gets kind of lost, especially the low end. The Vandersteens sounded really great--when I was one room away--and I kind of miss that. And the Harbeths don't have the detail in the highs, although I can goose that out of them somewhat if I use a transister amp...but it's the age-old problem (okay, maybe the one-generation old problem) of lots of treble detail exposing all the faults in so much of the source material that I want to hear.
For whatever reason, in my room, the sweet spot is very specific, to where I can move my head a couple of inches, and get a different perspective. I don't know if this helps much, but it was over 5 or 6 weeks of occasional tweaking that I found the right positioning. Small shifts matter (at least in my room)...and a slight toe-in--but not too much--really matters. For example, my listening position is about a foot off-center, just because of the way the room is laid out, and I had to break my visual tendency for order by toeing in the left speaker more than the right speaker before I got the image to center just right. But once I did that, my ears got happy.
I'll add, too, that I noticed (as apparently you did), that volume level is pretty important...and that backing off the volume a notch can make a surprising difference in how much I enjoy the sound--as can bumping it up a notch, too, depending.
The buzz about the Harbeths makes it seem like they're not very particular, but if you're using them for "audiophile" purposes, that just doesn't seem to be the case, in my experience. For what it's worth, here's what I'm using with them: A Linn CD12 or GNSC modded Wadia 860, a GNSC modded ARC LS15, and an ARC VT100 MKIII, all wired together with Cardas Cross cable. Without the mods, the Wadia and the LS15 would be too analytical and trebly (for my taste).
This discussion may be too detailed for everybody else, but feel free to email if you want to bounce around more ideas.