What makes you build a system around an amplifier?

Roger Modjeski of Music Reference is now offering an ESL of his own design and build, with a dedicated direct-drive tube amp---the amp has no output transformer, the ESL no input transformer, the ESL panels driven directly by the output tubes! THAT is the ultimate, perfectionist way to do it.

Its a way that can yield excellent results but I would not call it perfectionist. If you encounter any circumstances that mean a change is needed (such as more power, want deeper bass, etc.) you have to start over from scratch.

OK, I'm sure I'll be brow-beaten for this...

First of all, the older I get, the more I'm convinced that the better systems are a "synergy" of the various component parts, including cables.  Having said that, if I had to select between amps and speakers being the thing I'd build a system around, my vote would be for the speaker.

When I last tormented myself shopping for speakers, I was struck by how the McIntosh MC452 didn't seem to be "picky" about the speakers it was driving.  So.....here comes the "brow beaten" part, I suspect I'll have my MC452 for a long time!

My first speakers, many years back, were Quad esl57s. At the time, there was no other speaker in the world that was as neutral and revealing. However, they had a nasty habit of not liking (i.e. destroying or being destroyed by) the vast majority of amplifiers at the time, and Quad’s own 303 was the perfect match, so that is what I bought. Anyway, in 1971 in the Netherlands, the choice of high quality amplifiers was extremely limited.
In the next iteration of my system, about a decade ago (I don’t suffer from audio nervosa upgrade itches), it was once again the speakers that drove the decision, this time together with the room. I had decided to move up to the new 2805 stats, and these were less efficient than the old els57’s. I had also moved into a new house with a rather larger living room. It took me a while to realize that what I did not like about the sound was the simple fact that on more dynamic music played at higher levels the amplifier was strained. So I bought a competely refurbished (every resistor and capacitor replaced) 2x140 watt Quad 606-2. At lower levels there was no audible difference, but with symphonic repertoire at more realistic levels, there was a clear improvement with a much cleaner sound. To be honest, the speaker and room could perhaps still benefit from the even bigger power of two QUAD QMP monoblocks.
The last iteration was the addition of a B&W PV1d subwoofer. The bigger room was begging for more bass, and now that I had completely given up on vinyl I also had the source quality to exploit. The last thing I needed to do was to tame the sub with a DSpeaker Antimode 8033 room eq unit. I have now decided that the next step will be a second PV1d sub, for even smoother bass, and more power.
So in retrospect, all my decisions were driven by my clear preference for the Quad electrostats, and later the addition of a sub. I just bought the amplification that I needed to drive the speakers. I must add that I am not convinced that good amplifiers have much of a sonic signature, if any. They need to be powerful enough, however, because that is what power amplifiers are: sources of power, and preferably big power. Or to put it in Peter Walker’s words: straight wires with gain.