+1 for the Luxman speaker selector switch. I got a used one from Japan a couple of weeks ago. You can connect 3 amplifiers to 1 set of speakers, OR, 1 amplifier to 3 sets of speakers. Connectors are (3-4 way) speaker binding posts (incl. banana). You can select 1 connection path (A, or B, or C) at a time. Simple, but works well. Heavy construction, good quality. Obviously there are many other and much cheaper speaker selector switches on the market but none as high quality as the Luxman.

Luxman AS-55 Passive Speaker Level Switch

What actually goes wrong, if anything?  Speakers generate BackEMF which amplifiers withstand.  Why should signal affect another connected (but unpowered) amplifier?

And what would happen if both amplifiers were playing different music?

Please only answer if you actually know

I would also like to know actual outcome.  My great nephew asked me about possible liabilities connecting 2 amps and using 1 at a time a few years ago.  I strongly discouraged doing so but could not supply a good answer as to why other than the certainty that at some time somebody would goof and power up both amps.

I have also seen this question posed before on Agon with no specific answer.

Can anyone here with electronic chops give a good explanation?

Bill

Because you would be backfeeding one amp's signal into the other's outputs, a treatment for which no amp is designed. Amplifier output signals can be considerable; if you're listening to inefficient, low-impedance speakers really loud, you could see 40-50V and tens of amperes during transients.