Impact of phase inversion by preamp

In a balanced preamp the phase switch simply redirects the inverted phase to the non-inverted circuitry and vice versa. Its completely passive.
With many recordings the correct position is inaudible, owing to the recording being multi-tracked and mixed. The phase may be different depending on the instrument playing! So usually you only hear it on true stereo recordings, those done with only 2 or 3 mics (the latter being the Mercury process or Decca stereo tree, that sort of thing). The problem is that 50% of all recordings regardless of the media are not in absolute polarity while the other 50% are. Switching your speaker cables around to deal with this is a pain which is why we included the switch since we were making a balanced line preamp anyway.
You just have to try it and see. I seem to hear it easier with brass instruments.

In a balanced preamp the phase switch simply redirects the inverted phase to the non-inverted circuitry and vice versa. Its completely passive.

Nothing is completely passive. The more revealing the circuitry, the more it exposes component 'flaws'.

Experience with phase flip switches in active and passive balanced hardware often left something to be desired in terms of routing and hence inaudibility.

Making both halves of a balanced circuit identical is EXPENSIVE. Add in tube vagaries and the probability of a sonic delta is fairly certain. Not to all listeners, but to some.

Decades back it was demonstrated there were distinct sounds in a system on the same source:

1.  Normal polarity to speakers
2.  Invert channel A at source* and speakers - same Φ as 1
3.  Invert channel B at source* and speakers - same Φ as 1
4.  Invert both A&B at source* and speakers - same Φ as 1

*Balanced out - polarity swapped with identical but inverted polarity cables.

Theoretically all should sound the same, but they didn't. If one subscribed to OOPS, at least 2 & 3 should sound the same, but they didn't. It was postulated that the loudspeaker is an asymmetric load and depending on the other hardware, affects the sound.

ieales

Nothing is completely passive. The more revealing the circuitry, the more it exposes component ’flaws’.

Semantics. There is passive circuitry, and there is active circuitry. There’s no value in confusing the two.

Experience with phase flip switches in active and passive balanced hardware often left something to be desired in terms of routing and hence inaudibility.

Whose experience? With what hardware?

Decades back it was demonstrated there were distinct sounds in a system on the same source ...

When was that, and by whom? Where was the demonstration, what were the source and other components used?

@cleeds 
Nothing is completely passive. We may think of C in terms of uF, but there are L&R factors which pertain. Pull apart a switch and examine the contacts. One set may have a contaminants/wear/geometry that the other does not.

A great recording & electronic design engineer built an incredible sounding recording console that used mercury wetted relays as close to the active circuitry as possible to eliminate passive colorations from running traces [audible] to surface switches [audible] and back.

Mic pre's, EQ, recording consoles in the studio.

In the early 80's. Then state of the art LA recording studio. Participants were well regarded engineers and IIRC, some industry sales reps. Source was hot-rodded ATR-102 ½" master tape feeding Bryston amp stack and time aligned studio monitors. Test was change or no change. Blind. Most got most of the changes.

ieales

Nothing is completely passive ... Pull apart a switch and examine the contacts. One set may have a contaminants/wear/geometry that the other does not.

A mechanical switch is a passive component.

I’m sorry that you don’t have any details on the listening test:

... state of the art LA recording studio. Participants were well regarded engineers and IIRC, some industry sales reps. Source was hot-rodded ATR-102 ½" master tape feeding Bryston amp stack and time aligned studio monitors. Test was change or no change. Blind.

Without details, your claim really doesn’t mean much, except perhaps to you.