but I've never heard an amp just sound awful then sound great on a pair of normal speakers.That's because it couldn't handle the ESL's loading.
You needed to hear the ESL's again with an amp that can take 1ohm loading and do it comfortably without rolling off or becoming unstable, then you would have been very impressed with the highs and their extension.
Cheers George