The perfect speaker ... lets see, I can describe it, but it cannot be built. We'll start with a massless pulsating point-source sphere with linear frequency response, perfect dynamic and temporal linearity.
Dynamic linearity is whereby a change in input level equals an identical change in output. Many speakers and speaker elements are not terribly linear, rather they compress the sound. Even drivers themselves vary - a very linear tweeter with a nonlinear woofer will sound quite different, dynamically, than a speaker with the opposite situation. Horns typically have great dynamic linearity; we hear that as 'punch or 'drive' but also horns have their own inherent issues with coloration.
Temporal linearity is simply the ability of the speaker elements to respond to an impulse simultaneously. Ideally, this happens at the same time, same phase, same level, free from hysteresis (the phenomenon in which the value of a physical property lags behind changes in the effect causing it, as for instance when magnetic induction lags behind the magnetizing force). THe speaker elements are delayed while the voice coil - and crossover elements - 'charge' enough to overcome the various mechanical and electronic resistances and impedances, and reluctances (the property of a magnetic circuit of opposing the passage of magnetic flux lines), all of which hinder the temporal linearity of the speaker. Stereophiles famous impulse tests demonstrate this.
I think that when people talk about 'pace', and 'speed' of a speaker they are really interpreting the effects of hysteresis. By not having to overcome those cumulative delays, the speaker better follows the music.
So what about tall dipoles? They have three basic issues: 1) Dipole radiation patterns makes them very placement sensitive. 2) They can suffer from poor dynamic linearity as a function of the inverse square law (doubling the distance from the magnets to the wires in the panel reduces the magnetic field by 1/d^2, so doubling the distance from the magnets reduces the strength of the driving field by 4X). This is mitigated by making the gap from magnets to panel large, relative to their excursion. In turn this reduces sensitivity. 3) Off axis performance - as the listener moves off axis, the relative distances from the point of origin change, especially at higher frequencies. This in turn causes phase issues with the bass panel. Off axis on one side the woofer is closer to the listener, or the wall, Off axis on the other the tweeter is closer. Mitigation in this case is made by making the driver narrower relative to the frequencies propagated, so the tweeter panel is an inch wide, and the woofer panel a foot. Again, this reduces sensitivity because of the smaller radiating area, so the speaker is made taller. Full disclosure, I love Magnepans, but 'slam' is not their high point.
Electrostatics have all the same issues, unless the stators are on both sides of the panel, for #2 above; 1 and 3 are still shared.
MBL Audio Radialstrahler is an ambitious attempt to recreate the pulsating sphere, but even a quick glance demonstrates that while addressing one issue, they still have others.
Likewise the Ohm 'Walsh' driver is an attempt the address the 'pulsating sphere', solving one issue while creating at least one more.
So, there really is no possibility of a perfect speaker, there is only balancing the tradeoffs of any number of imperfect options.