There are many decent books on the topic.
Floyd Toole's _Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Room_ is the best synopsis of 50+ years research into how human hearing interacts with loud speakers and rooms.
It'll help you understand what good speaker design goals should be.
Vance Dickason's _The Loud Speaker Design Cookbook_ is the canonical text on speaker design. If you want a copy you'll do better getting it from a speaker building supply place like partsexpress.com or madisound.com.
On-line, Siegfried Linkwitz (as in Linkwitz-Riley filter, Linkwitz Orion speakers, etc.) has some decent coverage of speakers + rooms
as well as John Krevosky, especially the polar/power response illustrations
As far as first order cross-overs the phase distortion of even fourth order Linkwitz-Riley cross-overs is inaudible on blind tests with musical signals. First order designs with non-coincident drivers have a broader shallower power response notch than higher-order cross-overs. Excursion continuing to double with each octave below the cross-over point forces higher cross-over points which moves where that notch is on first order designs. The higher cross-over point in turn precludes using light stiff cone materials due to resonances and therefore makes stored energy less likely. You end up with big audible differences from power response approximating what you hear (to be more pedantic, your brain adds up the spectra from everything it identifies as a reflection) and a lower chance for stored energy. That's all quite audible.
The ideal speaker is flat on-axis with a monotonically increasing directivity index and off-axis curves that look a lot like on-axis. People prefer this regardless of listening background, preferred musical genre, and country of origin (Harman has a computer controlled speaker mover allowing blind comparisons which Toole and Olive have used to reach interesting results). Preference becomes significant when you depart from that, with differences in what distortions people object to less.
> "monotonically increasing directivity index"?
> I assume that directivity increases with frequency, but I am unsure what is meant by "monotonic" in this context. Can you shed some light?
Monotonic means that it never decreases along the way resulting in broader dispersion before narrowing again (this is an audible problem with most 2-way speakers).
Wiki is a good place to start and the reference links seemed unbiased.
Never understood the "time and phase coherent" argument myself when one side is 90 degrees out of phase and all the other positioning and design issues. I wouldn't presume to argue with Dunlavy and Thiel and I've heard their remarkable results but I've also heard enough to know that a complex question deserves more than simple dogma. The simplest is to keep the xovers beyond the vocal frequency range, where our ears are most sensitive and wherever those lines are drawn, but that's rarely practical and involves other compromises.
Inductors are probably the worst offenders in passive xovers and they get correspondingly bigger for lower frequency, low pass filters.
A crossovers selected to work with total design. 1st orders sound wonderful but are hard to pull off proper. Other networks are also of use to designers for various reasons. So no type of network is really superior to another just best to use networks thats optimal for designers end goal. More to loudspeaker design than just crossover. It should be considered as a whole transducers cabinet crossover etc.