speakers bottoming out

"Bottoming out" happens when the driver tries to exceed its excursion limits, typically (as you noted) while trying to reproduce low bass at high volume. For each octave lower you go in bass, the driver has to move four times further to reproduce the same volume (in a sealed enclosure). In a vented enclosure, such as the Soliloquy, the driver excursion actually decreases at resonance, then increases dramatically below resonance. Powerful bass signals significantly lower in frequency than the system resonant frequency can drive the woofer into over-excursion, resulting in mechanical "bottoming out". In addition, there is a tradeoff between midrange extension and efficiency on the one hand and bass output (including long excursion for high volume) on the other. The Solioquy 5.3 is intended to work well with fairly low powered amplifiers, and so the design trades off some bass excursion capability for higher efficiency.

Okay as far as wattage rating - perhaps a speaker can handle 100 watts at 1 kHz. That would be the thermal limit - any more and the voice coils melt. Somewhere in the low bass power handling becomes displacement limited - in other words, at some frequency 100 watts will cause the speaker to bottom out. I get the impression that the Soliloquy is tuned to a lower than normal system resonance for its 5 1/2" drivers. It will probably be displacement limited to less than 100 watts somewhere in the half octave or so above resonance, back up to 100 watts thermal limited at resonance, and then below resonance the displacement power handling limit will plummet to maybe just a few watts at 20 Hz or below. The specifics are speculation, but the pattern holds for many reflex enclosures.

What are you playing to bottom out the speakers? I have had the 5.3's for several months and have not had this happen. My amp is rated a 100 watts rms. The speakers are supposed to handle 200 watts rms (I think).

You're playing them too loud, so you need a bigger speaker, or else turn the volume down. IF YOU CAN GET A CATALOG LIKE THE SOLEN ONE, you can usually look up the maximum excursion limits of the woofers (if they're spec'ed and in the catalog). Dynaudio drivers have a progressive non linear suspension, SO THEY DON'T BOTTOM. This is not necessarily good if you operate them near their excursion limit very often, though, because dynamics are compressed. YOU'LL ALWAYS BE ABLE TO EXCEED THE EXCURSION LIMIT BEFORE THE THERMAL LIMIT OF A WOOFER, if the driving signal is low enough in frequency...or ESPECIALLY if it's concentrated enough around the box alignment's least damped cone motion frequency, which will vary from one speaker to the next.