How long does it take a decent quality speaker to "wear out"?

In general, a long time… well over 20 years… but, it depends. For instance I had a pair of ESS AMD 1Ds… which had relatively inexpensive woofers with material around the woofers exterior that wore out after about 12 years… similarly my B&W subwoofers needed the cone suspension material replaced after 25 years. The smaller B&W 805 Matrix speakers (in my home theater) sound and are great after 25 years.

I think if you are talking about high end speakers (not ESS AMD 1D… only good, not audiphile) thirty years or more is not unreasonable. But, as in most things, it depends.

@ghdprentice  , as a matter of fact, my speakers are the older 805 Matrixes.  I think I've owned them about 25 years.

(I did have to redo the cone suspension on my M&K sub twice in those 30 years.)

My 35 year old Spendors and 56 year old Quad ESLs still sound excellent. 

My 42 year old Infinity RS-1.5's are still going strong. The woofer and midrange surrounds have been replaced. Otherwise, trouble-free and sounding fantastic...

There is a very wide range from about 10 years to infinity. With regular dynamic speakers like yours 30 years would be about the norm. Certain types of loudspeakers like modern ESLs can be virtually immortal.