Cleaning outdoor speakers

I have a pair of Niles OS-20 that were mounted under our front deck. This was not my ideal place of course but at the time I had no other options. I bring them indoors each winter and clean them up but this year I noticed that the woofer has some dried dirt on it and appears to be stuck to the woofer material. They sound ok but I would like to clean them up. Would it be safe to wet the dirt/woofer to loosen the crud and then gently wipe off? Or, since they sound ok should I leave well enough alone?
Could you use a small to medium paint brush to whisk the dirt away gently so you dont risk damaging the speaker ?
Or try a can of compressed 'air' and carefully blow it parallel to the surface of the cone.