Speakers sound better without grilles?

I've noticed that my NHT SuperOnes subjectively sound better when i remove the grilles. Specifically, the off axis sound from the tweeter seems better. This is most noticeble if i'm sitting in between the speakers very close to the speaker wall (since the speaker drivers aren't pointing directly at me) The highs just sound clearer and brighter. Is this normal? Also, are speakers supposed to have the grilles on or is just for looks?
Depends on the speakers, but certainly many speakers do sound better without their grilles. Some manufacturers explicitly recommend removing the grilles for serious listening. Others (e.g., Paradigm) design their grilles to actually enhance dispersion - my Paradigms definitely sound better with the grilles in place. For most speakers, grilles are just for appearance.
LS3/5a's are supposed to be listened to with the grills on since it was designed that way. Some cannot tolerate listening to speakers with the grills on and invariably will remove it due to habit irrespective of what the speaker builder says.
It is rare that a speaker sounds better with the grills on. One exception was a pair of NEAR model 10M I believe; the grills actually tamed the harsh metal tweeter.
My experience has been just the opposite of Xiekithen's. I've owned quite a few different speakers and have only found a couple that sound better with grills removed and they were specifically designed to be used sans grills. Meadowlark Kestrels and Sheawaters are what I'm referring to. I owned NHT Super Ones and SB-3s and the SB-3s definitely sounded better with grills on, never tried the Super Ones without grills. I've also tried Acoustic Energy speakers which were recommended to be used sans grills for critical listening but I preferred the sound with grills in place. All of this is a matter of taste I suppose.
Nothing is so absolute. Many speakers are designed to perform better with the grills on to minimize edge diffraction on the drivers as Racarlson pointed out with his Paradigms. In contrast Israel Blume's Coincident speakers are designed without grills. Enjoy!
My NHT 3.3 sounded way better with the grills off. The older Wilson's were spec'd for listening with the grills off, and the newer Wilsons are voiced with the grills on.

Grills just minimize children from seeing the driver and poking it in! But then again I have had to run off adults for trying to do the same thing.