Why should some speakers need a big room?

It's often said that some speakers need a big room to perform well. Couldn't you just turn down the volume? And if the speakers don't sound nice at low levels, isn't this a fault with the speakers?
(And I never heard any say a pair of headphones needed a big head...)