My headphone AKG K340 give me a sound which comes to my ears like if i was listening to speakers out of the head... No fatigue... And i listen at relatively low sound level...
Then the problem is very bad headphone fatiguing one like the Grado i never heard again after the first and last time i listened to them...It was not a high level model... Like the low cost Stax model i disliked for his artificial sound...
Then the danger is not headphone , it is bad design paired with high level sound pressure and some unbalanced set of frequencies..
Headphone well chosen, optimized, at low level for listening,for sure pose no problem...
Public concert of rock, pop etc and speakers designed for this kind of music at high level is the problem ...
Pop music at high level in the ears for sure = deafness...
Beethoven quartet through headphone will not make me deaf .... 
I prefer speakers by the way to headphone.... But now in my local condition i enjoyed my headphone too ...