My nearest neighbors (on both sides) are less than 10' away, and I don't know if they even know I have a huge system.
I too, have a few windows in my audio room, but most significantly behind the speakers. It is an 8' sliding glass door. (all windows are double pain.) Surprisingly, it doesn't seem to make any difference in the sound quality. I have tried to cover with different materials, but don't hear any difference. ... Surprised me!
Bottom line, at least relating to my situation--->DON'T WORRY.
By the way, my room is 25X15X8 and it is fantastic. I place my system along the long wall. I used Robert Harley formula to see if there are many peaks or valleys in the frequencies, and there are virtually none.