What's that buzz, tell me what's a happening...

If you are handy with tools taking the woofer out shouldn't be a problem. Hopefully it's as simple as a wire slightly out of place and is vibrating against something. Knowing exactly where it's supposed to be is difficult.
Be sure to note the correct orientation of the driver before removal.

Perhaps you are simply over driving it...this is after all a small woofer and low frequencies take huge amounts of excursion on a small woofer?

It may also be a lose dust cap or perhaps the driver wire is touching the woofer cone when driven hard.....so yes do open it up and take a look!

What amp are you using?

First, try tigthtening all the woofer screws a little to see if one is loose. This happened regularly on my Spica Angeluses, exactly as you describe. Important: Don't overtighten! If the screw doesn't turn easily, then it's probably not the problem. If you get to one screw that turns easily, turn it until it's snug, then play the troublesome track.

Since you say the problem is in only in one speaker, I would first swap the speaker cable connection R->L, L->R, and see if the problem moves to the other speaker.

This will tell you a couple of things. First, it will confirm if the problem is just in the one suspected speaker. If you don't get the buzz with the cables swapped, you know the issue is/was in that suspected speaker. If the buzz does move to the other speaker, then the problem could be an issue with the design of that particular speaker model, or it could possibly be something in the chain before the signal reaches the speaker.

After the experiment, don't forget to swap the cables back to the original position and reconfirm that you still have the issue before you start "digging into" the speaker.

Anyway, it's free and it's easy to make the swap to insure the issue is really in that one speaker.

Enjoy,

TIC