Amp, sorry to say. When it goes from magical to painful, it's clipping in the amp, almost always.
Can you borrow an amp with 100 watt or more? (90dB sensitivity isn't exactly inefficient, but it's going to push your 25 watt amp to it's limits, particularly if your room is fairly large and has a lot of plush furnishings).
My speakers are rated at 91dB sensitivity and they didn't really start to sing until I got up around 200 wpc. I have lots of big band recording, big orchestra and jazz with powerful bass. I can't imagine playing that around 85dB average SPL with only 25-watts.
One question, I'm assuming that you're keeping the input level fairly constant and your source is the same for small groups and big band. If that's true, then I'm still betting on the amp. If it changed with source, then the input level might be too hot from one of your sources, but I'll bet you're using one source at around the same level for all types of music.
I lied, one more question, is this happening with a digital source or a turntable? If TT then you could have cartridge mistracking from too little downforce, or just a underachieving cartridge.