@lukaszwk If your speakers have dual woofers that might be where things are getting in trouble. One disadvantage of low impedance speakers is the speaker cables become far more critical. The series resistance they have while seemingly slight, has a far greater impact on the speaker performance if its impedance is already low.
This means that if you have 4 Ohm speakers its a good idea to keep your speaker cables as short as you can! 6' is a common maximum.
If its a nominally 4 Ohm speaker and uses dual woofers, they may be in parallel thus bringing the impedance of the bass array down to 2 Ohms. It might be a bit higher in practice due to box resonance. This is speculation since so far you've not mentioned what speakers you own.
The other possibility is the crossover has a dip and/or there's an odd phase angle in the bass region that asks more of the amp. If the latter you simply have to get an amp that really doesn't care about the load.
There are class D amps about that tick the boxes you require and are also load invariant. My recommendation in that regard is look for one that is 'self oscillating' as they tend to be load invariant while those that are not of that type are less so.
There is one other thing. If your speakers are in a room of regular dimensions you may be dealing with a standing wave (like I was) which is killing the bass. If you move about the room are there places (like along the walls) where the bass is better?