Extra speakers in a room can color the sound. If you have to do it place the drivers close to the wall. It is probable that you won't notice any difference
In addition to Alan’s comments you can also short the drivers.
Install a piece of conductive wire across the +/- terminals of the extra "passive" speakers.
Easy enough to listen to the before/after results of such, and if you hear a difference you can then consider using the extra speakers as bass/mid-bass traps.
I remember years ago, visiting the Good Guys listening room and it was filled with shelves of speakers. I listened to some plastic horn Klipsch speakers, which I don't normally care for, and they sounded real good. I later mentioned this to an audiophile friend who had worked at the Good Guys, and he said that the multiple sets of speakers would vibrate to the sound waves of the active speakers and create a pleasing effect that greatly enhanced the sound of the speakers.
Has anybody ever heard of this effect?