Clearly you should do what sounds better to you. Although I would recommend listening to music (not the system) for a couple weeks or so through the bypass, then switch for a couple weeks and go back. This would let your subconscious soak up the different sounds and help you choose. Sometimes it is easy to make a decision based on the treble or some other dynamic and not notice the rhythm and pace is sucked out of the sound. This causes a reduction in how musical it sounds. Longer listening will help you determine which really sounds better.
So, I would make sure you are really comparing the sound fully… then use which ever sounds better.