I rarely use center channel with processor

I wrote about this effect in my blog, but I should reall expand on it.  

You are right, with one major exclusion.  A well matched center channel being on or off is inaudible but only when you listen in the ideal listening location.  What the center channel should do is let you listen from a variety of listening angles (relative to the television) but still have the dialog come from the center. 

By the way, I have a very well matched center, so no, it does not sound better or worse when the center is on or off for me so long as I'm directly in front of the television.  Lying down the nearest speaker dominates and shifts the sound stage. 

https://speakermakersjourney.blogspot.com/2024/06/what-good-is-center-channel-speaker.html

Does anyone else not use a center speaker because it is better to use only the mains? 

Definitely!  One should use the center speaker only when the source material has discrete center channel content.  OTOH, don't eliminate it if there is such content.  Such manipulations can achieve only chance, sporadic success.

If you are talking about playing music through your home theater system, then just allowing it to act as a stereo can sound better. If movies, then probably not. In movies more than 80 of the dialogue comes through the center channel. and you're looking to recreate something usually created in 360 degree sound. 

PS - I've experimented with the use of analog matrix systems to extract a center, including Dolby Surround/ProLogic and PL II.  Among them by far the best sounding for music was the DTS Neo:6 system.  Really sounded great and with a center channel gave very realistic imaging. 

Of course, use what you'd like, the point is enjoyment. 

I think maybe what it gets down to is the nature of how the recording was done. If the engineers did a good job with five channels and isolating voices in the center channel then it may work out. But if they mix a lot of non-voice stuff in the center channel and let a lot of voice stuff inappropriately mix into the main channels it mixes things together in a bad way for sound purposes.

Also if you have a system that resolves detail really well that's really impactful too and you notice these things.  

So it gets down to quality of the recording and whether you truly have a good multi channel recording.