I rarely use center channel with processor

In my experience I prefer the more direct sound of a dedicated center channel over the phantom center when both are done well. For me this applies to music as well as movies. But I'm not setup with the amp channels or processor, or speakers space to do a center channel right now so I get by with my phantom center, which  sounds very clear and vividly located in the center so long as I'm sitting in the center. It ultimately lacks in tonality due to inherent comb filtering issues with creating a phantom center. But I feel in the past I've paid too much attention to it. If I don't do direct A/B comparisons my brain/ears adjusts very well to the tonality. Room reflections pretty much mask it.

I got rid of all my surround sound equipment a few years ago after many years of trying. I discovered that I could understand most of the dialog in movies much better with a good stereo system! And I could also enjoy music more without all of those extra speakers that cause imaging problems. That's just me.

I only use a center channel, a competent one for movies with center spread enable (not the full dump into the center channel).

i upmix a lot of stereo music recordings into 4 Channel (2 fronts, 2 surrounds), primarily with the the DTS, auro & 360 reality upmixers (not much a fan of dolby).

I've had musicians audit a/b with the stereo and upmix in my house. They've preferred the upmixed experience, not one picked the stereo, which is a bit eye opening.

There are speaker and and amp designers themselves who are happy to sell you 2 channel gear, but, do a lot of their personal listening with stereo upmix or multichannel. 

So, What exactly is a "serious audiophile" and a "non-serious audiophile" again?

There's a learning curve and requisite quality of equipment to do multichannel correctly. If you can't figure it out, stick with the ancient stereo. Those who wish to enjoy other things might try to figure it out.

I'd agree with those here that are saying two channel music (plus maybe subwoofers on) will typically give the best results for stereo recorded music. Although I do know some serious audiophiles that use a center channel, I know none that use full surround for two channel music. Of course you can enjoy your system however you want. Just curious if you're pushing two channel music to your surround via a surround algorithm or if youre using surround formats for your media playback.

Measure your room to get an idea of your peaks and dips. I already have excess bass at 30Hz. Neither I or room correction can do much about it without creating a massive hole from 60Hz-150Hz. With real content, it's not an issue because below 30Hz, it's more about feel and you want more of it. Therefore, if you have excess bass at 100Hz and use phantom center, the dialog can sound boomy.

I'm using an 802D2 and HTM2D2. I think of it like a 3-way vs 2-way speaker. You get more control over each channel. For example using a center, you can boost the bass on the LR without boomy dialog. Without the center, the dialog is boomy.

If I have a significantly subpar center, I would consider using phantom center with those speakers.