How do I know if I need a sub woofer?

As do I. Again, extension and a reduction in room modes. I think it is the ultimate approach.

@atmasphere wrote:

I use the Swarm subs as well and can also recommend them. That despite the fact my main speakers are flat to 20Hz. That meant I only needed a pair of the Swarm subs to do the job. What I like about them is they are designed to sit directly against the wall to take advantage of the room boundary effect, which allows them to be compact while flat to 20Hz and out of the way.

Which Classic Audio Loudspeaker model are you using - the T-1.5 Reference? If so then these main speakers of yours have what effectively is a built-in sub with an 18" down firing woofer per speaker, in addition to a front firing 15" ditto. They adhere to physics alright with size, high sensitivity and large displacement area, and so by adding the two spaced Swarm subs by Duke you have 4 bass sources with the predominant role of the dual Swarms to acoustically smooth out the response, much more so than pressurization anyway - a clever approach. Actually the Swarms don't need to extend to 20Hz flat to do that, but it certainly doesn't hurt either. 

What I'm saying is this: most don't come with a pair of main speakers that like yours have the kind of low end foundation they do, and so your speaker setup is hardly representative. Rather it's typically the inverse scenario where the subs need to do the pressurization and extension down low, but similarly to the approach by you they could the leave their main speakers running full-range (at least ~40Hz extension is needed) and space out the (ideally large) subs to make for a DBA and to have proper low end fill and smoothness. 

Well you guys are tapping into my insecurities big time,,my therapist says I may have audio nervosa and while she recommends more pills I can't stop thinking about the swarm and getting flat response down to 20Hz. It should boost my testosterone as well. 

Which Classic Audio Loudspeaker model are you using - the T-1.5 Reference?

@phusis (+1) I have a custom version of the T3, which is taller than the normal version to obtain the same box volume as the T1, which was too wide for my room. 

It employs dual 15" woofers (one is the TAD 1602; the other is a custom field coil powered woofer made by CAR). 

One of the Swarm subs sits to my left with its driver facing the wall (to insure its in the room boundary effect; moulding along the floor describes its distance from the wall). The other, also with the driver to the wall, is behind me and slightly to the right. Both are well out of the way of room traffic due to the small size and required placement.

Its very easy to demo how well they break up the standing wave in the room. At the listening position there's no bass impact without them on, although that bass is there along the room boundaries and elsewhere in the house. When I turn on the subs, the bass is the same anywhere in the room. Its not overpowering, its just there. I have them crossed over at about 50Hz to make sure they have no output above 80Hz, so as to avoid them attracting attention to themselves. 

I’ve got very good audiophile equipment and use bass traps and had help with setup - and to me the system sounds very good but not great!  I’ve hit a wall trying to make a less than ideal (family room) into to a great listening room. After attending Axpona the last few years, I am seeing some very high end Co add subs to their very expensive setups. I believe it’s for the same reason that I need them - poor room acoustics firming a less than ideal listening area. 
Like a few of the comments said, it’s less for deeper bass and more to affect standing waves that are limiting superior sound.