SET the best?

Is SET amplification where we should all end up? I keep reading posts where people tell of their journeys from plenty power to micro power, and how amazing SET amplification is 45 set 211 set 845 set otl, and usually, ....with the right speaker. I have yet to read of anyone who has gone the other direction from SET, to High watt beast class A amps or others.
If your speakers can be driven by minimal wattage, is this the most realistic, natural sound we can achieve? versus say, 86db sensitive speakers and a 1000w amp?
Is the end result solely based on speaker pairing? circuit? tubes?

I am in the process of changing my direction in my search for realistic sound, just because, and wondering if this really is the best direction to be going.
From what I have been reading I think it may be.

What do we get with SET? What do we give up?

What's you favorite color?
I have a system that includes a modded and fully tube-rolled 3-watt Moth s2A3 amp, Omega Grande 8R speakers, Von Schweikert subwoofer, tubed CDP, and all Jade wires. I also have a system that includes Mac 500-watt MC501 monoblocks, C2300 preamp, Marantz sa11s2, Montana EPS2 speakers and all Wireworld wires. I love the 2A3 system. The SS systems beats it in every parameter. Many roads to Dublin.
I know no SS that can do tone like SE DHT like a 300b. If you didn't like it probably running incompatible speakers.
Post removed 
I prefer Pure Class A SS amps over tube amps, including SET, after a long and varied tube amp journey.