I've been an ARC dealer for about 11 years and a fan for much longer. I've owned the LS5mkI, mkII, and mkIII, Ref 1, Ref2, and the Ref2mkII. I've also spent years listening to and selling all of the others you've listed. Generally, the newer models have always been better in a few ways. They are more transparent, more detailed, more extended at the frequency extremes, and more neutral in their tonal balance.
Some may prefer the sound of an LS5mkII to a REF2mkII, but they are choosing a more colored sound and are, I think, in the minority of ARC fans. ARC is a company known for neutral (read: not overly tubey) sound. If you want tubey sound, there are tons of other manufacturers to choose from. Once you get used to life with a REF2mk, the LS5mkII sounds too dark, muddy, opaque and dynamically restrained by comparison.
Personally, I prefer the REF2mkII to any previous REF, including the "all tube" REF2mkI. I like the LS16mkII and LS25mkII much better than the MK1 version of either. The sound improved dramatically with the change to the 6H30. Many of the people who claim that the MK1 versions sound better may not have access to both units side-by-side as I do. Some also just like the ability to play with NOS tubes, something you can't do with the 6H30 (with the exception of NOS versions for $50). I will even go so far as to say that the LS16mkII is better in certain important ways than the LS25mkI. Sure, the LS25mkI will be superior in that is has more weight and slam due to its bigger supply, but the LS16mkII will beat it in terms of clarity and transparency. So in the end, there are trade offs.
PS. For the record, the VT100mkIII is much better sounding than the MKII for all of the same reasons.
BTW, the VT100mkIII also sounds much better than the MkII for the same reasons.