I've done many many comparisons, with wide varrieties of music, played through lot's of different "passive set up's"..which is what you're discussing. My conclussions, thus far, has been that, you get more sonic purity in terms of potential transparancy doing what you're trying(as long as impedance between components is workable that is), but dynamic potential suffers! For me, this isn't usually acceptable for my music tastes overall.
No matter what any "mello-music listening audiphile" has tried to tell me about how wonderful their "passive" set up is sonically(i.e, no preamp), I always find there to be a lack of authority, weight, and dynamic ability to these systems! I've PERSONALLY and DIRRECTLY compared many many set up's in terms of dyanmic ability(at the very least), with systems using many different amps and varriable audio out CD players and DVD players alike, and NONE TO MY CURRENT ATTENTION could match what you could get from a good active preamp set up dynamically! I've tried had many such passive set up's in my systems over the years...the McCormmack TLC, Pass Aleph L and P's preamps, home made passive preamp's, source components(from many manufactures..both high end and otherwise) with volume controls built in connected dirrectly to MANY DIFFERENT SOLID STATE AND TUBE AMPS,and serveral other passive line control devices which I've forgotten the names of off hand! And if you're listening to dynamic rock, Techno, R&B, most Pop, or anything else containing demanding upbeat and weighty dynamic content, you'll likely be dissapointed compared to what a good active preamp can provide you with!...so it's got trade-off's. For me, I just won't settle for dyanamic "whimpiness!"
Hope this helps.