Are you talking about the Gen 1 ATV or the current one? If Gen 1, music streamed or stored?
I've had an ATV 1 for about 3 years now. Nothing but great things to say about it. If you have and iPod Touch, iPhone or or iPad, it makes a great remote. No need for a TV, and quite honestly it makes finding music a bit easier.
Toslink isn't usually the best connection, but it does have its advantages - it makes a ground loop and electrical noise impossible between the ATV and DAC. Get a glass cable, and it'll sound great. I use a $25 one from Amazon and it easily beat a $75 or so fiber cable.
There was a thread where a bunch of us shared what we were doing to maximize sound quality. Noting expensive or ridiculous.
I don't think the ATV will pass a high-res signal. I think it downsamples everything to redbook. About the only downfall I know. Then again, I don't have any high-res stuff, so no big deal to me.
It's a great self-contained server IMO. It just works. With my Rega DAC which is NOS and reclocks, my system gets out of the way and plays music. That's what it's all about in my book.
I hear the Mac Mini is better, especially with Pure Music. Can't afford that, and I don't regret it either.