I don't have a plasma but I do have a 42" SHARP LCD TV hooked up to a Mac Mini. Sound quality is as good as the DAC you use. It's great having all your music in one place and being able to easily organize and access it the way you want without having to deal with CDs. Video quality is as good as the source. The only problem I have is that I haven't found a way to make on-screen text big enough in Itunes, menubars or Finder to read easily from my couch 10 feet away. You may be ok being closer.
Also, it is very tricky to get it to output 1080p, at least with my model ("Late 2006") Mini. You might want to google Mac Mini 1080p and also check out the Mac Mini user groups.
The Apple remote works fine with Front Row but without a keyboard and mouse you're limited in what you can do. I've heard you can use an Ipod Touch or Iphone but that seems pricey if you just want to control the mini.
If the Mini goes down, you can't access your Itunes music. I like OS X more than Windows but it's still a computer.
All in all, I'm happy with it. It's not a seamless media center but then it's not designed to be. For example, when I watch DVDs, I have to change my audio midi settings if I want to use the TV speakers instead of the stereo. Not a big problem if you have a mouse handy but still a little annoying.
On the other hand, it can do a lot more than just play music and videos. And Macs hold their value very well so if you ever want to sell it, you shouldn't have a problem.
Just my 2 cents.