How to manage a big itunes library?


I have had a mac mini running itunes for about 2 1/2 years now. The inevitable has happened, my first drive has failed, at least it's icon has stopped showing on the desktop and I can't access the files.

I had a feeling this would eventually happen. I have the files backed up, but this does create a headache to fix it. The files were originally ripped on 250GB drives and the backups are on 500s. I have 5 250GB drives and one 500GB drive, all pretty much full. I have a pretty big collection, about 5000 or so CDs.

I need advise on a solid solution for this type of setup. There are some pretty big drives comming out but I just don't know what would be the best option.

Also this might seem stupid, but I am wondering if there would be any loss in quality in a copy of the original rip vs the original???
128x128ejlif
Ejlif,
I feel your pain. I had 40,000 songs on a hd that failed me a month ago. I didn't have it backed up so all was lost.
I found that i didn't listen to a lot of what was on there. I did have to re-rip a bunch of cd's but i only put on there what i actually want to hear. Makes it easier to find what i really like to listen too.
I haven't bought a cd in a long ,long time. I grew tired of buying a $14 cd for one or two songs. I end up surfing itunes and buying music there. I know it's lower quality but i would rather listen to good music poorly recorded than the other way around. Sorry for the rant!
If you want a more seamless backup you may want to consider one of the less expensive RAID solutions that use very simple interfaces. In this scenario you would have a slightly larger RAID unit as your primary music library. The RAID unit contains several raw hard drives which easily slide in and out of the unit. The hardware / software can make one or more of those drives a 'redundant' backup of the main drive, and continually back it up as the main drive grows (within the capacities of the enclosed drives of course). In the event of a drive failure, which is inevitable with any drive, you'd simply slide out the failed drive and slide in a new one. The RAID system should never miss a beat and would immediately put the backup into use should the main fail, and would immediately backup to the drive you slide in to replace the failed drive. At best it is a constant ongoing process. Drawbacks are the larger size, more noise (not an issue if you are using a WIFI solution), and greater initial expense (one of the current crop of affordable RAID solutions may run around $800 initially, but then you only will have to replace raw hard drives (cheap). Another suggestion would be to buy the slightly more expensive and longer-lasting drives like G-Tech, but that doesn't mean they'll necessarily never fail - all drives will eventually fail. In general the larger TB drives are less reliable overall than the moderate drives. 500GB is my flavor of choice. TB drives are more often two or more 500's in a case together. They make more noise, are slower read/write, and are more prone to early failures in general. A quick read of any real-world reviews should prove that out. Not to say that there are not exceptions.

Good luck.
One other suggestion is consider adding hard drive coolers to each drive. These are fan-based coolers that mount directly on the drive. I picked up 4 recently for my systems; they were $5 each. They went on quickly and lowered the hard drive temp of every drive by 10 to 15 degrees centigrade. Since heat speeds up the failure process it is not a bad idea (though it does not replace backups.)
I am wrestling with the similar issues. If you haven't done so, I suggest visiting Computer Audiophile for a fairly active discussion of HD alternatives, backup strategies, etc.
I've had my eye on one of these for awhile and may pull the trigger before the year is out.

http://www.drobo.com/

It's basically a simplified RAID system that utilizes up to four drives of any size. There is redundancy built in so when one drive fails your data is protected and the bad drive can be replaced on the fly.

I'm thinking of buying the newer Firewire 800 model and buying four 1TB Seagate or Maxtor drives for total storage of 2.7TB.
I am looking at the Drobo too. I want a solid solution. I am guessing I will have a library of well over 3 TBs when finished so I need something big which means 6-8TBs with backup.

Do you know if the Drobo is loud?

It is nuts to think this stack of drives I have is basically going in the garbage. Not really but I won't be using them, nor would I trust them.
The Drobo appeals to me also. But I think it's loud enough that you wouldn't want it in your listening room.
6-8TBs is one big library!

The reviewers at Newegg and Amazon don't seem to think the Drobo is TOO loud, however the new unit seems to be quieter AND runs cooler.

In your case, you may have to wait awhile for 1.5TB drives to become more affordable and even more reliable. That would give you a little more that 4TBs on a Drobo, but you may need to of them dark, shiny babies.
Good point, Drubin.

I haven't actually heard a Drobo unit, but four spinning hard drives with a fan can't be dead silent.

Then again, any computer based system will have to contend with some level of hard rive or processor noise.
I have a Drobo and it's a wonderful product. I don't think you would want it in your listening room. It's not as loud as some computers/drives, but it still has a "presence".

I got one of the new FireWire 800 drives and new network option. I place it in a large closet where I keep my CDs and it works perfectly.

It doesn't need service and does all the "RAID" work for you. Highly recommended if you can keep it out of the listening room.
Yo Dean - Boy, the reviews on Amazon are really spotty for those units. I dunno, if you are going for something in the room I might look for a potentially quieter solution (seems to be a frequent complaint). There was a similar unit that all the photogs were raving about on the ASMP list a while back. Noise obviously wasn't an issue there so there were no comments I recall on that subject. I'll try to get the name of it. If you're on a tight budget and want to try RAID, you might want to check out the Newer Tech Gmax which I know absolutely nothing about other than it is cheap. It does not appear very modular though. I'll try to find the name of the one the shooters were using. I actually just have to cheap, quiet LACIE 500gb drives that are redundant for my music. I'd prefer a RAID solution that did it in the background automatically and may go that direction also.

Marco
Keep in mind that about the only scenario a RAID setup helps with is a single drive failure.

There are lots of other things than can take out all the drives or even a whole system: a power surge, theft, fire or smoke damage, a broken water pipe and so on. Even typing the wrong command can take out a RAID.

The primary benefit of a RAID system is for computers that need 24/7 access (web & email servers, systems for businesses) where a RAID will allow a partially crippled system to keep running until a drive gets replaced.

For a large music collection there is no substitute for a backup copy of the drive. Re-ripping and tagging can be a very slow process. I actually keep two backups with one of them off premises. That makes a lot more sense to me than a RAID. I only use that for my business computer.
There is also Neil Van Berg's Music Vault which is a standalone solution (no computer interface needed). It backs up (automatically I think) to an external USB drive(s) of your choice. It is compatible with both WIFI And USB interfaces like Tranpsorter, Squeezebox, AE, etc. No direct experience to speak of. Six Moons wrote it up in a positive light, I think.

Yeah, that's a good point about keeping an off-site backup, especially if you have a huge library. I also do the same thing.
Whoa, Marco!

I hadn't read the Amazon Drobo reviews in a few months. I just became aware of the release of the FW800 version a few days ago, and it seems the newer unit for some reason has very poor reliability. What happened??!!

The other thing I was looking at is this:

http://eshop.macsales.com/item/Other%20World%20Computing/MHU27S3.0T64/

But this unit still carries the greater risk that larger capacity hard drives have associated with them.

I've used NewerTech processor daughter cards in my old PowerMac 8500. It's good to see the company is still around.
Dean - The OWC stuff is highly respected for using quality parts. My friend has a 500gig Mercury Elite Pro drive and it's pretty noisy so that's a downside to them. Other than that they are used by many photogs, as are G-Tech, for their reliability. I found the RAID solution that was recommended on the ASMP list (no direct experience myself): It is by Accordance Systems. As you know, us photographers are a pretty demanding lot when it comes to dependable storage/backup as our livelihood is on those spinning discs (this of course goes for many other occupations). When it comes to building hard drives definitely go with something that is well vented/cooled as has already been pointed out, and with a company known for using quality parts (controllers, power supply, etc.). Alternatively it is not that difficult to get all the parts and build your own with quality parts. Any way you do it, the drives will still eventually fail, it's just a matter of how long they last.

Marco
Another very tedious backup solution is to back up to dual-layer or single layer DVD's. That is best facilitated using a backup program that allows backup to multiple discs. If you have a library of of 5000 CD's that would take about 330 DVD's or half as many dual-layer (SLOW to burn) DVD's, and to restore your library would mean transferring those all back to another hard drive in the event of failure of the original copy. Still, it is yet another potential backup strategy for an additional hard-copy on fewer discs.
PS If you do backup to DVD, my calculations, I think, were for WAV files. If you use Apple Lossless or some other compression scheme you'd be using much fewer DVD's. Also, if you back up to DVD, I'd strongly recommend buying high quality media such as Taiyo Yuden, as opposed to most of the crap they sell at the mega-office warehouse stores, most of which is made in China, has poor lamination process and spotty reliability in burning. I've had half a cake-pak of those type of CD's fail to burn. YMMV, but I always use Taiyo Yuden now and have had no problems at all. One can easily see the difference in lamination process in comparing a TY disc to the alternatives. Again, a tip from one whose backup is his livelihood. Genuine TY media is available from The Supermedia Store
Over the last three years, I've bought a lot of equipment from Other World Computing. Good people and good stuff!

I can definitely vouch for Taiyo Yuden DVDs and DVD-DLs as well. And, Mitsui for CDs.

The number of CDs I've ripped to Apple Lossless is only about 40GBs, so I don't have nearly the challenge Ejlif is faced with. A 250GB fanless hard drive might be a solution for me.

Ejlif, you may have to consider a small RAID system placed in a different location from your listening room.
Wow I was ready to pull the trigger on a 4TB Drobo. I only needed to find out if it was to noisy. Sounds like that idea may be out the window. Now it sounds like it isn't reliable. Damn someone needs to come out with a great big drive that is reliable. They'd get rich it sounds like.
Ejlif - I don't think it's the drive that's unreliable, it is the RAID device itself and the interface with the drives. These are not "great big drives", as you intimated. A RAID unit usually has multiple slots that house many duplicate-sized hard drives - you buy whichever SATA (or whatever type the RAID device uses) raw hard drive that you choose and just slide them into the slots. The RAID hardware is what you are paying for in the case of these modular boxes. The hardware is what allows you to create and maintain seamless redundant copy drives in multiple slots and keep the system running in the the event of a drive failure. It also would have to keep all those drives cool as multiple large drives are bound to generate some heat. As someone pointed out, in the event of a disaster such as a flood, fire or theft you'd be SOL as all copies would likely be damaged/lost (they'd all be contained within that device unless you occasionally pull and replace a drive with a replacement and put the copy drive in a safe, off-site location. The RAID devices DO make the process of keeping/maintaining a backup of large amounts of data a very easy process if the device works properly. I don't know what the problems folks are having with the Drobo units are, but you should be able to use any brand of SATA drive you choose within the box itself. If the thing is noisy it is likely a problem with the cooling fan(s) and or the specific drives that are chosen by the user (some drives are noisier than others), as well as how the case of the device may amplify that noise.

Marco
I agree with Mlsstl's take, which is that RAID is not really important. What I need is to periodically backup the library to a separate drive and, ideally, store it offsite. I've just ordered a WB MyBook Studio II 2TB. If it works out, I'll need to get a second one, or something like it, to use for backup. The Studio II has Firewire as well as USB (and eSATA) connections. I'll try it in the music system connected via FW. If that doesn't work, I'll either place it on the other side of the wall and run FW through, or attach it to the router's USB port and access it wirelesly (I think that will work).

I have read that, if you are running USB from your computer to a DAC, you should not attach storage via USB also. Too much contention for the USB bus, or something like that.
I've been talking with Ejlif about this and doing some research on drives and backup and all that. Seems like it comes down to this:

1. Setup needs to be quiet as possible (or) removed to another room, ventilated closet, etc.
2. Setup needs unlimited* growth potential.
3. Setup needs to be backed up easily. Said backup needs to be easily transportable to another site (i.e. work, friend's home, etc.)

Given that I was thinking of getting the Western Digital MyBook Mirror Edition. The drive is set up with two drives, one which mirrors the other RAID 1. Here's a link to the drive:

http://www.westerndigital.com/en/products/products.asp?driveid=466

From there, I copy all my music in iTunes over to this drive and I'm good to go. iTunes reads from one disk while the second disk acts as a local backup. I can then take a new third disk (same size) and, once parity is achieved between the two disks, swap in the new disk, take the old disk and store it offsite. BAM! Local backup, offsite backup. Ahh...

But what about adding more disk space? Take advantage of iTunes...you can add new drives as your library grows and it will remember the previous drives you've used. It's an intuitive feature but it does work.

These drives are fanless, *green* and really solid. The downside here is that these drives are USB. Though I'm not certain how much of a downside that really is.

Thoughts?
Jwynacht - It's a sound strategy, yes. USB should be just fine as streaming music is not demanding of a fast interface. It will not be any problem at all in that respect. It will slow things down when you start copying large amounts of files from one drive to another since the interface is comparatively slow. The other disadvantage may be in using a 'green' drive, with no cooling fan, your drive life will likely be shortened as heat is an enemy of drive life. Still, those drives are certainly the most quiet (quiet as a church mouse) and I use them myself (Lacie 500 gig version of basically the same thing). Yes, iTunes can look at as many drives as you care to store your library on. In turn you can consolidate your library onto one large drive with just that command: "Consolidate Library" - in this case iTunes will COPY all of your music files to whichever hard drive is currently pointed to as the main library. It will leave all the original files in tact on whatever drive they were originally on. You, of course, need to have enough space on the drive you point it at to store all those files. Here is a case where a USB interface would seriously bog things down with a large library. As long as you can walk away from it you'll be fine, but if you plan to hang around and wait, you better get a good book.
OK, so I am going to get the WD mybook, the one with the firewire connection. I will just keep all my Lacie drives for now as backup. I have ripped my collection and as each drive filled up have gone to the next one. I have a total of 5 250gb drives and one 500 now filled so I will have a little room to grow and will be backed up with all the Lacie drives I have now, besides they aren't worth crap anyway so I might just as well save them for my backup. If the WD drive is nice I'll get another one and use it for new rips and backing up new rips.

It has a 5 year warranty so they must be somewhat sure that it will last.
If you are running USB out of your Mac Mini to the Paradisea, you may hear an improvement if your storage is connected via Firewire rather than USB. At least that's what some people claim.
And speaking of storage, good articles to read here:

http://hardware.slashdot.org/article.pl?sid=08/10/21/1716207

http://hardware.slashdot.org/article.pl?sid=08/10/21/2126252
This sounds like a good solution, and I see good deals currently on that WD Mybook. If you are planning to upgrade your Mac in the next couple of years you may want to keep the following in mind about Firewire drives.

Apple just phased out Firewire in their new aluminum MacBooks. So far their other current models still have Firewire, but there is speculation that they are planning on eliminating FW from their other model upgrades. See this article:
http://pogue.blogs.nytimes.com/2008/10/30/the-macbook-makeover/?em
If you think the hard drive to PC via firewire sounds good... you should hear firewire to dac. WOW...

USB was never intended to transmit misuc. Firewire is.
Hopefully, the omission of Firewire 400/800 from the MacBook might just be a prelude to newer Apple devices utilizing Firewire S1600 or Firewire S3200.

I've never been a fan of USB. USB 1.0 was terrible, but 2.0 is better. It'll be interesting to see how things sort once USB 3.0 is in mass usage.

In pure performance terms Firewire S3200 and USB 3.0 should be almost 4 times faster than current Firewire 800, so it'll be interesting to see how they stack up in real world usage.

Are there any DACs that can utilize a fibre channel equipped Mac?
The best solution would be to use a networked attached storage chasis, onboard RAID controller or expansion slot based RAID controller and build what is called a "RAID 1" array.

RAID stands for redundant array of inexpensive drives. Raid level 1 is designed so that all of your data is recorded to one drive and is automatically mirrored to the second drive. In the event that one of your drives fails, you can rebuild your array and recover all of your data by just replacing the single drive.

This requires two identical enterprise class hard drives, in this case I would use either two 750GB drives or 2 1TB drives and RAID 1 them for a secure vault for your music files. Both drives would have to fail simultaneously for you to loose data. Note that your total storage capacity remains at 750GB or 1TB, depending on the size of the drives used, because the second drive is merely a mirror of the first.

This is how IT professionals store data securely--although they do it on a larger scale. Your network and operating system will see the drive array as a single drive.
I should also note that you can use RAID 0 to create a single logical drive with the combined storage capacity of multiple drives and then mirror that array to an identical setup in RAID 1. (RAID 0+1) So you could chain together, for example, two 1TB drives in RAID 0 creating a single logical drive of 2TB and then create a second RAID 0 array made up of two identical drives, and then use RAID 1 to have the second array mirror the first for security.

The point of RAID 0 is that you can use the drives you want to create the capacity you need. You could take your four 250GB drives and create a single 1TB logical drive. I only point it out because sometimes large capacity single drives are expensive and it is more economical to use multiple drives. In addition, sometimes single platter smaller drives are more reliable, run cooler and quiter, ect. This provides flexibility.

Note however that RAID 0 has double the mean failure rate of a single drive. However, if you also mirrored the array or otherwise backed up your data this would not be a big deal.

All of the arguments against using RAID setups are well taken but it is an option that provides flexibility and ease of use in that the result is an entire music library on one logical drive and a built in backup system to guard against drive failure.