Leaving iTunes in 24/96?

I'm accumulating a few HD tracks in 24/96 (converted from flac to Apple lossless). I know iTunes does not switch sample rates automatically. Any big difference in just leaving set at 24/96? I know the redbook tracks would get up-converted all the time.
Use Media Monkey instead of ITunes. You won't have to worry about converting.
Much more superior soft to iTunes IMHO.
You can spend $300 on Amarra...it switches automatically. There is also a remote app for iPad that allows it to be used as a track pad remote and has a program for Amarra...pretty cool!
Experiment and see if the upconversion sounds okay to you.

If you are using a Mac, remember that every time you change the sample rate in Audio Midi, you need to restart iTunes for it to take effect, otherwise it will just up- or down-convert. Don't know whether or not this is also true in Windows. This step is not required if you are using Amarra or Pure Music (both are Mac only).
I'm extremely happy with iTunes and after experimenting, I cannot hear a difference thru my Yamaha HS80 studio monitors. Now, my converter is a lower end Presonus, but to be honest I think it does a decent job.
I didn't know you had to restart iTunes. How'd you find this out? Just wanted to make sure this was the case.
It's what the computer audio experts all say.