Best way of removing rust from silver or other

I have a pair of Audioquest cables with silver banana plugs that have tarnished due to humidity and time. I wonder which is the best way of removing this tarning without damaging the plugs permanently....Btw, is this sort of tarnishing detrimental to sound or just something that is not aesthetically pleasant?
The tarnish on silver and copper is a form of oxidation (so is rust), but it forms a patina on the surface and does not actually destroy the metal in the same that rust will cause iron and low-carbon steel to decay and eventually crumble. Any good silver cleaner/polish will do a good job of cleaning your silver banana plugs, although you should be careful to remove all of the cleaning agent (Q-tips are helpful here).
There used to be a two-step contact cleaner from Cramolin. Don't know if it's still available, but it came in two small bottles, one red and one blue. It was wonderful for removing oxidation and laying down a thin layer of contact enhancement film that prevents further oxidation. You use it sparingly so the bottles last a long time. I'm still using the stuff I bought 10 years ago...
I use a Dremel rotary tool with a carbon steel wire wheel. DO NOT use a stainless steel wire wheel, it is too stiff and will leave scratch marks. I then follow up with SuperContact or similar contact cleaner. This process works great with AC plug prongs also.
Silver tarnish (oxide) doesn't affect the electrical properties of the metal at all. So, not to worry in that regard.

Thanks Ed!
Agree with Ed. Did a quick search on the internet and turned this up:

CuO (copper oxide) has a magnetic susceptibility of +267 while AgO (silver oxide) has a magnetic susceptibility of -19. 

By way of comparison, the magnetic susceptibility of FeO (ferric oxide, (rust) is 293.  

The much higher magnetic susceptibility of CuO means the skin depth will be *very* thin, and net skin resistivity very high.  Meanwhile, the negative value for AgO means that the skin depth will be greater than anticipated.

As with steel versus copper we have to look beyond DC resistance to find out which material is better suited.  It turns out that oxidized copper is almost as bad as oxidized steel That is why silver plating is often suggested for conductors.  

Although the conductivity of silver is only 6% better than copper, when each surface is compared in an oxidized state, silver is a far superior conductor than copper.

Short answer, if you like the sound of copper, keep it free of oxidation. My own suggestion to polish and renew either material is Flitz polish from Germany. The best Flitz is called Polier and second best is Flitz metal polish.
Be careful, most polishes contain abrasive cutters and they certainly do the job. Household furniture polish might be the only exception. For soft metals, i.e. copper, aluminum, silver, and gold use a cleaner like Meguiar's Cleaner/Wax. It's automotive clearcoat safe so won't abrade surfaces like anything labeled polish. Run screaming from grandma's favorite pink silver polish as it's loaded with grit.

For hard metals, 0000 steel wool is a safe bet on the likes of pitting chrome (amp chassis) or specially hardened aluminum alloys (bike chainrings).
You can use an eraser to rub off the oxidation. It is cheap, safe, and leaves no chemical. You should use the one for removing ink from ballpoint pen, not the one for removing pencil marks.