To unplug and then plug back in does not clean anyhthing. It just moves stuff around.
As Tobias said, you can buy electrical contact cleaners such as Caig DeoxIt. DeoxIt Pro Gold is designed for cleaning gold connectors. I stopped using Caig products years ago because it leaves a film that can cause an ugly build up.
Radio Shack has electrical contact cleaners that do not leave a film, they just leave a clean surface. I have also tried some of the commercial cleaners with very good results.