Sound when volume off

I have an older preamp but one that I have been happy with for a while. Recently I noticed when I turn the volume all the way off there is still some sound that comes through the speakers. On mute there is silence. Is this a problem? speecifically I thought I once heard that caps might need to be replaced in this situation. All replies are appreciated
It sounds like a problem with the volume pot -- not the caps, surely. The mute switch stops (floats) the output connection, so you wouldn't have any sound at all, that's normal.

I would suggest you change the volume pot -- it's easy & not very costly. However, you need to know its resistive rating (how many ohms does it specify?). Or, you could do nothing, where the only danger I can think of is that the pot deteriorates completely and plays full blast all the time -- that's rare but dangerous for your speakers, obviously.
Another possibility is dirt/grime in the pot preventing a full off. You might try one of the Caig cleaners.
Or a noisy output stage, which comes after the pot but before the mute (wich is the normal arrangement).