Best way to handle this ground loop

This might have been questioned a lot but I can’t find a proper answer for this: 
I have two components that form a ground loop together. Both my integrated amp and my cd player are grounded and they’re connected through RCA cables.
I did all sorts of things to try and prevent this ground loop (all power cables in one strip, tried different wall sockets, even different power circuits), but nothing helped except removing the ground plug from the cd player (or the amp of course). As I saw it, it now has one route to ground, right?
So my question is, did I do the right thing here? Can’t it damage the equipment like this? Or should I have taken an other route?
Are there for example RCA cables that break this ground loop also? Please advice..