Best way to handle this ground loop

This might have been questioned a lot but I can’t find a proper answer for this: 
I have two components that form a ground loop together. Both my integrated amp and my cd player are grounded and they’re connected through RCA cables.
I did all sorts of things to try and prevent this ground loop (all power cables in one strip, tried different wall sockets, even different power circuits), but nothing helped except removing the ground plug from the cd player (or the amp of course). As I saw it, it now has one route to ground, right?
So my question is, did I do the right thing here? Can’t it damage the equipment like this? Or should I have taken an other route?
Are there for example RCA cables that break this ground loop also? Please advice..

As I said, the splitsecond there’s current leaking to ground, a fuse will be blown. That means no more current running through anything. 
A current of 50mA (barely enough to make a low wattage lamp even glow) is sufficient to send your heart into a state called ’ventricular fibrillation’, where the heart muscles are all working out of synchronisation with each other. Little or no blood is pumped, and you will die within about 3 minutes unless help is immediately at hand.

If ground pin is disconnected, for safety reason please at least use a GFCI adapter.
The GFCI will “sense” the difference in the amount of electricity flowing into the circuit to that flowing out, even in amounts of current as small as 4 or 5 milliamps. The GFCI reacts quickly (less than one-tenth of a second) to trip or shut off the circuit.
... the splitsecond there’s current leaking to ground, a fuse will be blown.
Perhaps. But you do not want your body to be part of that electrical path.

FYI: It's called a "safety ground" for a reason.
Although hififan's link describes the different ways a ground fault can occur, he never states that there is a difference in ground potential; a different impedance between the earth grounds.