In my mind there are 3 possible causes -
1. phono is faulty
2. phono is overloading
3. there are resonant peaks from the cartridge/arm/cable upsetting the phono
phono overload
most MM phonos run 40db - 50/56 is very high, particularly with your cartridges at 4-5mV.
I would suggest using the lowest setting 50db, which means you have to run single ended RCA's. On the INT60 for unbalanced you need to use inputs 3 or 4. If you use input 1 or 2 for a single ended input you have to use a shorting plug on the INT60. When you tried RCA's to get 50db did you set up the INT60 correctly ?
Resonant peaks generated by tonearm cable and cartridge
After setting the input to 50db, using unbalanced from phono to INT60, if you are still getting the problem - try increasing the capacitance loading, this will roll off the high frequency from MM's and will dampen ultra high frequency resonances that may be upsetting the phono.
Trying a different tonearm cable may work - bit hit or miss on this - testing changes to the capacitance load will tell you more.
If these steps do not work, your phono is either faulty, or unsuitable for MM cartridges with 4-5mV output.

