It may pay you to start all over with the installation and double-check the arm mounting and settings. Also gently move the arm horizontally to and fro over the normal playing arc and "feel" for any binding, particularly around the point you first notice the distortion/change. In addition to swapping the L/R interconnects also try swapping the L/R connections at the cartridge itself to see if that "hum/buzz" changes channel. Also try connecting up before you mount the cart in the headshell (being very careful of the cart) and move around as much as you can to see if the "hum/buzz" is not being introduced externally. When you have it all set-up again, have a good look at the cantilever as it plays and watch for any obvious mechanical deviations if you can.
I hope this helps and best of luck, regards, Richard.