Using both the Balanced and Unblanced inputs

Would there be any issue if I used both the Balanced and Unbalanced inputs on a MC275V amp to run EITHER video (RCA) and Audio (Balanced) and use the switch to select which format I wanted to use at the time, to avoid the need to plug in and out every time I wanted to change between CD and TV?