There are 2 explanations I can think of:
1) The higher gain setting will likely result in some sort of rolloff, probably at both ends of the frequency band. This could provide a filter effect, i.e. a synergistic effect that might be pleasant in some systems, through the mechanism of reduced bandwidth. Often that points to ultrasonic noise or other out-of band information (like actual audio information) that the power amplifier lacks the speed or power to reproduce without distortion.
2) TVCs can ring and are more easily loaded if used in the gain setting. It might be to your advantage to find out where critical damping occurs with the device, although that is probably tricky as it changes with each setting of the control. Hopefully the designer recognized this problem and built the loading into the unit, if so then #1 is your answer.