1. Do you find this deficiency on both channels when you play the second side? The outer tracks, which is there the left channel lives, are more likely to show a loss of fidelity since they are at the edge of the tape and will be most prone to less than perfect alignment.
2. Good tape to head contact is essential. Even if azimuth and zenith alignments are correct, there may be a slight deviation from a perfectly vertical front-to back head angle. You might try this experiment: While the tape is playing, using a Q-tip, press the tape gently to the play head and see if the sound improves.
What is puzzling me is why you find that this loss of quality only appears on the reverse tracks.
What the head sees on your tape should be exactly the same from the forward and revers tracks, any alignment would be mirror-image, either both bad or both good. Same with height.
See if that Q-tip diagnostic tell you anything.

