Mr. Modjeski seems to endorse passivesYes his amps are loved by passives and with my customers that use them.
My power amp is a CJ Evolution 2000 its input impedance is 100k ohms, and input sensitivity is 900 mV to full output.
It should also love passives, with that 0.9v input sensitivity. And that 100k input impedance you could of had a 20k Alps passive, not just 5k,
A 20k Alps pot would have been fine for just about any source then, which by the way you have not told what you have as the source.?? because if it has high’ish output impedance it won’t be comfortable with that 5kohm Alps pot as it will load down it's output and also create a voltage divider.