See what you think with no loading at all (which should be 47K).
My understanding is that the Pass Labs phono sections are pretty stable. It gets a bit technical, but the loading is often required for the benefit of the preamp, not the cartridge. This is because the inductance of the cartridge and the capacitance of the tone arm interconnect cable form a resonant peak, often at several MHz. If the preamp isn't RFI immune and also quite stable, it will react poorly to the Radio Frequency (RF) bursts that come in when the cartridge energizes this resonance.
Loading the cartridge causes the cantilever to become stiffer and therefore less able to trace high frequencies (and more likely to wear out the vinyl).
So IIRC, the Pass Labs phono sections are pretty good in this regard so you may find it sounds best with the stock 47K loading. A side benefit: you may also notice you're getting less ticks and pops, as many of those are caused by errant phono sections and not actually the LP surface! Have fun!