RCA or XLR?

Balanced differential, XLR, cables don’t necessarily sound better than single ended cables but they do remove any noise, electromagnectic, picked up along the signal path in transmission.  That is their main advantage.  Metallurgy, impedance, capacitance impacts from the insulation, can affect both XLR and single ended cables.

I use XLR - smart people at Boulder do not include RCA inputs on their equipment. So even with my not fully balanced phono stage I use XLRs. No complaints. 
Oh and to add - XLR and RCA ins/outs have different gain and impedance when it comes to fully balanced gear. So when you’re comparing XLR to RCA even using same cables, take this into account. It’s all about matching. 
So use whatever you feel sounds best. 

@052rc  "Balanced and single ended equipment is not compatible."

That is simply not true. It is true internal topologies can be unbalanced, balanced or a hybrid. The outputs require conversion to allow both balanced and unbalanced inputs and outputs. Also, that most audio components supply both. They can easily have the same sound quality or the one that reflects the internal topology may sound better. It depends a lot on how well the manufacturer does the conversion. As I mentioned when interconnecting my ARC gear I could not hear a difference. 

Interconnecting different gear is the same thing. You really just have t do try it. While theoretically it is better both units have the same topology and are connected with interconnects supporting that topology, it may not be so in real life and there is absolutely nothing wrong with doing it and getting excellent results. 

"RCA or XLR?"

in ideal environment, where there is no ground noise, RFI/EMI, RCA is better (w/ good cables). Balanced interconnects address noisy, less ideal environment, by rejecting noise, plus better impedance match between driver-cable-receiver, leading to less ringing/reflections. 

OP, your phono stage—the Pass XP-15—was released in 2009. For earlier models like this, the internal circuitry may not be fully balanced or symmetrical, unlike more recent designs such as the XP-17, which was released in 2020. You may want to confirm this with Pass Labs. If that is the case, the XLR connectors were likely provided for convenience, while the actual circuitry was optimized for RCA. Therefore, the RCA connection may sound better.