A general view says higher impedance will yield more sparkle or sizzle. This can compensate for a dead room or exacerbate a live one. Unless you've got a moving magnet, start at 100 Ohms and go up in 500 Ohm increments, then come back down in smaller increments. Go with what sounds best to your ears, stick with that for a month or so, then go up or down to see if you hear a positive difference.
The trick with the XONO is to only use as much gain as you need to drive the cartridge and hold down preamp tube rush (if you have a tubed pre), but no more . The unit will sound edgy when gain is higher than needed. The 61dB setting will strongly drive a .5mv moving coil. Experiment - lower gain may yield less distortion.
Tim