What's better, one conductor or two conductors for an RCA interconnect?

Perhaps mentioning "KHZ" is pedantic, but it make me suspicious about your experience in electronics.  None of my fellow engineers would write it like that.  As for "deflecting" from technical discussion - it deflects less than bragging about PhD title.
(to be really pedantic:  it is 16k  not 16K)

kHz would be it, but I needed to make the K lower case as the spell check caps it.

That is probably the simple explanation of why it got all uppercased.

If the current goes the other way, one side sees the impedance of the amplifier. The other (ground) sees the the impedance of the amplifier, but also sees a parasitic path through through that ground and out to "somewhere". Now you have a differential voltage caused by the common mode noise.

What do you mean current goes the other way?  It is the same current - not two different cases.  If it cancels at the speaker (no current thru speaker) it means net current in the loop is zero.  It will also be zero on the amplifier side (no current in the wires).   Impedance in the loop is the same for both wires - you cannot separate them - wires are in series.  Presence of the speaker doesn't change anything - we can short it.  It is the same wire where two induced currents flow in the opposite direction.  What might flow to ground is differential current that is zero. 

Draw two exactly same current sources of opposite polarities in series and close the loop with resistor.  Current in the loop, as well as voltage across resistor will be zero.  You can ground it at any point and it won't make a difference - it is floating circuit (no reference to anything).  Only connecting it at two points will unbalance currents (alternative path). 

As for the case of phono cartridge - as long as it is grounded on one side only it is still loop.  When you ground one end of the cartridge it won't make a difference as long as the other end is floating, for instance transformer.  It is the same case as with our headphones example - cartridge is inherently balanced and twisting wires will make sum of induced currents zero.  If it is zero at the cartridge it will be zero at the amp - no matter if and where you ground one of the wires.  Advantage of balanced input will be rejection of common mode noise picked by the cartridge.  Perfectly twisted wires will offer great common mode rejection in either case - balanced or single ended.

There are other reasons for driving headphones with balanced drive.  One is to get double voltage (often needed with 600ohm headphones), another is to avoid crosstalk between channels by using common return (GND).

@jumia   Sorry, for hijacking your thread - I won't write anymore.  To your question I can only say what I think is right.  I would use two wires with shield connected on the source side only.  Many SE cables have an arrow pointing direction from source to receiving end.  You can also unscrew shells to see and mark the end with shield soldered to case (GND).  

Your thinking is flawed @kijanki. You have an erroneous understanding of how noise induces (or does not) induce current into these circuits and hence have drawn incorrect conclusions about what the result will be. If the noise is flowing in a loop, through the speaker (or cartridge), it is not common mode noise, it is differential.

Let us change the problem to a phono cartridge and a pre-amp. By your understanding, even if one side of the phono amp input is grounded, the system is "differential", and hence will receive the full benefits of twisted pair wiring. That is wrong. I noted that @atmasphere who supported you above uses a true differential input on the phono inputs to his preamplifier (for improved noise rejection).

The flaw in your logic is that with common mode noise, the current does not (have to) flow through the turntable cartridge (or speaker as the case may be). A simple case is electrostatic coupling which capacitively couples a current onto both wires (common mode). No wire "loop" is required to induce current as the parasitic capacitors forms the circuit. However, since you have one side of the phono pre-amp "grounded" the current in that wire flows into the ground (or some portion of it depending on whether hard ground, resistor, or capacitor. The current in the other wire flows into the loading network, offset by whatever current did not flow to ground (from the other side) but also flowed into the loading network. The result is of course a noise voltage. Coax has superior electrostatic noise rejection compared to even shielded twisted pair. Without a proper fully differential system, there is no benefit of twisted pair.

For audio I would assume we can ignore RF antenna coupling onto both wires (common mode), or at least we can ignore the effects which could be similar to electrostatic coupling. If we include them, again, the single ended connection defeats any benefit of twisted pair wiring as above, while the twisted pair has less rejection than coaxial, even shielded.

For magnetic EMI coupling, there would need to be a differential current generated at the point of interference as a complete electrical loop, for practical purposes, would need to exist to have a magnetically induced current. A twisted pair does, obviously, reject magnetic coupling of EMI, but so does a coax (through arguably a different mechanism) and it does it rather well. In most practical cases, because the differential current from magnetic interference happens at the point of interference and not at the load/receiver, the balanced connection does not matter for magnetic interference. (note the experiment link I provided that showed in that case superior rejection of magnetic interference with coax).

So, @jumia, I go back to my original statement, still true, explained in great detail, apparently in agreement with others knowledgeable about the subject, that typically coax will be the superior cable for single ended connections.

@jumia , this is not touting my horn, but questioning the competence of many cable companies, but I expect few of them, based on the technical quality of their communication, have the understanding to either refute or agree with what I write. Some of them I am sure do have that understanding.

@jumia 

Is it better to connect the ground to each side of the RCA cable?

I think you mean the SHIELD, i.e. if there is one, is it better to connect it on both sides. No, it's best connected (to ground) on one side only.

It is often best to connect the shield on only one end. Is it always best?