Based on my experience, the main concern I would have is that the contacts within 3.5mm jacks often (but not always) do not have enough tension to provide a connection that is reliable over time. Especially if there are frequent removals and insertions of the mating plug, or if the weight of the cable and/or adapter is such that significant pressure is exerted on the contacts, or if the cable happens to be yanked on occasion. In some cases I’ve found that simply jiggling a 3.5mm plug with light to moderate pressure can put a huge transient through whatever electronics is downstream, due to intermittency of the ground connection between the plug and jack.
That tends to more true on consumer equipment than on professional audio and video equipment, IME. (And, yes, 3.5 mm jacks are sometimes used on professional equipment, to allow connection of headphones for monitoring, or to provide outputs to recorders, or for other purposes).
The suggestion of using a quality adapter in conjunction with RCA cables strikes me as likely to be a good approach. The concern I would have with respect to the RCA-to-3.5mm cables you mentioned is that depending on their design they MIGHT incorporate an outer shield that is grounded at only one end, which if the cable is truly intended for connection of a 3.5mm output to an RCA input would be the wrong end to be optimal in terms of noise rejection. Whether or not that would make a perceptible difference in your application, though, is anyone’s guess. But you might want to ask some of the manufacturers you mentioned if their adapter cables are designed in that manner.
So to answer your question I don’t think going RCA-to-3.5mm will NECESSARILY compromise either sonics or reliability, but as is usually the case in audio it depends, and is hard to predict.