DAC Comparisons using AI

OP, in my opinion, the key issue isn't AI involvement but the questions themselves. Sonic comparisons are system-dependent variables: amplification, speakers, room acoustics, cables, and listener perception and taste all affect the outcome. If you don't specify those parameters, the AI can only synthesize reviewers' and posters' generalizations. And it will do it confidently with attractive descriptions.

But in all of that word salad, it will be unlikely to flag whether your questions are missing critical information. That's a characteristic AI limitation.

So...why not try asking AI an additional question to the one you're posing – 

"Gemini, before answering, identify what information about my system and listening context is missing from this question that would be necessary to give me a more accurate and personalized response."

That kind of metacognitive prompt turns AI into an interlocutor rather than an oracle. It forces it to reveal its own assumptions before generating what might otherwise be a fluent but under-specific answer.

Garbage in garbage out.

AI accumulates data from unvetted sources and consolidates it into a cogent format- this is the problem. It looks rational but the primary source data is not vetted. 

Just for fun imagine you wanted to gather comparative data on a relatively unknown product and one that is that is heavily sponsored in mainline audio magazines, you tube videos, social media, etc. via a big budget.  Paid reviews. Guess which one AI is going to highlight? 

I wish all you guys would stop using AI, so I can get a few more sata ssd’s for my server at a reasonable price!

The problem of asking a question such as: 

What sonic difference would be achieved by upgrading the Aries Cerat Helene to the Kassandra Reference II?

AI doesn't have ears, it doesn't know, all it can do is summarize what others might have said about this upgrade. And while there are a million testimonials comparing <$1000 DAC's, the number of people who have done this upgrade, or any upgrade at this level, is probably less than 50 on the planet, and I'll bet no more than 2 of them have posted these comparisons on the internet. If those 2 people cannot put what they hear into words effectively, which is quite understandable, how can AI translate and recite that back in any form for someone else to understand it? 

Other than simply "the extra $10K is worth it". Well, I guarantee if someone has the spare $10k and the willingness to spend it on this upgrade from Helene to Kassandra, they will say this without even listening to it. 

I too have productively used AI - ChatGPT specifically - to assist me in thinking through upgrades and moves between systems. And, yes, of course, it’s important to check sources and ultimately use your own ears.  It’s also very helpful to allow your AI “agent” to know you very well. Mine knows every component and interconnect in my main system in two houses as well as two secondary systems. It also knows my room dimensions and features. Over time, my commentary on what I like or don’t like is further incorporated and used as reference to new queries. 
 

Without getting into the details, recently AI kept me from spending a lot of money on an upgrade that’s been tickling my brain. It really just reminded me of what I have in front of me and told me to sit back and enjoy the music for a while. Pretty startling turn to the conversation. I bought it a beer. 

I was researching the best input level for my new M3 DAC to my BHK pre. I thought it was just a sensitivity / compatibility question, but for my pre (and perhaps others), it uses both a potentiometer & resistor ladder. Both AI’s recommended reducing the DAC output to achieve a higher resistor ladder.
PS Audio confirmed the ladders, said I would probably see better performance from the higher ladders, but said their ‘official’ position was DAC at standard output, ignoring the ladders. 
The higher ladders sound fuller & more dynamic at the same volume. This is not documented anywhere I can find, & only through AI did I discover it. Good stuff!