"Many times I’ve asked AI to compare two models of speaker that I own/owned. In approximately 2/3rds of these queries, it suggested a preference for the speaker that I determined was inferior in real-world experience."
That doesn’t surprise me. It may be a classic case of ’garbage in, garbage out.’ If you haven’t given the AI enough information about the characteristics you personally value, it’s going to produce recommendations that may not align with your priorities. I might have chosen the same speaker AI did. Audio isn’t a right-or-wrong, binary pursuit—different listeners value different things, and it seems you may still be refining exactly what your own priorities are.
Steve Huff frequently drives me crazy in this regard. He understands that ’good’ is subjective, but he doesn’t always describe why something is good or for whom it is good. I don’t hang on every reviewer’s every word, but I still enjoy checking in from time to time. The Audiophiliac, in my opinion, often provides even less useful detail but I enjoy his reviews for entertainment purposes.
Because of that, I consider every source of audio information to be imperfect. What has worked surprisingly well for me is spending the time to teach ChatGPT exactly what I value in audio reproduction—not just what I like, but what I dislike and, more importantly, why I like or dislike something. Once it understood my priorities, it became much better at filtering and ranking equipment for me. It still isn’t perfect—nothing is—but these days it’s usually very close.
When I first started using it, it regularly recommended gear that held little interest for me. After enough feedback, however, it became well calibrated to my preferences. It can still be wrong, but not by much.
Of course, I’ve spent decades working with computers, so I have a good sense of how to communicate with them and how to get the best results. Changing employment this year, my exposure has dropped considerably, but that experience still helps.
Your mileage may vary.