Reviewers: Let's End the AI-Generated Content


I recently came across the review of a tube amp that I was excited to learn more about, a review on a well-known site by someone whose reviews I've enjoyed in the past. But as I began reading, something was not quite right. Phrases like "the dulcet tones of the amplifying device revolved around the listening environment" and "the KT170 amplifier tubes were more than worthy in generating timbres of truest fidelity" were peppered from beginning to end. WHAT? I am seeing these AI-generated sentences more and more in reviews, and my message to these reviewers is threefold: 1] You only make yourself look bad when you choose to incorporate AI into your writing 2] Your readers are not stupid 3] Your readers deserve better. And to the editors who have a responsibility as content gatekeepers, step up and hold reviewers accountable for original content...your readers will thank you.

bojack

@richardbrand 

I had the time and energy to find the article I’m referencing.  As mentioned, I can’t prove the use of AI but as the original article was written, it was poor if not unethical journalism:

https://forum.audiogon.com/discussions/low-frequency-directionality

I did write to the publication and pointed out the issues I had with it, so it’s possibly been updated/improved with citations and caveats.

Does this make me suspect the author was lazy and used AI for the entire thing? Absolutely I suspect it, but it’s not like authors will deliberately let you know they are doing it.  The articles easy to spot are the one’s with completely fabricated data points, links, citations, non-existent case law.  So, yeah, I’d put this up there in the category of "what is going wrong with audio writing today?" 

Also, FTR, I do not have a problem with the uncited original article which I think I found.  AFAIK the researcher did everything correctly and the publication is worth reading.  It’s that I expect a journalist to take that into context.  1 article, no matter how well written and how well the tests were designed, without peer review and context of other articles is rarely proof of anything.  It is that conversion, which borders on conspiracy generation, that I take an issue with, besides ethics of failing to give proper attribution. 

This last part is honestly a problem with technical journalism far outside of audio.  Consumers of publications should be more suspect of the data, the sources and conclusions, or we risk some weird article about rabbit poop proving cancer is caused by the letter P. 

 

@ghdprentice,

You bring back old memories. My K&E and Pickett are long gone, however.

@erik_squires  "...The issue is bad journalism becomes socially accepted fact quickly. "

Now you have hit on the issue. Lack of critical thinking among the public. It is critical thinking that filters the truth from all the junk. This is what has been lost allowing mass misinformation / illusion to be taken as truth / reality.  

@erik_squires 

I think your answer may have been directed to @faustuss .  I did not ask you for references, etc.

However, what you have dug up is illuminating.  The so-called scientific paper is a thesis for a Masters degree at a Finnish university.  It is heavy reading - you have to get to page 24 before the TLA MAA is explained.   I have no idea what MAILD and MAITD mean but they refer to ITD and ILD - no bloody help at all.  So I hope the thesis was rejected - I personally loathe unexplained Three Letter Acronyms (TLA) which should be spelt out in full at the first usage.  I deliberately did not do this here to see if it annoys anyone else!.

The thesis was written in 2022, before Large Language Models became widely available.  Also the experimental design measured 15 peoples' perception in an anechoic chamber - about as far from an ordinary listening room as you can get.

I am not anti-Finn - Gradient produced dipole subwoofers specifically for the Quad ESL-63 after all!

Now for The Absolute Sound (TAS) article.  There are TAS authors today that I don't bother reading except to reinforce my prejudice about the symbiosis between magazines and advertisers.  Funny how manufacturers who don't change their products often quickly drop from the radar,  For example, the Quad ESL-63!

How many cables can one author gush over?

@gdhprentice is spot on about critical thinking.  This is where AI can be a real timesaver

The thesis was written in 2022, before Large Language Models became widely available.  Also the experimental design measured 15 peoples’ perception in an anechoic chamber - about as far from an ordinary listening room as you can get.

@richardbrand  - Yes, as I mentioned, I have nothing against the original 2022 paper and have NO reason to suspect the author of using AI for his research nor do I believe he plagiarized or failed to cite his sources.  As far as I can tell, this is legitimate thesis grade work, and he is careful to explain his methodology. 

All my issues is how that was appropriated for TAS, including my suspicion that it was written in great part with the use of AI.  The TAS article, as I read it, gave ZERO credit to anyone, and, if this is the source paper failed to discuss exactly how the testing was done.  It just cherry pics very specific numbers (that's how I found the 2022 thesis) as normalized facts.