Study about readers' reactions to AI

[onehitwanda]

My rule of thumb: if you didn't spend the effort to write it, I'm not going to spend the effort to read it. And if I discover that you tricked me, I will shun you.

In my day job I've been forced to adopt LLMs. I am treating them like idiot savant interns, and so far my caution and skepticism are entirely justified. And then I look at the "engineers" in my industry who are deploying AI-generated code directly to production with no testing bar what the LLM promises it's done, and I look at my stores of alcohol and realise that it will never be enough.

 

[EmilyMiller]

And then I look at the "engineers" in my industry who are deploying AI-generated code directly to production with no testing bar what the LLM promises it's done, and I look at my stores of alcohol and realise that it will never be enough.

That’s prohibited in my origanization.

 

[onehitwanda]

That’s prohibited in my origanization.

Your org is saner than many out there, then.

My profession has been revealed as a large-scale cargo cult overnight. I should have been a paramedic instead.

 

[EmilyMiller]

Your org is saner than many out there, then.

My profession has been revealed as a large-scale cargo cult overnight. I should have been a paramedic instead.

The IT people I talk to have been whipped into using AI by managers who were consultants in previous lives, not coders.

 

[TheLobster]

My profession has been revealed as a large-scale cargo cult overnight. I should have been a paramedic instead.

Always has been.

The software industry is at its core a pyramid scheme, where actors are ceaselessly working around the artifacts produced by other actors. LLMs have only made it more evident, as every project is now subject to a DDoS attack from the changes around it (changing libraries, APIs, etc.) that it has to adapt to. And because those changes can now be churned at an astonishing rate by AI "agents", you need a flock of agents of your own to 'defend' against them, just to maintain a baseline level of productivity.

But hey, at least we aren't wasting all this computing power on Blockchain anymore.

 

[Callum_M]

Your org is saner than many out there, then.

My profession has been revealed as a large-scale cargo cult overnight. I should have been a paramedic instead.

The IT people I talk to have been whipped into using AI by managers who were consultants in previous lives, not coders.

Taking these two points together, AI is seeping into healthcare as well and it’s being pushed by managers who do not understand it. To the point that organizations are allowing staff to have AI summarize patient visits and write in the medical chart and it terrifies me.

What if the AI hallucinates something that didn’t take place and now that’s just part of someone’s medical history?

“Well it says right here you deal with this and that?”

“But I don’t deal with this and that.”

“It says it here though.”

Ahhhhh!!!!

I write in medical charts and I refuse to use any AI for it. Though I worry that may get taken out of my hands one day when someone decides we can just click a bunch boxes to summarize an encounter to prompt AI to write the note, leaving no option for me to write it myself.

 

[EmilyMiller]

Taking these two points together, AI is seeping into healthcare as well and it’s being pushed by managers who do not understand it. To the point that organizations are allowing staff to have AI summarize patient visits and write in the medical chart and it terrifies me.

What if the AI hallucinates something that didn’t take place and now that’s just part of someone’s medical history?

“Well it says right here you deal with this and that?”

“But I don’t deal with this and that.”

“It says it here though.”

Ahhhhh!!!!

I write in medical charts and I refuse to use any AI for it. Though I worry that may get taken out of my hands one day when someone decides we can just click a bunch boxes to summarize an encounter to prompt AI to write the note, leaving no option for me to write it myself.

The Pitt covered this, with AI summaries causing a surgery to be delayed.

 

[Callum_M]

The Pitt covered this, with AI summaries causing a surgery to be delayed.

Oh did it? Perfect example then!

I’ve yet to watch that show. I’m no good with medical dramas after working in healthcare for nearly 20 years now. Anytime I see a show or a movie where the doctors attempt to revive a patient for all of 20 seconds before calling it, I’m pacing around, grumbling “this wouldn’t happen!!!”

Someone gave me their summary of The Pitt and while it sounds medically accurate it also sounded like it’s years worth of disasters crammed into a single day?

 

[EmilyMiller]

Oh did it? Perfect example then!

I’ve yet to watch that show. I’m no good with medical dramas after working in healthcare for nearly 20 years now. Anytime I see a show or a movie where the doctors attempt to revive a patient for all of 20 seconds before calling it, I’m pacing around, grumbling “this wouldn’t happen!!!”

Someone gave me their summary of The Pitt and while it sounds medically accurate it also sounded like it’s years worth of disasters crammed into a single day?

I’m not a medical professional, but it’s verisimilitude is high. They generally have a justification for what happens. E.g. there was a mass shooting in season one - all too possible .

I read that every drawer on the set has actual medical supplies in it. They don’t do magic medicine, as far as I can tell. It’s more competency porn.

 

[AG31]

Well, since this thread has happily sprouted some very interesting side paths, here are my 2 cents.

Oh, this is a problem overall with technologies - engineers/designers who don't use their product.

How true! I often wish the designers had consulted me before they put the damn thing out to the public. From gadgets to plain old road signs.

Back in the medieval era, when I managed a team of programmers, I used to tell them to treat every bit of confusion they encountered in our new software like a nugget of gold. There's a very small window where they'll experience what the average end user experiences and can take the trouble to fix it.

To the point that organizations are allowing staff to have AI summarize patient visits and write in the medical chart and it terrifies me.

My PCP told me that they'd begun having AI write up summaries of the visits, but that they carefully reviewed them before signing off. I think that's better than watching her type frantically as we're supposed to be communicating.

 

[OverconfidentSarcasm]

Personal opinions of AI aside, there was a study with three parts meant to tease out readers' reactions to AI writing ( link to study ) ( link to data ). My goal here is not to provoke yet another discussion about the merit or lack thereof to using AI but just to share the results of randomized control trials.

The toplines that I drew from this:

* Readers preferred AI writing to human writing
* Readers preferred being told they were reading human writing
* Readers without "significant AI expertise" were less likely to be able to tell AI from human writing than if they just flipped a coin (this is the most dooming result for us as writers, IMO)
* Readers with "significant AI expertise" were more likely to be able to tell Ai from human writing than if they just flipped a coin (this makes sense, since most of us who have had to deal with AI writing can spot it)

Two GIANT caveats ... the stories were in the 1000 word range (so AI's inability to hold a thread for more than about 2-3K words didn't kill it) and it was literary fiction vs AI, not any kind of genre fiction where clarity is usually rewarded.

My personal reaction to this is "interesting, but not yet relevant." The most interesting (in a bad way) result to me was that the average reader (even ones experienced with literary fiction) couldn't tell the difference. My suspicion is that if you went into a towering rage by the end of the title of this thread, you've got "significant AI experience".

To me, the real test happens when AI is no longer funded by investors and the end-users have to pay the actual cost of using it.

Yeah, I discussed that study on another site before. And it kinda annoys me.

In short: They took ~1.700 people and gave them ONE short-story to read, creating four groups:

1. Read a human-written story they were told is human-written
2. Read a human-written story they were told is AI-Generated
3. Read an AI-generated story they were told is human-written
4. Read an AI-generated story they were told is AI-Generated

Then they were all given a questionnaire to determine whether they paid attention while reading, and a second questionnaire to determine whether they can remember core parts of the story, before they were asked whether they think it was AI-written and how much they liked it.
Now, the guy conducting the study saw people claim the AI-Written story is better, unless they were told it's AI-written, so he suggests AI isn't actually bad at writing, but humans have a bias.

But here's my problem with the whole thing:

First, the "story" they were reading was ~1,000 words long, as OP pointed out already. There already are studies that show AI gets harder to detect the shorter a text is. And I find 1,000-word stories to not be... well... stories.

Second, about 15% of the participants failed the attention and memory tests after reading the story. FIFTEEN percent couldn't remember core events in stories merely 1,000 words long. But there's a real chance the true number of people not having properly read the stories is WAY higher, because the questionnaires were multiple-choice, the false plot-points were VERY obvious (like "She yells obscenities and punches a nurse", which is something you should remember if it happened), and you had to claim to have seen at least two false plot-points to fail the test. That should tell you about how "engaging" and "understandable" the stories truly were.

Third, throughout the study, I couldn't find any metric for what constitutes "expertise with AI". Am I an AI-expert because I had a ChatGPT subscription two years ago, or do I need to be able to build my own transformer?
I think they literally just had the participants check a box saying, "Would you consider yourself experienced in AI/fictional literature?", going by their overall procedures... though the whole "study" was conducted for a psychology master's thesis, so I guess I shouldn't expect any "expertise" anyway.

And lastly... the worst thing the University essentially put the debunking of their study's validity into the Summary themselves, but didn't seem to realize that. "Self-reported expertise with AI, but not with fictional literature, was positively correlated with correct story identification."

What they're saying, in essence, is: The less you read, the more likely you are to think AI-generated texts are human-written.

 

[onehitwanda]

What they're saying, in essence, is: The less you read, the more likely you are to think AI-generated texts are human-written.

Which is itself a fun(ny) little corollary to the Dunning-Kruger psychosis we're seeing everywhere.

 

[OverconfidentSarcasm]

Which is itself a fun(ny) little corollary to the Dunning-Kruger psychosis we're seeing everywhere.

I'd agree... after I had to google what "corollary" means...

 

[Callum_M]

My PCP told me that they'd begun having AI write up summaries of the visits, but that they carefully reviewed them before signing off. I think that's better than watching her type frantically as we're supposed to be communicating.

That’s good. Genuinely! However, I can’t help but think someone, eventually, will not give a chart note the careful review it deserves or a review at all. People get complacent, they think “hey this is always accurate, and I need to get out of here on time.” And that’s all it takes.

I have seen so many forms of laziness and easily avoidable problems in multiple healthcare settings. It worries me how this could be abused.

 

[SkyBubble]

Oh, this is a problem overall with technologies - engineers/designers who don't use their product.

Marketing people who don't understand the product they're marketing.

 

[Bramblethorn]

That’s good. Genuinely! However, I can’t help but think someone, eventually, will not give a chart note the careful review it deserves or a review at all. People get complacent, they think “hey this is always accurate, and I need to get out of here on time.” And that’s all it takes.

Last year I had a trip to the ER. Long story short, doctor missed some stuff that could have been important and gave incorrect follow-up advice which could have had bad consequences, it sucked for a bit, I'm good now.

Some time later I visited my general practitioner, because things weren't feeling quite right, which turned out to be related to the thing that sent me to the ER. He looked through the hospital notes, and the follow-up advice recorded there was different to the advice the ER doctor actually gave me. What they'd recorded was the advice the ER should have given me, but didn't.

(FWIW, when I got that bad advice, I was very tired and dealing with a pain hangover, but I made a point of repeating the ER doc's advice back to her to make sure I hadn't missed anything. So I'm pretty sure that the advice recorded is not the advice they gave me.)

IDK for sure whether AI scribing was involved, but recording standard advice in place of the non-standard advice that was actually given is exactly the kind of mistake that I'd expect a LLM to make.

 

[Callum_M]

Last year I had a trip to the ER. Long story short, doctor missed some stuff that could have been important and gave incorrect follow-up advice which could have had bad consequences, it sucked for a bit, I'm good now.

Some time later I visited my general practitioner, because things weren't feeling quite right, which turned out to be related to the thing that sent me to the ER. He looked through the hospital notes, and the follow-up advice recorded there was different to the advice the ER doctor actually gave me. What they'd recorded was the advice the ER should have given me, but didn't.

(FWIW, when I got that bad advice, I was very tired and dealing with a pain hangover, but I made a point of repeating the ER doc's advice back to her to make sure I hadn't missed anything. So I'm pretty sure that the advice recorded is not the advice they gave me.)

IDK for sure whether AI scribing was involved, but recording standard advice in place of the non-standard advice that was actually given is exactly the kind of mistake that I'd expect a LLM to make.

Glad you're doing better. Truly. Dealing with ongoing pain is miserable and you don't get a break which can wear a person down mentally. So it's commendable that you had the presence of mind to repeat everything back to the doctor and yet they still gave you the wrong advice.

Whether or not it's AI related, it's a still an example of their mistake having huge consequences for you, and it's an avoidable mistake. That's what always irks me the most about working in this setting.