Inconsistencies with AI Detection - Yet another gripe

[nvsibl3]

I hate to be the guy that starts YAAIT (Yet Another AI Thread) but I'm boiling right now. I submitted 4 stories in the span of 48 hours - a fit of ADHD focus let me finish two that I had been dragging around for a while and I had one just spring nearly fully-formed from my brain.

2 stories publish and do really well.
2 stories kicked for AI.

All 4 stories written with the same process as everything else in my catalog (including a dozen started but not finished stories).

All 4 stories reviewed by human editors who have done wonders for catching my terrible punctuation, spelling, and continuity errors.

I really don't know what I'm hoping to get in response here. We know that the AI "detection" process is prone to false positives and negatives. We know that it will remain fairly black-box to keep bad actors from gaming the system.

I'm not famous, or even necessarily an exceptionally great writer. But my stories rate okay, get engagement, and I enjoy the feedback and comments I get from readers. I've definitely read things that have been published and have a pretty solid suspicion that they are AI "enhanced" at least.

The inconsistency is maddening.

 

[EmilyMiller]

I hate to be the guy that starts YAAIT (Yet Another AI Thread) but I'm boiling right now. I submitted 4 stories in the span of 48 hours - a fit of ADHD focus let me finish two that I had been dragging around for a while and I had one just spring nearly fully-formed from my brain.

2 stories publish and do really well.
2 stories kicked for AI.

All 4 stories written with the same process as everything else in my catalog (including a dozen started but not finished stories).

All 4 stories reviewed by human editors who have done wonders for catching my terrible punctuation, spelling, and continuity errors.

I really don't know what I'm hoping to get in response here. We know that the AI "detection" process is prone to false positives and negatives. We know that it will remain fairly black-box to keep bad actors from gaming the system.

I'm not famous, or even necessarily an exceptionally great writer. But my stories rate okay, get engagement, and I enjoy the feedback and comments I get from readers. I've definitely read things that have been published and have a pretty solid suspicion that they are AI "enhanced" at least.

The inconsistency is maddening.

Sorry to hear

Were there any differences between the stories that got published and those that didn’t? Tone? Narrative view point? Tense? Vocabulary? Category? Length? Anything really?

 

[MetaBob]

A suggestion:
https://forum.literotica.com/thread...-story-anymore.1648707/page-12#post-102275280

One result:
https://forum.literotica.com/threads/thoughts-on-ai-checkers.1651405/page-2#post-102520014

 

[SmilingLez]

This is how flawed AI tools are. I wrote three paragraphs each with three simple sentences. An AI detection tool determined everything was AI generated except for the last line which had to do with emotion - something AI is incapable of writing about

 

[txblush]

Here's something I wrote above my recent work in progress: Any detection of AI in this document is a false positive. All structural grammar mistakes are my own. I understand the rules but have my own writing style. I used ProWritingAid for its critiques at the beginning and took a few of its suggestions on how to improve the story. I went back to rewrite those scenes and edited the story along the way as I created it without an outline and chose to compose portions of the chapters as I practiced the craft of writing short paragraphs one sentence at a time.

 

[MrPixel]

This is how flawed AI tools are. I wrote three paragraphs each with three simple sentences. An AI detection tool determined everything was AI generated except for the last line which had to do with emotion - something AI is incapable of writing about

...

Samples are entirely too small to be a valid analysis. It's a failing of this tool to not return, "Insufficient data to make a determination."

 

[txblush]

Samples are entirely too small to be a valid analysis. It's a failing of this tool to not return, "Insufficient data to make a determination."

Hi, I'm going to step out on a limb here and propose the following as an example of what I wrote as a header about a short story: I hope this helps others who are struggling to understand how to address the policy of this site's policy on No AI written material.

Dear Lit Admin:

All structural grammar mistakes are my own. I understand the rules but have my own writing style. I used ProWritingAid for its critiques at the beginning and took a few of its suggestions on how to improve the story. I went back to rewrite those scenes and edited the story along the way as I created it without an outline and chose to compose portions of the chapters as I practiced the craft of writing short paragraphs one sentence at a time. This story is loosely based on a main male character I created from memory years before I even knew AI existed. But I could never finish it due to writers block.

I started this work in progress with just the name of the female character and an idea about mental health issues, cults, mind control, critical thinkers, liberty, PTSD, addictions, the fight for autonomy and justice for all who need a helping hand, symbolism, myths and legends, and vows spoken between an honest woman and an honorable man in old world language. They will protect and serve each other like a King and Queen playing a game of chess.

The chess piece of a King moves one square at a time on the sixty-four squares of a chess board. The Queen can move in any direction within those checkered squares. The Queen's Gambit usually evolves into a strategic game rather than an all-out tactical battle. It's the opening move of the White Queen. Queen's Gambit - Chess Openings - Chess.com

 

[Husky_Embrace]

As a non-native speaker my English will never be on par with AI, so it isn't a problem for me.

That being said, it is hard for me to judge who is using AI and who isn't.

My default state is assuming they used AI. The reasoning is simple:

We all know plenty of people will try to cheat the system. Using AI and then saying they didn't. Complaining might help alleviate the strictness in various ways. It is safe to assume a big chunk of complainers is using AI. AI is now so prevalent and easy to use it is nearly unthinkable not to use for many people.

There's also plenty of people who use it without knowing it. Not everyone is a native speaker, and even native speakers can have troubles with writing. They use tools to aid them, which I do not fault anyone for. If tools can aid your creative vision to be put on paper in a way that's readable to others then have at it! Editors are also part if that toolbox (I don't mean to say they're tools though). The problem is that many now have AI embedded in them. So they're using it without knowing. Some might know, and just hope it won't be a problem.

Then there's the people who legitimately get flagged. There's no data on it I trust, so it's entirely my feelings towards it. My gut feeling is that it will be extremely rare that you're writing uses words, phrases and grammar in line with AI so often that you'll get flagged. It's an aggregate of all writing on the internet with some AI algorithms splashed over it. What are the chances you'll fall into that narrow margin consistently? I suspect people can fall out of that margin if their personality changes, like writing when hungry, tired, or super horny.

Not to mention I expect most writers to add some creativity with the sentences at times to properly convey the message. If you know the rules to such an extent that you can write like AI, you'll likely know when you want to deviate from it.

So my default is that anyone complaining about AI has used AI. It is not fair, as normally people are innocent until proven guilty, but the AI van has probable cause written all over it.

If you are legitimately not using AI and getting flagged I do understand your frustration. Putting so much effort into a story only for it to be rejected must be devastating.

 

[txblush]

I'm going to step out on a limb here and say last July I spent 30 days using Claude.AI as a tool to learn how to outline, and understand grammar structure. On an original story I wrote and posted on Lit.

Prior to that and years ago, I asked Lauren to remove it from the site because I wanted to take the story to the next level by paying for a critique. The editor gave me one compliment on my BDSM world building, but there was a lot of other issues. Her bottom line was I'd need to rewrite the story.

Personal Note - I kept those two constructive criticism remarks in mind for a decade. I worked a demanding job as an account receivable and payable

Then one day.... BAM! inspiration for a rewrite on that original story I posted here started coming together. I completed two different versions of my original novel. The total word count was 300k. But it was the 2nd version that gave me the confidence I needed because I used Grammarly as a learning tool. Then I came here and asked for a volunteer editor and the person wanted me to do an outline. Which was something I did not know how to do at all. So, I looked for ways to learn by AI technology.

I love to learn the craft of writing. I practice writing and read books in between for the genre I enjoy writing. I've been an avid reader since I was a teenager. I'm retired now and the joy I feel when everything clicks into place came to me a few weeks ago when I penned my first short story based on a name and a host of ideas. I used ProWritingAid as a tool for the first part of the story and used some of its suggestions, but the words that I typed were carefully crafted with consideration for this website's policies.

I appreciated the sites policies because it made me a better writer. And that's all I ever wanted! I started learning the craft of writing in my mid-thirties. I'm retired now. It's a form of self-expression that cannot be duplicated by artificial intelligence. But as a learning tool it was helpful and I had a lot of fun.... while it lasted.

Thanks, Lit Admin, and the writers of Author Hangout for your multiple posts on the AI debate that is still ongoing.

 

[RockOn2019]

A lot of it comes down to detectors looking for patterns that overlap with normal writing styles. That’s why even genuine work can get caught. https://cleverhumanizer.ai/paraphrase-tool shows how easy it is to tweak phrasing and avoid those patterns, which kind of proves how flawed detection is. It’s not really measuring authorship, just surface-level structure. That’s why so many people are pushing back on it.

 

[Trionyx]

“All 4 stories reviewed by human editors who have done wonders for catching my terrible punctuation, spelling, and continuity errors.”

One idea crossed my mind… How do you know the editors did not use AI?

 

[AlexBailey]

I often buy lottery tickets from a vending machine.

Sometimes I can put the same dollar bill in over and over, only to have it spit right back out at me. It can even happen with new bills. Other times I can put an old ragged one in and have it work the first time.

 

[Husky_Embrace]

I often buy lottery tickets from a vending machine.

Sometimes I can put the same dollar bill in over and over, only to have it spit right back out at me. It can even happen with new bills. Other times I can put an old ragged one in and have it work the first time.

I understand. You're saying that you're trying a counterfeit bill over and over again, sometimes it looks brand new, and it is rejected. A legitimate bill works without issue, however ragged it looks.

And then you can gamble.

 

[Five_Inch_Heels]

its crazy

Bot edit incoming ...

 

[LegendInMyOwnMind]

I understand. You're saying that you're trying a counterfeit bill over and over again, sometimes it looks brand new, and it is rejected. A legitimate bill works without issue, however ragged it looks.

And then you can gamble.

No, he is not saying that at all, and were I him, I would find what you said offensive. I myself have had that experience with many vending machines with indisputably legitimate currency. Even after all these years, bill acceptor technology is inferior to coin acceptor technology, and the latter is far from perfect.

 

[AlexBailey]

No, he is not saying that at all, and were I him, I would find what you said offensive. I myself have had that experience with many vending machines with indisputably legitimate currency. Even after all these years, bill acceptor technology is inferior to coin acceptor technology, and the latter is far from perfect.

Thanks, but I’m truly not offended. I was being very glib.

My somewhat obscure point was that the vending machine, designed to take in money while also rejecting all possible counterfeits, is in conflict with itself - not quite the same but still similar to how an AI is instructed to produce writing that is unmistakable from human while also being instructed to identify AI writing with 100% certainty.

Both machines seem to have a similar track record and seem to be weighted more toward rejecting counterfeits than for accepting the real thing, and I can’t seem to predict whether a bill will be rejected or not until I actually try it.

Sometimes it rejects the real thing so many times that I give up with that particular dollar bill even though the potential reward could be $millions.

Yeah, it’s a sloppy metaphor, but it still makes sense in my head.

 

[SmilingLez]

For shits and giggles, I chose a random new story (published today) and ran it through an AI detector, and it came out 98% AI generated (first 100 words). So I guess literotica's AI/human automatic detection tool is faulty in both directions

Edit - my two (3/18 and 3/22) flash fiction/hybrid stories remain in pending purgatory. Perhaps I am on the naughty list

 

[IWroteThis]

Edit - my two (3/18 and 3/22) flash fiction/hybrid stories remain in pending purgatory. Perhaps I am on the naughty list

My 2/27 story is still pending, but my 3/20 story was approved on 3/27 and posted on 3/28.

My preferred theory to the naughty list is that Laurel holds back stories she really wants to read, for when she has time to read instead of just skim for approval.

 

[SmilingLez]

My 2/27 story is still pending, but my 3/20 story was approved on 3/27 and posted on 3/28.

My preferred theory to the naughty list is that Laurel holds back stories she really wants to read, for when she has time to read instead of just skim for approval.

I do not believe Laurel actually reads any story (in its entirety)

 

[EmilyMiller]

I do not believe Laurel actually reads any story (in its entirety)

I have had rejections with specific text suggesting the total opposite. These obv weren’t AI rejections.

 

[SmilingLez]

I have had rejections with specific text suggesting the total opposite. These obv weren’t AI rejections.

Curious, how long was that story pending until you got a rejection note?

With all the poorly written and now ai generated stories that get pushed through, it makes me wonder if well written stories are getting the shaft

 

[EmilyMiller]

Curious, how long was that story pending until you got a rejection note?

With all the poorly written and now ai generated stories that get pushed through, it makes me wonder if well written stories are getting the shaft

I’ve not had a rejection recently - then I last published something for Winter Holidays.

The note I was referring to was about the age of a participant. Laurel said something like, “I’m sure if you do the math it’s clear she’s 18, but could you make it more obvious?”

Given I had no intent for my FMC to be any younger than 18, I added a more explicit reference to her last birthday party and it was approved within hours.

 

[SmilingLez]

I’ve not had a rejection recently - then I last published something for Winter Holidays.

The note I was referring to was about the age of a participant. Laurel said something like, “I’m sure if you do the math it’s clear she’s 18, but could you make it more obvious?”

Given I had no intent for my FMC to be any younger than 18, I added a more explicit reference to her last birthday party and it was approved within hours.

So you're saying your story was flagged due to a character's age and someone sent you a message about fixing it. This doesn't mean your story was read, it only means that somebody checked the automated tools log, and realized that it could be an under 18 story and can get the site shut down.

 

[EmilyMiller]

So you're saying your story was flagged due to a character's age and someone sent you a message about fixing it. This doesn't mean your story was read it only means that somebody checked the automated tools log, and realized that an under 18 story can get the site shut down.

I’m kinda skirting around the precise details of the convo, which we are not meant to divulge, but it was apparent from context that she had at least skimmed most of the story (she actively referred to more than one occasion where age came up, and also mentioned narrative arcs in between). You don’t have to believe me. Your prerogative of course. I’m not going to replicate the text verbatim.

 

[SkyBubble]

For shits and giggles, I chose a random new story (published today) and ran it through an AI detector, and it came out 98% AI generated (first 100 words). So I guess literotica's AI/human automatic detection tool is faulty in both directions

I've been writing here since 2006, and for fun, I ran a couple of my stories from 2006-09 through AI detectors. They came back between 30-80% AI, but they were written before AI writing was a "thing."