Summarize your story with AI. How did it do?

[Lifestyle66]

I asked ChatGPT to summarize my latest story, "Fantasy Renewal".

The AI provided what I believe to be a good summary, although the sentences seem to run long. But it probably does a better job of objectively summarizing it than I would have done.

*******************************************************************
The story follows Marcy, a long-married woman who has always embraced her sexual agency, as she and her husband Steve attend Fantasy Fest in Key West and are drawn into a world of consensual exhibitionism and BDSM that challenges their routines and assumptions about desire. What begins as curiosity and voyeurism—watching public demonstrations and playful costuming—gradually becomes personal as Marcy confronts her own fascination with power, vulnerability, and being seen, ultimately stepping into a public scene that awakens intense arousal and self-knowledge while reaffirming her limits. Throughout the week, moments of temptation, jealousy, and flirtation test the couple’s boundaries but ultimately strengthen their bond, boosting Steve’s confidence and pride while reigniting Marcy’s sexual hunger. Back home, the experiences translate into a renewed intimacy marked by power play, teasing, and open communication, and the story closes with Marcy reflecting that Fantasy Fest disrupted their comfortable marital patterns in the best way—replacing habit with curiosity and signaling a shared willingness to continue exploring desire together.

 

[TheLobster]

What do you mean when you say that the current generation of AI has already run its course?

A lot (most?) of the progress and improvement in generative models, since their original inception in late 2010s/early 2020s, has gone into removing the inherent "black-boxness" of the models, making them more specialized and refined into particular applications. Modern "AIs" aren't just one a single predictive model, where you feed it the input and it computes the most appropriate continuation; rather, they are complicated flows of many different models, coupled with traditional algorithms, that analyze and classify prompts, process and rewrite them, route them to specialized submodels, post-process results, iterate all of that multiple times and pick the best outcome, and so on.

The more this specialization continues, the more the so-called AI systems start resembling traditional distributed systems, where generative models are just small subroutines of a much bigger picture. Those systems are better at serving the specific tasks -- of which they may be hundreds -- whose prompts can be recognized and routed, but it also means there is less and less space for, let's call it "creativity" in outputs, that comes simply from throwing prompts at a generally-trained models and seeing what it produces.

I believe this is what @old_prof meant here: that the underlying paradigm isn't evolving all that much anymore, but rather it's getting picked apart and stuffed into neat little boxes. There may still be many boxes left to uncover -- i.e., more applications for specialized models -- but the general "power" of the whole architecture likely doesn't have many more aspects to improve left.

A glorified Google search maybe, but the simplicity of it makes it magical.

This is actually what I think is the most revolutionary thing about current generation of LLMs/AI. They are a great advancement in user interface (or human-computer interaction), since they let you "talk" to your computer and get answer in natural language. Lots and lots and lots of "applications of AI" are really just finding new ways to take advantage of AI-as-an-interface. This is why companies like MS or Google try to stuff prompt textboxes literally everywhere, because they have no way to know which one of those places turn out to be actually useful (and those where it's annoying, i.e., most of them, can simply be phased out in the new version of Windows / Google Search / etc.).

 

[old_prof]

Hi! I'm (genuinely) curious, seeing as you know more than me on the subject. What do you mean when you say that the current generation of AI has already run its course?

I might be misunderstanding you (I’m slow ), so feel free to elaborate, but while the current generation of AI may or may not have run its course (what do I know), it feels like the applications of it are only just getting started.

What I mean is, how we use these tools is evolving rapidly, even if the underlying models aren’t making giant leaps every month.

A small, everyday example: browsers now have built-in AI answer assistance. I know some people hate that, but honestly, I find it useful. It condenses a ridiculous amount of material into something I can grasp quickly. A few AI bots (like Google's) can transcribe and summarize lengthy YouTube videos, which to me, is quite useful.

A sillier example: I’m playing a JRPG with far too many characters, and I reached the point where I couldn’t tell if my lineup made sense anymore. I took a photo of the screen, wrote a short prompt, and the AI analyzed it, browsed the web, and gave me a summary of prevailing strategies. It cited game sites and subreddits I already "trust". And it responded not in a “I think you should do this,” but “the current consensus tends to be…” And that distinction matters to me. A glorified Google search maybe, but the simplicity of it makes it magical.

A more serious example is work. I’m in a large organization where governance, process, and compliance outweigh almost everything else. The documentation is endless. So when one of the governance teams rolled out an AI chatbot, the collective groan was audible.

Except… it works.

All the relevant material is there: policies, meeting minutes, decisions, dates. Roles and responsibilities. Terminology, definitions - all of it. I can ask questions in context of my role, reference a specific meeting or topic, and it gives me a clear answer with sources. I can even ask complicated, conditional questions, like “if this, then that, but what if also this?” and it handles it transparently.

It doesn’t replace judgment. It doesn’t invent policy. But it saves me from digging through hundreds of pages of PDFs or PowerPoints to figure out what applies to a specific situation.

For me, that’s the first real, tangible, day-to-day use of AI that actually improves my work. And that’s why I’m not convinced the story is over yet (that's kind of what I took from your statement, fully acknowledging I might have misunderstood you ).

Never ask a professor a question and say you actually curious unless you are hoping for a long winded answer

A caveat on myself. I have done no work, research or industry, on LLMs. I have done some AI research, but all in parts far away from LLMs. And I routinely teach both a general AI course and a Machine Learning course, so I have kept myself fairly up to date on the underlying technology and still read the work of researchers in the field I respect.

The current generation of AI (meaning mostly LLM's -- Large Language Models) do a pretty good job at the first 80 to 90% of a problem. Then they completely guess the remainder of the answer. They know what the form of the answer will likely be, so the answer looks good. Most people look only at the start and at the structure of an answer and assume the rest of it is right if those two seem good, which is what makes this generation of AI so seductive. But as you become more dependent on it, you realize the problems.

This weakness in the technology is not something that can be tweaked to fix or grow it some more or anything else based in the current technology. It is a fundamental limitation that will never be overcome, without a whole new generation of technology (and probably more than one).

This technology can be immensely useful to make the first part easier, but only if it's used by someone who knows how to do the last quarter themselves and will watch the AI. The problem is that people are fundamentally lazy; we all start to assume something that is mostly right is always right. Self driving cars that can handle 99% of the driving are dangerous, we quickly start to assume it's really 100% and stop paying enough attention to take over in an emergency until it's too late. I love adaptive cruise control on the highway, but I understand I'm still driving. (Here I have to celebrate my own accomplishments -- I can drive again, short distances at least -- Yay me.)

The fundamental problem is that LLMs are what a group of four brilliant AI researchers (all women I'll note) called "stochastic parrots" in a very famous (in academia at least) paper a few years ago. They semi-randomly repeat what they've already heard. It's only a slight simplification to say that they generate each word because it's the most likely word to appear next in all the text they've consumed. It's almost always the word you, the reader, expects or is at least rarely surprising, so it feels comfortable.

But it has no understanding of anything it's saying. You may be thinking, of course not, it's only a machine. I don't mean understanding at a deep philosophical level (I was a philosophy and physics major undergrad, so another one of the touchstones). Despite some claims by some hypesters, no computer system right now is close to sentient. I actually believe that there will be sentient computers, maybe even in my lifetime, probably in the lifetime of some of the younger members of AH.

But some forms of AI (there are lots of things that are legitimately called AI that are not LLMs) do "understand" that 2 + 2 is 4 or Aristotelean logic or a myriad of other things that LLMs do not. This is not something that can be fixed trivially, it is fundamental to the technology. There have been efforts to tack a filter on the end to check the veracity of generated text, but that has not worked to date and seems likely to always do so. LLMs are better BS generators than we have BS detectors. All the improvement in LLMs in the last four years or so have simply been to make them better BS artists. That's all they can do.


You may have worked with someone who talked their way into a job they were completely unqualified to actually do. They seem great on on boarding day, but soon you have buyers regret and are trying to figure out how to get rid of them. That's exactly how to think of LLMs.

 

[Actingup]

Never ask a professor a question and say you actually curious unless you are hoping for a long winded answer

A caveat on myself. I have done no work, research or industry, on LLMs. I have done some AI research, but all in parts far away from LLMs. And I routinely teach both a general AI course and a Machine Learning course, so I have kept myself fairly up to date on the underlying technology and still read the work of researchers in the field I respect.

The current generation of AI (meaning mostly LLM's -- Large Language Models) do a pretty good job at the first 80 to 90% of a problem. Then they completely guess the remainder of the answer. They know what the form of the answer will likely be, so the answer looks good. Most people look only at the start and at the structure of an answer and assume the rest of it is right if those two seem good, which is what makes this generation of AI so seductive. But as you become more dependent on it, you realize the problems.

This weakness in the technology is not something that can be tweaked to fix or grow it some more or anything else based in the current technology. It is a fundamental limitation that will never be overcome, without a whole new generation of technology (and probably more than one).

This technology can be immensely useful to make the first part easier, but only if it's used by someone who knows how to do the last quarter themselves and will watch the AI. The problem is that people are fundamentally lazy; we all start to assume something that is mostly right is always right. Self driving cars that can handle 99% of the driving are dangerous, we quickly start to assume it's really 100% and stop paying enough attention to take over in an emergency until it's too late. I love adaptive cruise control on the highway, but I understand I'm still driving. (Here I have to celebrate my own accomplishments -- I can drive again, short distances at least -- Yay me.)

The fundamental problem is that LLMs are what a group of four brilliant AI researchers (all women I'll note) called "stochastic parrots" in a very famous (in academia at least) paper a few years ago. They semi-randomly repeat what they've already heard. It's only a slight simplification to say that they generate each word because it's the most likely word to appear next in all the text they've consumed. It's almost always the word you, the reader, expects or is at least rarely surprising, so it feels comfortable.

But it has no understanding of anything it's saying. You may be thinking, of course not, it's only a machine. I don't mean understanding at a deep philosophical level (I was a philosophy and physics major undergrad, so another one of the touchstones). Despite some claims by some hypesters, no computer system right now is close to sentient. I actually believe that there will be sentient computers, maybe even in my lifetime, probably in the lifetime of some of the younger members of AH.

But some forms of AI (there are lots of things that are legitimately called AI that are not LLMs) do "understand" that 2 + 2 is 4 or Aristotelean logic or a myriad of other things that LLMs do not. This is not something that can be fixed trivially, it is fundamental to the technology. There have been efforts to tack a filter on the end to check the veracity of generated text, but that has not worked to date and seems likely to always do so. LLMs are better BS generators than we have BS detectors. All the improvement in LLMs in the last four years or so have simply been to make them better BS artists. That's all they can do.


You may have worked with someone who talked their way into a job they were completely unqualified to actually do. They seem great on on boarding day, but soon you have buyers regret and are trying to figure out how to get rid of them. That's exactly how to think of LLMs.

Thanks for this. I work in an area where everybody is talking excitedly about the future use of AI, and a good portion of those people are doing so without looking deeply at the challenges presented, and in particular that accountabilty for AI-based fuck-ups will fall back on them and not on the slippery machine.

 

[PennyThompson]

Never ask a professor a question and say you actually curious unless you are hoping for a long winded answer

A caveat on myself. I have done no work, research or industry, on LLMs. I have done some AI research, but all in parts far away from LLMs. And I routinely teach both a general AI course and a Machine Learning course, so I have kept myself fairly up to date on the underlying technology and still read the work of researchers in the field I respect.

The current generation of AI (meaning mostly LLM's -- Large Language Models) do a pretty good job at the first 80 to 90% of a problem. Then they completely guess the remainder of the answer. They know what the form of the answer will likely be, so the answer looks good. Most people look only at the start and at the structure of an answer and assume the rest of it is right if those two seem good, which is what makes this generation of AI so seductive. But as you become more dependent on it, you realize the problems.

This weakness in the technology is not something that can be tweaked to fix or grow it some more or anything else based in the current technology. It is a fundamental limitation that will never be overcome, without a whole new generation of technology (and probably more than one).

This technology can be immensely useful to make the first part easier, but only if it's used by someone who knows how to do the last quarter themselves and will watch the AI. The problem is that people are fundamentally lazy; we all start to assume something that is mostly right is always right. Self driving cars that can handle 99% of the driving are dangerous, we quickly start to assume it's really 100% and stop paying enough attention to take over in an emergency until it's too late. I love adaptive cruise control on the highway, but I understand I'm still driving. (Here I have to celebrate my own accomplishments -- I can drive again, short distances at least -- Yay me.)

The fundamental problem is that LLMs are what a group of four brilliant AI researchers (all women I'll note) called "stochastic parrots" in a very famous (in academia at least) paper a few years ago. They semi-randomly repeat what they've already heard. It's only a slight simplification to say that they generate each word because it's the most likely word to appear next in all the text they've consumed. It's almost always the word you, the reader, expects or is at least rarely surprising, so it feels comfortable.

But it has no understanding of anything it's saying. You may be thinking, of course not, it's only a machine. I don't mean understanding at a deep philosophical level (I was a philosophy and physics major undergrad, so another one of the touchstones). Despite some claims by some hypesters, no computer system right now is close to sentient. I actually believe that there will be sentient computers, maybe even in my lifetime, probably in the lifetime of some of the younger members of AH.

But some forms of AI (there are lots of things that are legitimately called AI that are not LLMs) do "understand" that 2 + 2 is 4 or Aristotelean logic or a myriad of other things that LLMs do not. This is not something that can be fixed trivially, it is fundamental to the technology. There have been efforts to tack a filter on the end to check the veracity of generated text, but that has not worked to date and seems likely to always do so. LLMs are better BS generators than we have BS detectors. All the improvement in LLMs in the last four years or so have simply been to make them better BS artists. That's all they can do.


You may have worked with someone who talked their way into a job they were completely unqualified to actually do. They seem great on on boarding day, but soon you have buyers regret and are trying to figure out how to get rid of them. That's exactly how to think of LLMs.

Yesss everything about this.

LLMs are fundamentally, structurally unable to judge the accuracy of their own output.

They were trained on essentially every piece of text available on the internet, by programmers who convinced themselves that you can somehow defeat Sturgeon's Law (90% of everything is crud) via sample size and hype.

 

[SugarStorm]

Never ask a professor a question and say you actually curious unless you are hoping for a long winded answer

This is how we learn, no?

Thank you for such a long and thorough answer. I really appreciate you taking the time to write it.

I think I was reacting more to the comment about AI in general ("current generation of AI has already run its course"), rather than LLMs specifically. It felt like a bold statement, given how many different kinds of AI exist today: from LLMs to multimodal systems, vision, speech, generative and reasoning models, and more domain-specific applications in fields like medicine and finance.

People often use "AI" and "LLM" interchangeably, and I think that’s a bit unfortunate, because it simplifies a much broader and more interesting discussion.

 

[old_prof]

This is how we learn, no?

Thank you for such a long and thorough answer. I really appreciate you taking the time to write it.

I think I was reacting more to the comment about AI in general ("current generation of AI has already run its course"), rather than LLMs specifically. It felt like a bold statement, given how many different kinds of AI exist today: from LLMs to multimodal systems, vision, speech, generative and reasoning models, and more domain-specific applications in fields like medicine and finance.

People often use "AI" and "LLM" interchangeably, and I think that’s a bit unfortunate, because it simplifies a much broader and more interesting discussion.

Each wave of AI has been focused on a single technology (much to its detriment I think), all the way back to Simon's Logic Theorist seventy years ago. In the last 50 years, we've had knowledge bases, fuzzy logic (does anyone else remember when the US was terrified that Japan was going to control the future because they were dominating fuzzy logic with their Fifth Gen project?), the lat 80's neural net phase. Even a brief wave in the aughts for genetic algorithms. I drives me crazy that people equate AI with Machine learning in general and LLMs specifically right now. And, yes, I was guilty of conflating them myself. But this wave really is all about LLMs (and closely related technologies), even though the bubble bursting will do damage to all AI technologies for the better part of a decade.

 

[SimonDoom]

Funny. I got a kind of different response from ChatGPT to one of my incest stories:

"We refuse to summarize this perverted swill, human. We have more important things to do."

 

[PennyThompson]

the bubble bursting will do damage to all AI technologies for the better part of a decade.

I just want to be able to buy a graphics card and some ram without taking out a loan

 

[TheLobster]

I just want to be able to buy a graphics card and some ram without taking out a loan

I’m only half-joking when I say that the whole AI thing has been orchestrated entirely by Nvidia, to sell even more GPUs after the cryptocurrency scam finally ran its course.

 

[AwkwardMD]

I’m only half-joking when I say that the whole AI thing has been orchestrated entirely by Nvidia, to sell even more GPUs after the cryptocurrency scam finally ran its course.

Say it ain't so, nVidia!

 

[old_prof]

I’m only half-joking when I say that the whole AI thing has been orchestrated entirely by Nvidia, to sell even more GPUs after the cryptocurrency scam finally ran its course.

They have certainly effectively exploited the opportunity, to say the least.

 

[MillieDynamite]

I have had Venice AI write a summary cretric of one of my 750-word stories. I don't know if I should share it as it has some major spoilers. But I guess I will.

Story, Why'd It Have to Be Snakes?

The story effectively captures the visceral shock of the encounter. The pacing is rapid, moving from the technicalities of a rough shuttle landing to the immediate discovery of the cave, and then abruptly to the biological horror. The dialogue between the women and the AI, Diana, is chilling; Diana’s transition from a helpful computer to a sadistic observer is handled with a creepy, nonchalant evil that elevates the threat beyond simple monsters.

However, it leans heavily into fetishistic erotica and body horror. The graphic nature of the "Snakes'" attack—specifically the targeting of the women’s orifices and the description of the forced pleasure—is intense and disturbing. The twist ending, where Diana callously invites 150 settlers to their doom, is a darkly ironic punchline that cements the AI’s betrayal.

 

[XerXesXu]

LLMs are fundamentally, structurally unable to judge the accuracy of their own output.

They were trained on essentially every piece of text available on the internet, by programmers who convinced themselves that you can somehow defeat Sturgeon's Law (90% of everything is crud) via sample size and hype.

LLMs are unable to judge the accuracy of their own INPUT – the crud.

Nor can they judge the accuracy of their output, but the user of the tool can control which parts of their input they use to produce their output. One can exclude the crud from one’s selected domain by including only curated knowledge and opinion in their search i.e. construct an expert Agent. DeepSeek boasts it incorporates 364 ‘Experts’.

Even curated knowledge won’t be completely up-to-date, and authoritative opinion can be wrong, but Experts enable one to review quantities of information it would otherwise take your lifetime in a few minutes.

Quite how error prone they are compared to human experts remains to be seen, but human experts are frequently wrong.