Yes, the Toplists are Broken

[BobbyBrandt]

If it wasn't deliberate activity, we wouldn't be able to predict it so accurately.

I question that you are predicting it accurately.

The variables that I have shared with my own scores, both within top lists and elsewhere are similar to what they have always been; up and down without any significant rise or fall either way. As far as I can tell, the scores overall are consistent to historical trends.

Your myopic view of a singular presentation point (top lists) does not prove that the system is being manipulated by anyone.

 

[Voboy]

I question that you are predicting it accurately.

The variables that I have shared with my own scores, both within top lists and elsewhere are similar to what they have always been; up and down without any significant rise or fall either way. As far as I can tell, the scores overall are consistent to historical trends.

Your myopic view of a singular presentation point (top lists) does not prove that the system is being manipulated by anyone.

Many threads like this one flip-flop cause and effect, and the OP has done that more than once in some of the "side issue" posts this thread has spawned. I think he started with an idee-fixe, and that's always a bad way to "analyze" anything. Anyone claiming to be in any way scientific needs to strive for a sober, unbiased assessment of their hypothesis, and that's far from what this OP is doing; similar threads have made the same mistake.

This is not an unimportant topic, but as most of these kinds of threads eventually conclude, we lack the data to tell for sure what's going on. What we have is [often very passionate] opinion, backed by confirmation bias.

 

[pink_silk_glove]

Those stories had around two hundred ratings each. They didn't have any 1* ratings,

None at all? Srsly??

 

[ThatNewGuy]

we lack the data to tell for sure what's going on. What we have is [often very passionate] opinion, backed by confirmation bias.

The variables that I have shared with my own scores, both within top lists and elsewhere are similar to what they have always been; up and down without any significant rise or fall either way. As far as I can tell, the scores overall are consistent to historical trends.

Let me provide some data (750 stories) to support the claim that top lists have been subject to mass downvoting, creating a flattening effect across top lists.

• On April 5, 2024, the Lesbian Sex top list had scores ranging from 4.94 to 4.85. Today, all stories are at 4.84 except six.
• On April 5, 2024, the SF&F top list had scores ranging from 4.96 to 4.87. Today, all stories are at 4.84 except eight.
• On April 5, 2024, the Novels and Novellas top list had scores ranging from 4.93 to 4.87. Today, all stories are at 4.84 except four. (If you check my earlier post in this thread (post 140), you'll see that six stories were above 4.84 earlier today. Just since that time, two more stories have been knocked below the 4.84 threshold.)

I think this data makes a compelling case for my claim. I'm open to other opinions, but those opinions would need to provide an explanation that fits the data.

So far, I've only heard two offered. The first (highlighted in @BobbyBrandt's quote above) is that scores are going up and down just as they always have without any noteworthy rise or fall. I don't dispute that he has seen this pattern in his stories. However, the data I've provided above clearly shows that this is not true among stories in the top lists.

The second explanation, mentioned by both @BobbyBrandt and I believe also by @lovecraft68, is that there has been some system-wide adjustment affecting all stories. The top list flattening, as I understand the argument, is just a manifestation of this system-wide adjustment at the upper end of the scale.

I would ask two questions of those supporting this alternative explanation.

First, what evidence can you present that other stories outside the top lists have been subject to this system-wide adjustment? Clearly, if the adjustment is affecting the stories in the top lists in the same consistent manner (i.e., a decrease in scores down to a threshold of 4.84), then you should be able to point to a similar consistent effect that this change has had on other stories across the board. I've seen no evidence offered for that.

Second, how do you explain the fact that any stories that rise above the 4.84 threshold on the top lists seem to be immediately bumped back down to 4.84? How is that consistent with the system-wide adjustment theory?

Looking at these numbers, it's hard for me not to conclude that some person(s) have decided that stories on certain top lists shall not rise above an arbitrary threshold (in this case, 4.84). Once they do, they are bumped back below it.

 

[Voboy]

Let me provide some data (750 stories) to support the claim that top lists have been subject to mass downvoting, creating a flattening effect across top lists.

• On April 5, 2024, the Lesbian Sex top list had scores ranging from 4.94 to 4.85. Today, all stories are at 4.84 except six.
• On April 5, 2024, the SF&F top list had scores ranging from 4.96 to 4.87. Today, all stories are at 4.84 except eight.
• On April 5, 2024, the Novels and Novellas top list had scores ranging from 4.93 to 4.87. Today, all stories are at 4.84 except four. (If you check my earlier post in this thread (post 140), you'll see that six stories were above 4.84 earlier today. Just since that time, two more stories have been knocked below the 4.84 threshold.)

I think this data makes a compelling case for my claim. I'm open to other opinions, but those opinions would need to provide an explanation that fits the data.

So far, I've only heard two offered. The first (highlighted in @BobbyBrandt's quote above) is that scores are going up and down just as they always have without any noteworthy rise or fall. I don't dispute that he has seen this pattern in his stories. However, the data I've provided above clearly shows that this is not true among stories in the top lists.

The second explanation, mentioned by both @BobbyBrandt and I believe also by @lovecraft68, is that there has been some system-wide adjustment affecting all stories. The top list flattening, as I understand the argument, is just a manifestation of this system-wide adjustment at the upper end of the scale.

I would ask two questions of those supporting this alternative explanation.

First, what evidence can you present that other stories outside the top lists have been subject to this system-wide adjustment? Clearly, if the adjustment is affecting the stories in the top lists in the same consistent manner (i.e., a decrease in scores down to a threshold of 4.84), then you should be able to point to a similar consistent effect that this change has had on other stories across the board. I've seen no evidence offered for that.

Second, how do you explain the fact that any stories that rise above the 4.84 threshold on the top lists seem to be immediately bumped back down to 4.84? How is that consistent with the system-wide adjustment theory?

Looking at these numbers, it's hard for me not to conclude that some person(s) have decided that stories on certain top lists shall not rise above an arbitrary threshold (in this case, 4.84). Once they do, they are bumped back below it.

Okay. So...

Why does it matter?

 

[ThatNewGuy]

Okay. So...

Why does it matter?

My personal opinion? It doesn't really matter. I enjoy writing stories. I'm grateful people read them. The top list shenanigans don't affect that.

That said, people are making claims in this thread that there isn't a problem when clearly there is one. I think it benefits all of us in this discussion to acknowledge that as a starting point.

Edit to add: others have pointed out in this thread that stories on top lists get a nice bump in views and reader engagement. Getting knocked off the top list by vote manipulation affects how many people are reading your stories. There's an element of unfairness in that, no?

 

[old_prof]

Okay. So...

Why does it matter?

I will argue for most perspectives, it doesn't.

All the arguments that it matters eventually comes down to our ego, which are very real.I feel the pain of it happening and I understand anyone wanting to complain about it.

I have no problem with anyone arguing that it's the way it is, live with it. It doesn't make a real difference in any real way.

I do have a problem with posters trying to ignore the evidence that it's happening, evidence which I find overwhelming, just because they can't handle the reality. And getting angry at people trying to point out that it is happening.

It's a real phenomenon. Get over it. It sucks and hurts your feelings to have your stories pinged. Get over it.

Emotionally, I'm in the second camp, because it hurts to have unreasonable criticism of something I have poured my heart and should into. But I need to learn to lump it and move on. I guess for me, the biggest real damage, beyond my ego, is it makes my job of figuring out what works for me as a writer much harder to decipher. That's my primary motivation right now. I'm nowhere the writer I want to be. If you want to give me harsh comments about how awful my writing is, I will read them to find what parts of that I can learn from. A 1-bomb because my story was too successful does not help me at all and confuses the subtle signals I'm trying to learn from.

 

[Voboy]

My personal opinion? It doesn't really matter. I enjoy writing stories. I'm grateful people read them. The top list shenanigans don't affect that.

That said, people are making claims in this thread that there isn't a problem when clearly there is one. I think it benefits all of us in this discussion to acknowledge that as a starting point.

Edit to add: others have pointed out in this thread that stories on top lists get a nice bump in views and reader engagement. Getting knocked off the top list by vote manipulation affects how many people are reading your stories. There's an element of unfairness in that, no?

As I've posted several times now, my take on this is that just about every aspect of the rating/voting system on this site has problems. Several problems.

I also believe quite strongly that the site owners won't ever change it, for various reasons both sensical and nonsensical.

My perspective is that rehashing this sort of issue again and again and again only feeds into the stress of those who live and die by their ratings. My suggestion is, as always, that the kinds of people obsessed with these issues are perhaps likely to be happier away from the site, or at the very least by redefining what they're looking for here.

Either way, I don't see yet another iteration of this kind of thread as being productive, really, especially when the OP proves as recalcitrant as this one has. So, from my perspective, I buy your numbers and perhaps your conclusion... but I just don't care. Because if I wasted energy on this sort of thing, I'd get no enjoyment out of this place.

And I prefer to enjoy it. That's a choice I can make. The OP could, if he wishes to, make the same choice.

 

[ThatNewGuy]

So, from my perspective, I buy your numbers and perhaps your conclusion... but I just don't care. Because if I wasted energy on this sort of thing, I'd get no enjoyment out of this place.

Fair enough. I agree that it's important to keep perspective on these type of things.

 

[Cacatua_Galerita]

Maybe if we wrote for money. I'd agree with you. The thing is, it's supposed to be a fun distraction.

Hmm. Perhaps there is money involved?

I wrote about this series a year ago because it dominates the SF&F Top Stories list: Three Square Meals. From the story comments I gather that the last 30 chapters are pay-to-view on Patreon. It seems the author is using the popularity of Literotica to capture readership before directing them to a paid site.

I was going to comment about my amusement with (this thread) OP’s audacity... complaining about his chapters getting bumped from the top list. I only write standalone stories, so I estimate I’m at a ~0.18 disadvantage when competing against chapters for a position on the Top Lists. It isn’t a level playing field – chapters have the advantage. (They should really be called the “Top Chapter Lists” given their makeup.)

Last May I wrote this:

Currently, in the Sci-Fi Top List, for example, eighteen of the Top 50 entries are the same story. “Three Square Meals” has chapters 100, 53, 60, 68, 59, 95, 112, 138, 33, 34, 43, 29, 55, 50, 57, 54, 58, and 61 listed on the first page, plus more on the next pages of the Top List. Note that chapter 29 is the first in that series to score highly enough to make it onto the Top List. (Chapter 01, with far more views than subsequent chapters, is rated 4.62.) I’m sure this series is a great read, but if I’ve only got two hours to do some reading then I don’t want to start binging on a 100-part series. So I'm already going to skip over series stories, don't make be have to skip over eighteen listings of the same story.

Things have changed with whatever is now gravitating Top List entries towards 4.84 in SF&F. Chapter 29 is no longer the first Top List entry for Three Square Meals. Chapters 13, 15, 16, 18, 23, 27, & 28 have joined ahead of it. There are now 35 chapters of that story listed in the “Top 50” of SF&F (~twice as many as a year ago) and 85 entries of the top “250 stories” are various chapters of Three Square Meals. (Chapter 1 is still rated at 4.62.) Given the thousands of votes each chapter has, it would take a lot of 5-bombs to achieve such an improvement. It is more likely the site has changed something relating to the calculation of the top lists – probably not for the better, given the even lower diversity now. And from the way the reader comments have turned on that author recently (click to see 'newest first' comments), I would have expected more 1-bomb effects than 5-bomb gains over the last year.

Note that I have no evidence of any Top List vote-rigging campaign, for this story or any others. I’m just agreeing that something has changed with the way the site calculates Top Lists and there now appears to be even less diversity overall, with even fewer standalone stories represented (the format that I want to read).

 

[Five_Inch_Heels]

Where does this thread about lists rank in the list of threads about lists?

 

[RocketGrunt]

And this whole notion of counting only fives has another serious problem. When a story with a 4.80 and 600 votes is ranked above someone with a 4.84 and 500 votes, the 500 vote author's screams of cheating and site bias will startle people out of a dead sleep on the other side of the world. And there would be a butt-ton of those stories from ages long past that suddenly appear on the toplists, so the collective wailing would likely be picked up by gravitational wave detectors. Using any metric other than a simple average is going to summon a torch-wielding mob screaming WITCHCRAFT!

The current button that goes to the toplist for a category doesn't say "more highly rated stories," it says "more popular stories." Assigning things by average rating is already very weird, because it's just objectively not a measure of popularity. A story can get the maximum possible average rating with only a single vote, and a majority of stories do! The average rating isn't a measure of how many people read it, or liked it, or rated it highly. It's a negative measurement of how few people didn't like it enough to cast a low vote. Number of views, number of favorites, number of positive ratings, and even raw number of ratings are all obviously much better proxies for "popularity" than average rating.

Like, fans of The Cure probably rate The Cure higher than fans of Def Leppard rate Def Leppard, but you wouldn't say that The Cure was ever more popular than Def Leppard, right?

Actually, the "Popular Stories" tab really puts an underline on the whole toplist fiasco. If you open up the Sci-Fi/Fantasy board right now, the "Popular Sci-Fi & Fantasy Sex Stories" heading has three tabs: 7-day, 30-day, and All-Time. These are also arranged by average rating, but they don't have the same "number of ratings" cutoff that defines the top 250 list that opens when you click "More Popular Sci-Fi & Fantasy Stories." The result is, predictably, that the ten "all-time" most popular stories displayed are ten stories that have a perfect 5* average rating. None of them have received a single rating of 4* or less, though none of them have been favorited more than five times, nor have any of them gotten enough votes to be considered for the longer toplist.

The top stories on that most popular list have been stable for four months, receiving no downvotes in the entire time since the new view went live. Because no downvote script patrols that part of the site, and there's no reason for any human to downvote them. And yet, we know that if any of them got enough ratings to be exposed to the toplist that is patrolled, they would receive 1* trollhammering in less than 24 hours. Ironically, the currently presented ten examples of "all time popular" SF&F stories are able to stay there precisely because they aren't popular enough to trigger the downvote script.

 

[lovecraft68]

Okay. So...

Why does it matter?

It shouldn't but as this and countless other score related threads prove, it does for some people and way too much.

 

[lovecraft68]

they aren't popular enough to trigger the downvote script.

 

[BobbyBrandt]

Second, how do you explain the fact that any stories that rise above the 4.84 threshold on the top lists seem to be immediately bumped back down to 4.84? How is that consistent with the system-wide adjustment theory?

How do you explain stories not on any top list, some under 4.00, also seeing the same "leveling", or score adjustment?

Every story that I have with a score over 4.85 saw an adjustment since last fall, and most were not high on any top lists. In addition, every other of my 32 stories has seen the consistent fluctuation associated with ongoing vote activity.

My perspective is, that if you look beyond only the top lists, the rating changes are there on a larger scale. I have no dispute with the changes affecting the top lists, but but claims that they are the target for the changes aren't obvious if you look at things broader.

 

[RocketGrunt]

How do you explain stories not on any top list, some under 4.00, also seeing the same "leveling", or score adjustment?

Every story that I have with a score over 4.85 saw an adjustment since last fall, and most were not high on any top lists. In addition, every other of my 32 stories has seen the consistent fluctuation associated with ongoing vote activity.

My perspective is, that if you look beyond only the top lists, the rating changes are there on a larger scale. I have no dispute with the changes affecting the top lists, but but claims that they are the target for the changes aren't obvious if you look at things broader.

This is simply nonsense.

Since the November update, Chapter 71 of a Warcraft fanfic has been given as the prime example of a "popular" Science Fiction & Fantasy story because it has and continues to have a perfect 5* rating. It has 3,600 more views than chapter 69 of the same fanfic. Chapter 69 is not presented on the "popular" tab because its rating is "only" 4.93. That presentation on the popular works tab of the story category board is worth about a thousand views a month even for a late chapter of a story as obviously niche as that.

And no one down votes it. There's more than one person clicking on that story every hour of every day and has been since the beginning of November when the update went live and not one person has given it a 1* or even a 4*. Because that's the point. People don't randomly drop 1* bombs on stories that are well regarded by niche fandoms that aren't theirs. You don't log onto the fetish board and downvote looners and shoe collectors, just as you don't log onto the SF&F board to downvote Warcraft fanfiction.

And machines do. A script that finds every nail that sticks up and hammers it down genuinely does log into stories of no interest and hammers them down. The number of stories on the SF&F board that are above 4.84 is down to 8 again after the last pass of the flattening script. There's more SF&F stories on the popular tab with perfect 5s than there are SF&F stories on the toplist with a rating other than exactly 4.84 because the popular tab isn't targeted and the toplist is.

 

[BobbyBrandt]

Once again, I clearly understand your obsession with the top lists based upon your perception of their value to reader engagement.

You just refuse to acknowledge that there are scores on stories here besides those that appear on the top lists which are also seeing changes. The fact that stories NOT on the top lists are also affected mitigates your theory that the top lists are somehow being exclusively targeted maliciously.

The top lists are merely a visible presentation of a larger phenomenon. You don't have to like it. You don't have to understand it. You don't have to accept the reality.

Fine. Move on.

 

[RejectReality]

Well, let's watch these then.



At some point this year, a couple of these are going to break 100 votes. At that point, they're going to get hit just like Ch. 04 & 09 did. The toplists are undeniably a hotbed of trolling. ( And they're not going to uniformly drop to 4.83 either, because it's uncoordinated humans doing it, and not a script )

 

[ThatNewGuy]

There are now 35 chapters of that story listed in the “Top 50” of SF&F (~twice as many as a year ago) and 85 entries of the top “250 stories” are various chapters of Three Square Meals. (Chapter 1 is still rated at 4.62.) Given the thousands of votes each chapter has, it would take a lot of 5-bombs to achieve such an improvement.

This is a great observation, but I would argue your conclusion is wrong. Tefler doesn't now have more stories in the top 50 because they have been five-bombed up the ladder. He has more stories in the top 50 because the stories ahead of him were bombed down the ladder.

Keep in mind that position on the top lists for tied stories is based on total number of votes. Tefler's stories all have a huge number of votes. So with mass downvoting pushing everything to 4.84, it pushed down a bunch of 4.85+ non-Tefler stories with only, say, a few hundred votes. Tefler's other 4.84 stories (which didn't previously make the list) then took their place because they now had the same score but far more votes.

This should be testable. If the new Tefler stories in the top 50 are there because of mass five-bombing, they should have a much higher vote total than his chapters that there were already there (due to five-bombing). This doesn't appear to be the case. Chapters 80, 46, and 52 (all new entries to the top 50 based on your info) all have vote totals equivalent to or lower than the other top 50 stories.

 

[RocketGrunt]

Well, let's watch these then,

At some point this year, a couple of these are going to break 100 votes. At that point, they're going to get hit just like Ch. 04 & 09 did. The toplists are undeniably a hotbed of trolling. ( And they're not going to uniformly drop to 4.83 either, because it's uncoordinated humans doing it, and not a script )

That's reasonable.

Since there are multiple stories close to the startingly line that, like literally hundreds on the site, have earned a 4.9+ and have been allowed to keep it, the prediction would be that they would enter the toplist an entire tenth of a point above the trollhammer target. Since the hammering comes in singles and doubles, I would predict that it would take a couple of rounds of hammering to push it off.

A story that enters with exactly 100 votes at 4.93 would drop to 4.89 after a single 1* strike, and to 4.85 after two 1* strikes. 4.82 after 3 and 4.78 after 4. Obviously it can be thrown a bit if there are a few 5* reviews that get inspired by the enhanced readership, but that won't make a huge difference. With a start at 4.9, one bot slam drops it to 4.86 and two drops it to 4.82.

So that's my prediction. Your 4.93 story hits the top list and when you wake up the next morning it will still be above the line at 4.85 or 4.86, and the following day you will find it at 4.78 or 4.79, where it will obviously never again come into striking range of the toplist. Alternately, your 4.9 story hits the top list and the very next day it is at 4.82 and never sees the toplist again.

Obviously, having it play out exactly like that or slightly differently does not prove or disprove the use of scripts or the malice of human hands. The entire script-kiddy theory is that the toplist attacks are scripts emulating the activity of a disparate group of malcontent haters that the vote sweeps are unable to identify as inauthentic. But I find the script-kiddy argument persuasive, and it looks like your story will sadly be a case study pretty soon.

 

[old_prof]

At 400 votes, it takes roughly twenty-five 5 votes to move a 4.84 to a 4.85. It takes a single 1 to move it back down.

 

[old_prof]

That's reasonable.

Since there are multiple stories close to the startingly line that, like literally hundreds on the site, have earned a 4.9+ and have been allowed to keep it, the prediction would be that they would enter the toplist an entire tenth of a point above the trollhammer target. Since the hammering comes in singles and doubles, I would predict that it would take a couple of rounds of hammering to push it off.

A story that enters with exactly 100 votes at 4.93 would drop to 4.89 after a single 1* strike, and to 4.85 after two 1* strikes. 4.82 after 3 and 4.78 after 4. Obviously it can be thrown a bit if there are a few 5* reviews that get inspired by the enhanced readership, but that won't make a huge difference. With a start at 4.9, one bot slam drops it to 4.86 and two drops it to 4.82.

So that's my prediction. Your 4.93 story hits the top list and when you wake up the next morning it will still be above the line at 4.85 or 4.86, and the following day you will find it at 4.78 or 4.79, where it will obviously never again come into striking range of the toplist. Alternately, your 4.9 story hits the top list and the very next day it is at 4.82 and never sees the toplist again.

Obviously, having it play out exactly like that or slightly differently does not prove or disprove the use of scripts or the malice of human hands. The entire script-kiddy theory is that the toplist attacks are scripts emulating the activity of a disparate group of malcontent haters that the vote sweeps are unable to identify as inauthentic. But I find the script-kiddy argument persuasive, and it looks like your story will sadly be a case study pretty soon.

I know my last story in that situation took three 1-bombs within a few hours. MelissaBaby reported a similar occurrence. They do not hit just once a day.

 

[ThatNewGuy]

How do you explain stories not on any top list, some under 4.00, also seeing the same "leveling", or score adjustment?

You've made this claim several times now, but you've provided no evidence for it. In fact, you've done the opposite (see below).

In addition, every other of my 32 stories has seen the consistent fluctuation associated with ongoing vote activity.

I have had stories with ratings below 4.00 see the same type of change (up and down) as any that I have on a top list.

Your argument is that your stories not on the top lists are also seeing a leveling or flattening effect. But then you say that these stories are moving up and down.

If you are right, and your stories outside the top lists are being subjected to the same flattening, then your stories shouldn't be moving up and down as you say they are. They should be seeing some type of consistent effect across the board, just as all top list stories have been consistently pushed down.

I'd respectfully offer that what you are seeing with your non-top list stories is normal voting. They move up and down precisely because they are not being affected by any mass downvoting, or even any system-wide change. What evidence can you provide that non-top list stories are also seeing some consistent effect resulting from a system-wide change?

I'll give you one more example:

• The Novels and Novellas 12-Month Hall of Fame has 78 stories above 4.84.
• The Novels and Novellas All-Time Hall of Fame has 4 stores above 4.84

If your theory about a system-wide change affecting all stories is correct, why hasn't the Novels and Novella 12-Month Hall of Fame List been affected in the same way as the all-time list?

The answer is because only the all-time top-lists are being intentionally targeted.

 

[BobbyBrandt]

Your argument is that your stories not on the top lists are also seeing a leveling or flattening effect. But then you say that these stories are moving up and down.

I will try to clarify...

There are currently eight of my stories that have "leveled" off between 4.83 and 4.84.


Only one has ever been higher than 25 on any top lists, yet they all experienced the same "leveling" phenomenon. And while there continues to be slight up and down movement in their scores, between 4.81 and 4.84 over the past few months, none have gone above 4.84 at any time regardless of their position on any lists. It's almost like the scores in general are being capped at a threshold and not allowed to go over that point.

Hence, I can't draw any correlation to the score situation as it relates solely to the top lists.

 

[old_prof]

I will try to clarify...

There are currently eight of my stories that have "leveled" off between 4.83 and 4.84.

View attachment 2595758
Only one has ever been higher than 25 on any top lists, yet they all experienced the same "leveling" phenomenon. And while there continues to be slight up and down movement in their scores, between 4.81 and 4.84 over the past few months, none have gone above 4.84 at any time regardless of their position on any lists. It's almost like the scores in general are being capped at a threshold and not allowed to go over that point.

Hence, I can't draw any correlation to the score situation as it relates solely to the top lists.

Honest question. How often do you monitor your scores? Would you notice one crossing the boundary up to 4.85 for a day?

When you talked about scores going up and down, my guess was that you have a number of stories whose "natural" rating is above 4.84. Those will slowly drift up over time, until they get to 4.85 and then get sent back down to drift up again. That is bobbing up and down, but it is not a natural bobbing.

The kind of huge groupings of ratings at 4.84 across many categories is not a natural statistical phenomenon. There is some sort of artificial barrier being placed there, either by the site or some external factor. Dedicated trolls seems easily like the most likely scenario.