Yes, the Toplists are Broken

[RocketGrunt]

So a few hours ago, I made a prediction:

But of course, getting enough ratings at a high enough average response level is necessary but totally insufficient to stay on the toplist. You also have to have your rating randomly place you at a point that is low enough that no one decides to shove you down to protect the place of something that they want to keep on the toplist. Because the curation trolls have complete power over what STAYS on the toplist. At the moment I'm writing this, the top two stories on the SciFi board are chapters of serials that have ratings of 4.87 (which is a normal starting point for later series chapters, since obviously most of the readers of "Part 57" are already fans of the series), and less than 200 votes that came out in the last couple of days. They will be troll hammered within a few days, maybe a few hours. And given that a single 1* review would knock them down to 4.85, and falling to 4.84 would push them all the way off the top 250, they will probably be all the way out of the top 250 by the next time this argument comes up.

So both of those works have been pushed down to 4.83 or 4.84, off the toplist entirely. It's as reliable as clockwork, because someone presumably has a python script that is literally set to the clock.

The thing is that a rating of 4.87 or so is simply a normal rating for a later chapter from a popular series to have. There's almost no reason for people who don't like a series to read chapter 25 or chapter 55 of a series. As long as a popular series maintains quality and doesn't make any sharp turns that offend a section of the fanbase, getting hundreds of votes where 7/8ths or 9/10ths of the ratings are 5* is simply the expected result. And that's why we get new works passing the 100 vote threshold at 4.87 or 4.88 almost every day.

And yet, the reality is that every piece that sprints up that hill is going to be smashed down again. Because the trolls have complete control. Nothing can stay above whatever arbitrary rating they choose to set their whack-a-mole script to. The way math works, it takes a trivially small number of people to hold an average rating below an arbitrary target, even in the face of hundreds or thousands of fans voting something up. The trolls have a system where they can't actually be stopped, and the result is that ratings on the entire site have been lowered substantially to make sure everyone limbos in under whatever entries they are protecting.

We know this is happening because we've all watched it happen to our own stuff. We know it's happening because even a casual look at the toplists shows the unmistakable fingerprints of an unnatural voting by the massive plateaus at arbitrary rating values. And we know it is happening because we can predict when it will happen next and to which stories with great accuracy.

The average ratings cannot be protected with sweeps. The trolls have won. Completely. The toplists are broken in a way so fundamental that they cannot be based on average ratings and continue to have any meaning.

 

[lovecraft68]

Broken or finally corrected? Maybe all the cheerleader and high five free votes have been removed?

Regardless, there's three things we know for sure.

We will never know, because the site is never going to say what changed

Everyone is going to be dealing with the same thing

This will be the latest source of handwringing and the gnashing of teeth as the stat obsessed will now have something else to be upset about.

I'm wondering if the answer for some people here might not be to shut their scoring off to protect their sanity

 

[SugarStorm]

Or - or - it’s Laurel and Manu’s way of keeping the top lists moving. You know, because nobody wants to see Random Series Part 43 permanently cemented at #1.

Turns out the real trolls were the friends we made along the way.

 

[StillStunned]

It's unfair, but so what? The top lists are such a minor part of the writing experience. None of the scores of any stories reflect quality, so it's no surprise that the top lists don't either.

Is it nice to be on a Top List? Sure! Is it frustrating to be bombed off it? Of course. Can we do anything about it? Will Lit do anything about it? No.

So might as well shrug and move on. Keep writing the stories you want to write, to the best of your abilities. And remind yourself that "fair" has no power at Lit.

 

[RocketGrunt]

Broken or finally corrected? Maybe all the cheerleader and high five free votes have been removed?

Uh... you know that's not what happened. We all know what happened. Don't insult our intelligence by pretending otherwise. It's not a change in policy at the top. It's a change in tactics by the trolls.

We know what it looks like when inauthentic 5* votes get removed, because it happens all the time. The effect is very small, as mathematically it must be. If there are hundreds of votes and the average is high, one 5* vote more or less has pretty much no effect. One extra 1* vote has an immediate and obvious effect on the average. That's how averages work when the average is close to the top.

We can see the votes that kick things off the toplists. We can predict them with a watch. We know they are added low votes and not subtracted high votes. I mean, mathematically that was obviously how it had to work, but also if you watch the ratings on any particular day you'll know that that's definitely what's happening.

 

[ElectricBlue]

The toplists are broken in a way so fundamental that they cannot be based on average ratings and continue to have any meaning.

Ten years ago the Top Lists were dominated by stories from the first ten years of Lit, against which nothing can compete. Over time, the age effect of older stories perpetuates this, so they're meaningless.

I don't understand why people care more so much about these lists. If they cause so much angst, why don't you just ignore them?

 

[pink_silk_glove]

So both of those works have been pushed down to 4.83 or 4.84, off the toplist entirely. It's as reliable as clockwork, because someone presumably has a python script that is literally set to the clock.

Who cares?

 

[RocketGrunt]

Pink Silk, you've already established beyond doubt that you are lazy and dishonest. That you are totally unprepared for any discussion that involves numbers of any kind, yet happy to pretend that the results of simply arithmetic are somehow in doubt so long as it affords you the opportunity to mouth off. I don't care what your opinion is on this or any other issue. Please vacate my thread.

 

[Voboy]

I don't care what your opinion is on this or any other issue. Please vacate my thread.

Lol.

Your first day on the internet or something? That's not how it works. If you're going to post your gripe publicly, the public is going to chime in. If you don't like that, you can leave too.

I'm with those who say you should quit worrying.

 

[BobbyBrandt]

Someone should take the time to graph out the performance of a story of theirs over a one, two or three year period. I'm pretty sure that it would show that over time, the rating of a story finds its level spot and will deviate little from that point.

Loyal followers have had their one vote, trolls have lost interest, hurt feelings get forgotten, sweeps come and go, and life moves on.

 

[Erozetta]

Here's the thing:

The readers can vote however they want, even if it's trolling, which is unfortunate, but part of life. Some people have enough time on their hands to make it their "responsibility" to protect their favorite authors or stories on those lists. It's brutal, but it happens often and not just here.

Anything with a voting system will be able to be manipulated by someone with enough time to figure it out.

Personally, scores are nothing more than a marker of hitting the right note with readers or not. If someone reads the first line of my work and hates it, it's their right to give it a 1 and move on. If they open it because they hated the title and tagline and wanted to give it a 1 for that reason alone, it's their right. If they see tags that set them off in a bad way, they are perfectly allowed to vote a 1 based entirely on that. Hell, they can vote a 1 because they don't like my username! Their vote is theirs and whatever reasoning they have for it is on them.

Probably the least manipulatable metric for top lists would be view to vote ratio. A high ratio on that would mean people interacted because the story hit something right or wrong for them. Add in the comment # (and no longer including author comments in total or recent activity metrics) for ties and you have a three point system that would require incredible effort to manipulate. Still doable, but less so because it's not based on a single set of raw numbers, but different elements working together and it adds a ton of effort to attempt to manipulate.

Hell, to make it less able to be manipulated, you could add in a time frame for the lists as they are currently broken down into different time periods. You could make a story not even eligible for the all time list until it's been on the site for x years to cut down on the whining of current writers if you wanted to be passive aggressive about it.

But none of that is going to happen. The readers here like the system as it is. They like being able to lift and lower works as they see fit. For some of them, it's their entire purpose here. It's incredibly sad, but once we put a story out there it's out of our hands.

Do I wish there was a system in place to prevent or minimize fraudulent votes? Yeah, not just for top lists, but for everything. I also wish series had their own lists because I prefer one shots and trying to find something to read by using the top lists means wading through a hundred chapters. I've also found that many stories on the top lists simply aren't all that good (with some exceptions) and seem to only be on the list due to how long they've been around to accumulate the number of votes they have.

Ultimately, top lists are curated by readers for readers, whatever their methodology or intent behind that curation is up to them. The site has approved said method by not intervening in the 25+ years it's been live. They know the manipulation tactics, they are aware of the problem, they don't see it as egregious enough to change for whatever reason. I'm not going to pretend to know why, but my guess would be "Not a hindrance to their intended purpose, not a vital failure. They function, that's good enough."

For what it's worth, I've seen my scores drop (anywhere from 2-5 votes will come in on low vote total stories that haven't seen a new vote in months, it's kinda blatant and I don't even watch my scores or track my stats.) after a couple of other specific writers complain about trolling on their stories. And when they have a story in a competition alongside mine, which is hilarious because I'm not a threat in that regard *at all*. I have a strong suspicion of who is doing this because they also did it to my previous account. I, admittedly, baited them a few years ago to see if they would, and they did.

They vote negatively as a protective measure toward a few other writers who, I would wager, have no idea they are doing this and wouldn't approve if they knew. Every time I see it happen, I get amused. I'm quite positive they think I down vote the authors they are "protecting" but I've literally only ever given a single one to a story here. I don't even know if I've given anything under a 4 to be honest. I don't have time to hate read anything to vote lower. I don't even have time to read all of the stuff I want to read. Maybe once I'm retired, like they are, I'll have as much time on my hands as they do, but I'm probably not gonna spend my retirement trolling younger writers who won't flirt with me in favor of the ones who do.

Trolls exist. Assholes exist. We can get upset by them being a minor inconvenience to us, or we can be amused by the tremendous amount of effort they put into their antics for such a meaningless thing. I choose the latter, because the former is just ... Sad. I have better things to focus my time and attention on. Like writing and reading. Which I should be doing now.

 

[StillStunned]

I'll confess here that I've done my share of griping over the troll-curated Top Lists. So I understand where @RocketGrunt is coming from, and I do sympathise with them.

I'll chalk up my changed attitude now to being tired of seeing the issue rehashed again and again recently (there's nothing like hearing your own complaints coming from someone else to show you how futile they are), mixed with some more cynicism towards Lit, its readers and its writers.

 

[RocketGrunt]

Someone should take the time to graph out the performance of a story of theirs over a one, two or three year period. I'm pretty sure that it would show that over time, the rating of a story finds its level spot and will deviate little from that point.

Loyal followers have had their one vote, trolls have lost interest, hurt feelings get forgotten, sweeps come and go, and life moves on.

This is not what happens anymore.

Stories used to achieve an equilibrium value and become more and more thoroughly whatever that rating was. The inertia of a piece would be higher with 500 votes than with 100 votes, higher still at a thousand. Eventually reaching the thousands of votes where a few high or low votes wouldn't move the needle even .01 in either direction. The ratio of 5* ratings to 4* and lower ratings would be pretty constant over time, which would iron out the effects of any transient campaign to raise or lower the score.

And I say "used to" because that's not how it is anymore. Because what happens now is that someone has a trolling script that whack-a-moles any story which gets onto the toplists with a rating above the target. This creates a plateau where everything on every board is hobbled automatically if it rises above the target rating. It is now normal for the toplist in every category to have over a hundred stories with the same rating. On the Science Fiction and Fantasy toplist, 243 out of 250 stories have the exact same rating because only 7 stories are allowed to have a rating higher than that.

 

[WhiteTailDarkTip]

Here's the thing:

The readers can vote however they want, even if it's trolling, which is unfortunate, but part of life. Some people have enough time on their hands to make it their "responsibility" to protect their favorite authors or stories on those lists. It's brutal, but it happens often and not just here.

Anything with a voting system will be able to be manipulated by someone with enough time to figure it out.

Personally, scores are nothing more than a marker of hitting the right note with readers or not. If someone reads the first line of my work and hates it, it's their right to give it a 1 and move on. If they open it because they hated the title and tagline and wanted to give it a 1 for that reason alone, it's their right. If they see tags that set them off in a bad way, they are perfectly allowed to vote a 1 based entirely on that. Hel, they can vote a 1 because they don't like my username! Their vote is theirs and whatever reasoning they have for it is on them.

Probably the least manipulatable metric for top lists would be view to vote ratio. A high ratio on that would mean people interacted because the story hit something right or wrong for them. Add in the comment # for ties and you have a three point system that would require incredible effort to manipulate. Still doable, but less so because it's not based on a single set of raw numbers, but different elements working together and it adds a ton of effort to attempts manipulate.

Hell, to make it less able to be manipulated, you could add in a time frame for the lists as they are currently broken down into different time periods. You could make a story not even eligible for the all time list until it's been on the site for x years to cut down on the whining of current writers if you wanted to be passive aggressive about it.

But none of that is going to happen. The readers here like the system as it is. They like being able to lift and lower works as they see fit. For some of them, it's their entire purpose here. It's incredibly sad, but once we put a story out there it's out of our hands.

Do I wish there was a system in place to prevent or minimize fraudulent votes? Yeah, not just for top lists, but for everything. I also wish series had their own lists because I prefer one shits and trying to find something to read by using the top lists means wading through a hundred chapters. I've also found that many stories on the top lists simply aren't all that good (with some exceptions) and seem to only be on the list due to how long they've been around to accumulate the number of votes they have.

Ultimately, top lists are curated by readers for readers, whatever their methodology or intent behind that curation is up to them. The site has approved said method by not intervening in the nearly 25+ years it's been live. They know the manipulation tactics, they are aware of the problem, they don't see it as egregious enough to change for whatever reason. I'm not going to pretend to know why, but my guess would be "Not a hindrance to their intended purpose, not a vital failure. They function, that's good enough."

For what it's worth, I've seen my scores drop (anywhere from 2-5 votes will come in on low vote total stories that haven't seen a new vote in months, it's kinda blatant and I don't even watch my scores or track my stats.) after a couple of other specific writers complain about trolling on their stories. And when they have a story in a competition alongside mine, which is hilarious because I'm not a threat in that regard *at all*. I have a strong suspicion of who is doing this because they also did it to my previous account. I, admittedly, baited them a few years ago to see if they would, and they did.

They vote negatively as a protective measure toward a few other writers who, I would wager, have no idea they are doing this and wouldn't approve if they knew. Every time I see it happen, I get amused. I'm quite positive they think I down vote the authors they are "protecting" but I've literally only ever given a single one to a story here. I don't even know if I've given anything under a 4 to be honest. I don't have time to hate read anything to vote lower. I don't even have time to read all of the stuff I want to read. Maybe once I'm retire, like they are, I'll have as much time on my hands as they do, but I'm probably not gonna spend my retirement trolling younger writers who won't flirt with me in favor of the ones who do.

Trolls exist. Assholes exist. We can get upset by them being a minor inconvenience to us, or we can be amused by the tremendous amount of effort they put into their antics for such a meaningless thing. I choose the latter, because the former is just ... Sad. I have better things to focus my time and attention on. Like writing and reading. Which I should be doing now.

You’re also supposed to be staying off the forums. You said it, not me …

 

[Erozetta]

You’re also supposed to be staying off the forums. You said it, not me …

.


Yeah, I have a story to read...

 

[Bazzle]

My view on this is that someone likes to have the list the same way every day.

Whether it is through being nasty or because of spectral issues.

When a new story arrives and gets over the 100 votes, and disturbs his list. He or she down votes it. Curating the list as they see it.

In E&V anything greater than 4.8 is voted down. I have had it, @StillStunned has had it, and I noticed @Djmac1031 has had it with his latest extremely good story.

The piece of work sits at 4.8x until it creeps over 100 votes and then down it goes. Someone doesn't want the list changed.

 

[WhiteTailDarkTip]

I'll confess here that I've done my share of griping over the troll-curated Top Lists. So I understand where @RocketGrunt is coming from, and I do sympathise with them.

I'll chalk up my changed attitude now to being tired of seeing the issue rehashed again and again recently (there's nothing like hearing your own complaints coming from someone else to show you how futile they are), mixed with some more cynicism towards Lit, its readers and its writers.

I’m kinda with you on this. One of my stories went up and gradually accumulated good ratings right up to the point where it suddenly popped up in the GOAT rankings. I didn’t know what was going on to begin with because my views spiked. Then over the course of a couple of weeks, enough 2* and 3* votes were lobbed in with metronomic regularity to push me sub level. If this had happened earlier in my career I’d have been crushed, but I’ve been at this lark for about five years and I’m much more confident in my abilities. If Laurel and Manu won’t adopt a system such as Liz describes then we just have to live with it as it is and ignore the gameplaying

 

[pink_silk_glove]

Pink Silk, you've already established beyond doubt that you are lazy and dishonest. That you are totally unprepared for any discussion that involves numbers of any kind, yet happy to pretend that the results of simply arithmetic are somehow in doubt so long as it affords you the opportunity to mouth off. I don't care what your opinion is on this or any other issue. Please vacate my thread.

I'm only trying to help you. You will have a much more stress-free experience on lit if just stop giving a fuck about the top lists. It is impossible to come up with a scoring system that is remotely accurate to award top scores on a free porn site with hundreds of thousands of people clicking while touching themselves. Improvements can be made but it;s pissing into the wind. Trolls will always win on a site like this. Siskel and Ebert aren't rating your story, so who the fuck cares about a toplist?

Am I lazy? Well, I certainly don't put in the effort to chase dead argument stances as you do. : P But I am hardly dishonest. The very reason that people (like you) get pissed at me is because I'm not afraid to speak blunt truth - truth that spoils little fantasy worlds like the one that you have built for yourself where you are somehow entitled to hit the top of the board. Why don't you start your own site and show us just how easy it is to make it troll free. Countless sites have branched off and started with the ambition of being more 'author friendly' and troll free. They all fizzle because snobby authors who deserve to write troll free are generally not very reader friendly and a site like this is dead without reader traffic.

All of these wanky convoluted ideas to 'fix' the scoring system have nothing to do with scoring accuracy. They have everything to do with self-serving writers who believe themselves to be unworthy of trolls, so that their works can get the 4.99s that they deserve. I don;t give a fuck about that and you shouldn't either, unless you really are the snob that you sound and act like.

You are not entitled to the top of the board. You are not entitled to a fair shot at the top of the board. You are not entitled to win a contest. You are not entitled to a fair shot at winning a contest. You are not entitled to Red Hs. You are not entitled to a fair shot at Red Hs. The top of the board, contest wins nor even Red Hs mean that you are an awesome writer anyways. There are tons of total shit stories that have achieved all of these things. Get over it. You will enjoy your time here much more.

 

[RocketGrunt]

I'll confess here that I've done my share of griping over the troll-curated Top Lists. So I understand where @RocketGrunt is coming from, and I do sympathise with them.

I'll chalk up my changed attitude now to being tired of seeing the issue rehashed again and again recently (there's nothing like hearing your own complaints coming from someone else to show you how futile they are), mixed with some more cynicism towards Lit, its readers and its writers.

The thing is that defeatism isn't warranted. It hasn't "always been like this." Stories had the kind of dynamic distribution you'd expect... in January of 2025. The arms race between the site's trolls and the system of vote sweeps shifted in favor of the trolls about twelve months ago.

In November, the site did a radical overhaul of the site's presentation, and while I am in the camp of people who say that that update was a "fucking disaster" it nonetheless shows that Laurel and Manu are working to change things. One of the things that the new site outlook promises is a different way of presenting "most popular works" from a category. A way that has three tabs: 7-day popular, 30-day popular, and all-time popular. Now those tabs don't work properly, because that update was a fucking disaster and all, but it definitely shows that they are thinking about this exact problem.

There are, in fact, ways to fix the lists so that the trolls don't win. One way would be to whack-a-mole the trolls themselves. There almost certainly aren't that many people curating the lists, and they only came up with a working system in the last year. If the sweeps could be made to find and excise the ratings of those bad actors, the lists would go back to presenting something that was defensible as something you might want to present. Another way would be to go scorched earth, and simply show stories by number of high votes given within the defined timeframe (weekly, monthly, or all-time).

I favor the latter one for two reasons. One, I do not believe that the sweeps are going to be able to win against the trolls. The curation trolls have held the upper hand for twelve months and they can update their python scripts faster than the Manu can change the sweep code. And secondly, a "fair" average rating is still going to favor niche fandoms, and that doesn't seem like the kind of thing you want filling up the toplists you present to new readers. That is to say, a high average isn't maintained by having a lot of people rate something highly, but by having few people rate something low. That means that a story that is only read by people who like it is destined for a higher rating than something that is read by a lot of people.

To see that effect in action: the broken "most popular" tabs on the scifi/fantasy category show you stories that aren't on the toplist but do get an average rating of 5*. Those are stories that don't have one hundred ratings and thus haven't been exposed to the toplist vandalism script. They haven't gotten any downvotes because they only get read by fans. The #1 and #2 by that metric are chapters 70 and 71 of a World of Warcraft fanfiction. And while a rating cutoff of 100 votes would keep those ones out specifically, the kinds of stories that are going to be over 4.9 and over any arbitrary number of votes are going to be conceptually like that - stories whose titles ensure that only dedicated fans are going to read, but ones with enough fans and followers to get over the arbitrary threshold.

 

[Erozetta]

One thing I haven't seen mentioned around this is:

If the change has only happened in the past year, it would point to a targeting of the weak points within the system to prove a point rather than reader curation.

Which brings up the question of who has been the most vocal about the problem as it was and do they gain anything by making the problem bigger than it was previously?

Do you really want to go down a path that may ultimately start conspiracy theories and finger pointing toward people who may have nothing to do with the issue at all, but look guilty because of the coincidences you're drawing attention to?

 

[StillStunned]

If the change has only happened in the past year, it would point to a targeting of the weak points within the system to prove a point rather than reader curation.

My E&V stories were being bombed off the Top List long before that.

 

[pink_silk_glove]

There are, in fact, ways to fix the lists so that the trolls don't win.

No there aren't. At least not without decimating reader engagement and feedback. Certainly, trolling can be mitigated to a degree, but never significantly. There is just too much trolling going on. The only way to stop it is to put every click, every vote, every comment through a gauntlet of security and captcha bullshit which will decimate feedback. This site lives because of its traffic. The traffic is there because they come here for free. The writers contribute because of levels of feedback and engagement that aren't found anywhere else due to that traffic. If the writers can't get their feedback because a clique of cry-babies are afraid of trolls and a gauntlet of protection is set up to kill the feedback, they stop submitting. That would be wet dream time for you because it would pave your way to the top of the board. But then the traffic would leave because instead of 200 stories posted per day there are only 20.

Go to lush where they have one-tenth of the traffic but they protect the precious writers from trolls. Anti-troll and low traffic/feedback go hand in hand.

 

[TheLobster]

I favor the latter one for two reasons. One, I do not believe that the sweeps are going to be able to win against the trolls. The curation trolls have held the upper hand for twelve months and they can update their python scripts faster than the Manu can change the sweep code. And secondly, a "fair" average rating is still going to favor niche fandoms, and that doesn't seem like the kind of thing you want filling up the toplists you present to new readers. That is to say, a high average isn't maintained by having a lot of people rate something highly, but by having few people rate something low. That means that a story that is only read by people who like it is destined for a higher rating than something that is read by a lot of people.

The third good reason is that unlike average ratings, which show increased resilience to trolls the older the story is (and therefore disadvantage newer stories), the raw number of 5*'s can be made fair(er) across stories of all ages. The all-time toplist can can still count every single vote of that 20-year-old story, but after scaling the result against this long span of time the story can still be fairly positioned among those published only a few months ago.

This is not the case now, because even the newer ones who manage to climb to 4.85 or whatever it's needed, against all odds and trolls, will then be placed within the same 0.01 strata and thus digned because of their low(er) number of votes overall. Some toplists are very tight (0.05 difference or less), so the sheer number of votes becomes important when that 0.01 interval is like 50 sequential ranks on the list.

 

[RocketGrunt]

My E&V stories were being bombed off the Top List long before that.

I'm sorry to hear that.
However, there's multiple kinds of vote bombing. Individual targeted vote bombing has obviously been a thing for years. People get fans who read their stuff when it comes out and give 5* ratings before even finishing the story - but they also get anti-fans that go through their catalog and 1* everything. Sometimes the sweeps pick these up as inauthentic and sometimes they don't. Haters gonna hate, and they've been with us for decades. Similarly, contests whip up large numbers of fake votes in both directions and the sweeps always purge a lot of votes around the contests.

The new thing is the mass flattening of the toplist contenders. 243 of 250 stories on a toplist having the same rating is new. And it's utterly insane. That's not the result of someone hating on an author, that's the result of a script that is evading the vote sweeps. And because it's obviously a script, it could literally be one person. And because it's a script, it doesn't unfairly target one person, it unfairly targets everyone.

 

[pink_silk_glove]

That means that a story that is only read by people who like it is destined for a higher rating than something that is read by a lot of people.

Not if it hits 4.8 as you claim. In fact, the fewer the votes, the easier it is to knock down. This is your own theory at work here.