I see my score is 4.18

[pink_silk_glove]

As a pragmatic question for the category specific H levels, does the percentile ever get recalculated?

Absolutely, the percentiles must recalculate whenever scores change. I would recommend daily.

 

[TheLobster]

The more they denigrate those "raw" scores, the more adequate and fitting it is to say about the proponents of stack ranking that they are fetishizing the categories.

It's basically magic. You can take those allegedly very flawed, totally imperfect, almost random scores; and the moment you line them up along the more or less arbitrary lines -- which people with the vested interest, i.e., authors, choose -- those "useless" scores suddenly transform into a great measure of story quality. Just like that!

Indeed, the magical effect of category division is so potent that it even allows you to compare stories across categories!

Excuse my exasperation but this is frankly ridiculous. There are so many tangential, systemic factors that influence scores that it's preposterous to assume a coarse-grained filter like stack ranking can meaningfully discern. Some of those factors, funnily enough, are readily admitted to exist by proponents of SR, such as the fact that readers rate stories by their kink appeal rather than quality.

But hey, apparently percentage buckets deal with all that!... Yeah, no.

 

[StillStunned]

I can't be arsed to go through everyone's maths, but if the Red H was linked to percentiles, we can all guess what will happen. People will start loading their own stories with 5s, bombing other stories that come close, and generally becoming even pettier than they are now.

 

[pink_silk_glove]

The more they denigrate those "raw" scores, the more adequate and fitting it is to say about the proponents of stack ranking that they are fetishizing the categories.

Not sure what you mean by fetishize, but if it is what I think you mean then no, it's actually the opposite. Category percentiles level out the categories. As it is now, certain categories (Romance, Fetish) are super easy to score highly in while others (LW, BDSM) are more difficult. I don't know if fetishize is the right word, but it certainly makes writing in these certain categories more attractive to get a higher score. By using percentiles within each category and basing Hot levels on those percentiles, the enticement to each category becomes more even.

You can take those allegedly very flawed, totally imperfect, almost random scores; and the moment you line them up along the more or less arbitrary lines

What could be more arbitrary than 4.5? The number was chosen largely out of thin air thinking that it would represent the top echelon of scores, yet in practice it has just inflated scores and promotes 45% of all stories (!!) as the cream of the crop. The 4.5 bar has been a huge flop in every way.

Indeed, the magical effect of category division is so potent that it even allows you to compare stories across categories!

Please explain this. It does not make any sense.

Excuse my exasperation but this is frankly ridiculous. There are so many tangential, systemic factors that influence scores that it's preposterous to assume a coarse-grained filter like stack ranking can meaningfully discern. Some of those factors, funnily enough, are readily admitted to exist by proponents of SR, such as the fact that readers rate stories by their kink appeal rather than quality.

Absolutely they rank by kink far more than by quality. That's why it's so much easier to score well in Fetish than in EC for example. No one is arguing that a percentile ranking will determine quality of a story on lit. Nothing will ever isolate quality in the scores of lit stories. But at least we can equate the reaction of readers to stories across categories, which we cannot do with any accuracy at all as it is right now. That would be a huge plus. You say coarse-grained. What could be more coarse than an arbitrary 4.5?

 

[pink_silk_glove]

I can't be arsed to go through everyone's maths, but if the Red H was linked to percentiles, we can all guess what will happen. People will start loading their own stories with 5s, bombing other stories that come close, and generally becoming even pettier than they are now.

Things can't get much pettier, so I doubt that that behavior would make much impact. No system will every stop contest bombs and multi-account voting which is already rampant. Not much room to get more rampant, so let them try.

 

[SimonDoom]

I can't be arsed to go through everyone's maths, but if the Red H was linked to percentiles, we can all guess what will happen. People will start loading their own stories with 5s, bombing other stories that come close, and generally becoming even pettier than they are now.

This is pure speculation. It doesn't make sense to me. Some people will do this whatever the system, but it seems to me they are much more likely to do it with the system that we have now than a percentile system. If you know that a red H is tied to a percentile, say 75%, then it loses some of its talismanic power. Knock it down to 74%, and readers still know it's so close to 75% and can make their decisions. But with the system we have now, and judging by the comments, people obviously have no idea what 4.5 means. They have no idea that in some cases it's a sub-median score, and in some cases it's a 90 percentile score. A percentile ranking isn't perfect, obviously, but it conveys more useful information than what we've got now, and that's the point of a numbering system.

 

[old_prof]

I don't know why it would be much more difficult to constantly update percentile rankings than to constantly update other data.

Most of the data they use is strictly local. Percentiles are fundamentally global, meaning it needs to scan an entire category for each display. More likely to cache the answer and recalculate it periodically.

In other words, it is MUCH more difficult.

 

[BlazonG]

From my perspective as someone new that barely knows writing the red H is somewhat helpful. Of course those with established followings or at chapter 15 of their story with consistent readership are going to immediately score over 4.5 when they post.

But, when I score over 4.5 with a standalone story and no followers at least that tells me I’m not totally stupid. The red H is a nice motivator to get better.

 

[pink_silk_glove]

Most of the data they use is strictly local. Percentiles are fundamentally global, meaning it needs to scan an entire category for each display. More likely to cache the answer and recalculate it periodically.

In other words, it is MUCH more difficult.

No, not much more difficult, very slightly more difficult. All that lit has to do is stop once per day and take a minute to recalculate everyone. In fact the site probably already stops a couple times per day to reshuffle the top lists anyways, so this would be pretty much nothing.

Recalculating percentiles of scores would be less server strain and far less manual admin (read: none) than making 200 new story submissions go live every night.

 

[Actingup]

No, not much more difficult, very slightly more difficult. All that lit has to do is stop once per day and take a minute to recalculate everyone. In fact the site probably already stops a couple times per day to reshuffle the top lists anyways, so this would be pretty much nothing.

Recalculating percentiles of scores would be less server strain and far less manual admin (read: none) than making 200 new story submissions go live every night.

Once again, I’m agreeing with PSG. The graph in the discussion above took me some hours, but that was because I had no access to the database. A good coder (not me) with decent statistical knowledge could produce a whole range of measures, including daily percentile ranking per category, in a flash.

I’m treating this discussion as a thought experiment though. I wouldn’t anticipate actual change.

Reasons:
- percentile scores do nothing about the 1-bombing or 5- fluffing problem, other than highlight it. Many of our 2024 stories will have risen significantly in score since that snapshot, due the sweeps that attempt to correct score manipulation. The score manipulation behaviour is a disincentive for ‘honest’ authors to post stories, and I’m not sure that the Admin team would want to highlight it further.
- seeing how you actually rank (rather than ‘everything is awesome’ Red H) may also cause waves of consternation amongst authors. It wouldn’t necessarily sail through in a democratic process.
- there is no democratic process
- inertia. The site is managed by (wonderful) people with long ‘to do’ lists and particular skill sets.

That said, it’s a an interesting discussion.

 

[Voboy]

I can't be arsed to go through everyone's maths, but if the Red H was linked to percentiles, we can all guess what will happen. People will start loading their own stories with 5s, bombing other stories that come close, and generally becoming even pettier than they are now.

I wouldn't "link the Red H to percentiles," largely because I know the site will never change the Red H system.

But it would be helpful if some math weenie recalculated the percentiles periodically, so that folks trying to compare the numbers could gain another data point about the relative quality of their story in categories where the Red H is functionally meaningless.

I am not that math weenie, but PSG has made the methodology make sense to me. As an additional way for us to think about stories, at the very least.

 

[Writer61]

I prefer that my story be judged based on scores received (4.5/5) rather than in relation to others (90th percentile).

It feels wrong that a story could fall below 'H' because of somebody's view of an entirely different story.

 

[SimonDoom]

I’m treating this discussion as a thought experiment though. I wouldn’t anticipate actual change.



That said, it’s a an interesting discussion.

My attitude. I have more than my share of "If only this place acted the way Simon wants it to" posts, but I don't get too invested in them. I like the idea of change, but I don't expect it, and my reasons for not expecting it are perfectly benign.

 

[crookedletter]

I find the H a little misleading. I guess this being an erotica site, "HOT" makes sense as just a story most of its readers liked. But when mine first grabbed that label I assumed it meant it was "hot" in the sense that it was getting a lot of views/ratings quickly. Then over time it would inevitably "cool" and lose the H. Here it is nearly a year later, having thoroughly cooled off as far as engagement, but that little H is still sitting there as bright red as ever.

Not that I mind, as apparently it's a kind of badge of honor, but as a relatively new user it's not what I would expect.

 

[old_prof]

No, not much more difficult, very slightly more difficult. All that lit has to do is stop once per day and take a minute to recalculate everyone. In fact the site probably already stops a couple times per day to reshuffle the top lists anyways, so this would be pretty much nothing.

Recalculating percentiles of scores would be less server strain and far less manual admin (read: none) than making 200 new story submissions go live every night.

I am not going to get into it about who is better equipped to evaluate this. I will make some comments, though.

I fully agree that the task of approving the incoming stories is massive. But this is complelely irrelevant here. This does not change that and uses a completely different resource.

It is certainly not outrageously expensive IF YOU CACHE, which you are suggesting (unlike I think Simon was), but caching is notoriously tricky to get right for all conditions. Development mindset is another precious resource I am guessing. This is notably harder, both in development effort and server time, than is an open request from author's for a histogram of vote totals.

If you do not cache, the easiest implementation is to recalculate the percentile for a group every time any vote is cast by any reader. , which would be tens of thousands of times more expensive on the server than simply adding the vote total.

 

[Actingup]

I am not going to get into it about who is better equipped to evaluate this. I will make some comments, though.

I fully agree that the task of approving the incoming stories is massive. But this is complelely irrelevant here. This does not change that and uses a completely different resource.

It is certainly not outrageously expensive IF YOU CACHE, which you are suggesting (unlike I think Simon was), but caching is notoriously tricky to get right for all conditions. Development mindset is another precious resource I am guessing. This is notably harder, both in development effort and server time, than is an open request from author's for a histogram of vote totals.

If you do not cache, the easiest implementation is to recalculate the percentile for a group every time any vote is cast by any reader. , which would be tens of thousands of times more expensive on the server than simply adding the vote total.

If it was done as an author's resource rather than as part of a ranking system, what you could do is regularly (daily or monthly) construct a percentile chart of some kind using an automated version of what I did in the other thread , and then just write a little 'how does my story compare?' button that displays your story's performance against the most appropriate comparison graph. This could then link to some of Literotica's FAQ/Help resources around 'tips to help make your writing even better' - articles about grammar, use of beta readers, respect for the readers, respect for your characters, writing techniques, and so on. Basically, you could use the process to help stimulate continuous improvement rather than shallow 'hot or not' comparisons.

 

[pink_silk_glove]

I wouldn't "link the Red H to percentiles," largely because I know the site will never change the Red H system.

But it would be helpful if some math weenie recalculated the percentiles periodically, so that folks trying to compare the numbers could gain another data point about the relative quality of their story in categories where the Red H is functionally meaningless.

I am not that math weenie, but PSG has made the methodology make sense to me. As an additional way for us to think about stories, at the very least.

To your first point, if you were in charge, replacing the folks who won't change it, you would change it. ; )

To your second, if one of us manually recalculated percentiles, it would be nice for the few of us who care, but it won't change how the readership search the stories and choose what to read.

Think about it, a reader looks at your profile and sees your Romance story with a Red H on it and your LW story with no Red H on it and decides to read the Romance because it's rated HOT! But let's look at the reality. The LW crowd is difficult to please because it is so fractured and the different factions are terribly judgemental. 4.5 in LW is probably the 80th percentile. On the other hand, the Romance crowd is easy to pander to and so long as you tick the boxes correctly you get an easy A. 4.5 in Romance is probably the 40th percentile! (have you seen some of the garbage that scores 4.8 in Romance??) Your LW story scored 4.22 which is around the 70th percentile, say, while your Romance scores a 4.62 which is roughly the 50th percentile, say. Your LW story is actually better received in its category than your Romance story is, yet the Romance gets the Red H and draws more attention to readers who search by profile.

 

[pink_silk_glove]

It is certainly not outrageously expensive IF YOU CACHE, which you are suggesting (unlike I think Simon was), but caching is notoriously tricky to get right for all conditions. Development mindset is another precious resource I am guessing. This is notably harder, both in development effort and server time, than is an open request from author's for a histogram of vote totals.

If you do not cache, the easiest implementation is to recalculate the percentile for a group every time any vote is cast by any reader. , which would be tens of thousands of times more expensive on the server than simply adding the vote total.

Of course we don't recalculate on every vote. That would be silly. Just update once per day. The site already does this, we can tell because depending which page you look at (your own profile, the new list, the category list, etc) the scores update at different times.

This is not tricky at all. There is no development mindset and no precious resources. All of the data necessary is already there. You spend an hour or two writing up some code, test it out and stick it in. The extra server time is tiny - a blink - compared to what the server already does.

 

[pink_silk_glove]

I prefer that my story be judged based on scores received (4.5/5) rather than in relation to others (90th percentile).

It feels wrong that a story could fall below 'H' because of somebody's view of an entirely different story.

It's not about your story's Red tag. It's about making the Red tag meaningful for readers to search for something that they like. Right now, it's not.

If you want it to be about you and your stories, by all means enjoy your meaningless Red H.

 

[BlazonG]

If you want it to be about you and your stories, by all means enjoy your meaningless Red H.

PSG! That’s so mean. Maybe you should earn one before you talk so bad about them.

 

[Voboy]

To your first point, if you were in charge, replacing the folks who won't change it, you would change it. ; )

Maybe? Maybe not? It's not something I think seriously about, because I don't have the mindset to be in charge of a site like this. Honestly, I'm not sure how I'd do it. I like to think I'd try to solicit opinions before pulling any triggers, but that's probably bullshit. I suspect I'd end up pissing off droves of people, no matter what I tried.

The status quo has one benefit: simplicity. It has many detriments.

To your second, if one of us manually recalculated percentiles, it would be nice for the few of us who care, but it won't change how the readership search the stories and choose what to read.

Think about it, a reader looks at your profile and sees your Romance story with a Red H on it and your LW story with no Red H on it and decides to read the Romance because it's rated HOT! But let's look at the reality. The LW crowd is difficult to please because it is so fractured and the different factions are terribly judgemental. 4.5 in LW is probably the 80th percentile. On the other hand, the Romance crowd is easy to pander to and so long as you tick the boxes correctly you get an easy A. 4.5 in Romance is probably the 40th percentile! (have you seen some of the garbage that scores 4.8 in Romance??) Your LW story scored 4.22 which is around the 70th percentile, say, while your Romance scores a 4.62 which is roughly the 50th percentile, say. Your LW story is actually better received in its category than your Romance story is, yet the Romance gets the Red H and draws more attention to readers who search by profile.

Fair. "The few of us who care" is why I don't think this should be a sitewide change. I would see it as a thread here in the AH, reporting periodic percentiles.

Of course, that's easy for me to say. I lack the brainpower and tech savvy to help with that.

 

[Actingup]

4.5 in Romance is probably the 40th percentile! (have you seen some of the garbage that scores 4.8 in Romance??) Your LW story scored 4.22 which is around the 70th percentile, say, while your Romance scores a 4.62 which is roughly the 50th percentile, say. Your LW story is actually better received in its category than your Romance story is, yet the Romance gets the Red H and draws more attention to readers who search by profile.

Goddammit, don't invalidate my 4.8s in Romance!!! I'm very proud of those!

However, this is partly why I found it personally liberating to look at the percentiles for my own Loving Wives stories and think about them in terms of how they could have connected more and improved their relative standing rather what raw score they got. The other aspect was just the process of digesting the raw numbers and understanding that with nearly 700,000 stories on the site, our efforts aren't around being *the* best, appearing on a top list or winning a competition, but just being better and learning more.

 

[ElectricBlue]

I find the H a little misleading. I guess this being an erotica site, "HOT" makes sense as just a story most of its readers liked. But when mine first grabbed that label I assumed it meant it was "hot" in the sense that it was getting a lot of views/ratings quickly.

It's the average at any point in time, but not rated to acceleration. Straight average, not delta average.

 

[pink_silk_glove]

Fair. "The few of us who care" is why I don't think this should be a sitewide change. I would see it as a thread here in the AH, reporting periodic percentiles.

I meant the few of us who care about an an accurate rating. The vast majority of writers here care deeply about their ratings (some of us even more than our stories themselves) but very very few care about an accurate rating. They only want a higher rating.

The great benefit of a more accurate rating system is that it helps the readers find the stories that they want more quickly.

 

[Voboy]

The great benefit of a more accurate rating system is that it helps the readers find the stories that they want more quickly.

To find what they want most quickly, all they usually have to do is unzip. Lol.

I see your point. As noted by all of us, it'll never be implemented. But it's a good point.