Rating and Raters

[ElectricBlue]

How did we get from 9620 to 9640 in the 5 example?

Edit: Sorry I see Bramblethorn already caught that.

Oops - transcribed my scribble calcs incorrectly...
Should be:

Answer = 6 and 30

4.37 x 2200 = 9614
9614 + (6 x 1) = 9620
9620/2206 = 4.36

9620 + (30 x 5) = 9770
9770/2236= 4.37

 

[8letters]

I think you have an error there? Otherwise not sure how you're getting from 9620 to 9640.

For a story sitting at 4.37, it should take about 5.35 five-star votes to cancel out a single one-star. The exact formula is (R-1)/(5-R).

This ignores rounding; a story that displays at "4.37" could be anywhere between 4.365 and 4.374999, so if you want to get precise you need to look at different possibilities over that range.

And that's the problem. If you story is at 4.365, a single 4 vote will knock it do to 4.36. And I think that LitE never adjusts the rating for a single vote. The less votes you have, the more volatile your rating will be. And then sweeps adds even more randomness to your rating. Focusing on day-to-day changes will drive you crazy.

 

[NotWise]

And that's the problem. If you story is at 4.365, a single 4 vote will knock it do to 4.36. And I think that LitE never adjusts the rating for a single vote.

I've monitored votes for years now, and I think Lit adjusts the rating on every vote, but not every vote changes the rating.

The less votes you have, the more volatile your rating will be. And then sweeps adds even more randomness to your rating. Focusing on day-to-day changes will drive you crazy.

The early volatility of the rating is one of the things that encourage people to think they're being trolled, but it's really just inherent in having a small number of votes.

 

[ElectricBlue]

I've monitored votes for years now, and I think Lit adjusts the rating on every vote, but not every vote changes the rating.

I'd guess the maths is done to many decimal places (like any digital calculation) as soon as a change happens - we're just seeing the score rounded to two decimal places. It's fine data but a coarse display.

 

[MichaelGH]

Yeah I was wondering if there’s a better rating system because a 1.0, 2.0 or 3.0 carry so much more weight than a 4.0 and 5.0.

But I guess at the end of the day we’re all subject to the same rating system. So there’s a certain fairness in that.

 

[gordo12]

Yeah I was wondering if there’s a better rating system because a 1.0, 2.0 or 3.0 carry so much more weight than a 4.0 and 5.0.

But I guess at the end of the day we’re all subject to the same rating system. So there’s a certain fairness in that.

They certainly can. I'd like to see a rating system that is more friendly to readers. Starting @3, 3.5, 4, 4.5, 5.

When people read a good story and it's above 4 it's hard to hit that 5 button because the story probably isn't perfect. 5 says it is. A lot vote 4 then and drive the score down. It's actually a downvote on the story. 3.5 and 4.5 would give the readers a little more flexibility.

 

[RejectReality]

The reality is that most people who read won't vote. Those who do tend to be on the polar ends of the scale. It's either a five or a one. Even given an option for more nuanced scoring, most won't take advantage of it. I've seen it play out on another site, and they ended up rolling things back, because it didn't change voting patterns.

As SR has been suggesting for years, the best thing that could happen for both readers looking for stories and author anxiety is either to add a second H level from 4.0 to 4.49, or eliminate the Hs entirely.

Now that the scores are available everywhere, getting rid of the bright red "Look at me!" would cause people to not skip over 4.49 stories as much as they do now, leading them to a lot of good work that's just under the radar.

It would also eliminate that easy target for trolls. That H barrier is an easy mark when they want to hurt an author. The more they have to blast a score, the more traces they leave, and thus the more likely they are to have their votes scrubbed the next time the Hoover comes around. Not having that easy target would likely force more aggressive vote-bombing to ( temporarily ) achieve their goal.

It could be worse. On one of the sites where I post, a single 4 can be enough to reduce your long-term readership by more than half. The scores are so inflated that nothing short of a perfect 5 gets you anywhere near first page of the Popular tab listings where a great deal of your long-term views comes from.

And that's with me holding #1 spots in at least three different categories, meaning that I pick up a lot of peripheral traffic from those that others aren't getting.

They literally talk about 4 bombs there in the same way we talk about 1 bombs here, with the same 1-5 scale.

 

[MichaelGH]

The reality is that most people who read won't vote. Those who do tend to be on the polar ends of the scale. It's either a five or a one. Even given an option for more nuanced scoring, most won't take advantage of it. I've seen it play out on another site, and they ended up rolling things back, because it didn't change voting patterns.

As SR has been suggesting for years, the best thing that could happen for both readers looking for stories and author anxiety is either to add a second H level from 4.0 to 4.49, or eliminate the Hs entirely.

Now that the scores are available everywhere, getting rid of the bright red "Look at me!" would cause people to not skip over 4.49 stories as much as they do now, leading them to a lot of good work that's just under the radar.

It would also eliminate that easy target for trolls. That H barrier is an easy mark when they want to hurt an author. The more they have to blast a score, the more traces they leave, and thus the more likely they are to have their votes scrubbed the next time the Hoover comes around. Not having that easy target would likely force more aggressive vote-bombing to ( temporarily ) achieve their goal.

It could be worse. On one of the sites where I post, a single 4 can be enough to reduce your long-term readership by more than half. The scores are so inflated that nothing short of a perfect 5 gets you anywhere near first page of the Popular tab listings where a great deal of your long-term views comes from.

And that's with me holding #1 spots in at least three different categories, meaning that I pick up a lot of peripheral traffic from those that others aren't getting.

They literally talk about 4 bombs there in the same way we talk about 1 bombs here, with the same 1-5 scale.

That's the thing I don't like about voting. It's not about how many people love your story, it's about how many people don't like your story.

 

[ElectricBlue]

That's the thing I don't like about voting. It's not about how many people love your story, it's about how many people don't like your story.

It's a spectrum. I can see where readers are rewarding my writing, where they're comme ci, comme ca, and where they clearly don't like it much. On the whole, though, I see more like than dislike, especially when they comment.

 

[Bramblethorn]

I'd guess the maths is done to many decimal places (like any digital calculation) as soon as a change happens - we're just seeing the score rounded to two decimal places. It's fine data but a coarse display.

Oddly, the Halls of Fame rank stories by the rounded scores, with number of votes as the tiebreaker. Seems like it'd be easier and less arbitrary to use the unrounded scores.

 

[SimonDoom]

Yeah I was wondering if there’s a better rating system because a 1.0, 2.0 or 3.0 carry so much more weight than a 4.0 and 5.0.

at.

But that isnt true. It only seems that way because we've become accustomed to relatively high scores. But theres nothing objectively right about high scores. The most common vote is a 5, so theres a lot more love than hate in Lit voting.

 

[TxRad]

Scores are in the eyes of the beholder but what do they mean. A 5 is a perfect 100% score. A 4.5 is a 90% score. A 4 is a 80% score. What is a passing grade these days? A 70 was passing back in mine. That is a score of 3.5.

Write your stories and move along to the next one. The scores will be what the scores are. Learn as you go. Forget the trolls. They are a waste of your time.

 

[Voboy]

I’ve said this before, and it’s a good reality check for me as a writer:

I read here for well over a decade before I posted anything. Before I joined, even. In all that time I read thousands of stories and liked a great many of them. And I never once cast any sort of vote on any of them.

Until I became a writer here, I didn’t care how the votes worked. I’d imagine a lot of readers are the same.

 

[gordo12]

The most common vote is a 5, so theres a lot more love than hate in Lit voting.

LOL, from your lips to LW readers ears. I don't think anyone's told them that!

Scores are in the eyes of the beholder but what do they mean. A 5 is a perfect 100% score. A 4.5 is a 90% score. A 4 is a 80% score. What is a passing grade these days? A 70 was passing back in mine. That is a score of 3.5.

Write your stories and move along to the next one. The scores will be what the scores are. Learn as you go. Forget the trolls. They are a waste of your time.

But is this about trolls? They are a problem, but the dilemma I face all the time is the 4-5 conundrum.

It's a good story. Not good enough for a 5* But a 4* downvotes it.

So do you say screw it and vote the 5? Or take the lesser route and knock the author's work down?

So how are you all handling that. Are you giving the 5* or the 4*?????

 

[SimonDoom]

LOL, from your lips to LW readers ears. I don't think anyone's told them that!

Ha! Gordo, I've had plenty of hard words for a certain clique of LW voters, but in their defense I imagine there are many stories that they do like lot -- probably the kind I don't care for, and my guess is they shower plenty of love on those stories. So even that bunch has some love in its heart.

But is this about trolls? They are a problem, but the dilemma I face all the time is the 4-5 conundrum.

It's a good story. Not good enough for a 5* But a 4* downvotes it.

So do you say screw it and vote the 5? Or take the lesser route and knock the author's work down?

So how are you all handling that. Are you giving the 5* or the 4*?????

As Reject Reality aptly points out, this is a problem created in part by the Red H system. I suspect without that system there would be more "4" votes. I've sometimes advocated converting the present system to a 10 point system where 9 would be a red H, and that way you could have two classes of stories (9s and 10s) that would get red Hs, allowing voters to differentiate between worthy stories and truly exceptional stories. But it's possible that voters would eventually just fall into patterns of gaming that system too.

I think the long term goal should be to move away from the red H system and toward more complex and nuanced search functionality and personalized home pages for Lit members that are tailored to their reading desires. I think that would substantially reduce the incentive for gaming behavior.

 

[Inkhorn]

How I Rate:

No Vote: It just doesn't stand out. My eyes skipped over a couple of paragraphs when it bogged down. I'll probably forget that I read it.

1: How the hell did this get approved? Bad punctuation, bad grammar, badly written, and too confusing to follow. Did this author actually pass elementary school English?

2: I skimmed through the story (or at least the first installment) but couldn't keep going. It's flat. There's no emotion, no pathos. I can't get immersed in the story or care about the characters.

3: Ok, not bad! I got through the whole thing, and I saw some stuff that I genuinely liked. I can see how it can be better, though, and I get a slight itch in my fingers rewrite it in my own way to write my version of it.

4: Hey, pretty good! I liked what they did there! Damn, I didn't see that coming! Sharp ideas and some interesting characters. Might favor if it's a genre or topic I like. Let's see what else this author has to offer!

5: I have read this person's work and come to the conclusion that I suck. I suck, I suck, I suck, and I have no idea why I bother writing anything because this person just puts me to shame. So of course, I am going to favorite them and/or their story and keep coming back for inspiration and to flog myself for not being a better writer.

 

[TarnishedPenny]

How I Rate:

No Vote: It just doesn't stand out. My eyes skipped over a couple of paragraphs when it bogged down. I'll probably forget that I read it.

1: How the hell did this get approved? Bad punctuation, bad grammar, badly written, and too confusing to follow. Did this author actually pass elementary school English?

2: I skimmed through the story (or at least the first installment) but couldn't keep going. It's flat. There's no emotion, no pathos. I can't get immersed in the story or care about the characters.

3: Ok, not bad! I got through the whole thing, and I saw some stuff that I genuinely liked. I can see how it can be better, though, and I get a slight itch in my fingers rewrite it in my own way to write my version of it.

4: Hey, pretty good! I liked what they did there! Damn, I didn't see that coming! Sharp ideas and some interesting characters. Might favor if it's a genre or topic I like. Let's see what else this author has to offer!

5: I have read this person's work and come to the conclusion that I suck. I suck, I suck, I suck, and I have no idea why I bother writing anything because this person just puts me to shame. So of course, I am going to favorite them and/or their story and keep coming back for inspiration and to flog myself for not being a better writer.

Good on you for trying! Even if I don’t quite agree across the board, it adds a bit of objectivity to a completely subjective matter. The lack of commonly-agreed-upon standards or definitions certainly leads to this sort of discussion.

I cannot (sorry Tex) agree that a 5 is a ‘perfect 100%’ score, for I don’t believe perfection is possible for mortal man (or, Nadia Comaneci notwithstanding) woman. To hold to that would simply lead to no 5s being given.

I still maintain that the best way of dealing with trolls (and they certainly exist) is to require people to register to vote. That wouldn’t stop it entirely, of course, but it’d make those chinless wonders work for it.

But as to voting, I tend only to give 5s and only to those I feel are significantly better than the norm.

 

[Voboy]

Yeah.

I think some of us think MUCH more about the nuances of voting than most of the readers do.

 

[gordo12]

How I Rate:

No Vote: It just doesn't stand out. My eyes skipped over a couple of paragraphs when it bogged down. I'll probably forget that I read it.

1: How the hell did this get approved? Bad punctuation, bad grammar, badly written, and too confusing to follow. Did this author actually pass elementary school English?

2: I skimmed through the story (or at least the first installment) but couldn't keep going. It's flat. There's no emotion, no pathos. I can't get immersed in the story or care about the characters.

3: Ok, not bad! I got through the whole thing, and I saw some stuff that I genuinely liked. I can see how it can be better, though, and I get a slight itch in my fingers rewrite it in my own way to write my version of it.

4: Hey, pretty good! I liked what they did there! Damn, I didn't see that coming! Sharp ideas and some interesting characters. Might favor if it's a genre or topic I like. Let's see what else this author has to offer!

5: I have read this person's work and come to the conclusion that I suck. I suck, I suck, I suck, and I have no idea why I bother writing anything because this person just puts me to shame. So of course, I am going to favorite them and/or their story and keep coming back for inspiration and to flog myself for not being a better writer.

Roughly what I used to do too. However as years pass and I'm on the other side of the issue now I ignore 1&2 stories and only vote on 3+.

With regard to #5 maybe it's time we start a movement to ban authors like that. No scribbler here should be made to feel like they suck. Just like schools do now, we all deserve the blue ribbon. Let's ban 'em all. Tweet them into twitter hell

 

[SimonDoom]

Yeah.

I think some of us think MUCH more about the nuances of voting than most of the readers do.

I'm sure that's true, because, unlike readers, we are keenly aware of how voting affects our writing colleagues. Non-writing readers don't give that a moment's thought.

 

[Hypoxia]

I'll say again: Obsessing over scores is a route to madness. Write and move on.

 

[BelleCanzuto]

Gordo wrote: "It's a good story. Not good enough for a 5* But a 4* downvotes it."

I just don't look at it that way. A 4* doesn't downvote anything if it's given as the reader's honest assessment of the story. If you're only happy with perfection, then you've come to the wrong life.

Like TxRad said, a 4.5 means that 90% of the people who decided to vote on the story gave it 5* (on average). A 4.0 means 80% of voters gave it 5*. I just don't see that as something to be unhappy about or blame on trolls.

And like Voboy wrote, I read here for a long time before I set up an account, and probably voted on half the stories I read. I don't know about anyone else, but I'm much more likely to click the stars on a story I like or sort of like than I am a story I didn't enjoy. Most of the stories that I think deserve 2* or 3* I don't vote on at all. And for me to leave a 1* means the story has pretty much got to be godawful.

 

[SimonDoom]

Like TxRad said, a 4.5 means that 90% of the people who decided to vote on the story gave it 5* (on average). A 4.0 means 80% of voters gave it 5*. I just don't see that as something to be unhappy about or blame on trolls.

I don't think that's what TxRad meant, and that doesn't work mathematically. If your story has a score of 4.5, then it's likely that 60% or so gave it a 5, most of the rest gave it a 4, and a few nasty people gave it something less than that. I think TxRad is trying to make the current grade system correspond to school grades.

The problem is that doesn't correspond to actual numbers. While the figures vary somewhat from category to category, a 4.5 generally is somewhere around the 75 percentile, or maybe a bit less than that. It's good, but it's not outstanding. So if you reserve a "5" for, say, the top 5% of stories, and you give lower grades to stories that don't satisfy that standard, you are grading much more harshly than the way most readers grade. You are free to do that, of course, because the site offers no guidelines for what scores mean, but you might want to be conscious of the fact that by voting in this way you may penalize an author whose story is better than other stories that other readers gave higher grades.

 

[BelleCanzuto]

I don't think that's what TxRad meant, and that doesn't work mathematically.

Lissen, Simon, your math might technically be CORRECT and all, but I like my math better.

So if you reserve a "5" for, say, the top 5% of stories, and you give lower grades to stories that don't satisfy that standard, you are grading much more harshly than the way most readers grade. You are free to do that, of course

I don't know if this bit was directed at me, but I'll just say that's not really how I look at it. Obviously, I can't reserve a 5* for the top 5% of stories on the site, since I don't read that many. Of the stories I vote on, I've probably given 25% 5*, 50% 4*, and the remaining 25% is some combination of the rest. My default is to simply not vote on stories to which my reaction is "meh". So, I read a few stories every week and I don't vote at all on most of them.

I still don't think of 4* as downvoting anything.

 

[TarnishedPenny]

I'll say again: Obsessing over scores is a route to madness. Write and move on.

Pray point to three sane writers on this site...