Some programatic validation for ratings

minacia

perpetually sour
Joined
Jun 22, 2020
Messages
531
Points
133
This was sort of a random idea because of that thread about troll ratings, but......

Since ScribbleHub already has a means to identify “valid reads”, what if we linked this to the rating system?

In other words, a rating would only count if the reader has at least one valid read in the series.

The idea of this is to diminish “troll ratings” and genre ratings that come from people who glanced at the synopsis and didn’t really click into the story/read a single page.

To combat trolls adapting to this strategy, I think Tony shouldn’t tell us if this kind of approach is ever implemented. Instead, I think trolls should be allowed to leave “shadow ratings” that only they themselves can see (but aren’t visible to others). This would give trolls the impression that they are leaving ratings but in reality it isn’t being counted.
 

Moonpearl

The Yuri Empress
Joined
Dec 25, 2018
Messages
764
Points
133
Would this cause problems if someone removes it from their reading list after rating, though? Do they then technically count as having read 0 chapters?
 

Devils.Advocate

An objectionable existence
Joined
Jan 1, 2019
Messages
181
Points
133
diminish “troll ratings”
hmm.. part of me want to define, not just troll ratings but all ratings first.

how are people rating, is the stars, likable~ness or some other subjective metric. like "well-written"~ness.

if the stars have a hover bubble that make them have a unified definition. would it help?
 

LostLibrarian

Well-known member
Joined
Jan 27, 2019
Messages
709
Points
133
I mean RoyalRoad has the same feature where you have to read the story (so open one chapter) before a rating. And guess what, it doesn't help.

Do you really think, that people who want to rate your story for whatever reason won't be able enough to open a chapter or 2 in another tab for a few seconds?

And for shadow ratings? Just check it with another browser and it'll be clear. There'll be a "my rating doesn't show up"-thread after 10 minutes and everyone knows anyway. Because there are also enough 5 stars thrown around without reads...

Imho, it's just another discussion that blows a small problem out of proportion. And if you really think the system is at fault, this won't change anything anyway. Shit remains shit even with a layer of paint over it...
 

minacia

perpetually sour
Joined
Jun 22, 2020
Messages
531
Points
133
hmm.. part of me want to define, not just troll ratings but all ratings first.

how are people rating, is the stars, likable~ness or some other subjective metric. like "well-written"~ness.

if the stars have a hover bubble that make them have a unified definition. would it help?
I personally don't really think there's an issue with people rating the way that they want with different scales or subjective measures, but that's my opinion.

I mean RoyalRoad has the same feature where you have to read the story (so open one chapter) before a rating. And guess what, it doesn't help.

Do you really think, that people who want to rate your story for whatever reason won't be able enough to open a chapter or 2 in another tab for a few seconds?

And for shadow ratings? Just check it with another browser and it'll be clear. There'll be a "my rating doesn't show up"-thread after 10 minutes and everyone knows anyway. Because there are also enough 5 stars thrown around without reads...

Imho, it's just another discussion that blows a small problem out of proportion. And if you really think the system is at fault, this won't change anything anyway. Shit remains shit even with a layer of paint over it...
Out of curiosity, are you of the opinion that RR should revert back to before they had the read-one-chapter rating requirement or do you feel like it's fine now?

Obviously there is no "solution" for everything, but there are different steps that programmers can take to do their best to reflect something that they think is important.

For instance, on SH, view counts are rate-limited per IP address at a certain number per hour (iirc it's something like that), so you can't refresh your page infinite number of times to increase your view count. However, you can still visit multiple times per day from the same IP address to artificially inflate your views.

My point is that there are always ways to game, cheat, and circumvent systems, but the question is to what extent a programmer is willing to implement some kind of automatic check...

...or do nothing at all, if that is your preference.
 

LostLibrarian

Well-known member
Joined
Jan 27, 2019
Messages
709
Points
133
Out of curiosity, are you of the opinion that RR should revert back to before they had the read-one-chapter rating requirement or do you feel like it's fine now?
I think it doesn't make any difference. If you really want to give your .5 rating, you'll do that with one click more.

So going by that, we have 2 kinds of 1 star ratings on here. People who think you should get a 1 star after they read it and what so many call here "trolls". The first one wouldn't feel anything and for the second group it is one or two clicks more to do exactly the same. So if we talk troll: Do you really think that a person who is petty enough to give the entire frontpage 1 star ratings will stop because he has to do one click more?


And yeah, you can do something against pushing your own story views and the like, but this also brings us into the discussion of "when is a rating valid?". After one chapter? After one paragraph?

If someone reads a story where the MC rapes a girl in the first story and thinks that deserves a 1star rating, why would he have to read X more chapters so that his rating is valid? If someone publishes a 10k word chapter, is the reader forced to "read" 40k words of shit, because we decided that "4 chapters" sounds like a good cut-off? What do you do with one-shot stories? So do we use "1 chapter valid read" as cut-off? Then that's one click more to throw a 1 star rating around...

The big problem is this: pageviews are objective. Ratings aren't. Some user might want to rate a 1 star to a story that gets mostly 5 stars. So that isn't something an automatic check can decide. So how do you decide what is a good check for a subjective feeling?

...or do nothing at all, if that is your preference.
The problem isn't whether someone wants to do something or not. The problem is that people have a fundamental problem with 1 star ratings and the ratings system. And that won't change because you add one more step to add such rating. RoyalRoad still has drive-by .5 stars, massive bad ratings for popular stories, and the same toxicity that a small subset of their users bring into the game. It didn't go away. And the same would happen with your proposal...

It's like those "Are you over 18?" dialogues on certain websites. Those doesn't change the fact that underage people visit them. At best, they learn how to distract 18 from the current year...


In all honesty: If you can find a working automatic check that makes sense, take my upvote and blessing. But I personally can't see any improvement following this specific proposal...
 

minacia

perpetually sour
Joined
Jun 22, 2020
Messages
531
Points
133
I think it doesn't make any difference. If you really want to give your .5 rating, you'll do that with one click more.

So going by that, we have 2 kinds of 1 star ratings on here. People who think you should get a 1 star after they read it and what so many call here "trolls". The first one wouldn't feel anything and for the second group it is one or two clicks more to do exactly the same. So if we talk troll: Do you really think that a person who is petty enough to give the entire frontpage 1 star ratings will stop because he has to do one click more?


And yeah, you can do something against pushing your own story views and the like, but this also brings us into the discussion of "when is a rating valid?". After one chapter? After one paragraph?

If someone reads a story where the MC rapes a girl in the first story and thinks that deserves a 1star rating, why would he have to read X more chapters so that his rating is valid? If someone publishes a 10k word chapter, is the reader forced to "read" 40k words of shit, because we decided that "4 chapters" sounds like a good cut-off? What do you do with one-shot stories? So do we use "1 chapter valid read" as cut-off? Then that's one click more to throw a 1 star rating around...

The big problem is this: pageviews are objective. Ratings aren't. Some user might want to rate a 1 star to a story that gets mostly 5 stars. So that isn't something an automatic check can decide. So how do you decide what is a good check for a subjective feeling?


The problem isn't whether someone wants to do something or not. The problem is that people have a fundamental problem with 1 star ratings and the ratings system. And that won't change because you add one more step to add such rating. RoyalRoad still has drive-by .5 stars, massive bad ratings for popular stories, and the same toxicity that a small subset of their users bring into the game. It didn't go away. And the same would happen with your proposal...

It's like those "Are you over 18?" dialogues on certain websites. Those doesn't change the fact that underage people visit them. At best, they learn how to distract 18 from the current year...


In all honesty: If you can find a working automatic check that makes sense, take my upvote and blessing. But I personally can't see any improvement following this specific proposal...
This is so interesting! ^^

I think for me (I mean, coming from the other thread), my actual opinion is that a majority of 1-star ratings that most people receive are "genuine" (which I personally define as a unique rating coming from someone who at least read a little bit of the story). There's a tendency for authors to automatically think that it must be a "troll rating" coming from malicious readers... but to some extent I kind of doubt how common "troll ratings" really are.

There's not really a good way to measure how prevalent this is either.

I think authors tend to care more about their own ratings then readers do. Like in the other thread, we were talking about hypothetical readers going through the effort to make clone accounts to dump multiple negative reviews on the same story but... seriously...? Are there really that many readers who are going to dedicate that much effort to express their negative feelings about a story?

Personally, I think some authors probably get really paranoid and presume that readers hate them more than they really do...

I think most readers don't really care that much... they leave 1-star and move on...

-----

Personally I think a rating is valid even if a reader only reads the first paragraph.

But to me, a rating isn't a rating if they didn't even look at the story...

Hence to me, it makes perfect rational sense to have the "you clicked on the first chapter" => your rating counts.

If it doesn't take that much effort to implement that check programatically, I personally don't see any harm in having that kind of check. Even if it's just to annoy trolls, I don't really mind. It's sort of like how some websites have one-account-per-email policies. Every web developer knows that it's extremely trivial and easy for people to go and make additional emails if they wanted to, but it's enough of a hassle that it does somewhat disincentivize multiple account usage.

I don't think the goal is to eradicate malicious behavior.

It's extremely easy to implement certain boolean programatic checks, and 30 minutes of a programmer's effort cumulatively adds up to a lot of time that trolls need to waste later on.

Furthermore, Tony already does a lot of manual review-removing on ScribbleHub.

I'd be surprised if he didn't already start looking for ways to automate it... since a lot of people request him to remove certain ratings. For all we might know, maybe he already has some kind of algorithm that is detecting potentially spam-ratings.
 

LostLibrarian

Well-known member
Joined
Jan 27, 2019
Messages
709
Points
133
but to some extent I kind of doubt how common "troll ratings" really are.
To me, we readily accept that the majority of stories has overwhelmingly many 5 star ratings and have no problem with that. At that point, rating isn't about objective quality anymore but just subjective thinking, which makes "giving a 1 star for any reason" more or less valid.

That said, I can see the reasoning behind "at least read the first chapter and don't 1star the story for a certain tag". I just came from a more pragmatic view where it wouldn't change the core of the "troll discussion".

I don't think the goal is to eradicate malicious behavior.
If the goal is just to annoy trolls and add that one click more (maybe make it harder for bots to mass-rate the entire frontpage), then it's fine. To me it just didn't touch the "core of the problem" (but if that wasn't the goal anyway, that's that... :D)

I'd be surprised if he didn't already start looking for ways to automate it... since a lot of people request him to remove certain reviews. For all we might know, maybe he already has some kind of algorithm that is detecting potentially spam-reviews.
I hope he does given the size of the page. But that's something you have to regulate over mass ("X 1 star ratings in time Y", "X% or more of 1star ratings", "time between registration and rating").

But the "troll discussion" often also goes into "that one single rating" which you can hardly catch with automatic checks...
 
Top