Require reviews when rating

Skim

Member
Joined
Oct 17, 2020
Messages
17
Points
18
Thanks to all the feedback from those willing to reply I've got an answer as to why we don't require reviews when rating. The main reason for anyone wondering is, as of yet, there is no solution that is much better than the current system. Anything will have its ups and downs, and with the current system the flaws are evident, but the benefits are just as strong if not stronger.

I know this isn't the first discussion about the topic, but I was just hoping for a clear answer as to why something like requiring a review when rating a story low hasn't been implemented. It could even be made so that the review had to be at least 30 words. A few reasons I have to support this are(Didn’t mean to make this sound so snobbish):
  1. Whether someone rates your story high or low, feedback is always appreciated. Replies/ratings should be from people who have read enough of your work to voice genuine complements/critiques.
  2. It would reduce the number of bad reviews people give without good reason.
  3. It would help the rating system become more trustworthy.
  4. It would allow people to understand if people are rating a series objectively or subjectively.
  5. Before clicking on a story the number of ratings isn't shown, so fewer reviews wouldn't have a negative impact here.
Disadvantages(so far)
  1. Leaving bad reviews could lead to targeting and unwarranted backlash on series of the reviewer. Allowing anonymous reviewing might be necessary.
If anyone else has points I will add them to this list.

I also understand that Scribblehub isn't a daycare and criticism is a large part of being a writer, but the feeling of getting a bad review without knowing the reason can be difficult to deal with. A fresh feeling for me as I just got my first 1 star, which is obviously the reason why I felt like writing this post. I fully accept that I probably deserve the rating I got, but I would just like to know where I could improve. And as much as my writing is fueled by annoyance I also feel that this is a reasonable request, although I'd feel bad if it increased Tony's workload by too much.

Another alternative which Discount_Blade mentioned was:

“...do as RRL does and add stars as apart of the actual review process instead of it being separate, meaning you can ONLY LEAVE STARS WHEN WRITING A REVIEW. And the reviews have to be of a certain length though I don't remember how many characters it was.

Someone once mentioned it had to be no less than 450 characters in a review though I can't swear to the accuracy of that.”
 
Last edited:

Skim

Member
Joined
Oct 17, 2020
Messages
17
Points
18
No. We like the site's mechanic as it is right now. Does it suck? Yes. Does your idea make it suck even more? Yes.

So take it as the lesser of two evil. So far, nobody's given a better idea, so we're sticking to this shitpen we've got until someone docks their penis into another guy's foreskin and gets that Eureka™ epiphany.

'Till then, shut up and chill with us. We've got a cocktail bar, potluck, and that ballpit from Dashcon.
😂
I know my post is a nuisance you don’t have to tell me.
Do consider Discount_Blades point about switching to RRL style reviews, what’s your opinion on that?
 

CL

Well-known member
Joined
Jan 30, 2020
Messages
507
Points
133
Say.... for the uneducated... Like I... who won't know what the RRL system is even if I am trapped in a paper bag with it for a whole month in solitary confinement...

What is the RRL system?

Or would you rather I make tracks, go over there... make an account and try using it?
What I will do (like right now) is go over there and see whether or not if RRL has any documentation on how the rating/review system.

...

I'm going to just copy what I found out about their rating/review system:

Can you make it so users have to leave reviews to rate?​

There are several problems with making the review content compulsory. It would significantly reduce the number of ratings on the site. This would pose several problems, including:
  • We'd need to remove existing anonymous ratings, as the reduced number of reviews would mean that top list entries would be very, very hard to overcome, if not virtually impossible unless the site's viewership suddenly increased tenfold.
  • At the same time, removing existing ratings would mean that we'd make a gigantic mess of the site. The top list would get completely scrambled, and manipulating the top list would become much easier due to the significantly reduced number of reviews to judge rankings by.
  • Removing reviews, actually, does not stop drive-bys. An excellent example of this is Amazon. We recommend doing quick research on the topic, as "spam reviews" are a significant problem with the system either way.
Additionally, most users that do not review, according to statistics, would not rate if they had to write a review to do so. Another problem with forcing reviews is that drive-by 0.5-star ratings would amount to a significant level of written hate reviews, which looks really bad and isn’t fair to the author. This would both make moderation increasingly hard and authors more likely to quit due to the negative feedback.

(November 2020: We are beta-testing this with a few stories.)

What are the rules about Ratings and Reviews?​

Reviews aim to inform prospective readers if the story is something they will want to read. This can be done by communicating how well a story delivers what was promised in the premise (as outlined in the synopsis and opening chapters), by judging how well the story is written (both in terms of grammar and prose) and by highlighting the highs and lows of a reader’s experience with a story.
You can verify the validity of a review by checking the reading history of the reviewer. You can also check the upvotes and downvotes given by other readers to show how accurate or good they think the review is to the story
If you read the story, there are three levels of feedback you can leave:
Ratings are plain 0.5-5 star scores. No written feedback or justification for the score is required.
Basic Reviews provide an overall 0.5-5 star score along with a brief overview of your opinion of the story, providing some justification for the score.
Advanced Reviews provide a score for the overall fiction in addition to the following subcategories: Style, Story, Grammar, and Characters.
  • Style: The general feeling of the piece as created by the author's word choice, literary devices, and unique way of writing.
  • Story: Plot development and pacing (i.e. how well a story is put together).
  • Grammar: The technical aspects of the writing, including spelling, grammar, and punctuation.
  • Character: The characters in the story, how well they are depicted, how realistic and three-dimensional they are. This applies to both main and side characters.
    ___
Rules:
  1. Justify the Rating
    1. The Review needs to justify the rating, regardless of how high or low the score is.
    2. In an advanced review, you must write a detailed review that covers the subcategories being rated (Style, Story, Grammar, and Characters) and to justify the overall score.
  2. Stay on Topic
    1. Your review should address the story itself, and only the story.
    2. Do not write about unrelated content, including comments about the author's activities, other people, other reviews, etc.
    3. Reviews and ratings may not be based solely on superficial information such as cover, synopsis, tags, title, story’s popularity, its position on the ranking lists, or similar things that have little to do with actual story content. If you wish to review this regardless, especially the genre and tags, it cannot be the main focus. You may say how certain elements of the genre were poorly done or how they take away from the story, but that cannot be the central topic of a review.
    4. Reviews comparing the story to another according to the following rules will be removed:
      1. Accusing an author of plagiarism in reviews is forbidden. Any story violating Royal Road terms of use(such as plagiarism) should be reported directly using the report system via the report button on the violating fiction or chapter.
      2. Don’t discourage readers from reading a story by comparing it to another similar story.
  3. Be Fair and Respectful
    1. Directly insulting the author or the readers is strictly forbidden. You may criticize the writing in constructive ways if you have a genuine desire to help, but do not direct your feedback at any author specifically, especially if it is a personal attack on their character.
    2. Discouraging an author from writing is viewed as taboo on Royal Road.
    3. Note that while the review could generalize the story as a whole, it can’t intentionally depict a story as something entirely different from what it is. Such as stating that a story includes instances of horrific actions when it does not. Examples such as pedophilia, incest, necrophilia, racism, etc.
    4. Do not instigate or incite any harmful acts in the review.
  4. Use Appropriate Language
    1. Reviews can’t contain profanity, vulgarity, sexual content, or descriptions of gore. This includes, but is not limited to; vivid descriptions of sexual acts, torture, violence, and other acts or situations that may be considered “gore”.
    2. Slander and discrimination is completely forbidden, and violations are viable to get the user banned. This includes but is not limited to attacks on ethnic, religious, political, sexual orientation, gender or racially discriminatory reviews.
  5. Spoilers:
    1. A review may contain spoilers under the appropriate spoiler tag(from the toolbar).
    2. Spoiler content may not exceed a third of the overall word count.
    3. Discussing spoilers can’t be the focus of a review.
    4. Spoilers can’t be the majority of the plotline summarised into points, nor can they mention any major revelations in the story.
      Examples: a character's death, success or failure in achieving a main goal. Among others.
    5. Quoted content can’t break any of the review rules.
  6. General
    1. Reviews must not contain meaningless characters and words used with the purpose of inflating the character count and review length.
    2. Reviews must not contain any spam.
    3. Reviews must be written in English or in the language the fiction is written in.
    4. Offering to release chapters based on the scores of reviews and ratings is considered ranking manipulation and is prohibited.
    5. Campaigns encouraging others to upvote or downvote reviews, whether positive or negative are forbidden.
    6. Rating and reviewing from multiple accounts is prohibited. We have software to catch people who try to manipulate the ratings by doing this.
    7. Members of the same household or friends may review the story as long as they clearly state their relation to the author.

That is what I gathered from the FAQ on RRL.
 

BenJepheneT

Light Up Gold - Parquet Courts
Joined
Jul 14, 2019
Messages
5,344
Points
233
😂
I know my post is a nuisance you don’t have to tell me.
Do consider Discount_Blades point about switching to RRL style reviews, what’s your opinion on that?
Personally, I think it's boring. The attraction for SH's reviews, for me at least, is its simplicity, freedom, and accessibility. You can be concise, irrelevant, informative and straight-up defamatory. It's easy to access for absolutely anyone with an email-account and a short-term memory capable to remembering their password.

Everytime I see a review I think of it as a roll of a dice. Will I get an in-depth review, a troll, or just a on-the-move, stream of consciousness type shit? That's what I like.

I realize my standards towards a review mechanics make no sense whatsoever but it's what I consider to be interesting. The only close second is Royal Road's reviews. It's basically the same thing, but with a more restrictive guideline and a more sectioned scoring system.

But SH's more fun for me anyway.
 

Skim

Member
Joined
Oct 17, 2020
Messages
17
Points
18
What I will do (like right now) is go over there and see whether or not if RRL has any documentation on how the rating/review system.

...

I'm going to just copy what I found out about their rating/review system:

Can you make it so users have to leave reviews to rate?​

There are several problems with making the review content compulsory. It would significantly reduce the number of ratings on the site. This would pose several problems, including:
  • We'd need to remove existing anonymous ratings, as the reduced number of reviews would mean that top list entries would be very, very hard to overcome, if not virtually impossible unless the site's viewership suddenly increased tenfold.
  • At the same time, removing existing ratings would mean that we'd make a gigantic mess of the site. The top list would get completely scrambled, and manipulating the top list would become much easier due to the significantly reduced number of reviews to judge rankings by.
  • Removing reviews, actually, does not stop drive-bys. An excellent example of this is Amazon. We recommend doing quick research on the topic, as "spam reviews" are a significant problem with the system either way.
Additionally, most users that do not review, according to statistics, would not rate if they had to write a review to do so. Another problem with forcing reviews is that drive-by 0.5-star ratings would amount to a significant level of written hate reviews, which looks really bad and isn’t fair to the author. This would both make moderation increasingly hard and authors more likely to quit due to the negative feedback.

(November 2020: We are beta-testing this with a few stories.)

What are the rules about Ratings and Reviews?​

Reviews aim to inform prospective readers if the story is something they will want to read. This can be done by communicating how well a story delivers what was promised in the premise (as outlined in the synopsis and opening chapters), by judging how well the story is written (both in terms of grammar and prose) and by highlighting the highs and lows of a reader’s experience with a story.
You can verify the validity of a review by checking the reading history of the reviewer. You can also check the upvotes and downvotes given by other readers to show how accurate or good they think the review is to the story
If you read the story, there are three levels of feedback you can leave:
Ratings are plain 0.5-5 star scores. No written feedback or justification for the score is required.
Basic Reviews provide an overall 0.5-5 star score along with a brief overview of your opinion of the story, providing some justification for the score.
Advanced Reviews provide a score for the overall fiction in addition to the following subcategories: Style, Story, Grammar, and Characters.
  • Style: The general feeling of the piece as created by the author's word choice, literary devices, and unique way of writing.
  • Story: Plot development and pacing (i.e. how well a story is put together).
  • Grammar: The technical aspects of the writing, including spelling, grammar, and punctuation.
  • Character: The characters in the story, how well they are depicted, how realistic and three-dimensional they are. This applies to both main and side characters.
    ___
Rules:
  1. Justify the Rating
    1. The Review needs to justify the rating, regardless of how high or low the score is.
    2. In an advanced review, you must write a detailed review that covers the subcategories being rated (Style, Story, Grammar, and Characters) and to justify the overall score.
  2. Stay on Topic
    1. Your review should address the story itself, and only the story.
    2. Do not write about unrelated content, including comments about the author's activities, other people, other reviews, etc.
    3. Reviews and ratings may not be based solely on superficial information such as cover, synopsis, tags, title, story’s popularity, its position on the ranking lists, or similar things that have little to do with actual story content. If you wish to review this regardless, especially the genre and tags, it cannot be the main focus. You may say how certain elements of the genre were poorly done or how they take away from the story, but that cannot be the central topic of a review.
    4. Reviews comparing the story to another according to the following rules will be removed:
      1. Accusing an author of plagiarism in reviews is forbidden. Any story violating Royal Road terms of use(such as plagiarism) should be reported directly using the report system via the report button on the violating fiction or chapter.
      2. Don’t discourage readers from reading a story by comparing it to another similar story.
  3. Be Fair and Respectful
    1. Directly insulting the author or the readers is strictly forbidden. You may criticize the writing in constructive ways if you have a genuine desire to help, but do not direct your feedback at any author specifically, especially if it is a personal attack on their character.
    2. Discouraging an author from writing is viewed as taboo on Royal Road.
    3. Note that while the review could generalize the story as a whole, it can’t intentionally depict a story as something entirely different from what it is. Such as stating that a story includes instances of horrific actions when it does not. Examples such as pedophilia, incest, necrophilia, racism, etc.
    4. Do not instigate or incite any harmful acts in the review.
  4. Use Appropriate Language
    1. Reviews can’t contain profanity, vulgarity, sexual content, or descriptions of gore. This includes, but is not limited to; vivid descriptions of sexual acts, torture, violence, and other acts or situations that may be considered “gore”.
    2. Slander and discrimination is completely forbidden, and violations are viable to get the user banned. This includes but is not limited to attacks on ethnic, religious, political, sexual orientation, gender or racially discriminatory reviews.
  5. Spoilers:
    1. A review may contain spoilers under the appropriate spoiler tag(from the toolbar).
    2. Spoiler content may not exceed a third of the overall word count.
    3. Discussing spoilers can’t be the focus of a review.
    4. Spoilers can’t be the majority of the plotline summarised into points, nor can they mention any major revelations in the story.
      Examples: a character's death, success or failure in achieving a main goal. Among others.
    5. Quoted content can’t break any of the review rules.
  6. General
    1. Reviews must not contain meaningless characters and words used with the purpose of inflating the character count and review length.
    2. Reviews must not contain any spam.
    3. Reviews must be written in English or in the language the fiction is written in.
    4. Offering to release chapters based on the scores of reviews and ratings is considered ranking manipulation and is prohibited.
    5. Campaigns encouraging others to upvote or downvote reviews, whether positive or negative are forbidden.
    6. Rating and reviewing from multiple accounts is prohibited. We have software to catch people who try to manipulate the ratings by doing this.
    7. Members of the same household or friends may review the story as long as they clearly state their relation to the author.

That is what I gathered from the FAQ on RRL.
Thanks for finding this.
 
  • Like
Reactions: CL

Skim

Member
Joined
Oct 17, 2020
Messages
17
Points
18
Personally, I think it's boring. The attraction for SH's reviews, for me at least, is its simplicity, freedom, and accessibility. You can be concise, irrelevant, informative and straight-up defamatory. It's easy to access for absolutely anyone with an email-account and a short-term memory capable to remembering their password.

Everytime I see a review I think of it as a roll of a dice. Will I get an in-depth review, a troll, or just a on-the-move, stream of consciousness type shit? That's what I like.

I realize my standards towards a review mechanics make no sense whatsoever but it's what I consider to be interesting. The only close second is Royal Road's reviews. It's basically the same thing, but with a more restrictive guideline and a more sectioned scoring system.

But SH's more fun for me anyway.
Interesting, I kinda get where you are coming from. Ever since I first visited SH it’s felt very simple and free to access. Making reviews mandatory with ratings would definitely take away from this aspect. I wrote this post in the spur of the moment as a way to just get an idea that came by out, it’s great to hear your opinion on this. Cools my head and makes me almost appreciate the 1 star 😁
 

LostLibrarian

Well-known member
Joined
Jan 27, 2019
Messages
709
Points
133
I mean switching to RR's system won't change anything. While the "detailed stars" are only usable with a review, the overall rating can get its 0.5 to 5 stars without any review at all. So the only reason to write a review would be to give stars for Style, Story, Grammar, and Character. But the Overall Score is just as easily brought down by drive-by 0.5 star ratings in RR.

There is honestly no way to get "unfair bad ratings" out of the system, because people will just say "bad story meh" in their 1 star "review" and that's that. No way to check if the stuff in it is even true, given that the site will get flooded with short reviews without meaning. The only thing RR has going for it, is that people can upvote/downvote reviews which is just as easily played as the normal rating system.


I would honestly go the different route:
Use the room on the frontpage better and display highly upvoted reviews of the last X days and display a review score more prominently on the fiction's site. That way, a story with "meh"-rating due to 1 stars but a better review-rating might seem more attractive for users. At the same time, reviews might also help smaller stories to get some views, so it'll give people who enjoy a story a reason to care enough to write a review.

Don't try to get toxicity out of the system. It doesn't work. Even RR didn't manage to do it. Even their "reviews have to be impartial" rules only led to more support tickets with authors complaining (whether they were right or not).

Instead of fighting against something that no site can win against, try to enforce good behaviour instead. Make reviews something that helps stories, give users a prominent place to display their feelings, and above all make them visible (!).

Right now you can read here for months without even knowing that a review system exists...


-------------------------------------------

On a different note: all those "forced reviews will make bad ratings go away" topics are also off. Those changes will make the overall rating of all stories drop. Because the people who care to invest their time and write reviews are either those who really are invested in the story or are really against it/trolling.

So yeah, you might have only 2 instead of 10 1star ratings. But you'll also only have 5 instead of 60 5 star ratings, because all those who just "liked it enough to press a button" won't rate your story anymore. So congratulations, you now have 7 instead of 70 ratings and your ratings dropped from 4,4 to 3,8.

Nothing has changed besides the fact that the site looks half-dead (now nearly no ratings on top of rare comments for most stories) to new readers ...
 
Last edited:

Skim

Member
Joined
Oct 17, 2020
Messages
17
Points
18
I mean switching to RR's system won't change anything. While the "detailed stars" are only usable with a review, the overall rating can get its 0.5 to 5 stars without any review at all. So the only reason to write a review would be to give stars for Style, Story, Grammar, and Character. But the Overall Score is just as easily brought down by drive-by 0.5 star ratings in RR.

There is honestly no way to get "unfair bad ratings" out of the system, because people will just say "bad story meh" in their 1 star "review" and that's that. No way to check if the stuff in it is even true, given that the site will get flooded with short reviews without meaning. The only thing RR has going for it, is that people can upvote/downvote reviews which is just as easily played as the normal rating system.


I would honestly go the different route:
Use the room on the frontpage better and display highly upvoted reviews of the last X days and display a review score more prominently on the fiction's site. That way, a story with "meh"-rating due to 1 stars but a better review-rating might seem more attractive for users. At the same time, reviews might also help smaller stories to get some views, so it'll give people who enjoy a story a reason to care enough to write a review.

Don't try to get toxicity out of the system. It doesn't work. Even RR didn't manage to do it. Even their "reviews have to be impartial" rules only led to more support tickets with authors complaining (whether they were right or not).

Instead of fighting against something that no site can win against, try to enforce good behaviour instead. Make reviews something that helps stories, give users a prominent place to display their feelings, and above all make them visible (!).

Right now you can read here for months without even knowing that a review system exists...


-------------------------------------------

On a different note: all those "forced reviews will make bad ratings go away" topics are also off. Those changes will make the overall rating of all stories drop. Because the people who care to invest their time and write reviews are either those who really are invested in the story or are really against it/trolling.

So yeah, you might have only 2 instead of 10 1star ratings. But you'll also only have 5 instead of 60 5 star ratings, because all those who just "liked it enough to press a button" won't rate your story anymore. So congratulations, you now have 7 instead of 70 ratings and your ratings dropped from 4,4 to 3,8.

Nothing has changed besides the fact that the site looks half-dead (now nearly no ratings on top of rare comments for most stories) to new readers ...
Sound logic! I'm probably going to adjust my post, considering I don't know how to delete it, to just saying my original question has been resolved.
 

Shard

Keeper of Fluffy Tails
Joined
Jan 18, 2019
Messages
282
Points
103
Personally, I'd say the RR review system would be great, but something more like youtube would be better for ratings, just a like/dislike rather than a scale. I can't tell how many very low quality stories I've seen get multiple five-star ratings. If essentially 90% of ratings are only the bare minimum or the absolute maximum, then there is basically no point in having any other option, because those who put effort into a detailed rating are drowned out by mindless min/max. In that case, just switch so everyone is equal on min/max, and have detailed ratings for those willing to leave reviews. Likewise, you can then have a system set to promote stories with good reviews and an indicator of percentage of positive ratings with a count of total ratings, so that a story with a single good rating doesn't appear better than one with hundreds of good and one bad. Currently, you have to go to a story's page to see how many total ratings, which means older stories almost universally have a worse first impression via rating because there is nothing to distinguish 4.5/5 with 80 reviews vs 5/5 with one review, except the 5/5 looks better unless you open the story page.

And as others have said, trying to enforce being good doesn't work, it is best to simply reduce the impact of trolls as much as possible while improving the effects of good behavior.

And on that note, I really ought to go leave some reviews and ratings myself... *mutters under breath* Bunch of good stories I was just too lazy to do anything with.
 
  • Like
Reactions: CL

Skim

Member
Joined
Oct 17, 2020
Messages
17
Points
18
Personally, I'd say the RR review system would be great, but something more like youtube would be better for ratings, just a like/dislike rather than a scale. I can't tell how many very low quality stories I've seen get multiple five-star ratings. If essentially 90% of ratings are only the bare minimum or the absolute maximum, then there is basically no point in having any other option, because those who put effort into a detailed rating are drowned out by mindless min/max. In that case, just switch so everyone is equal on min/max, and have detailed ratings for those willing to leave reviews. Likewise, you can then have a system set to promote stories with good reviews and an indicator of percentage of positive ratings with a count of total ratings, so that a story with a single good rating doesn't appear better than one with hundreds of good and one bad. Currently, you have to go to a story's page to see how many total ratings, which means older stories almost universally have a worse first impression via rating because there is nothing to distinguish 4.5/5 with 80 reviews vs 5/5 with one review, except the 5/5 looks better unless you open the story page.

And as others have said, trying to enforce being good doesn't work, it is best to simply reduce the impact of trolls as much as possible while improving the effects of good behavior.

And on that note, I really ought to go leave some reviews and ratings myself... *mutters under breath* Bunch of good stories I was just too lazy to do anything with.
Your take on the whole conversation seems very viable. Personally, I would completely agree with you if I didn't like seeing stars better than likes/dislikes :ROFLMAO:
 
D

Deleted member 45782

Guest
yeah. hopefully if one day SH did incorporate required reviews, that will go for all ratings. there will be some flaws as nothing is perfect thu. until then.
 

Ral

Well-known member
Joined
Oct 15, 2019
Messages
604
Points
133
Personally, I'd say the RR review system would be great, but something more like youtube would be better for ratings, just a like/dislike rather than a scale. I can't tell how many very low quality stories I've seen get multiple five-star ratings. If essentially 90% of ratings are only the bare minimum or the absolute maximum, then there is basically no point in having any other option, because those who put effort into a detailed rating are drowned out by mindless min/max. In that case, just switch so everyone is equal on min/max, and have detailed ratings for those willing to leave reviews. Likewise, you can then have a system set to promote stories with good reviews and an indicator of percentage of positive ratings with a count of total ratings, so that a story with a single good rating doesn't appear better than one with hundreds of good and one bad.
This system sounds very much like how FimFiction.net does it.

Currently, you have to go to a story's page to see how many total ratings, which means older stories almost universally have a worse first impression via rating because there is nothing to distinguish 4.5/5 with 80 reviews vs 5/5 with one review, except the 5/5 looks better unless you open the story page.
Yeah. The overemphasis on the overall rating kinda sucks. It is kinda one of the most useless statistic there is but one that is the most visible.

yeah. hopefully if one day SH did incorporate required reviews, that will go for all ratings. there will be some flaws as nothing is perfect thu. until then.
I don't think they would incorporate such implementation.
 
D

Deleted member 45782

Guest
This system sounds very much like how FimFiction.net does it.


Yeah. The overemphasis on the overall rating kinda sucks. It is kinda one of the most useless statistic there is but one that is the most visible.


I don't think they would incorporate such implementation.
True, thats way I had the if there. Likely it won't happen, but if it does.
 
Top