Changing the rating system

[Shiver]

So, I've mentioned this before in other threads, but I'd thought I'd make a thread to ask for a change in the 5-star system used on SH and add my reasoning to it.
Apologies if this has already been discussed and turned down.

Going by the number of threads about 1-star ratings and trolls I've seen appear over the six weeks since I started posting here, it has to be apparent that the 'give your 1 through 5-star rating' for a story causes a lot of frustration among authors when they see those inevitable 1-stars.
Now I know Tony can check out the ratings and will remove those that look like trolling, but that gives him more work to do and doesn't solve the underlying issue.

Youtube published the figures back in 2009 when they switched from their 5-star rating system;
The gist of it is that while a few people carefully consider their star ratings, the overwhelming majority of people use a 5 star to show they like something and 1 star to show they dislike something.
I do not think it's different here.

So I propose a like-dislike binary system.
That would make it clear what the reader thinks of the novel, and every account's opinion carries the same weight.
So no more single one-stars tanking the novel's rating and demotivating the author and/or giving Tony more work.

Now I understand the worth of a visual five-star rating in choosing which novels you want to look at -not to mention whatever math magic the site uses it for to figure things out with-, but switching to a like-dislike binary system doesn't have to change that.
A simple percentage-based system of the number of likes/dislikes compared to the total amount of votes would give an excellent place to base the star rating on.
A 100% of accounts liking the novel would give a 5-star visual. 1 in 10 liking it would give a 0,5 star visual, etc.

That's it, cheers.

 

[Maple-Leaf]

How does the rating system we have now affect the reviews anyways? Youtube comments aren't made better or worse by the like/dislike feature.

Youtube comments have a significant freedom in what they talk about. You don't scroll down to see the comments expecting a full review of the video. And you don't go to the reviews page expecting people to talk about a specific part of the story. I don't know about review systems that actually use like and dislike buttons, but I don't think it's a very good idea. Reviews, or at least in my opinion, are written to convince, or dissuade a potential reader from reading a book. Limiting that review system just makes it the slightest bit more difficult for that potential reader to tell whether or not the critic likes the book. I feel like because of all this "leave a review and go" we forget the real purpose of "reviews" is giving criteria. Removing the stars from the review system hinders its original purpose, and it's not really solving anythink major. It would take away more than it would fix.

Anyway, I think people would just write the number of stars they'd give the book in the review itself so it probably doesn't really matter.
It's not worth taking the time to change.

 

[Shiver]

Already suggested a few times before. The answer is still the same.

The difference between the star system and the like/dislike system (or any other system you can come up) is minimal and will barely make any difference.

Unless someone can convince Tony that a change would be worth his time, it's not happening. There are simply better features to implement than a change in the rating system in which you change 6 for half a dozen.

In absolute numbers, I agree there isn't much of a difference. But I'm not talking about numbers; I'm talking about how those numbers are perceived and received by the authors.

Now I'm not one to make decisions about how others spend their time, and I have no idea what the site's code is like, so I can't make a reliable estimate, but I'd gladly donate a few hours to help out if needed.
(I'm a test-coordinator with 19 years of project experience across the dozens of systems and platforms, it'll cut into my writing time, but If it helps...).

I would find that demotivating as a reader, since I'd have to give really great stories the same generic "liked this" as meh stories.

Would be better to make it "like", "dislike" and "this was okay". And by that point you're better off just using the star system anyway.

Isn't the 'meh' option covered by not voting?

 

[Moonpearl]

Isn't the 'meh' option covered by not voting?

Then that would reduce reader interaction and that would suck.

 

[Ral]

I would find that demotivating as a reader, since I'd have to give really great stories the same generic "liked this" as meh stories.

Would be better to make it "like", "dislike" and "this was okay". And by that point you're better off just using the star system anyway.

I don't know. In the first place, the original poster started this because readers are using the rating system as a like/dislike system. Those readers, at least, aren't demotivated.

Also, there are story publishing sites that implements like/dislike instead of ratings system and it seems to be used a lot by the readers.

Then that would reduce reader interaction and that would suck.

I don't think anyone who find a story meh would interact with it (much) anyway. I mean, how often does people rate 2, 3 or 4, the various shades of meh? Rarely.

 

[Shiver]

Then that would reduce reader interaction and that would suck.

I don't understand how that would reduce reader interaction.
I don't think the people who would give the 'meh' rating would take the time to give a 2, or 3-star rating anyway.
Those who think long enough to decide between something else than 1 or 5 stars will probably be polarized into the like or dislike camps anyway.

 

[Moonpearl]

I don't know. In the first place, the original poster started this because readers are using the rating system as a like/dislike system. Those readers, at least, aren't demotivated.

Also, there are story publishing sites that implements like/dislike instead of ratings system and it seems to be used a lot by the readers.

Not everyone does, though. And maybe those readers only use it as a "like" vs "dislike" system because they're only reading things they expect to be 5 star anyway.

Taking away the ability to be more nuanced from people who genuinely want to put the effort in to give back to the community would just be a negative.

Also, SH is not other writing sites and that's a good thing. Copying from other places isn't always the answer.

I don't understand how that would reduce reader interaction.
I don't think the people who would give the 'meh' rating would take the time to give a 2, or 3-star rating anyway.
Those who think long enough to decide between something else than 1 or 5 stars will probably be polarized into the like or dislike camps anyway.

I would and do consider carefully how many stars to give a novel. There are also many people who give three stars to works they find "meh".

And, as an author, I find a lot of value in the difference between one stars, two stars, and three stars as my feedback. "Like" vs "Dislike" would feel like a waste of my time to pay attention to.

 

[Shiver]

Not everyone does, though. And maybe those readers only use it as a "like" vs "dislike" system because they're only reading things they expect to be 5 star anyway.

Taking away the ability to be more nuanced from people who genuinely want to put the effort in to give back to the community would just be a negative.

Also, SH is not other writing sites and that's a good thing. Copying from other places isn't always the answer.

I would and do consider carefully how many stars to give a novel. There are also many people who give three stars to works they find "meh".

And, as an author, I find a lot of value in the difference between one stars, two stars, and three stars as my feedback. "Like" vs "Dislike" would feel like a waste of my time to pay attention to.

Possibly, but when I look at the ratings here they follow the same trend as most other sites with ratings;
highest amount given are of the max rating, followed by lowest rating, followed by a smattering of the in-betweens.

I think if people realy wanted to give back, they'd give a review, and if their ratings are uncoupled and kept to the stars it's fine. A simple binary system for most people, and if you realy want to 'give back' then a review will allow you to be more nuanced. best of both worlds.

 

[BenJepheneT]

True, except, that's not how the average person works.
If a tiny minority of people only uses the ratings 2,3 & 4, those ratings don't improve the system; they only muddle it.
I've worked in computer development for a long time, and one of the most critical lessons I've learned is that simple is better.

Thanks, but I still love my 2, 3, & 4 stars. I don't know about you but I value those minority ratings more than the average mindless 5 stars. Turning it into a binary system isn't gonna improve anything. It's just gonna streamline everything and make it even easier to be a mindless drone. Some opinions can't be summarised with a simple "like" or "dislike", and that's where the 2-4 stars come in.

 

[Moonpearl]

Possibly, but when I look at the ratings here they follow the same trend as most other sites with ratings;
highest amount given are of the max rating, followed by lowest rating, followed by a smattering of the in-betweens.

I think if people realy wanted to give back, they'd give a review, and if their ratings are uncoupled and kept to the stars it's fine. A simple binary system for most people, and if you realy want to 'give back' then a review will allow you to be more nuanced. best of both worlds.

Reviewing is a touchy issue here on SH because the authors can obviously see what you write, and there's already so much bad feeling about people leaving honest one star ratings.

I feel like it will be a long time before the review system is used frequently.

 

[BenJepheneT]

Possibly, but when I look at the ratings here they follow the same trend as most other sites with ratings;
highest amount given are of the max rating, followed by lowest rating, followed by a smattering of the in-betweens.

I think if people realy wanted to give back, they'd give a review, and if their ratings are uncoupled and kept to the stars it's fine. A simple binary system for most people, and if you realy want to 'give back' then a review will allow you to be more nuanced. best of both worlds.

here's the star system:

-5, 4, 3, 2, 1.
-easy to drop 1 star, as it is easy to drop 5 stars
-if your opinion on a piece of work sits on the middle of the fence, you have the 2, 3, 4 for varying degrees of fence-sitting

now here's a binary system:

-like, dislike
-easy to like as it is to dislike
-if your opinion on a piece of work sits on the middle of the fence, you're shit out of luck. you're fucked. either you give the like you don't want to give or give the dislike you don't want to give. no in-betweens.

simplicity is good but not when it takes out intricacy. this isn't some traffic monitoring site or some business venture. this is a site for writing works where results you get aren't always binary and are filled with opinions that can't necessarily be boiled down to "me like" and "me no like". plus, I don't see the improvement you're talking about for the binary system. all you're doing is taking out 3 extra options which make little to no difference. i can still dislike as much as i can drop one stars.

 

[Shiver]

Thanks, but I still love my 2, 3, & 4 stars. I don't know about you but I value those minority ratings more than the average mindless 5 stars. Turning it into a binary system isn't gonna improve anything. It's just gonna streamline everything and make it even easier to be a mindless drone. Some opinions can't be summarised with a simple "like" or "dislike", and that's where the 2-4 stars come in.

I don't wholly disagree with you, but what makes those 2, 3, or 4 stars 'better' is the assumption that people thought about them because they're different from the majority.
There's zero evidence that that is the case.

I value chapter comments and reviews more than any rating because there's actual quantifiable effort gone into those. Not an assumed effort.

Reviewing is a touchy issue here on SH because the authors can obviously see what you write, and there's already so much bad feeling about people leaving honest one star ratings.

I feel like it will be a long time before the review system is used frequently.

Wouldn't that just be a vote in favor of a basic binary system for the masses and a star rating for the reviews?
I'd love an actual 1 star review that takes the time to point out my flaws and I can't imagine any serious author who wouldn't want constructive critisism to help them improve.

here's the star system:

-5, 4, 3, 2, 1.
-easy to drop 1 star, as it is easy to drop 5 stars
-if your opinion on a piece of work sits on the middle of the fence, you have the 2, 3, 4 for varying degrees of fence-sitting

now here's a binary system:

-like, dislike
-easy to like as it is to dislike
-if your opinion on a piece of work sits on the middle of the fence, you're shit out of luck. you're fucked. either you give the like you don't want to give or give the dislike you don't want to give. no in-betweens.

simplicity is good but not when it takes out intricacy. this isn't some traffic monitoring site or some business venture. this is a site for writing works where results you get aren't always binary and are filled with opinions that can't necessarily be boiled down to "me like" and "me no like". plus, I don't see the improvement you're talking about for the binary system. all you're doing is taking out 3 extra options which make little to no difference. i can still dislike as much as i can drop one stars.

That's fair, except that you're forgetting the option to not give a vote or actually giving a review.
I think a star rating in a review uncoupled from a binary like/dislike for the novel would adress all the issues that every one of us has brought to the table.

 

[Moonpearl]

Wouldn't that just be a vote in favor of a basic binary system for the masses and a star rating for the reviews?
I'd love an actual 1 star review that takes the time to point out my flaws and I can't imagine any serious author who wouldn't want constructive critisism to help them improve.

Hahahahahahahahaha.

As the Yuri Garden creator who creates and referees our writing events, I've had the absolute joy of working with many genius writers, many of whom are very polite and eager for feedback.
I've also had authors practically declare me their sworn enemy for life because I told them that their work objectively didn't comply with the rules.

Honestly, I want to leave reviews everywhere I go, but don't because I never know how rational the writer is going to be.
Plus there's a lot of pressure when it comes to writing feedback that could emotionally affect the person who made the thing, even if the author is super nice and willing to listen to criticism.
It takes a brave person to leave an honest review, so ratings are a good compromise.

(Also, I would like to point out that making "like and "dislike" buttons might just encourage people to press the "dislike" button a lot more often than they currently "do".
There's a sense that the star rating is supposed to be somewhat objective and about the novel itself, even though whether you think it's good or trash might be subjective. But if you just ask for an undiluted opinion, people who straight up don't like the genre are perfectly justified in pressing the "dislike" button. There would be nothing dishonest and trolling about that.)

 

[Ral]

Not everyone does, though. And maybe those readers only use it as a "like" vs "dislike" system because they're only reading things they expect to be 5 star anyway.

But almost everyone. Why must we cater to the minority?

And that is only a maybe. I don't think you even believe that.

Taking away the ability to be more nuanced from people who genuinely want to put the effort in to give back to the community would just be a negative.

What is nuanced about the star rating?

I really don't agree with this argument. In the first place not only do readers mostly rate 1 or 5 which obliterates this nuance you are talking of when only two factors weight so heavily but authors themselves ignore this "nuanced" ratings because they don't carry much impact. I mean, what will affect an author more "this horrible" "it is okay" "it is really great?" I bet "it is okay" is going to be invisible.

Also, SH is not other writing sites and that's a good thing. Copying from other places isn't always the answer.

Uh . . . LOL.

SH is like other writing sites. Specifically, it is like a discount version of RoyalRoad.

The star rating is a really popular tool. It is almost everywhere. I'm sure ScribbleHub included a star rating system here because it is popular.

SH is not unique at all. Not to mention, being unique or different doesn't mean good or better.

 

[BenJepheneT]

I really don't agree with this argument. In the first place not only do readers mostly rate 1 or 5 which obliterates this nuance you are talking of when only two factors weight so heavily but authors themselves ignore this "nuanced" ratings because they don't carry much impact. I mean, what will affect an author more "this horrible" "it is okay" "it is really great?" I bet "it is okay" is going to be invisible.

This just screams "there's only one type of author in the world and it's me".

Let me tell you this: in the current system where it's flooded with two paragraph reviews consisting of varying renditions of "it is really great" and a straight up 2 sentences of "this horrible", I find myself focusing on more of the "it's okay" "it's got potential" "it could be better" reviews. And you won't be hard pressed to find like-minded individuals who prefer a Ying-Yang review rather than blind praise or slander.

 

[GDLiZy]

I don't understand why people say "if it's meh then don't vote" as an argument for the dislike-like system.

What if the story is meh? No rating for them? Does this mean the story is meh or no one read them?

 

[Ral]

This just screams "there's only one type of author in the world and it's me".

Let me tell you this: in the current system where it's flooded with two paragraph reviews consisting of varying renditions of "it is really great" and a straight up 2 sentences of "this horrible", I find myself focusing on more of the "it's okay" "it's got potential" "it could be better" reviews. And you won't be hard pressed to find like-minded individuals who prefer a Ying-Yang review rather than blind praise or slander.

Uhmmm . . . I was not talking about reviews?

I don't understand why people say "if it's meh then don't vote" as an argument for the dislike-like system.

This. While I'm for the like/dislike system, this isn't an argument for anything.

Though, if you don't like it or dislike it, you are pretty much indifferent. The argument here seems to equate like=5 star and dislike=1 star. No. They are totally different system.

What if the story is meh? No rating for them? Does this mean the story is meh or no one read them?

And here it is, liking and disliking is not rating the story, pretty much liking a post in the forum is you rating the quality of the said post. Like and dislike doesn't reflect the quality of the story, unlike star ratings which tries to be. A Dislike is not 1 star (which is meant to mean that the story is terrible) but the reader not liking the story. Same for a Like.

If there is no likes or dislikes, then that pretty much it, no liked or disliked it. No one cared.

Also, pretty much the same thing happens with star ratings. If the story is meh, almost no one rates it. Does it means the story is meh or no one reads them?

 

[GDLiZy]

And here it is, liking and disliking is not rating the story, pretty much liking a post in the forum is you rating the quality of the said post. Like and dislike doesn't reflect the quality of the story, unlike star ratings which tries to be. A Dislike is not 1 star (which is meant to mean that the story is terrible) but the reader not liking the story. Same for a Like.

If there is no likes or dislikes, then that pretty much it, no liked or disliked it. No one cared.

Also, pretty much the same thing happens with star ratings. If the story is meh, almost no one rates it. Does it means the story is meh or no one reads them?

Except that star rating gives options for making it known that the story is meh. In the like-dislike system there is simply no option for not great not terrible.

If the readers didn't rate then it's the fault of the readers and not the fault of the system.

 

[yansusustories]

I think I can see both sides and after reading all the arguments in the post, I'd actually think combining both ideas would be a good idea. I'd like to first give my opinion on some of the things that have been said before I get to that. I'll put that in a spoiler though so those who don't care can just jump straight to the idea I finally arrived at.

Spoiler

But just take a look at the author general, and you'll see a lot of authors taking them badly and getting demotivated. We're even coming up with things like 'you're not a real author until you get the lowest rating' to make it a badge of honor or something.

I can see how this might seem strange. But honestly, we also have to keep in mind that there are quite a few new authors here that don't have much experience. getting 1-star ratings and bad reviews is really a normal thing. I mean just look at ratings/reviews for bestselling novels or even novels that won prizes: They get the same bad ones we do because despite what everyone says, taste still plays a large role in this. The big names are just better at dealing with that truth than a lot of the people on here are.
The yeah, while I wouldn't call it a "badge of honor", it's really just part of being an author.

So, think about, the only way you could rate a story with stars is when it is completed, and by leaving a review for the completed work. It would encourage more writers to actually complete their stories, which is a good thing in my mind.

I am completely against only allowing people to rate/like/dislike/review stories when they are completed. Why? because it can take an effing long time to finish a story. Even if you update daily, if you have a story with 300, 400 chapters, or even more, you'll need a year plus to finish it. Are we supposed to wait that long for ratings/likes/dislikes/reviews? Actually, that would be much more demotivating and get us to drop the stories instead of completing them.
As a reader, I'm also against this: I don't need to finish reading a story to be able to say whether I (dis)like it or what I would rate it. I can do it after half the story or even after a third or so. By that time, I already know if I like the theme, the characters, the setting, the writing style etc. Why shouldn't I be allowed to rate/like/dislike/review at that time? Sure, I could change my mind later on but if I continue reading, then I can still change that on SH. If I don't continue, then, well, the story was bad enough in the beginning to imo warrant giving a bad rating/dislike/bad review.

Then everything regarding the 'meh' option:

I would find that demotivating as a reader, since I'd have to give really great stories the same generic "liked this" as meh stories.

Isn't the 'meh' option covered by not voting?

I don't think anyone who find a story meh would interact with it (much) anyway. I mean, how often does people rate 2, 3 or 4, the various shades of meh? Rarely.

I read 'meh' stories and sometimes even to the end. Some 'meh' stories turn out to be either really good or bad later on or they stay 'meh' until the end but I continue with them because there's nothing better to read atm. I would want to have an option to express that I thought they were 'meh' because others might also like to read a 'meh' story once in a while. Personally, I normally give those 3 stars (Harry Potter gets 3 stars from me, btw, just to give a reference).

I don't wholly disagree with you, but what makes those 2, 3, or 4 stars 'better' is the assumption that people thought about them because they're different from the majority.
There's zero evidence that that is the case.

I value chapter comments and reviews more than any rating because there's actual quantifiable effort gone into those. Not an assumed effort.

I actually wanna laugh at this. It's great you get comments an reviews. Personally, I have 11 stories on SH, 4 of those finished (a short story, two full-length novels (90k and 150k), as well as a 5-volume series (400k+)). I only have 9 reviews though, about half of them coming from a longtime reader and friend of mine. I do get comments (some more by now) but not many. Since I respond to almost all comments, I think I can gauge that I likely have about 1k comments overall. To calculate: This means I don't even have one comment on each chapter I have published here. Not even 1!

I do value reviews and comments just as much as anyone else but let's face it: Most authors here don't get many of those. The rating system is important for that kind of author because it lets us see (to a larger degree than just a like/dislike feature would) what people think. For some of us, that's the only way to have at least some kind of interaction.

That's fair, except that you're forgetting the option to not give a vote or actually giving a review.
I think a star rating in a review uncoupled from a binary like/dislike for the novel would adress all the issues that every one of us has brought to the table.

This goes in the direction of what I'm thinking of but I'm not quite on board with like/dislike feature yet. Not giving a vote really isn't much of an option and would take away one "rating" from the abovementioned authors that already lack interaction. You honestly think that readers that can't be bothered to rate anything but 1 or 5 stars would go out of their way to write a review for saying 'meh' when the story doesn't fit a binary system? Truth be told, I don't see that happening. I can only see 'mediocre' authors (not necessarily) getting even less feedback.

Though, if you don't like it or dislike it, you are pretty much indifferent. The argument here seems to equate like=5 star and dislike=1 star. No. They are totally different system.

If there is no likes or dislikes, then that pretty much it, no liked or disliked it. No one cared.

This is actually what made me think of combining both systems: It's not good to show authors that nobody cares since that's even more demotivating than the lack of interaction we already have. But I can see where a binary system can come in handy as an additional source of information.

So, my idea would be:

• To leave the 5-star-rating system so readers can give any star-rating they feel applies to the story in question with however much nuance they wanna add.
• Add a binary feature. Although I would - instead of going for just like and dislike - go for a recommend/don't recommend feature.

Why do I think this might work?

• 'Meh' stories can still be addressed with the star rating by giving 3-stars or whatever by lazy people.
• People who want to give nuanced ratings can still give 2, 3, or 4-star ratings.
• If somebody especially likes or dislikes a story, they can recommend that (additionally to rating or not give a rating and just go for the binary system).
  • We could even make a recommendation list like that that could be used for the 'similar series' feature that still seems a bit lacking right now since if several people recommend A and B both, there's a chance other readers that liked A might like B and vice versa.
• If this is going too far for your taste: Opportunities to recommend stories or not recommend them could be limited, like, just one story a day or a week or so to make sure people don't just recommend or not recommend instead of rating. (Not quite a concern for me but just added in case.)

Also, if you guys are worried about discouraging authors (which seemed to be the original worry), you could even just make a 'recommendation' feature where people basically say 'Oh, yeah, this is one of my absolute faves, I'm gonna recommend it!' (instead of also having a 'not recommended' option). That way, no vulnerable author fee-fees are hurt.

 

[Ral]

Except that star rating gives options for making it known that the story is meh. In the like-dislike system there is simply no option for not great not terrible.

Just as you say that "it is meh don't vote" is not a valid argument, "it gives an option" is not a valid argument either.

And as I said, like/dislike is not about the quality of the work. You liking something doesn't mean it is great nor you disliking something doesn't mean it is not good. Like/dislike pretty much don't have a middle option (indifferent) because it does not matter.

Stop thinking that like/dislike is the same as rating system with the middle options removed. That like=5 star and dislike=1 star.

If the readers didn't rate then it's the fault of the readers and not the fault of the system.

That doesn't matter. What we should be concerned is if it works or not.

The thing is, humans are not perfect. You are kinda saying that it would work if only we are faultless, perfect. But, we aren't and never will be.

 

[BenJepheneT]

Uhmmm . . . I was not talking about reviews?

Ah, my bad.