Frogboy Frogboy

The Rating System is going to be changing..

▼

The Rating System is going to be changing..

from WinCustomize Forums

Okay, we've made posts and news items trying to get people to rate things in a reasonable manner but all have failed.

As someone else correctly wrote, the rating system here is a JOKE.

There are basically two problems: People who rate everything high and griefers who go around sabotaging other people's ratings.

The new system WILL work as follows. Yes, most of you will be ticked off for different reasons. Tough. The system is broken, we're going to fix it. Play time is over. It will work this way:

There will be a WEIGHT used to calculate the ratings. A rating by a citizen will counts a 1Xtheir rating. An apprentice 2X their rating, Master Apprentice 3X their rating and so on.

So when an admin or moderator rates a skin, it matters.  Now, the default rating from moderation will still only count as a weight of 1. But as people deliberately put in the effort to rate something, it will matter. 

Yes, it's unfair. It's unequal, whatever.  But the current system has become meaningless.

I have seen excellent ObjectDock icons get ratings of 4 because some griefer comes along and gives a bunch of them 1 totally counter balancing an admin rating of 7 (where the admin in question makes icons for a living).

I'm not sure when we'll get to this. It's pretty high on the priority list. This will probably result in a more skins going down in rating than skins going up in rating. A rating of 5 is an average looking skin. NOT 8. Some people seem to rate things as if anything under an 8 is bad. No, 5 is AVERAGE. Most skins should be rated at around a 5. Not 8.

30,123 views 87 replies
Reply #26 from WinCustomize Forums Top
there were survivors?
Reply #27 from WinCustomize Forums Top
5.5 is the numerical mean of the numbers 1-10, but I must agree that I think in alot of people's minds 7 represents an average effort in a given endeavour. They obviously should all be rounded up in a large pen and forced to study statistical analysis.
Ashes of the Singularity II
Reply #28 from WinCustomize Forums Top
Well I hope this takes some pressure away from rating. I used to download and rate and comment a lot and then everything got so intense. Mostly these days I only rate things that I have tested for people (or for people I have tested things for previously) and I know I am probably a little biased. I always feel guilty about the pieces I don't dl and test... that never seemed fair either (that a person more well-known might get more ratings in general than an unknown -- but there are only so many skins you can dl and test) I do plan to start being more diligent, though, when things are settled down more here. Still, I feel guilty when I rate one thing and not another.

It is hard to know what is right. I have found Paxx's skin evaluation guide a great help. http://pax.gslcom.com/ratings/rating.html

Actually, this may be silly but evaluating things, balancing the social pressure with the desire to be fair and feeling like I was failing miserably at that really took a lot of the fun away from WC. Then again, it only feels a little better downloading, using, enjoying stuff without commenting and rating.
Reply #29 from WinCustomize Forums Top
As of now the casual non-registered user can only see 8 ratings or higher in the wallpaper section. This may be true in other sections, but i havent payed attention. Will this be lowered after the ratings change?
Reply #30 from WinCustomize Forums Top
hehe,sounds like we are all in agreement then...5.5 is a 6, which is avg here.
Reply #31 from WinCustomize Forums Top
ahhhhhh it all becomes clear

/me wanders away muttering "5.5=6, 5.5=6, 5.5=6..."

Reply #32 from WinCustomize Forums Top
Yup. I think I've been rating wrongly now that I think of it. Not that I rate all the time. I mostly forget to rate. But I always seemed to think a 5 is low and I try to be fair. And get this if I like something I rate high, and if I don't I rate low.

FEEDBACK PLEASE: What all should we take into consideration whenn rating. Skin complexity? Skinner skill level? What?

Or am I just nuts.
Reply #33 from WinCustomize Forums Top
#24 by Admin Jafo - 1/27/2004 11:21:39 PM In apportioning marks or ratings, if they start at 1 and end at 10 then the REAL average is 5.5 which in the case of this site is rounded up to 6.


at first I was like:
Jafo... not you! Are you loosing it???

but I just realized that in order for the average to be 5 the ratings would have to go down to 0

Average between 0 and 10.



anyway, I guess this is the way to go. we will see if the weighted system is better than the system we have now.
Reply #34 from WinCustomize Forums Top

I think that a '1' is something you consider impossible to live with, and a '10' is something you believe is impossible to better.

Everything [approved] falls between these two extremes.

How an individual determines where a work fits between these is down to personal preference and opinion....

Reply #35 from WinCustomize Forums Top
Joe....I 'may' be 'losing' it but nothing is really 'loose' at all....Spell checker
Reply #36 from WinCustomize Forums Top
jarget........ check out the link in goodmorphing comment......... a very good guide to rating. Notice you'll have to actually use a skin to use the guide. This is not directed at you, but at anyone that rates based on the screenshot.



Powered by SkinBrowser!
Reply #37 from WinCustomize Forums Top
funny it is that TSF has almost ALL 10's

I love their work!
[Message Edited]
Reply #38 from WinCustomize Forums Top
travis shall use paxx's little system more often. it is very nice
Reply #40 from WinCustomize Forums Top
SkinsFactory have the best of the best designer team though Kona.
Reply #41 from WinCustomize Forums Top
#36 by Wizop Koasati - 1/27/2004 11:44:10 PM
jarget........ check out the link in goodmorphing comment......... a very good guide to rating. Notice you'll have to actually use a skin to use the guide. This is not directed at you, but at anyone that rates based on the screenshot.


10-4 I'm on it.
Reply #42 from WinCustomize Forums Top
Hrm...If by the 'new rules' (or as the old ones stated, but implemented poorly) the average rating will be a '5', shouldnt the rating at which a skin/wall/whatever is visible be lowered also?

If Im not mistaken, it's currently at '7' and any lower will not be shown to the public. I think it'd be fair to knock it down a notch or two...
[Message Edited]
Reply #43 from WinCustomize Forums Top
Hmmm, I think a subscriber (user lvl 4) should have a little more pull. We do help support the site and we should have a bigger voice if you know what I mean--higher than a registered citizen but lower than an apprentice. Are many 'griefers' subscribers? The new system sounds good.
Reply #44 from WinCustomize Forums Top
Or how about us who are both apprentices AND subscribers?
Reply #45 from WinCustomize Forums Top
Nothing changes. I just think one who helps support the site should have a bit bigger voice than one who just registers here. One who supports this site I think wouldn't be too likely to drop ratings on purpose, as that's the main problem. How accurate that is I don't know.
[Message Edited]
Reply #46 from WinCustomize Forums Top
DJ.... Citizens who aren't subscribers can't rate..... problem solved.  



Powered by SkinBrowser!
Reply #47 from WinCustomize Forums Top
I was thinking that perhaps buying a subscription could add 1 'Voting Power-Point' to your current collection of points.

If a citizen buys a subscript, he/she will now have 2 voting powerpoints (one from being a citizen and one from buying a subscription)

An apprentice who bought a subscript would have three Power-Points (2 for being a Apprentice and one for buying a subscript) and so on...

Just an idea.
Reply #48 from WinCustomize Forums Top
Citizens who aren't subscribers can't rate..... problem solved.


But then how many total votes would we be getting into the system?

Unless every subscription buyer votes on every work, it wont be a whole lot.

I think having the public have a say in the rating of a skin is a good idea, provided that their power is limited...
[Message Edited]
Reply #49 from WinCustomize Forums Top
Just thought I'd offer my 2 cents as one who is relatively new to the site after only 6 months, is an Object Desktop subscriber but not yet a WinCustomize subscriber, and who is still learning and wanting to become an active member of the community by hopefully soon submitting my own skins, and will then be subject to the ratings.

I have not yet been able to rate anything, and even if I had been permitted, I would likely have refrained from doing so because I have seen so much controversy and emotion displayed over ratings, and frankly, I don't feel I've learned enough yet to consider my rating to be worth as much as those who have been involved in skinning longer and with so much obvious talent.

For those reasons alone, I like the proposed system outlined here, and would be more willing to rate, knowing that the weight of my rating would contribute to the whole yet not be weighted as heavily as those of individuals more experienced and knowledgeable than myself.

By the way, though 5.5 is the arithmetic mean of the numbers 1 thru 10, statistically it is highly unlikely to ever be the actual mean, median, or modal rating here at WC. I can provide some pretty interesting studies (well, interesting to geeks like me ) to prove the point if anyone really cares, but what it comes down to is a common cross-cultural reluctance to humiliate someone else.

In most cultures, it is considered rude to give severe or harsh criticism even when it is completely justified - people can usually find a reason to temper the judgement or rating. They give credit for the person's effort, intentions, or simply to encourage her/him to do better and not quit. There is also another common tendency toward leniency in any rating system, because the rater wants to be liked by the person being rated.

One last reason why the lowest rating is unlikely to be given is due to the halo effect of the moderation process. Most people will assume that since the item being rated has passed moderation, it must therefore have SOME merit, so is very unlikely to be rated as a 1, unless someone has a personal grudge, or some flaw is discovered that was missed in moderation, like it being a rip, partial rip, or just being rumored to be a rip.

At any rate, that's about a dollar's worth of my 2 cents - sorry for being long winded, and thanks for reading if you got this far.
Reply #50 from WinCustomize Forums Top
I think this is better if for no other reason it adds some reward for being promoted more than just in the name. Having ranks in user level is something I've always loved about this place.

I still believe that a user should have the option to reset their rating during an update though, possible to do that too maybe?