I suspect that this will probably get flagged as not a question soon enough, but I believe it a valid one. I have a passing interest in rating systems like those used in games to rank player's relative strength when you consider all players. I have observed that stackoverflow also has a rating system. My question is this: does stackoverflow's rating system strive to measure the performance of its members in a relative sense, or does it strive to do the same in an absolute sense?

The former is clearly the better choice because gauging a members absolute performance is sillyness. Take for example this excerpt from the wikipedia entry on the Elo chess ranking system : "Performance can't be measured absolutely; it can only be inferred from wins and losses." Before you dismiss the Elo chess ranking, consider the fact that several very smart people have written their PhD thesis on this topic.

The major problem in ranking in an absolute sense is that the number of total units or points of rank are not conserved. Ideally the total ranking points of the entire system should grow in linear proportion to the number of users joining it, spread amongst the members. If you rank in an absolute sense you encourage a user to increase their performance by merely increasing the quantity of posts, not the quality.

Anyhow, this is merely one question and a bunch of observations. Feel free to comment, but I really only need the andwer to that question.

share|improve this question
2  
That's a really good question, but it belongs on meta.stackoverflow.com –  Avi Aug 18 '09 at 7:49
1  
ask this at meta.stackoverflow –  Zed Aug 18 '09 at 7:49
    
right, can someone move it or should I just requestion? –  Anonymous Aug 18 '09 at 7:52
    
someone meta this shizm –  Jason Aug 18 '09 at 7:52
    
i agree that the points system is not good. i'd prefer if it didn't exist. –  please delete me Aug 18 '09 at 7:53
    
If the user no longer exists, does it make this question another sod in the astroturf? –  random Aug 18 '09 at 8:18
1  
The user asked for it to be moved to meta. It is a reasonable question, and an interesting topic. –  Avi Aug 18 '09 at 8:32
    
Looks like this user went from gmatt to Anonymous in five minutes. –  random Aug 18 '09 at 8:46
add comment

migrated from stackoverflow.com Aug 18 '09 at 7:54

This question came from our site for professional and enthusiast programmers.

5 Answers

Generally, rating systems are for well-defined and fairly homogenous arenas. For example, one's United States Chess Federation rating is directly related to one's skill at playing chess, and goes up and down in response to results in USCF-rated games one plays.

This isn't the case on SO. It isn't well defined, except in the sense that there are rules. It is easily possible to game reputation in ways that aren't real helpful, while complying with the rules of the site. It isn't homogenous. For example, Jon Skeet has his rep not only for his ability and diligence in quickly producing good answers, but because his areas of expertise are quite popular here. Somebody doing the same for Haskell or Lisp would gain far less reputation for it.

As something of a C and C++ maven, I don't have quite as much opportunity to gain rep. This doesn't bother me, precisely because it isn't a rating system like the USCF has. If we were being seriously competitive, or if people thought I wasn't as good as a C# guy with twice my rep, it would bother me. It might drive away people who answer questions on less popular topics, and then those who ask, and SO would be the poorer for it.

share|improve this answer
add comment

I do not think that reputation is in a strict sense a ranking system. It is a measure of trust, an expression of usefulness, and a way of gratification.

Can we compare trust? Is somebody 2.5 times more trustworthy then somebody else? I think that the trust part of reputation is in how many people upvoted a certain answer. The more people found an answer to be useful and correct, the more I can trust it.

The upvotes on questions tell me how many people found this question useful.

Even if a person with much reputation asked or answered the question, you cannot derive the usefulness or correctness of the question or answer. Does it add to the respective measure? In such a way as this person obviously has asked or answered usefully or correctly many times, it does.

In what way is reputation a way of gratification? I guess in the first place people find my questions to be useful and my answers correct. People seem to profit from me. Secondly only, maybe I make a game out of it: I have been more useful or correct then 99% of the people on SO, or just sunk and wasted more time in SO.

If somebody has a lot of ELO or game points, when I go to battle him I know to be careful as he obviously has battled some strong contenders succesfully before.

Do we battle on S[OFU]? I don't, even if I sometimes get a little kick when my anser scores higher then somebody's with more reputation...

share|improve this answer
add comment

There aren't any players on SO. There are questioners and answerers. Questions and answers are rated, not people. That the points are transferred to the people is only the case, because the system has to trust you somehow.

share|improve this answer
add comment

The data dump is a gold mine for analyzing voting relations. If I had a bit more time and interest, I would draw a voting graph and possibly find some interesting cross voting patterns.

If you have some expertise in an SO topic, you could - in theory - cast your votes on the questions, then compare that with the community's response. Plus, subtract the cross-voting anomalies to truly evaluate a question.

share|improve this answer
add comment

I don't think that the system would benefit from being competitive in the zero-sum sense. There are many ways to earn 100 reputation. You could answer ten questions really well, earning 10 upvotes each, or you could answer 200 questions very blandly, getting an upvote every other time.

SO doesn't care which route you choose. If you choose the quality approach, then you earn trust by virtue of providing thoughtful, high-quality answers. If you choose the volume approach, then you earn trust by virtue of participating so much that you are understood to have learned how the community operates.

Thus reputation measures both quality and participation, and doesn't distinguish between the two. That's by design. Either way, you've earned your trust.

share|improve this answer
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .