I have a passing interest in rating systems like those used in games to rank player's relative strength when you consider all players. I have observed that Stack Overflow also has a rating system. My question is this: does Stack Overflow's rating system strive to measure the performance of its members in a relative sense, or does it strive to do the same in an absolute sense?

The former is clearly the better choice because gauging a members absolute performance is silliness. Take for example this excerpt from the Wikipedia entry on the Elo chess ranking system : "Performance can't be measured absolutely; it can only be inferred from wins and losses." Before you dismiss the Elo chess ranking, consider the fact that several very smart people have written their Ph.D. thesis on this topic.

The major problem in ranking in an absolute sense is that the number of total units or points of rank are not conserved. Ideally the total ranking points of the entire system should grow in linear proportion to the number of users joining it, spread amongst the members. If you rank in an absolute sense you encourage a user to increase their performance by merely increasing the quantity of posts, not the quality.

Anyhow, this is merely one question and a bunch of observations. Feel free to comment, but I really only need the answer to the above question.

  • i agree that the points system is not good. i'd prefer if it didn't exist.
    – please delete me
    Aug 18 '09 at 7:53
  • If the user no longer exists, does it make this question another sod in the astroturf?
    – random
    Aug 18 '09 at 8:18
  • Looks like this user went from gmatt to Anonymous in five minutes.
    – random
    Aug 18 '09 at 8:46
  • I think this should be applied to the ordering of answers, not the ranking of users
    – endolith
    Oct 11 '14 at 22:08

Generally, rating systems are for well-defined and fairly homogenous arenas. For example, one's United States Chess Federation rating is directly related to one's skill at playing chess, and goes up and down in response to results in USCF-rated games one plays.

This isn't the case on SO. It isn't well defined, except in the sense that there are rules. It is easily possible to game reputation in ways that aren't real helpful, while complying with the rules of the site. It isn't homogenous. For example, Jon Skeet has his rep not only for his ability and diligence in quickly producing good answers, but because his areas of expertise are quite popular here. Somebody doing the same for Haskell or Lisp would gain far less reputation for it.

As something of a C and C++ maven, I don't have quite as much opportunity to gain rep. This doesn't bother me, precisely because it isn't a rating system like the USCF has. If we were being seriously competitive, or if people thought I wasn't as good as a C# guy with twice my rep, it would bother me. It might drive away people who answer questions on less popular topics, and then those who ask, and SO would be the poorer for it.


I do not think that reputation is in a strict sense a ranking system. It is a measure of trust, an expression of usefulness, and a way of gratification.

Can we compare trust? Is somebody 2.5 times more trustworthy then somebody else? I think that the trust part of reputation is in how many people upvoted a certain answer. The more people found an answer to be useful and correct, the more I can trust it.

The upvotes on questions tell me how many people found this question useful.

Even if a person with much reputation asked or answered the question, you cannot derive the usefulness or correctness of the question or answer. Does it add to the respective measure? In such a way as this person obviously has asked or answered usefully or correctly many times, it does.

In what way is reputation a way of gratification? I guess in the first place people find my questions to be useful and my answers correct. People seem to profit from me. Secondly only, maybe I make a game out of it: I have been more useful or correct then 99% of the people on SO, or just sunk and wasted more time in SO.

If somebody has a lot of ELO or game points, when I go to battle him I know to be careful as he obviously has battled some strong contenders succesfully before.

Do we battle on S[OFU]? I don't, even if I sometimes get a little kick when my anser scores higher then somebody's with more reputation...


There aren't any players on SO. There are questioners and answerers. Questions and answers are rated, not people. That the points are transferred to the people is only the case, because the system has to trust you somehow.


I don't think that the system would benefit from being competitive in the zero-sum sense. There are many ways to earn 100 reputation. You could answer ten questions really well, earning 10 upvotes each, or you could answer 200 questions very blandly, getting an upvote every other time.

SO doesn't care which route you choose. If you choose the quality approach, then you earn trust by virtue of providing thoughtful, high-quality answers. If you choose the volume approach, then you earn trust by virtue of participating so much that you are understood to have learned how the community operates.

Thus reputation measures both quality and participation, and doesn't distinguish between the two. That's by design. Either way, you've earned your trust.


The data dump is a gold mine for analyzing voting relations. If I had a bit more time and interest, I would draw a voting graph and possibly find some interesting cross voting patterns.

If you have some expertise in an SO topic, you could - in theory - cast your votes on the questions, then compare that with the community's response. Plus, subtract the cross-voting anomalies to truly evaluate a question.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .