I have a passing interest in rating systems like those used in games to rank player's relative strength when you consider all players. I have observed that Stack Overflow also has a rating system. My question is this: does Stack Overflow's rating system strive to measure the performance of its members in a relative sense, or does it strive to do the same in an absolute sense?
The former is clearly the better choice because gauging a members absolute performance is silliness. Take for example this excerpt from the Wikipedia entry on the Elo chess ranking system : "Performance can't be measured absolutely; it can only be inferred from wins and losses." Before you dismiss the Elo chess ranking, consider the fact that several very smart people have written their Ph.D. thesis on this topic.
The major problem in ranking in an absolute sense is that the number of total units or points of rank are not conserved. Ideally the total ranking points of the entire system should grow in linear proportion to the number of users joining it, spread amongst the members. If you rank in an absolute sense you encourage a user to increase their performance by merely increasing the quantity of posts, not the quality.
Anyhow, this is merely one question and a bunch of observations. Feel free to comment, but I really only need the answer to the above question.