What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 130 Stack Exchange communities.

Has anyone considered using a PageRank-like algorithm for the reputation system? This would help to address the common criticism that users gain more reputation for witty one-liners than detailed, technical answers. (Anecdotally, I find this to be true. I got more reputation for a one-line answer to a funny question than any detailed answer I ever gave.)

So why not apply some ideas from PageRank to Stack Overflow? Google's PageRank is so successful because a link from a respected site will give your webpage a lot more rank than a link from some obscure site. In the same way that Google doesn't count all links equally, Stack Overflow shouldn't count all up-votes equally.

An up-vote from someone with a lot of reputation should be worth more than an up-vote from somebody with no reputation. Intuitively, an upvote from Jon Skeet is worth a lot more than an upvote from (unknown yahoo user), because the community trusts him.

This would also help to mitigate a lot of other ways you can game the system.

Edit:

Ah, I thought my idea was original, but evidently not. Having read the reactions to the previous person who suggested this, it is clear the SO community at large is not in favor of this idea. Still, I don't find the reasoning very solid. Disturbingly, the main line of reasoning against weighted up-voting (WUV) also implies that reputation is almost entirely useless.

The main reasons against WUV are:

1) Reputation != expertise. It is mostly an indication of how much time you spent on SO.

Really? Then perhaps we should just replace the rep system with a simple uptime-like readout. But seriously, the SO FAQ specifically says "Reputation is never given, it is earned by convincing other Stack Overflow users that you know what you're talking about". Clearly, there is at least supposed to be a strong correlation between reputation and computer programming knowledge. And if there isn't a strong correlation between reputation and computer programming knowledge, then the whole rep system is basically useless; it becomes nothing more than a measure of social standing.

But if reputation is correlated with computer programming knowledge, it seems obvious that an upvote from someone with a higher rep is likely to be more valuable. Sure, there are edge cases where this won't be the case; for that matter, there are horrible webpages with a high PageRank on Google - but I'm not looking for perfection here.

2) Reputation does not mean you're an expert, because reputation can be earned by asking masses of low value questions.

True. But the solution to this is simple. Question upvotes shouldn't increase reputation. This makes sense, because the only reason for WUV is because we should value the answers given by more knowledgeable people. But it doesn't take any expertise to ask a question. So, upvotes on questions shouldn't increase reputation.

3) Reputation is too broad a measurement; it doesn't indicate that a particular voter is knowledgeable in that particular subject.

This is probably the only significantly difficult rebuttal. It's true that a person's overall rep score doesn't indicate they are knowledgeable in any particular subject. But that doesn't mean a WUV system couldn't take specific expertise into account. In fact, the SO tagging system already records the information necessary to resolve this. For example, if you look at my user-info page, you'll see that I've answered a lot of C++ questions, but no Lisp questions. Therefore, my upvote on a C++ question should be weighted more than my upvote on a Lisp question. Adding a WUV system which takes into account expertise in specific areas would really help to better identify answers which are more likely to be valuable to the questioner, as well as everyone throughout posterity who reads the question.

4) Adding a WUV system would make it more difficult for new people to gain reputation

A WUV system will only make things difficult for new people who don't know what they're talking about. People who actually know what they're talking about, and who provide useful answers, will quickly gain reputation in a WUV system. (Perhaps even faster than in the current system.)

And finally, a WUV system in general would help to mitigate many of the ways people are able to game the system to increase their rep points, or otherwise increase their rep in ways that don't correlate to actual expertise.

share|improve this question
2  
The trusted users made their rep with one-liners. Your idea is leaking. –  Ladybug Killer Nov 6 '09 at 17:28
1  
This idea has been thoroughly thrashed before: meta.stackexchange.com/questions/22151/… –  random Nov 7 '09 at 0:57
1  
+1 for great idea (this should be retagged with feature-request). There is a solution to people gaining rep from asking lots of questions, and to answers gaining huge momentum (usually the one-liners) just because they answered in the first 30 seconds. As for @LadyBug's comment: this isn't true after a rep-recalculation. –  BlueRaja May 13 '10 at 16:24

2 Answers 2

Why not use the algorithm described by Randall Munroe here: http://blog.reddit.com/2009/10/reddits-new-comment-sorting-system.html ?

It seems to be pretty effective, and intuitively, it does what you want it to do.

share|improve this answer

The problem I see here is that you have no way of gauging the quality of a reputation score. High rep doesn't always equate to high quality.

There are other users of Stack Overflow who have lower scores but who post very high quality answers or questions in specialised areas, perhaps less often, and where the topic is of less interest than say C# Generics, Java Date Time problems or SQL JOIN questions. Who's to say that these votes are less valuable than Jon's, Joel's or Marc's for example?

Also what happens when a high rep 'trusted' user casts a down-vote? Should that down-vote also be weighted by taking into account their rep? Votes can be cast for idealogical reasons rather than technical ones and that would introduce some very odd skewing.

I can see what you're getting at but I think this is a flawed idea. There would need to be extra 'quality' data applied to this ranking or weighting system which isn't captured by Stack Overflow presently.

share|improve this answer
    
True, but your criticism applies equally to the current state of the rep system. And of course, in the case of Google, there's nothing to prevent ideological interests from increasing the PageRank of a site. But overall, Google is pretty darn good. Similarly, with a system based on weighted up-votes, it's more likely that over time, answers from better qualified users will be given more value, mirroring the real-world value we invest in expert advice. –  Charles Salvia Nov 6 '09 at 17:42
    
We'd still need some extra data or quality screening to be able to do this. –  Kev Nov 6 '09 at 17:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .