What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 127 Stack Exchange communities.

While reading this question, Which accounts have more questions than answers?, I started wondering what ways are there that we can measure participation more accurately than reputation.

I've always been skeptical that a user's reputation score actually reflects their "reputation." That is your level of knowledge and ability to ask/answer questions may not be correlated to how well you "play together in groups," as they used to say in kindergarten.

On the theory that, if you can't say something nice about someone, you shouldn't say anything at all, I wonder if the ratio of upvotes to downvotes may be a better indication of your "plays well together" score than your reputation. Another potential measure may be the ratio of upvotes to downvotes on your own questions. Or flags to votes?

What other metrics, or would any of these metrics, be useful to expose for users on the site?

share|improve this question
7  
Unless I have misunderstood something, SO is about answering technical questions correctly. –  nb69307 Jul 5 '09 at 18:08
2  
One would hope that would be the case, but I can think of at least one situation where I was serially downvoted by someone because I happened to comment that I liked a humorous comment on one of their questions. That person has a high reputation score on the site, but not a particularly high reputation with me. –  tvanfosson Jul 5 '09 at 18:16
3  
I get a fair number of downvotes every day on both answers and questions that are by any measure technically correct. I live with it, I don't start trying to stop all downvoting, or suggest that everyone on SO must be "nice". –  nb69307 Jul 5 '09 at 18:26
    
you might be interested in this suggestion: meta.stackexchange.com/questions/2028/… –  Shog9 Jul 5 '09 at 18:30
    
@Neil -- I did make a suggestion that downvoting questions seems unnecessary since closing or commenting seems to be a better way to improve questions. In this question, though, I'm just interested in whether there is a better way to measure actual reputation. I suppose its more of an academic interest as someone's reputation, in a personal sense, is not particularly useful in evaluating their answers. I can see where it might be useful to a moderator in evaluating disputes. –  tvanfosson Jul 5 '09 at 18:30
    
@Shog -- I'm ok with the current system of handling serial voting. I brought it up just to say that "reputation score" != "reputation". –  tvanfosson Jul 5 '09 at 18:34
    
Ah, agreed. Given how many things can now affect reputation (including the built-in discrepancy between rep gained by upvotes and lost to downvotes), it would be interesting - and not to hard to obtain - a hard ratio of up/down votes given to a person's posts. –  Shog9 Jul 5 '09 at 18:49
add comment

7 Answers

Another potential measure may be the ratio of upvotes to downvotes on your own questions.

I could see that being interesting... Everyone has the same number of votes to give (whether or not they actually do), but the number of votes they can receive is much less bounded.

share|improve this answer
add comment

Another potential measure may be the ratio of upvotes to downvotes on your own questions.

This is an interesting way to measure but i don't know if it would work as intended. Since i've seen OP's that downvote all answers that don't solve their problem. And others who ask a question and upvote every answer they get (even though they are wrong)

share|improve this answer
    
I had intended this to be the number of votes you receive, but how you reward people who answer your questions is also interesting. –  tvanfosson Jul 5 '09 at 18:06
    
I think the latter is more a measure of how much of a [redacted] you are, not that that isn't interesting... –  bananakata Jul 5 '09 at 18:11
    
I think it's perfectly valid to downvote whoever doesn't answer your problem without being a [redacted]. After all, the whole point is for the OP to get an answer to his question. All the answers that don't help are noise that can be voted down with no hard feelings. –  Nathan Fellman Jul 5 '09 at 19:53
    
@annakata, [redacted] doesn't work since we all know you can't edit comments =P. I like your style though. –  devinb Jul 5 '09 at 19:56
add comment

On a personal level I've only ever down-voted twice on So and once here and then in one case it was borderline and only the "promise" of the critic badge tipped me over the edge.

Partly it's the cost in rep to myself that dissuades me, but mainly it's the thought that of itself down-voting won't achieve the desired result. I'd rather comment when I see something incomplete or wrong in an effort to get the OP to correct the post.

share|improve this answer
    
The reason that it's interesting to me is that when there is a on-going issue, one of the things I look at to get a feel for how a person interacts is their upvote/downvote ratio. The other thing I look at is their comment history -- comments seem to say a lot more than questions/answers about someone. –  tvanfosson Jul 5 '09 at 18:08
1  
But you don't look at the quality of their answers? –  nb69307 Jul 5 '09 at 18:42
    
@Neil -- the issues that I was talking about typically don't have anything to do with their technical capabilities, but rather their behavior. –  tvanfosson Jul 5 '09 at 21:21
add comment

All the ratios you talk about help tell the story of a user but fails to tell the whole story.

  • Answer/Question: There are people who are good at asking questions. A good question can contribute to the discussion and make us think. And there are the questions that are used to game the system.
  • Contributions/Net Votes: What if the user is giving well thought out answers to questions in narrow areas?
  • Up Votes/Down Votes: Do we want people who are only going to give safe answers?
  • Touchdown/Interception: What if you are a running game quarterback? What if your receivers aren't fast enough to Go All The Way?

I'm not saying that there aren't good reasons to monitor these things. Just don't take it too far.

share|improve this answer
    
Can you explain what you mean by Touchdown/Interception? –  Nathan Fellman Jul 5 '09 at 19:52
    
I was being a little sarcastic. I was reading something about an American Football Quarterback and it gave that stat. I was thinking about how flawed that number was. So I guess it was the seed of this thought. I guess it is an example of a stat outside this domain. I could have used Body Mass Index, or such. –  jrcs3 Jul 5 '09 at 20:25
    
Touchdown/Interception (or Homerun/Strikeout - which probably has a formal name given how stat-oriented baseball is) would indicate how much of a risk taker someone is. Although I don't know how you'd derive that stat here. </ignoring sarcasm> –  Dennis Williamson Jul 5 '09 at 20:45
    
@Dennis Williamson: Shog9's stat is a good candidate for risk taking in SO. –  jrcs3 Jul 5 '09 at 21:14
add comment

I certainly believe that you ratio of answers to questions is incredibly important and should be used more often as a metric along with reputation in determining who should be promoted to new privileges. This would stop users from asking hundreds of questions and no real answers from achieving enough reputation to be given mod abilities.

share|improve this answer
    
The OP's question was on ways to measure participation. As long as the questions are good, they contribute as much to the community as answers do. If someone is asking a lot of questions, they are participating in the community. –  codewaggle Jun 12 '12 at 7:42
add comment
Questions+Answers/Comments

How much non rep earning activity is the user willing to participate in.

I won't remove this (then the comments won't make sense) but I acknowledge that it is deeply flawed: "You could also look at Wiki questions and answers."

share|improve this answer
    
The tricky thing about wiki is that it can skew any measure beyond breaking point. Just look at the huge numbers of votes involved... but posting a single xkcd/dilbert comic doesn't mean all that much. To me, at least. –  Marc Gravell Jul 5 '09 at 21:16
    
If I was measuring selfless participation, I would count the # of posts not the phantom points. –  jrcs3 Jul 5 '09 at 21:38
add comment

This seems like a really hard thing to accurately quantify.

Ratio of [what is currently rep] to number of questions and answers seems like the most meaningful thing to me, but I remember looking at that a while ago and it actually didn't work out as expected given a few users with extreme profiles.

I'd love to see some serious research on this.

share|improve this answer
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .