What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 127 Stack Exchange communities.

Sometimes I'm reading an answer and it has a certain "smell" to it that I can't quite pinpoint, but makes me distrustful of the answer and I feel surprised to see it coming from a user with a high rep. The rep system is what it is, and I'm not arguing against it, but perhaps as a single metric it has too many interpretations. A high rep could mean someone who has a lot of quality questions and answers, but could also mean someone who scrapes a few rep from a large quantity of dubious-quality answers.

I heard once that in the baseball world, a player's batting average is taken into consideration more often than their ability to do homers. Regardless of whether that's true or not, would a "rep per answer" stat be useful? I think it should exclude questions with bounties, but of course that's debatable.

I also wonder if the actual display of such a batting average would cause more "meta-rep gaming" like strategic down-voting - griefers see a high batting average and automatically down-vote or some such...

Edit:
IMHO, a lot of the value in the rep system is to build literal reputation for a user. A high rep really does bring a bit of prestige, and user who has a high rep will probably (and rightfully) be proud of their rep. Having a high rep means something - why else do people do rep-whoring, or fastest-gun-in-the-west sort of stuff? Ask any user who has more than 5k rep if they'd be happy to have it reset to 1.

My point is that we shouldn't lead ourselves to believe that "the system is designed to produce good answers, nevermind if an answerer has high rep or not", because I don't buy it. Rep-whoring is a symptom of this, I think. A real reputation would be developed if a person is known to consistently produce good, thoughtful answers, and a single number at a glance just doesn't cut it. I guess my desire would be to have the users kept more honest with themselves, and I don't think a rep average is the answer but it does help with evaluating the users, not just the answers.

Only a very tiny percentage of users are on often enough to participate in the in-jokes and Jon Skeet fawning, the vast majority of users do not know that so-and-so is full of B.S. and that sort of thing. If you fast-forward a few years, SO will continue to get much more users and things like this will get more and more ambiguous. The average user will need a little more help figuring out who is trustworthy and who is not, and that need will increase as the community gets bigger and bigger.

Hopefully that makes sense. I feel like I've been ranting and that's never been helpful.

Edit 2:

devinb provided a very handy list for me to show why rep avg can be a great metric to use: His list is this:

  • There are people with many answers in niche tags who would have low rep-per-answer.
  • There are people who have many answers in very popular tags who aren't fast enough to get huge upvotes.
  • There are people who answer very few questions, but answer them very very thoroughly and get huge upvotes.
  • There are people who only answer joke-type questions and get huge (undeserved) upvotes.
  • There are people who put off-the-cuff answers on a lot of questions and get a few upvotes.

He also provides a list of things he would have to do in order to increase his rep average:

  • Delete my lower voted posts (even if they are still valid and helpful)
  • Ignore any low-view tags or questions
  • Create a sock-puppet user to upvote all my posts
  • Downvote other users to make my posts look more attractive

Here are my responses:

  • niche areas (gnostradamus also mentions this in his answer): if you're answering questions in a niche area, your rep is already low relative to the # of answers you've posted. You shouldn't be surprised by a corresponding low average. Working in a niche area and providing quality answers anyway despite a low rep score means you wouldn't care about a low rep average either.

  • many answers, but in popular tags and too slow to get huge upvotes: if they're slow, they're not getting rep from the huge initial upvotes either. And "slow" is relative, because fastest-gun behavior encourages fast (and often crap) answers. People with fast, crap answers would tend towards a lower average, which is where this metric can help out. See the last point on this list.

  • people who answer very few questions thoroughly and get huge upvotes: if they answer questions like that, they deserve their high rep and their high average. To me, this is the ideal case. For a site premised on providing great answers, don't we want a vast majority of people to answer like that?

  • huge undeserved upvotes on joke questions: the undeserved rep from those upvotes would lead to an undeserved higher average. This should be an argument against awarding rep from joke answers, not against adding a rep average.

  • lots of answers with a few upvotes: this is precisely the behavior I'd like to see less of. A person with a very high rep score coupled with a very low rep average would suggest this sort of person, and would immediately slow down the impulse to do fastest-gun type of behavior - if someone cares so much about getting rep, they'll also care a little bit about not having a really low average.

Now, about the behaviors one would have to do in order to raise their average:

  • delete lower voted posts: if you care about providing good answers, you'd leave it there. If you delete them, someone else who cares about good answers will put one up anyway despite not getting rep for it. It already happens - see those who work in niche areas.

  • ignore low-view tags/questions: this is the niche segment case.

  • sock-puppet to upvote self/downvote others: there's already algorithms to detect this sort of abuse, right? those upvotes get discarded anyway.

Also, note how all these behaviors DO NOT lead to the person providing crap answers. If someone wants a really high average, they'd have to have REALLY GOOD answers. Is that a bad thing?

Now to take things completely from my side.

The way things are now, you only have the one rep metric. The rep metric is the single largest motivator that the site provides to encourage people to provide answers. If you ask 10 different users what a high metric means to them, you might get 10 different answers, but the general idea is that all 10 of those would be something positive - e.g. a high rep means the person is smart, participates a lot, provides great answers, etc. Contrast that with a low rep, which just tends towards "newbie with no privileges". This is why so many people want a high rep, and play all those rep games - there should be no question about it, rep-whoring is not desirable.

Now add rep average to the mix. Now you have 4 general cases:

  • low rep, low average: this could be a newbie, but if you couple that with # of answers, this could indicate a niche segment as well.

  • high rep, high average: this is an indicator of a quality answer, because the person in the past has gotten a lot of upvotes for their answers. The bad part is that meta-behavior may lead towards upvoting due to an already high average, but that seems to be an already-present sympton with regards to high rep.

  • low rep, high average: this is also an indicator of a quality answer, because their rep came from fewer answers. If the current answer isn't high quality, their rep would fall faster than an established, stable "player".

  • high rep, low average: the greater the difference, the more I'd distrust this answer. To me, this identifies the type of user who rep-whores a lot.

share|improve this question
    
P.S. - after posting this question I saw this question about "hit rate" (at meta.stackexchange.com/questions/2238/…), but that's slightly different - it's asking about a percentage of answers which get accepted. –  weiji Sep 11 '09 at 1:51
    
I came up with an idea, but it seems to be disliked a lot more than I anticipated: meta.stackexchange.com/questions/98141/… –  GeneralBecos Jul 11 '11 at 15:51
    
@GeneralBecos - Unfortunately, after reading about your idea I became one of those who disliked it. My suggestion with this rep batting average is to reduce the importance of the user's rep; your idea of weighing votes with it will give the rep metric even more power. –  weiji Aug 11 '11 at 23:43
    
What about people who post both in niche tags and in mainstream tags (like me)? I'm posting a lot of answers in both small and big tags. (My 917 answers are in 802 different tags - still, most of them are also in the Java tag.) Actually, I have a lot of answers which are not voted a lot - these are mostly in the "too late" category, I would say. Do you want to encourage me to delete the posts with no votes (or only one), even if I think they provide a good answer? (I think if I'll start deleting, I'll be blocked soon from further deleting.) –  Paŭlo Ebermann Aug 12 '11 at 3:17
    
@Paulo - I cover a situation like yours - see under "Edit 2" in my question. Basically, if you are participating in niche tags, then you already have a low rep compared to popular tags, so a low rep average shouldn't mean anything to you. The desire to provide good answers should be greater than the desire to game a high average. –  weiji Aug 12 '11 at 17:06
    
But the effect is that people will think that my answers in the popular tags are not trustable, too. (I think "adding a second trust metric" was your goal, wasn't it?) Actually, a better idea would be not to show the reputation at all for each answer, instead show it only on the user profile page. There also could be several other metrics, like reputation/answers or average score for answers (and average score for questions). –  Paŭlo Ebermann Aug 12 '11 at 18:22
    
That's a good point, but I was actually thinking of a general rep/answer average. When I originally suggested this, I don't think any per-tag metrics had even been established yet. I'd like to think that for this situation, a good answer would still get upvoted alot, so if there's a per-tag breakdown the average should show up higher than an average-of-averages. It's funny you should mention removing the display of rep - I also suggested as a direct result of the amount of dislike from this question. Search for "Hide the rep and badge count" (or just go to my profile page) –  weiji Aug 12 '11 at 19:47
add comment

6 Answers

Go with your gut - if an answer stinks, it doesn't matter how much rep the author has. Plenty of high-rep users, even those who generally do write good answers, occasionally venture into territory they aren't quite familiar with... and make mistakes or write something misleading or sub-optimal as a result.

share|improve this answer
    
Thanks - but an expert wouldn't spend the majority of their time answering things outside of their field, right? A miss here and there would not affect their rep/answer stat very much. I'm expanding on my thoughts in a separate answer... hold on. –  weiji Sep 11 '09 at 1:56
8  
Doesn't matter - if you're trusting a number in the face of your initial distrust, that's bad. Better you should voice your misgivings and give the author a chance to address them - if he knows what he's talking about, you'll both be better off... and if he doesn't, then you've not misplaced your trust. –  Shog9 Sep 11 '09 at 2:20
1  
If you care to look it up, you'll find that I'm in the SO 10K+ club. I make mistakes and screw up from time to time. You can trust my answers to be honest and sometimes researched opinions of a person knowledgeable in C++ and some other fields. You can't trust them to be uniformly correct. –  David Thornley Sep 11 '09 at 15:16
    
@David - I don't understand your position on this rep avg suggestion - I'd think that your type of behavior would give you a rep avg to be proud of. If you really are as you describe yourself, I'd imagine your rep avg would be higher than the majority of junk answerers with high rep that I see on SE sites. I didn't suggest (nor do I want) that all your answers should have an equal amount of upvotes or rep. –  weiji Aug 12 '11 at 0:07
add comment

A "rep-per-answer" metric wouldn't necessarily be a very good measure of the quality of a users answers either. Some of us (like myself) are highly active in niche tags, which don't generally get many upvotes or views since they may not have very large followings. I estimate my "rep-per-answer" measure to only be a bit more than 30 (3 upvotes) for MATLAB answers I give, which is probably a lot less than people who answer Java or C# questions.

share|improve this answer
    
I wonder if your situation is the opposite - with a specialized field like yours, an experienced SO user might conclude your rep is under-inflated. I realized I'm concerned about users who have artificially high reps. –  weiji Sep 11 '09 at 1:54
3  
Wait a second! Let's do an example (ignoring that there is rep for questions): rep/answers: gnovice: 6790/251 = 27,1 - Jon Skeet: 95569/5287 = 18,1 - Now, where is your problem, gnovice? –  Ladybug Killer Sep 11 '09 at 8:59
2  
@John: Interesting numbers, but your total number of answers also includes answers to wiki questions (which shouldn't really be included in the total). I was looking at only the tag stats for matlab, which currently has me at 439 upvotes for 133 answers: 3.3 upvotes per answer (admittedly, this doesn't count Rep from accepted answers and bounties). However, it is nice to see a metric where I'm beating the Skeet. ;) –  gnostradamus Sep 11 '09 at 13:36
    
In your case, a low rep avg would correlate with a low rep, both relative to the number of answers you've provided. There is no real loss for you. You just brush it off as "I'm a specialist in a niche area". You still provide your quality answers, right? My suggestion is intended to mitigate the motivations behind crap answers. –  weiji Aug 12 '11 at 0:12
1  
As of 12/9/2011 your rep per answer is 54.2. jonskeet is only 21.1 :) I think RPA is more a measure of your audience than anything else. –  Will Bickford Dec 10 '11 at 1:29
add comment

Rep-per-answer is not a useful metric

  • There are people with many answers in niche tags who would have low rep-per-answer.
  • There are people who have many answers in very popular tags who aren't fast enough to get huge upvotes.
  • There are people who answer very few questions, but answer them very very thoroughly and get huge upvotes.
  • There are people who only answer joke-type questions and get huge (undeserved) upvotes.
  • There are people who put off-the-cuff answers on a lot of questions and get a few upvotes.

The point being that the rep-per-answer averages will vary wildly, but will not add any extra information about the user who is giving you the answer. Because rep is given out arbitrarily, there is not (and never has been) a way to firmly link rep-gain to usefulness.

Edit

This is from an answer I posted on a similar question, but it got closed.

Rep Average is a bad idea.

In order for me to control (increase) my reputation, I need to invest more time in StackOverflow. This is very beneficial to the community.

In order for me to control (increase) my average-rep-per-post, I need to do one or more of the following things. (only a partial list)

* Delete my lower voted posts (even if they are still valid and helpful)
* Ignore any low-view tags or questions
* Create a sock-puppet user to upvote all my posts
* Downvote other users to make my posts look more attractive

You'll note that "write higher quality posts" is not on the list. There's a reason for that. Even a high quality post has a chance of not getting upvoted, which means it would lower my rep-average, and then I'd have to delete it. Also, "answer questions very quickly" is not on the list for the same reason.

Every item on that list is not a desirable behaviour. So there isn't any benefit to showing the rep batting average.

share|improve this answer
2  
Thing is, if a person answers few questions but does those very well, any answer is likely to be excellent. Similarly, a person who answers a lot of questions, and most answers are okay, is likely to have lower rep-per-answer and their answers are less likely to be good. It seems to me that these are cases for rep-per-answer, not against. –  David Thornley Sep 11 '09 at 15:18
    
@David: You'd be encouraging people to answer only high-view questions. Answering a niche question will ALWAYS bring down your avg, and so people would be discouraged from doing it. Also, tumbleweed questions would have no reason to be answered either, because it will probably bring down your rep. –  devinb Sep 11 '09 at 15:30
    
@devinb - I think it's agreed that rep gaming can produce bad answers, which is undesired. But rep avg gaming? Does that produce bad answers? I've edited my question to address these. –  weiji Aug 12 '11 at 0:19
add comment

Some people have artificially high reps. Some don't. There can be only one Skeet.

Never trust the number. The number is a game. Look at rep not as knowledge of a domain expert, but rather participation on the site. It has more to do with how active someone is than what he/she knows, necessarily.

Never judge an answer by the person giving it. Judge the answer on its own merits. If you think it's wrong, downvote it -- that's what the tool is there for.

If you're reading an answer on a question because it's something you need, validate it before you use it in production. (This should be something we all do as a matter of course anyway, regardless of where our source material originates.) If it's wrong, downvote it.

If it smells, and there's better, upvote the better. (And of course, downvote the smell.)

Again -- the answers should be judged on their own merits. Sometimes very inexperienced users just happen to have the right answer right away. Sometimes really experienced users happen to be wrong. The answers speak for themselves.

(All that said, I wouldn't necessarily mind seeing this statistic. But I wouldn't use it for anything more than entertainment value.)

share|improve this answer
1  
I like this interpretation. However, this then makes me wonder if "reputation" is the wrong term to use. Slashdot, for instance, just calls it karma - it's arbitrary, kinda misunderstood, etc. But "reputation" is a little more well defined - it's like saying "respect" or "status". –  weiji Sep 12 '09 at 0:46
add comment

Just as another "don't use rep-per-answer" answer, if you mean actual rep, high-rep users typically miss out on a lot of rep due to the cap (i.e. a lot of questions with votes generate zero rep); so an infrequent user with a few good days could easily have a higher rep-per-answer rate than the hard-core high-volume user.

But as has already been stressed; a bad answer is a bad answer is a bad answer. I thought Jon wrote a bad answer once, but it turned out the answer was perfect and the question was wrong ;-p (substitute "question" for "universe", "compiler", etc to suit whimsy).

share|improve this answer
    
+1: This would definitely skew the rep-per-answer metric too low for those who answer lots of questions after hitting the rep cap. –  gnostradamus Sep 11 '09 at 14:39
    
how about a "rep-per-answer-pre-daily-cap"? :) –  weiji Sep 12 '09 at 0:44
    
That seems needlessly complex? –  Marc Gravell Sep 12 '09 at 8:49
    
@Mark - complex? Likely. Needlessly? No. I basically stopped paying attention to reps once I notice that MANY mid-rep (3k-7k) people's main scores came from early non-programming-related fluff threads and they had very few highly marked technical Q/As. That's why I previously advocated per-tag rep which was mercilessly voted down on Meta :) –  DVK Oct 7 '09 at 17:30
    
@DVK well, a year later I come back to this one, I've added a pretty big edit. A rep-per-answer-pre-daily-cap would be complex only because there's so much history to backfill. If the suggestion is turned down because of that, at least agree it's a good idea first - let's see if I'm persuasive or not :) –  weiji Aug 12 '11 at 1:20
    
Another way to look at a capped user is that people then start to understand what a "high" rep average should be, and someone with a much higher average is probably artificially inflated. So your argument actually supports my suggestion, because IT SHOWS THAT REP AVERAGE IS SELF CORRECTING. (All caps because I did not think of that without your prompt, thank you) :) –  weiji Aug 12 '11 at 1:27
    
One could also argue that you should raise the daily limit, and/or find a balance between when too much time per day on SO is unhealthy. Again, having a rep average is helping the community by providing a way of moderating burnout and curbing overindulgence. –  weiji Aug 12 '11 at 1:29
add comment

I think it’s a great idea. Just like reputation, it would be fun statistic. It doesn’t have to be used to help predict the helpfulness of an answer. Many of us programmers love statistics and data points, right? By the way, rep/answer would be more synonymous with baseball’s slugging percentage. Also, just upvotes on answers should count. A batting average equivalent would be answers with 1 or more upvotes / total answers. This would give a user a “hit” with just one upvote. Why not put this on the user’s statistics tab?

I don’t think including a statistic like this would cause users to delete “good” answers with 0 votes. They might go back and delete half hearted answers with no hope of an upvote, but I would think this is a desirable behavior to clean out the noise.

Here are some more fun statistics with its baseball counterpart.

  • accepted answer = Home Run (Did Skeet pass Bonds yet?)
  • accepted answer to a bounty question = Grand Slam
share|improve this answer
    
I don't follow baseball, so one coworker tried to explain it to me. "The great thing about the game is all the statistics". I replied "So, a baseball game is just a slow accumulation of stats?" :) –  weiji Aug 12 '11 at 1:06
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .