150

Gaining reputation from answering on Stack Overflow is getting increasingly difficult. There are many talented users to answer most questions within minutes, providing good enough answers immediately and editing in fuller answers within a very short time. Sometimes I feel like gaining reputation is somewhat of a race. However, I mean this in a good sense - it's a fun race, and it's great to have answers to most questions supplied most quickly.

But it saddens me to see how easily reputation can be gained by just asking questions. I know this has been discussed here on Meta Stack Overflow several times, but I want to highlight a pattern. Take a look at the Hottest Questions This Week (I tend to scan through this list on weekends). I don't want to point fingers, but many questions there have already been asked multiple times before. Stack Overflow veterans surely scan with their eyes, nodding - been there - over most of them. And yet, these questions gain lots of reputation. Some get closed in a couple of hours, but still manage to gain many votes until then.

This isn't a good situation, IMHO. Many members on Stack Overflow brandish high reputations and when you enter their profile it's a few 100s of questions, 3 downvoted answers on some semi-subjective topic, and nothing more. Is this the 4K user employers would like to hire based on high Stack Overflow reputation?

Perhaps to prove this, it's worthwhile to make an experiment - a user that asks semi-subjective questions over and over again, on topics which haven't been discussed in the past week or so. I have a feeling this can show some nice results, reputation-wise.

I know there are open issues on User Voice on this issue, but is someone planning to do anything about it? IMHO too much reputation is gained by asking questions, it is blatantly obvious, and it makes Stack Overflow a less fun place to be - as it partially invalidates the whole reputation system.


Why less fun? Consider this scenario:

Bob: So are you knowledgeable in programming?

Alice: Yeah, I have a Stack Overflow reputation of 8K!

Bob: Eh, I knew people get to such reputations just for asking hundreds of repeating questions there. So that doesn't count. Any other proof of your expertness?

Sounds surreal? I disagree. As an employer interviewing people quite often, such a scenario is real for me. I just value Stack Overflow reputation less because of this, period. Stack Overflow's administrators have to decide what they want out of it. Is it just a website for getting your questions answered? Great - it's a great one, indeed. But it appears they want to make something more, connecting to Careers, meaning reputation to actually be real. In such a case, allowing reputation farming from repeated questions is a self-defeating strategy.

22
  • 133
    I was going to answer this question, but then I realized it was a shallow attempt to gain reputation! Shame on you sir! For shame! :) Commented Dec 19, 2009 at 6:38
  • 11
    @Jeff :-) Yeah, I realize the pun here. But I don't care much about my Meta rep (don't take it personally...), and I made this a community wiki anyway Commented Dec 19, 2009 at 6:56
  • 6
    I noticed! And I'm certain I'll be cursing you once my answer is upvoted! stares at vote total intensely, as though willing it to increment
    – Pollyanna
    Commented Dec 19, 2009 at 7:06
  • 1
    If you know, that it was discussed here already, why don't you extend that discussion with a new answer, instead of cluttering the discussion with a new question? Commented Dec 19, 2009 at 8:23
  • 75
    The critical flaw in this question is that it assumes employers are too stupid to actually look at the underlying data and reach their own conclusions. Sure, a high stack overflow rep can be gained in dumb ways, but guess what? So can everything else in life: success, fame, degrees, etc. You probably don't want to work for an employer so clueless that they can't figure this out. Commented Dec 19, 2009 at 9:55
  • Isn't the point that if you seek to gain reputation via means that aren't as relevant to being a good employee (i.e. you ask more questions then you answer) that is in fact exactly the sort of employer such a person would want?
    – cyborg
    Commented Dec 19, 2009 at 10:49
  • @Jeff - depends on the pay. "We've already established what you are, madam, now we are haggling over price." /Churchill ;-D
    – Pollyanna
    Commented Dec 20, 2009 at 1:48
  • 4
    -1 I couldn't disagree with this more. There SHOULD be positive reinforcement for asking a question... the asker isn't the only person who benefits when a question gets answered: the entire community does.
    – TM.
    Commented Dec 20, 2009 at 4:52
  • 3
    @TM: That's only true if the question is worthwhile. Repeats, subjective questions, and some of the poorly worded monstrosities that can only be called questions by way of punctuation (if any) should not be encouraged in any way, shape, or form.
    – AnonJr
    Commented Dec 20, 2009 at 13:21
  • 1
    @AnonJr that's why the users get to decide whether or not to upvote! People shouldn't and won't upvote the "terrible" questions that you describe.
    – TM.
    Commented Dec 20, 2009 at 16:41
  • you can see my updated stats. Conclusion: data does not support your argument. Commented Jan 8, 2010 at 10:07
  • 4
    There's some point, most of reputation is gained from the simplest questions. Complicated questions on very specialistic subjects are rarely accessed, so gain relatively little reputation. Commented Feb 2, 2011 at 9:15
  • 3
    You can also ask one extremely popular question... Someone won the rep lottery here.
    – RomanSt
    Commented Nov 14, 2012 at 12:36
  • 9
    @romkyns: or ask only single question and never answer: stackoverflow.com/questions/79923 Commented Sep 24, 2013 at 9:33
  • 2
    You could always look at their accepted answers. That must show some knowledge.
    – Jamie Read
    Commented Nov 8, 2013 at 15:12

14 Answers 14

66

I (Jeff Atwood) eventually came to agree with the premise of this question, which is that too much rep is generated from questions:

2
  • 4
    @Jeff, thanks - I think you're doing the right thing Commented Mar 19, 2010 at 10:37
  • 1
    @Jeff This was definitely a good thing to do, as I think many people just see a users rep, and don't drill down to see the details. Just this morning I answered a question from a guy with higher rep than me. On his profile, I saw that he has ONLY GIVEN ONE ANSWER ever on SO, but he has asked 10 times more questions than I have. So I think you could go further with this (or maybe I should just ask more questions!) :-)
    – TrojanName
    Commented Oct 19, 2011 at 10:47
108

I've got a crazy idea: rather than just making stuff up, let's look at the data!

  • 5,954,332 actual rep generated from ~439k questions

  • 21,968,671 actual rep generated from ~1.4m answers

Almost 4× as much reputation has been generated by answers, so I take issue with your statement

… how easily rep can be gained by just asking questions …

The data certainly doesn't agree.

Since there were requests for per-post averages, here you go -- the second number excludes community wiki questions and answers.

Average question score

1.47, 1.33


Average answer score

1.96, 1.69


The median question and answer scores are 1 in all cases.

Many members on SO brandish high reps and when you enter their profile it's a few 100s of questions, 3 downvoted answers on some semi-subjective topic, and nothing more

"Many" members? really?

Total users with reputation > 4000

1494


Total users with reputation > 4000 who have asked more than 50 questions

250


Total users with reputation > 4000 who have asked more than 100 questions

85


Total users with reputation > 4000 who have asked more than 200 questions

28


For all users with > 4000 reputation:

questions (avg, median)

31.1, 14


answers (avg, median)

351.7, 246


Edit:

In honor of Stack Overflow users' combined reputation passing one billion last month, and approaching ten million users, I thought it would be interesting to revisit Jeff's stats from 8½ years ago.

To compare current stats to when Jeff posted the answer above, Stack Overflow had:

                   2009-ish        2018-ish
Users         ~100 thousand     ~10 million   ←  97.4× increase 😮 
Combined Rep    ~30 million      ~1 billion   ←  36.4× increase

Updated stats:

  • 6 million 165 million reputation generated from ~439k ~15 million questions
  • 22 million 843 million reputation generated from ~1.4m answers ~25 million

Almost 4× Over 5× as much reputation has been generated by answers.

Per-post average scores (second number excludes community wiki Q&A's):

  • Average question score: 1.47, 1.33 1.97, 1.95
  • Average answer score: 1.96, 1.69 2.74, 2.69

The median question and answer scores are still 1 in all cases.

Totals for users with reputation > 4000 ...

  • ...number of users >= 4000 rep 1494 39044 (26× increase over 2010 )
  • ...who have asked more than 50 questions 250 9532 (38×)
  • ...who have asked more than 100 questions 85 2993 (35×)
  • ...who have asked more than 200 questions 28 1955 (70×)
  • ...rep from questions (avg, median) 31.1, 14 51.9, 22
  • ...rep from answers (avg, median) 351.7, 246 312.5, 188

A couple of related charts:

13
  • 6
    this all goes back to reputation lotteries, reputation envy and bike shedding ... the numbers do not support this, reputation lotteries are rare. its hard enough to get people to vote in the first place
    – waffles
    Commented Dec 19, 2009 at 9:46
  • 33
    4x rep from answers than from questions - don't you think this proves my point, especially given that there are probably much more than 4x answers than questions. Commented Dec 19, 2009 at 10:32
  • 10
    Indeed. I'm actually surprised by how well the stats support the argument. Like I answered elsewhere, rep shouldn't be handed out for asking questions at all. meta.stackexchange.com/questions/1326/…
    – U62
    Commented Dec 20, 2009 at 2:16
  • Does the ratio change significantly if you only look at rep gained since 1 july 09?
    – Antony
    Commented Dec 20, 2009 at 5:42
  • 1
    Shore has cracked 3k stackoverflow.com/users/104015/shore and Gold is at 1.5K
    – waffles
    Commented Dec 20, 2009 at 22:33
  • 7
    Agreed, the argument is that it is easy to earn rep by asking questions rather than answers. None of the statistics given address that at all. The only metric which would apply is conspicuously missing: Avg reputation per answer vs Avg reputation per question. If they are comparable, and if we take as given that asking is easier than answering, then the OP's point is proven. Commented Jan 6, 2010 at 17:47
  • How many questions for that rep? How many answers? Reputation per post is an important consideration, I think.
    – ale
    Commented Jan 6, 2010 at 19:53
  • 4
    It's not a question of rarity. It's a question of the value of posting a question relative to posting an answer compared to the relative effort required for each. Please see my answer below: meta.stackexchange.com/questions/33398/….
    – jason
    Commented Jan 8, 2010 at 6:10
  • Am I misunderstanding something? How is the points per post lower when excluding Community Wiki posts? Shouldn't it be higher (same number of points in numerator but fewer posts in denominator)?
    – jason
    Commented Jan 8, 2010 at 14:18
  • 2
    @Jeff Atwood: Thanks for updating and replying, but I don't see how the data you added (which I essentially already had in my initial answer) support your point? I elaborated on why I think it supports the opposite; could you elaborate your position? Thanks.
    – jason
    Commented Jan 8, 2010 at 14:21
  • Its annoying being unable to comment (ask a related question to a thread) when you are beginner. I don't know it purpose or don't know the way to ask related question Commented May 19, 2014 at 12:31
  • Wow. Really good use of the numbers. Statistics, baby!
    – naiveai
    Commented Sep 8, 2014 at 11:31
  • @EliBendersky Very late, but you were off on the rep ratios. Rep/Question = 5.9e6/440e3 = 13.4 . Rep/Answer = 21.9e6/1.4e6 = 15.6
    – user373608
    Commented Oct 25, 2017 at 19:46
23

Shore, there is always Gold. But they are edge cases; they must have elephant skin because almost 50% of the votes they get are downvotes. Besides, everybody knows they are Jon and Marcs sock puppet accounts.

The surest way to gain lots of reputation points on Stack Overflow:

  1. Answer lots of questions (and be the first to answer them)
  2. Answer them well
  3. Rinse and repeat

More controversial ways:

  1. Answer lots of questions on lucrative tags (subjective, source control, homework)
  2. Answer them well
  3. Rinse and repeat

Proof:

select 
    case when ParentId is null then 'Question' else 'Answer' end as [Type],  
    COUNT(*), 
    (SUM(Score) / (COUNT(*) + 0.0))
    from Posts
    where CommunityOwnedDate is null and ClosedDate is null
group by case when ParentId is null then 'Question' else 'Answer' end
           Count     Rep       Rep/Post
Answers:   1194295   2040522    1.71
Question:   380461    529778    1.39

Conclusion

Most reputation points are gained from answers; you are likely to get more reputation from answering.

The statistics exclude closed and community wiki questions and answers (which when included only mean more people get more reputation points on average).

5
  • Bug in SO, formatting in preview does not match actual
    – waffles
    Commented Dec 20, 2009 at 6:06
  • 8
    Bug in SO, or, y'know, tabs in the input. Tabs are TEH DEVIL. Commented Dec 20, 2009 at 6:56
  • 2
    Which one is Jon's and which one is Marc's? It's a mystery to me.
    – mmx
    Commented Dec 20, 2009 at 23:41
  • why is asking 500 questions as users, "Shore" and "Gold" have done bad for SO's business? They only have 4,000 reputation points. This only puts them in the top 5%. Having trouble seeing the real threat here. Commented May 4, 2012 at 3:36
  • 1
    I'm really confused about the data arguments being made here because I don't think they address the actual concern, which is that it is easier to get rep by asking questions than by answering them. How much rep people are likely to receive from answering vs asking doesn't really matter when the difference there is completely overshadowed by the stark contrast in difficulty between the two activities. It is much easier to ask a question than to give an answer. Getting points for answers requires being fast and knowledgeable. Asking can be done on a huge scale without knowing much of anything. Commented May 14, 2015 at 11:22
17

Bob: So are you knowledgeable in programming?

Alice: Yeah, I have a Stack Overflow reputation of 8K!

Bob: Eh, I knew people get to such reputations just for asking hundreds of repeating questions there. So that doesn't count. Any other proof of your expertness?

Alice: My reputation comes mainly from answering questions. Take a look at my StackOverflow profile if you don't believe me.

(At least I would say that if I where in the same situation)

2
  • Incredulous bob...
    – juan
    Commented Feb 4, 2010 at 20:44
  • Most HR screening is done by computers now, there is no negotiating. Commented Mar 20, 2015 at 15:08
10

Maybe rewarding asking questions was more important in the beginning, to attract users (and useful questions) to the site? Now, however, I'm not that sure that rewarding questioners is that important anymore. Their "reward" should be getting an answer; wouldn't that be enough?

At least re-balancing the question/answer reputation gain should be considered.

1
  • 1
    Indeed, this has now been implemented.
    – Teddy
    Commented Mar 19, 2010 at 17:55
10

Early on, as Teddy says, it seems to me questions have more inherent value than they do for a large site. The larger the pool of existing questions, the greater the proportion of new ones that are not new knowledge or threads. That means new questions have a bit less value on average; it also means there is new value in coordination, consolidation, and organization of existing questions.

A few tangential responses:

  • Consider changing the weight given to posting questions, posting answers, rearranging/fixing metadata, etc. based on what is most in demand. This could be a slow-changing way of aggregating input from thousands of community members in simple metrics that let everyone see what is being most appreciated.

  • There's value in direct manipulation of the question namespace. The close/delete mechanism isn't a good one for many 'repeat' questions -- the new variant may have a few twists not covered before; it may get an answer better than one offered before; and the questioner may have been thinking about the question with different jargon, making it hard for him/her to find the original discussion. So change the system to make it possible for compulsive organizers and classifiers to do something more than add or unify tags across similar questions. Let them group questions together, directly merging them or redirecting one to another. Make it possible for someone to create an omnibus question that will turn up in the same searchable namespace, which can point to a score of related questions and provide context that lets a reader figure out which one pertains to what's on their mind.

  • Examples in current practice: Stack Overflow questions can be closed with "migrated to ([email protected])". Support similar closures as "merged with (question)", "redirected to (question)", "disambiguated via (omnibus question)".

2
  • 1
    +1 love the idea about question manipulation and grouping!
    – Aaron M
    Commented Jan 6, 2010 at 18:15
  • Grouping is a good idea - it allows richer taxonomies than just tagging, and producing FAQ/HOWTO type documents covering a discrete topic such as (for example) data warehousing or perhaps subtopics within it (perhaps a hierarchical grouping facility). It might also be a good tool for marking duplicates and almost-duplicates. Commented Jun 11, 2010 at 10:52
8

Bob: Eh, I knew people get to such reps just for asking hundreds of repeating questions there. So that doesn't count. Any other proof of your expertness?

Me: So, you're going to make an assumption about a specific scenario based on a belief about a general practice, when you could look it up in under 30 seconds?

Bob: Yes.

Me: Never mind, I don't think I'm interested in working for you.

3
  • 9
    This doesn't apply in the real world, where often there are dozens applicants for a position and HR does very shallow filtering before it gets to interview Commented Dec 20, 2009 at 4:52
  • If interviewers are not bothered with checking up simple things themselves before interview then doesn't that make them lazy and quite bad at interviewing? If the company representative is not excited about their applicants then why should I as an applicant be excited about working with the company?
    – Spoike
    Commented Jan 8, 2010 at 9:30
  • 2
    Just because the HR screener is lazy or incompetent, doesn't mean I don't want the job. Commented Mar 19, 2010 at 8:08
7

This is an incomplete argument.

First, it's mostly just trying to point out that the OP may be have been slightly loose with his facts, or perhaps a bit hyperbolic. I see just one assertion. This is the assertion that "the data certainly doesn't agree" with the observation of "how easily rep can be gained by just asking questions." But the data provided doesn't argue that. All the data provided shows is that most people are not earning reputation points this way. A first approximation to addressing the statement that "rep can [easily] be gained by just asking questions" is to compare average reputations points earned per question to average reputation points earned per answer. A second approximation would account for the effort differences between the two.

The data currently shows that the ratio is 1.69 points per answer versus 1.33 points per question.

I think that the gap between these two numbers should be significantly larger. Answering questions requires more effort, knowledge and time than asking questions. How much more is a difficult question. But the current ratio shows that answers are currently worth 27% more than questions. This seems far too small given the effort, knowledge and time required to answer questions is certainly 27% more than that required to ask questions. All you need to ask a question is a the ability to fog a mirror, an internet connection, and a vague semblance of the right words to incant. But to answer a question generally requires knowledge and experience as well as time. The better answerers likely have a breadth of knowledge or a depth of knowledge gained from years of experience and study. Only providing answerers with 27% extra compensation seems far too little.

Second, what is the reason for giving points for upvotes on questions? Question posers already have enough incentive to post (they often have a legitimate question that they want answered). I always thought the intent of reputation points was a signal of how much you are trusted by the community; awarding points for asking questions does not support this.

One idea I would support is awarding reputation points for questions that receive a lot of upvotes (obviously "a lot" needs to be specified). This would help bring the ratio of points per answer to points per question closer to a more meaningful number. As it stands, the ratio of 1.69 per answer versus 1.33 per question given the level of effort required for the former over the latter is too small.

4
  • Disagree; data does not support your arugment. See my updated stats. Commented Jan 8, 2010 at 10:13
  • 2
    I see 1.33 and 1.69 in your updated stats. How do they contradict this post? Commented Jan 30, 2010 at 21:03
  • Where's the magic button that lets me upvote this 20 times? Commented Mar 19, 2010 at 8:11
  • I eventually came around to agree with you on this; see my 2nd answer. Commented Mar 19, 2010 at 8:18
2

Here's another proof of concept. Stellar 12.7K rating from 655 questions and 44 answers.

6
  • 3
    Holy ****! There really are people out there in IT who can't work anything out for themselves, scary.
    – U62
    Commented Jan 6, 2010 at 16:42
  • 1
    Actually, I'd value the willingness to accept that you don't know everything and asking the right questions over know-it-all people every day of the year. Commented Jan 6, 2010 at 19:52
  • 5
    The user you refer to asks valid, meaningful questions. I don't know what the problem is. He has one question...ONE...with a negative vote. Half of the questions have zero votes on them, which means they don't contribute to his rep. The remainder of the questions have upvotes on them, in a pretty standard distribution, which means the community saw value in them. I don't see what the problem is.
    – user102937
    Commented Jan 8, 2010 at 7:48
  • 1
    If you really want to see the most egregious violator on SO, check out Shore's account: stackoverflow.com/users/104015/shore. His rep is only 3229, due to the fact that he asks crappy questions that get downvoted. It's a self-correcting problem.
    – user102937
    Commented Jan 8, 2010 at 7:50
  • 1
    To me, the most egregious example is mrblah: stackoverflow.com/users/68183/mrblah. His questions are duplicates, ill-informed, poorly-written, or entirely based on making no attempt on his own to solve his issue, but he's asked 692 of them (with a huge zero answers). He's been on a tear of around 20 questions a day for a bit now; my favorites are a question about using tool X that is followed by a question an hour later about how he was too frustrated by X and now he's trying to figure out tool Y. I'm actually frightened that his code might be in use somewhere!
    – delfuego
    Commented Jan 14, 2010 at 1:53
  • Now at 40k... (!)
    – hayd
    Commented Mar 25, 2014 at 20:19
2

I do not know if the amount of rep available from asking questions is "too much".

I do not even know if the amount of rep available from asking a question which has already been asked is too much.

However obviously there is some value in asking questions.

And I even think that there is some value to asking repeated questions -- though, again, it's not clear how much value, and there probably is some kind of logarithmic falloff here, in "conceptual value" even if that has not been implemented for "actual rep".

One advantage of repeated questions is that repeated questions can be easier to find than lone questions, and frequently asked questions tend to be asked frequently for good reasons. Another advantage is that repeated questions tend to shed noise that can accumulate on a page that presents a single instance of the question.

And, of course, their are downsides and costs also. A frequently asked question might not be a frequently answered question and a frequently answered question might not have the really good answer on all of its instances. And then there's the sheer annoyance factor. And no one in their right mind likes spam.

(Then again reputation can never be the only criteria used here, or reputation will stop mattering to people.)

1

As long as the questions are closed, and eventually deleted, then when the rep is recalculated this 'spurious' rep will disappear.

Therefore the system is ultimately self-healing. If you believe a particular user has been abusing this aspect of the system, flag one of their posts for moderator attention, and explain briefly.

They may delete the closed questions and push a rep-recalc to bring the user back to where they should be.

Beyond that - does it really matter that someone has more rep than they should? Yes, they are able to edit things, maybe close things, and perhaps they've attained 10k and can use the tools - but if they aren't abusing these powers, who does it really harm?

Reputation is meant to encourage participation. It's now being used for more than this, but I don't see the side effects being terrible - it's not a zero sum game where rep they gain means rep you lose.

3
1

I do believe handling questions and answers in a different way is going to add too much complexity.

One of the things I liked in SO's reputation system in the first place is it's simplicity. Terrible questions can be upvoted, but good ones are being too. I don't think we should try to address all these "edge" cases. I prefer to just let them be exceptions instead of creating more and more condition layers about reputation.

And we need questions in this place to exist. The "rewards" to ask or answer somthing IMHO have to be the same, so we can at least try to have a balance here between questions/answers.

1

OK, so you can't upvote your own answers with sock-puppet accounts, because that will be detected. However, you can potentially create a sock-puppet to conveniently "plant" questions that you have great answers prepared to. Then within a minute of the question being posted you give your great answer.

Sit back and wait for the plus votes and everyone to think what a genius you are. Of course if you can login remotely to another computer you can post the question and answer from two unrelated IP addresses.

(Waits for the downvotes to come pouring in...)

3
  • 6
    xkcd.com/810 - But what if the sock puppeteers actually contribute useful information to the site? Seriously though, if your question and answer is good enough, you can do this with a single account and still get reps from it - the self learner badge is there for a reason
    – Yi Jiang
    Commented Jan 24, 2011 at 13:58
  • Elaboration of xkcd 810. Commented Feb 10, 2014 at 2:51
  • back in January 2011 I had only just joined the site and was having a bad time with confidence at work and really wanted the rep more than anything else..
    – CashCow
    Commented Feb 10, 2014 at 14:41
-4

A solution to this problem (which I even doubt exists) is to make a feature where people with enough reputation can take away the reputations generated by a question. This way, even if the person generates a lot of reps before someone can get to it, the reps can still be revoked. (The number of reps to get this privilege would have to be high though.)

2
  • 2
    Huh? Why would you want someone to revoke reputation?
    – user176326
    Commented Jan 27, 2012 at 1:16
  • Well what I mean is the problem is that people repeat questions to get rep, so I was thinking that you could just revoke it from people who abuse this power. It was just an idea. It probably won't even be needed since the problem doesn't exist. I just felt like someone actually needed answer the question, not attack its premise. Commented Jan 27, 2012 at 20:41

Not the answer you're looking for? Browse other questions tagged .