6

If you are looking to hire the best possible developer, how much can you infer from the fact they have a high Stack Overflow score? Think about "best" as the person you think could have the biggest impact on making your project successful.

This question assumes you are already matching skill sets with the right problems which is required in any case.

Obviously Stack Overflow score is not a perfect indicator, but is the correlation between top Stack Overflow contributors and the best developers low or high?

| improve this question | |
  • 7
    Please define "best developer", "best possible developer", and "top developer", or define a system of objective measurement that will allow us to determine who is in that category. – Pollyanna Sep 7 '11 at 14:43
  • @Adam - Ok, I tried to clarify that part a bit. – whitneyland Sep 7 '11 at 14:50
  • It would probably be better to review the questions/answers that got the most upvotes as that demonstrates that a developer is held in high regard by other developers/enthusiasts. Also, a developer could be good at what they do but not necessary familiar with the full life cycle of software which would be necessary for the completion of a project. – James P. Sep 7 '11 at 15:01
  • Btw, you might be interested by StackExchange's Data Explorer: data.stackexchange.com – James P. Sep 7 '11 at 15:03
  • 1
    SO Careers has not been running long enough (and by inference, no one org or person has hired enough people through it) to be able to draw a statistically significant conclusion to this question. tl;dr: THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER. – AakashM Sep 7 '11 at 15:56
  • 4
    It's mostly a game. Some of what I consider to be my best answers are hardly noticed, while I get an occasional homerun answer that's really simple - thanks to the "bikeshed effect" – NullUserException อ_อ Sep 7 '11 at 16:26
  • Even with your "clarifications" this question is not objectively answerable. Voting to close. – Pollyanna Sep 7 '11 at 17:53
  • @Adam - voting to close? ... what about your q meta.stackexchange.com/questions/44995/… – StuartLC Sep 7 '11 at 21:18
  • @nonnb see Don't close questions where the user has requested that it not be closed – Pollyanna Sep 7 '11 at 21:56
  • @Adam - Honestly I had some doubts if it could be productive and answerable, but I think there has been useful insight and that I'll be able to mark it answered. Give us a bit more rope on this one... – whitneyland Sep 7 '11 at 23:56
  • 1
    Well too late I guess. – whitneyland Sep 7 '11 at 23:57
5

I would say that it correlates best with the ability to play the SO game. From experience, I can tell you that certain skills can improve your overall score.

  1. The ability to quickly discern the problem and the solution. Often this is not as easy as it seems. Being first or, at least, early is important.
  2. The ability to design a good strategy. I will try to write a short, but complete, answer, then follow up with an example. I also focus on just a few subject areas that I know well or that I'm learning. I tend to also only answer questions that don't have any answers yet.
  3. The ability to effectively use search to learn. I do this often when I answer questions in an area that I'm also learning about. Sometimes I use a question as a learning opportunity.
  4. Editing! Improve your answers. I frequently go back and improve the language or add examples. If someone makes a comment about something that would be better, I will add (with attribution) that information.
  5. Have a good body of general knowledge and specific knowledge on topics that SO focuses on. .NET used to be much more dominant on SO and that's where my background is. I'm ABD in CS and have 25+ years of IT experience. Jon Skeet, of course, wrote the book on C#.
  6. Be competitive! For awhile I was spending 2-3 hours a day on SO - some at work as I was learning a brand new technology for which both asking and answering questions turns out to be very useful. Mostly at home - while eating breakfast, watching TV. I know Jon Skeet has detailed how he spends his time on the train answering questions. My usage has slowed significantly. I still pop over with some frequency to check things out but my answers/day average is trending way down.
  7. Fast typist. Corollary to #1.
  8. Be right! Don't settle for incomplete or imperfect. Don't give anyone a reason to down vote you. That includes your code examples. Fix any errors that are pointed out; try not to make any in the first place.

It's certainly possible that some of those are also important to being a good developer. I am the Lead Developer on my team and spend a fair amount of time mentoring (a skill which SO helps me to develop and maintain). I certainly wouldn't put myself in a class with most of the others on the first page of SO all-time users. I've always considered myself too much of a generalist to really be a top-tier developer, though I'm certainly getting better (assuming that looking back and recognizing the code that I wrote 6 months ago was crappy compared to what I'm writing today means I'm improving).

| improve this answer | |
  • Accepted as best answer thank you. Did I not mention I would be taking 6 years to decide? You only have 4 votes but in hindsight I think have captured one of the most important points - that SO success probably corrrelates best with its own unique combination of required skills. One thing I don't think anyone emphasized enough is that SO success may place a lower bound on ones success as a developer. For example, a person with a 100k score may not be the best but is less likely to be the worst. The cynical view (expressed above) is that they're worse, I just disagree with that. – whitneyland Jul 16 '17 at 21:47
21

Here's an argument that they're the worst developers

The top answerers are so unfocused that they, like rats at a feeder, can't sit down for 10 minutes straight and think about hard problems without getting distracted by idle curiosities such as those presented in Stack Overflow questions.

They spend all day distracted by other people's problems, focused on an arbitrary reputation number rather than their jobs and careers. They get so distracted that they lose focus on their day jobs due to the addictive/fun/twitch nature of the system. They get distracted trying to react quickly to questions with good answers before anyone else. They might be smart and able to communicate well but their work day widdles away as they, with their IDE filled with arbitrary theoretical/superficial Stack Overflow problems, lose focus on the hard, in-depth problems their employer has paid them to solve. Problems they need people to really FOCUS on. Problems that can't be solved by people distracted by random interesting tangents like the ones found in Stack Overflow questions.

I don't necessarily agree that most people fit into this category, I'm just making a point that you can prove the opposite -- high rep may mean bad developers -- people who can't focus to solve hard problems. Whether or not that's true may be just as difficult to prove the opposite.

| improve this answer | |
  • +1, nice argument – iammilind Sep 7 '11 at 15:03
  • 8
    Did anybody read that, and not look at the score of the writer? – AndreyKo Sep 7 '11 at 15:06
  • 4
    @AndreyKo: I got distracted somewhere in the middle of paragraph 2. So yes, I didn't look until you mentioned it ;) – Piskvor left the building Sep 7 '11 at 15:14
  • @DougT - Interesting point. Do you think SO is different than prominent developers who blog a lot, or speak a lot at conferences? – whitneyland Sep 7 '11 at 15:20
  • 4
    Yep, Jon Skeet, Marc Gravell, and Darin Dmitrov are all terrible developers because they're so unfocused. – Peter Olson Sep 7 '11 at 15:25
  • 1
    +1. I was going to say they're people with too much spare time. Could be the greatest or could be the worst (in my case the latter, Jon Skeet however...) – GUI Junkie Sep 7 '11 at 15:28
  • 1
    Or that they have read "7 Habits of Highly Effective People". – Brian Sep 7 '11 at 15:32
  • 1
    Covered in the comments to this Spolsky post: programmers.stackexchange.com/questions/20407/… – Uphill Luge Sep 7 '11 at 15:51
  • 1
    I should add I should have been working when I posted this :) – Doug T. Sep 7 '11 at 16:12
  • 1
    I can add that folks with high rep likely have x jobs, where 0 <= x <= 1. My participation has slacked now that I added four part time jobs beyond my full time job. Granted when call volume was low I participated a lot at my part time jobs. – user7116 Sep 7 '11 at 16:31
  • Who never met a technical genius having answer to anything, knowing every design pattern and frameworks deeply, ... but was unable to deliver value by doing useless implementations, overengeneering ? – user150926 Sep 7 '11 at 18:05
  • 1
    Did you notice that to get high ratings its better to answer A LOT of EASY questions instead of A FEW HARD ones? – rickythefox Sep 8 '11 at 12:04
  • 1
    I really don't like these kinds of arguments; I'm quite saddened to see this kind of thinking. Firstly, why do you say they "can't" sit down for 10 minutes. Secondly, why do you think they care about the points. Consider instead this: they feel so good about helping people that they are willing to sacrifice their comfortable working flow for others. And because they are motivated, they make good answers, and as a side effect, get good points. Also, how do you know they don't solve hard problems and additionally help others (there's more than 8 hours of work in a day). – egaga Oct 2 '11 at 7:44
  • The value to this answer is that it's a great example of how being contrarian can force people to think and consider alternatate perspectives. I love that. However on the whole I agree with @egaga enough to find it unfortunate that it became the most popular answer. Also can't help but notice the counter example that I'm writing this on Sunday, not conflicting with work, only because mindless thinking is sometimes relaxing. Hard to be too contrarian, but for me this crosses a bit into cynicism which is not always productive. – whitneyland Jul 16 '17 at 22:01
11

Think about "best" as the person you think could have the biggest impact on making your project successful.

There's going to be no correlation at all. Look at the list of top users and you'll see a wide assortment of skills. Some of those people would have a large and immediate impact on your project, but some of them would be wasting your time because they simply have the wrong set of skills.

That doesn't mean you can't use Stack Overflow (Careers, actually) to find candidates to hire, it just means you have to look a lot more closely than just at their reputation score. Look at what people have written and you'll find all sorts of people who are good at communicating their knowledge. This is what you should be looking at when hiring a developer.

| improve this answer | |
  • 1
    Bill I would agree with you, but the question was meant to assume all else was equal, for example two developers both with matching skill sets, how much does SO predict success. – whitneyland Sep 7 '11 at 15:15
  • 1
    @Lee: You need to explicitly state your assumptions, but there will still be low/no correlation. One mediocre C++ programmer might participate on SO a lot and answer easy questions, while an expert hardly participates at all. Their scores won't represent their real skill with their chosen technology. – Bill the Lizard Sep 7 '11 at 15:32
  • I have updated the question to clarify this assumption. Regarding your answer on the correlation, if that's the case so be it. I've seen some companies boast of hiring high ranking developers - we as a community should decide if this is right thinking or misguided. – whitneyland Sep 7 '11 at 15:37
  • 2
    @Lee "I've seen some companies boast of hiring high ranking developers" Really? I would be very interested in references to these boasts. – Pollyanna Sep 7 '11 at 17:52
10

Not at all; in my experience, there is zero correlation:

  • There are users with high rep who are exceptionally good programmers
  • there are users with high rep who are exceptionally mediocre programmers who just answer a lot of simple questions
  • there are exceptionally good programmers (famous even) with low rep, as they only ever answered one question
  • and there are unknown users with low rep.

See also our excellent FAQ:

What is reputation?

Reputation is a rough measurement of how much the community trusts you; it is earned by convincing your peers that you know what you’re talking about.

Note that it says "trusts you" - not "how good you are"; and nowhere it is even hinted that the rep measures "impact on making your project successful."

Note also that "programming" is not a homogeneous topic - if your project deals with realtime, embedded, safety-critical programs, a PHP programmer dealing with large-scale distributed projects probably wouldn't fit into your team, even if xe had a score of one hundred thousand million rep points.

| improve this answer | |
  • 4
    I think "exceptionally mediocre" is an oxymoron. – Peter Olson Sep 7 '11 at 15:17
  • 1
    @Peter Of The Corn: I was wondering whether anyone would comment on that :o) It is a tongue-in-cheek phrase, obviously - I meant that you can't really get high rep with bad answers, but you can get there with lots of answers that are just good enough to not get downvoted under 0 (+1/-1 is still a net +8 rep). (although I still don't understand the motivation for that) – Piskvor left the building Sep 7 '11 at 15:21
7

I could have at least four times the amount of reputation that I have on Stack Overflow if I was interested more in reputation than learning something. I currently have a little over 14,000 points.

Initially, Stack Overflow promised me that most features of the site would be unlocked once I managed to earn 10,000 reputation points. That became a goal, but not my primary motivation for using the site. What programmer doesn't like a challenge?

My primary language is C and I've got quite a bit of experience at using it. However, seven out of ten times that I found an interesting question I could answer, someone else had answered it in a way that made me think a little differently about the problem at hand. I have no issue with citing those users in this answer:

.. Just to name a few. I'm sure I left some people out. I could have answered many of the same questions that they answered while earning more reputation, but what is the use of regurgitating the same answer while not really adding anything? I get enough of that watching the nightly news after spending an hour with my browser. What does that do for this thing that I (and many other self taught hackers) have grown to really appreciate? Would it not be better to just cast my votes to make something really helpful rise to the top and add comments or edit when you see someone has made a mistake?

Once in a while I happen upon an interesting question that doesn't have an answer conveying the same knowledge or experience that I can provide. When that happens, I provide an answer. I'm also known to up-vote answers that provide details that I omitted while answering the same question.

Well, at least I used to do that, lately I've been rather busy with this thing we call flags. I'm quite content with that, I'm able to help this thing I really care about quite a bit more after being elected.

Stack Overflow has never been, nor will it be about the number next to my name. In fact, the answers I'm most proud of received less votes than the answers I wrote in five minutes. The people that see Stack Overflow as some kind of competitive sport are completely missing the point, at least in my opinion.

And then, there are just users that have so much knowledge to share that it more or less leaks out of them, and we're grateful to have them here. But, as far as I know, those people aren't looking for jobs.

| improve this answer | |
2

One of the 'disjoints' in correlation between quality of developer and reputation lies with the popularity of the tags / skills which the developer professes - rep is generally biased toward the number of 'views', so those quickest on the draw in common languages such as C#, java, php etc typically get rep quicker. (Admitted, there are also more beady eyes on their answers, and mistakes will also be punished more)

In some of the less popular tags, there are excellent devs with relatively low rep. Tomas Restrepo in the BizTalk category for example.

| improve this answer | |
  • excellent point. so, even if there is a correlation direct comparisons must be qualified. – whitneyland Sep 7 '11 at 17:01
2

With great respect to some of the incredibly knowledgeable and helpful individuals who put a lot of time, effort, and expertise into answering SO questions and have rightly gained rep from doing so, the truth is that given enough time one can easily answer sufficient questions resulting in one or two upvotes to accumulate "high" rep. In other words, the most important characteristic one needs to generate SO rep is perseverance.

Secondly, note that the simplest questions sometimes generate ridiculously high levels of rep. Look at my own highest-scoring answer for a great example of this. I am no Java guru, but have a 50-vote Java answer that's obviously netted me a nice quantity of rep.

In short, and not solely based on those two examples, my opinion is that there's not necessarily a correlation between high rep and being a great developer.

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged .