Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 158 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

I am interested in using reputation and points systems (like the one used here) to improve how we recognize what people learn and the skills they have. There seems to be a disconnect between the innovation in web application development, where we see lots of experimentation with activity streams, badges, and reputation/ points systems - and the old-fashioned and traditional way of testing knowledge by making learners take, often meaningless, exams.

Is this already happening with stack overflow badges - are users getting jobs by pointing to their badges and points?

If anyone has done research on the topic, please leave a comment and get in touch. We are hosting a workshop on the topic in late September, and we'd love to invite someone who was involved in designing the Stack Overflow points system (or similar systems).

share|improve this question
Thank you all for replying. I now better understand the shortcomings of SO itself as a measure of skill and knowledge, and have some ideas how to use the different representations of participation in the context of learning. In my work, we are interested in addressing some of the shortcomings mentioned here, and I believe that the basic SO software could easily be modified to allow better recognition of learning. In case anyone is interested to carry on this conversation - or get involved in the work - please contact me directly. – Philipp Aug 24 '10 at 11:48

Reputation is specifically a measure of your contribution to the community, and as such, the extent to which the community "trusts" you.

If you try to apply reputation to knowledge, it fall short. It is possible to earn lots of points from very easy questions and few points from challenging ones. It is possible that the most knowledgeable users earn few points because they may choose to participate infrequently (or not at all). Some users gain almost all of their points by asking questions (and some of them have clearly demonstrated an inability to learn, too!).

share|improve this answer
Ignoring reputation, there are other ways of using Stack Overflow to learn about a user's skills (or at least areas of interest). For example, from your tags, I see you are well versed in .NET and C#, but not so much Linux development. – Dan Moulding Aug 20 '10 at 18:26
@Dan - True. However, I also know my way around VB6. I don't answer VB6 questions (since I'm so sick of it), though, so you wouldn't know that from my SO profile. – user27414 Aug 20 '10 at 18:30
Yep, but if I'm hiring a VB6 developer, I want someone who not only knows it, but is interested in it. I know Tcl but I'd hate to be stuck in a Tcl programming job. And it's evident from my profile that Tcl's not one of my interests. – Dan Moulding Aug 20 '10 at 18:33
I agree that the link between reputation and knowledge in SO is weak. Partly this is by design - I suspect that recognition of knowledge was not the primary design consideration. In other communities, where reputation is defined differently it might be more clearly connected (think chess players) and I'd be interested which activities one ought to track in order to recognize knowledge or specific skills. – Philipp Aug 24 '10 at 11:42

It's also a measure of your communication skill - Jon B touched on this (community "trust"), but I think that rep scores (particularly the stratospheric ones) have a lot to do with one's ability to:

  • Discern the question asked from the words posted

  • Communicate the answer clearly, concisely, and quickly

  • Fit into the community (post according to commonly accepted standards & traditions), which mainly just helps with the 2nd point

I suppose there's an argument that someone with good searching & communication skills could build up a pretty hefty rep on any rep-based site by answering the low hanging fruit. ;)

share|improve this answer
We have thought a little bit about the different dimensions of someone's ability to "answer a question" with respect to skills we'd like to recognize. That work is available on the P2PU/Mozilla School of Webcraft wiki here. - seems directly related to your point. – Philipp Aug 24 '10 at 11:45

I think you can get a general idea of a Stack Overflow user's areas of interest and/or experience pretty easily by looking at the tags associated with their account. For instance, just a quick glance at Jon B's SO account, I see that he is probably well versed in .NET/C#/Windows, but probably not so much in POSIX/C/Linux.

Experience or knowledge would be far more difficult (impossible?) to accurately gauge. But looking at tags, I think, can give you a pretty good general idea of someone's interests and maybe just a little bit of insight into their experience.

It's worth noting that, while Jon B might not have POSIX/C/Linux tags, that doesn't mean he isn't knowledgeable about those subjects. It might just mean he's not currently as interested in them (compared to .NET and C#).

BTW, sorry for picking on ya, Jon, but I needed a good example :-)

share|improve this answer
+1 for making my ears burn. – user27414 Aug 20 '10 at 18:35
I hadn't thought about tags yet, but focused on badges and overall reputation. Thanks for mentioning tags, they are interesting because it's the only "signal" that the user is completely free to choose him/herself. Reputation is automatically calculated, and badges are awarded. – Philipp Aug 24 '10 at 11:47

I can't speak to getting jobs, but the badges mainly signal that you have learned how to use the site. Whether that means you've earned the underlying skill (in this case, programming), is, another matter altogether, I think.

There is a useful connection between this site and getting jobs, but I'm not sure it runs through badges per se, or even points--unless the points were gained by 'answering' questions and gaining actual respect from peers as opposed to just garnering upvotes, which can be accomplished through a variety of paths and tactics.

share|improve this answer
This is a common problem with standards based testing. Exams and tests often measure a student's ability to successfully complete that particular exam or test - they often say little about the student's real understanding of the subject matter. – Philipp Aug 24 '10 at 11:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .