Meta Stack Exchange is a question and answer site for meta-discussion of the Stack Exchange family of Q&A websites. It only takes a minute to sign up.
Sign up to join this community
It seems to me (and I don't think I'm alone) in thinking that it should be possible to tweak the reputation/incentives on Stack Overflow to improve the quality of questions and answers, and bring Stack Overflow closer to its stated goals.
How do you think the reputation/incentives on Stack Overflow could be improved?
I think it is worth reflecting a little on why the question quality is low to begin with. (I actually think that the answer quality is reasonably high -- not surprising, as most of the reputation mechanisms on the site focus on promoting good answers.)
Here are two thoughts:
1.
There are always folks looking to get started and build reputation. The easiest (only effective?) way to do this is to watch the newest questions list and answer the easiest questions immediately. However, the easiest questions are frequently exactly the questions that most developers would consider low quality. There is thus no incentive for users not to research before asking questions -- there will always be someone out there willing to do the legwork they should have done themselves.
This suggests a mechanism for improving question quality: Give more reputation for providing an up-voted answer to an up-voted question than for a down-voted question. (For a bonus, make up/down-voting questions cheaper and more accessible so as to get more community feedback on them sooner.) That way, the best strategy for an incoming SO user is to answer high quality questions. If low quality questions visibly don't get answered, that may reduce the incentive to ask them.
2.
If you email a mailing list, you're taking up the time of all the recipients of the list. And many of those recipients jealously guard their time. This has led to a general culture of "I won't google things for you" and links to "How to ask a good question documents".
With SO, there is less of a sense of impinging on folks, and the culture of insisting on high quality questions is missing. This does have an upside -- the barrier to entry for new programmers is lower -- but it does mean more spammy questions.
SO could probably attempt to influence new question askers by e.g. linking prominently to "How to ask a good question" docs, but they're likely to be ignored by users.
Why doesn't the SO community actively police question quality? I think it is because of the incentive problem above, and also because there are no decent mechanisms to do so. The only one there is now is downvoting a question, which costs reputation yet has no really apparent impact on the question or questioner.
Without the ability to have community policing, it comes down to this: What is your incentive not to ask an unresearched/incomprehensible/etc. question?
In mailing lists, it is something like shame (for better or for worse). You may do it once, but after the very public response you get from the list, you're unlikely to do it again. If one wanted to go nuts along this lines, you could put SO users who ask poor quality questions (as judged democratically by other users) on question probation, visible to all -- a cone of shame. Then provide ample resources for them on how to ask better questions, and let them earn their way back out of question probation by improving their existing questions.
Clearly, neither of these is the whole story, but I think either could help.
The general point is: Provide mechanisms that reward users for asking good questions and for answering good questions, and that don't reward (or punish) users for asking bad questions and for answering bad questions.
I'd like to throw in an idea I had today. What would you think about some kind of mentoring program?
The principle is very simple. You need one experienced user and one or more new (not so experienced) users.
Mentors would get one or more trainees assigned with the aim to teach them how to ask good questions, write good answers, use the meta, comment etc.
Mentors would get notified of new questions/answers of their trainees so they can review the content and give them a feedback and/or correct/improve it.
I'm aware that this will be pretty time demanding for the mentors but if they decide to invest their time in improving the community why not?
Question: How would the mentor find/choose a trainee?
Proposal:
The first step would be to gather a list of participants. We'd have to find out how many potential mentors there would be available for how many trainees. Once that list is gathered we have to decide on an appropriate ratio. That is how many trainees per mentor. To be honest I really have not had any time to sit back and think about the time required per trainee yet, so I can't make any statement on that ratio.
Best case scenario, we manage to get a list up with an appropriate ratio then a mentor would get trainees assigned by a committee, staff or an automated system (which of course would have to be created first).
Question: Would a user add themselves to the list or participants, or would the system add them?
Answer:
The participation of all groups, mentors, trainees and staff would be on a voluntary basis of course. No one would be forced to participate. They would register/subscribe to that system.
No force on earth, no incentive structure, no deus ex machine, will prevent lazy students from dropping their homework onto these sites, nor confused (and often in-over-their-heads) sufferers from posting context-free questions, nor the just plain stupid from posting just plain stupid questions. Most of this material shows up as the very first post from someone.
Most, but not all, of the previous observation is that there is a stubborn fringe who will repeatedly post manure if not somehow inhibited. And, no, downvoting them didn't serve.
Prior-reviewing first posts would be a radical change and I think the management has rejected it, consistently. Instead, they strive to incent the rest of us to vacuum up the initial spills, and concentrate on identifying and blocking the repeat offenders.
So long as this is the approach, we'll see a a certain number of low-quality posts. So it goes.
One of the big selling points of SO was it's "wiki-like" nature; in reality, the incentives and the question/answer format reward adding additional answers rather than fleshing out already existing answers. Imagine if Wikipedia encouraged everyone to create their own sub-page on a subject, rather than edit the main page-- well, that's what we've got here.
I'd suggest taking a certain set of already-answered, non-active questions each week, and turning them over to the community for "summarization". Interested parties would then submit a one-page-ish summary of the best thinking on the subject, condensed from the answers. After a suitable period (a week? two?) to collect the various summaries, the community would vote on which summary is best (using the traditional upvote/downvote method.) After a suitable time period, the highest-ranked summary would become the "community accepted answer" for the question, and would go at the top of the answers, in "community wiki" mode. The author of the accepted summary could get, say, 50 or 100 rep points, and a badge.
I think it would be beneficial to encourage more upvoting and downvoting, to help the better answers rise to the top (and the poorer answers sink).
One way to effect this would be to allow users over a certain reputation threshhold to downvote without penalty, and to give them up to 5 upvotes per answer. This way, people who are dedicated members of the community, who can be trusted, can have some extra weight in these matters.
I would like to see the people who have upvoted (if all upvotes are from mrskeet et al or a tag specialist then you know its a rock solid answer).
Otherwise it might be nice to see the average rep of the people who have up/downvoted.
Just based on my experience, I think a better way to search the existing questions would be somewhat useful. Because usually people just give up on their first search and start posting questions, expecially when they need responses fast.
Also it would be great to be able to somewhat merge questions that are similar. I dont know if this has been implemented or not.
