Stack Overflow is going the way of Usenet.
No, I don't mean that the quality is going down. I mean that the moderation of questions on Stack Overflow is focusing more and more on the effort made by the asker to conform to the group norms, and less and less on the goal of “collectively increasing the sum total of good programming knowledge in the world”.
Stack Overflow sought out to “build a great library of canonical answers”. A key characteristic of Stack Exchange, which distinguishes it from other forms of knowledge sharing on the web such as discussion groups and blogs, is that its tidbits of knowledge are searchable. We don't bury gems in 100 pages-long threads. We don't tell people to search the web, 'cause I'm sure this was answered back in '09 (or at least we shouldn't), nor do we answer for the 43278th time: we close the question as a duplicate of the '09 question. We edit posts to make sure they stay up to date. If someone comes up with a better answer three years down the line, we tell them to share their knowledge on the existing question. It's the “Wikipedia of long tail questions” — the Wikipedia of the non-notable. And we filter questions: we only want the answerable ones, not “how long is a piece of string” or “should I wear the blue tie or the green tie”.
The answers… Stack Exchange is all about the answers. Questions are just an enabler. When you need to know something, you might search for a question, but the answers are what you're after.
And yet… asking a good question is hard. So we ask people to put effort in asking their question. We grant people
brownie reputation points for asking, albeit half as much per upvote as for answers.
I've sometimes wondered if halving the reputation for questions had been the right decision. I find that writing an answer is for the most part straightforward: 1. understand the question; 2. gather the facts; 3. find a solution; 4. explain. Asking a question is less forgiving: did I provide all the necessary background? Am I stating all the relevant facts? Have I not bogged down the question in unnecessary detail? Am I targeting the right audience? Have I sufficiently oriented the question so that I won't get inapplicable answers, yet not so much that I'd drive off non-obvious solutions? Reputation should in principle be an incentive to write good questions, like it is an incentive to write good answers.
Then again, what do askers care about reputation? By and large, askers don't want reputation¹. They want answers. So maybe reputation isn't so important an incentive anyway. Score is, though — a high-scoring question is more likely to attract answerers, and thus answers.
However score isn't such a good indication of a good question. A good question must be answerable, and score doesn't really indicate this. Plenty of well-written questions are unanswerable — because they're too subjective, too broad, off-topic, etc. Fortunately we have another tool to judge questions: closing. Closing determines whether a question is answerable.
Closing is clearly of value to the community — it's what determines whether the topic calls for answers, for those slices of knowledge that the site is about. Answer score has an obvious use in showing the good answers from the bad. What about question score? Question score has two major effects:
- The effect on the asker: the votes on a question factor in the asker's reputation.
- The effect on the community: the score on a question determines to some extent how much attention it gets.
If the asker's goal is aligned with the community's, then all is well: the asker wants answers, so the asker wants attention, so the asker wants a high-scoring question. But often there's a drift. Upvotes on a question are seen as a reward, and downvotes are seen as a punishment. Put effort into the question, get an upvote. Show no effort, get a downvote.
Yet the amount of effort by the asker is not a good indication of a good question.
If I spend a week debugging a 1000-line piece of code, and post it on Stack Overflow in desperation, that's a lot of effort². I may show my analysis, explain all the investigation steps I've taken… and the result will still be an ad hoc exercise in finding the needle in a haystack, useless to anyone else because no one else has exactly my code. On the contrary, a copy-paste of a homework or book exercise is the epitome of laziness³ yet is likely to be useful to many — well-designed homework exercises teach something, and building a canonical repository of solutions to exercises would be in fact a worthwhile use of Stack Exchange.
I've written thousands of answers on Stack Exchange, and I don't think that my most useful answers correspond to the best questions. In fact, detailed, researched questions often have a frustrating trait — they're buried in details that turned out to be irrelevant, which makes them poor duplicate targets, which makes posting detailed, canonical answers a waste. The most useful question for a canonical answer is one that gets to the point quickly and doesn't meander into side issues.
Over time, when a poor question has a good answer, one of two things tend to happen. Either the question gets closed, which is a waste of a good answer; or the question is improved, and thereafter upvoted (because it's now good), and the correlation between its score and the effort by the original asker is lost. Closing the question is detrimental to the community: the right thing is to forget about judging the asker and judge only the question.
When closing starts to be about effort, this does the community a disservice. Voting on questions is supposed to be about effort; closing is supposed to be about answerability.
One of the custom close reasons on Stack Overflow is “Questions asking for code must demonstrate a minimal understanding of the problem being solved.” One of the custom close reasons on Security is “Questions asking us to break the security of a specific system for you are off-topic unless they demonstrate an understanding of the concepts involved and clearly identify a specific problem.” I like these close reasons: in essence, they say “if you're so out of your element that you wouldn't be able to understand the answer, let the grown-ups talk”. There's no point in answering when the asker, or future visitors with the same question, wouldn't benefit from the answer. Unfortunately, on Stack Overflow, the close reason is often interpreted as requiring effort from the asker. No, that's not it. The asker may be trying very hard just to post that question, but if he's too far out of his depth, the question is useless. Conversely, the asker may have spent 30 seconds asking how to undo a mistake, and yet it's such a common issue, and fairly easy to fix, that this tidbit of knowledge is widely useful.
On a forum, when you write an answer, you're helping the asker individually. You don't want to waste your time on a lazy, demanding asker. On Stack Exchange, you're helping everyone with the same question, now or in five years' time. In five years, the amount of effort shown by the asker will be utterly irrelevant. What will matter is how many other people have found your answer useful.
Effort is a terrible thing to factor into question closure.
The problem you discuss in your question is real, and worrying. By insisting on effort, the Stack Overflow community is moving away from the goal of building a canonical repository of answers, and more towards being a forum with peculiar habits. But the question in your title is not the right one. We shouldn't be basing so many judgements on effort. Effort is the wrong problem.
I read the first couple of paragraphs and skipped your rant. Are you going to propose solutions or what?
That's a tough one.
When I started participating on Stack Exchange, I didn't see much point in downvoting questions. Either a question was answerable, or it should be closed. If the question was well-written and interesting I'd upvote it. Then gradually I started to want to convey a message that this question is bad and you should be ashamed. Because, yes, there are plenty of bad questions — not merely off-topic or ill-suited for Stack Exchange but what the hell are you on. Downvote.
Yet I often face the duality of the question score. If a lazy, badly written, poorly researched original post leads to a great answer, the question should be improved. We should downvote the asker but upvote the thread. Does that mean we should downvote or upvote the question? This dichotomy is very common.
Sorry, I'm falling back into rambling mode. Ok, so despite not using regular expressions, we have two problems.
- Our primary problem is to build a great library of canonical answers.
- Our secondary problem is to keep a decent signal-to-noise ratio on the demand front.
To solve our primary problem, the score of a question must indicate how useful it is. To solve our secondary problem, the score of a question must indicate how much effort went into it. We're back to this: question scores can't make up their mind whether they're rating the thread or the asker.
If it was only for problem #1 then we could do away with question score. We could even go further and do away with question ownership. What matters is the answers, so the question should belong to the community! Edit, edit away. Make each question awesome.
However I don't think this is realistic. Answering is hard work, we can't answer everything, we need quality control.
We could separate the two scores: have a score for the original question (contributing to the asker's reputation), and a score for the current state of the question (contributing to the thread's rank in searches). But maintaining two scores would be awfully complex. These scores would start out identical and gradually diverge over time as the question is edited. This system would be too hard to understand.
So I don't have anything radical to propose. Let's stick with the statu quo, with the schizophrenic question score. It's not too bad as long as we remember to distinguish score from answerability. Effort is a big part of question score. But it should not leak into closing.
Closing is all about answerability. Closing isn't about the asker, it's about the community. “No effort” is not, should not be, must not be a reason to close.
Answering no-effort questions is not “awarding ‘A’s”. Answering is imparting knowledge to thousands of people, of which a measly one happened to ask the question.
Stack Overflow needs to stop worrying about effort and learn to love the answers.
¹ Answerers aren't always after reputation either, but I think they're pretty much universally after positive attention, whether this attention takes a numerical form or not.
² I should cite an example at this point, but I'm sure you know what I'm talking about.
³ Ok, not quite. It's trumped by a scan of a homework exercise.
foobar()function but couldn't find it", that's what you know and you should say it.
...but every time I ask someone else to interpret them, I hear, "Oh, those are for questions that don't show enough effort"-- That's because people who put effort into their questions can usually describe their problem adequately and demonstrate a minimal understanding of the problem. The lack of these two things generally demonstrates lack of effort.
foobar()and it hasn't helped" a great lot. I do sometimes – not often, but it is a peeve – see "I couldn't find any good docs for
foobar()" which is nearly always a lie. (And prod the poster to say what documentation, bad as it seemed to them, they found, and what part they need explained, as a "blow the dust out of the socket" kind of move.)