What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

Stack Overflow has been wildly successful. And maybe in some ways too successful.

I am concerned that Stack Overflow is being inundated by a stream of low-quality questions from users who are accidentally poisoning our well -- by turning off and turning away the core answerers who do all the real work in the system.

In theory there is "no such thing as a stupid question" but in practice, there are:

  • users who can't be bothered to form sentences
  • users who don't do the most basic kinds of research themselves
  • users who barely even explain what it is they are trying to do

I mean a pattern of the above. Not an isolated incident, but 5-10 questions (or dozens or hundreds!) all showing the exact same negative characteristics over a period of days or weeks.

Now, a few of these questions is no problem for our community -- that's why we have voting, reputation, question closing, community moderators, flagging, etc. I am happy to intervene if there is a pattern of negligent, irresponsible, failure-to-learn-anything-at-all questions from a particular user. It's easily the #1 reason I mete out timed suspensions at this point.

All of these systems work, and have worked to date! That's the good news. That's why we have a community worth participating in, and a community worth visiting.

But.

I'm starting to see cracks in Stack Overflow as its popularity grows. At some point you have to face up to the hard reality: there are an infinite number of bad questions that can be asked in willful ignorance.

However smart our software, however smart our users -- we can't scale enough to defeat a million monkeys randomly typing. Not possible.

I worry that we're not doing enough to automatically filter out obviously bad / malicious / inept questions from the system, before the burden of having to deal with these questions lands on our talented audience of answerers.

It's an explicit goal to make moderation easy and effortless. I can't in good conscience say we're doing that, if users have to face down a neverending flood of truly horrible, careless questions.. and hope for an occasional gem to float along.

What can we do -- what do you suggest -- to detect and prevent these kinds of bottom-of-the-barrel questions from even entering our system in the first place? I am willing to sacrifice a small percentage of new questions (up to 10%) as collateral damage if necessary.

(hint as a starting point: think new user / IP address restrictions, around question asking, perhaps based on history..)

Now completed! See self-answer below...

share|improve this question
3  
So you want a "Hidden features to help with moderation"? –  random Jul 13 '10 at 0:56
4  
Ratchet it up to 25%, that's a good number these days. –  random Jul 13 '10 at 1:20
13  
You just noticed this, did you? :P –  Aarobot Jul 13 '10 at 1:40
24  
So wait a second, you recolor my post Are Duplicates Creating Broken Windows and then post a quite similar treatise that replaces the word 'duplicate' with 'easy question'? What do you think the duplicates are of? They aren't generally the hard stuff. –  George Stocker Jul 13 '10 at 2:01
3  
To be fair, I only read the first part of your post before my Someone is wrong on the Internet mode kicked in. Sorry 'bout that. I have thoughts on this; but I wanted you to know while it's not an exact duplicate, it's a related issue. :-) –  George Stocker Jul 13 '10 at 2:14
4  
But he has a diamond next to his name. –  Ward Jul 13 '10 at 2:54
3  
George's original post is here: meta.stackexchange.com/questions/52530/… –  Ether Jul 13 '10 at 3:12
42  
"we can't scale enough to defeat a million monkeys randomly typing" - I thought Perl questions were allowed... –  Steven A. Lowe Jul 13 '10 at 16:19
16  
A question with only a single sentence should require 1000 rep. A question whose text is the same as the subject should require 3000 rep. This simple metric automatically eliminates a good chunk of them. Redirect them to catb.org/esr/faqs/smart-questions.html –  Uphill Luge Jul 14 '10 at 7:37
8  
How about knocking off titles that start with a tag? I'm so tired of "C# <real title here>". –  John Saunders Jul 14 '10 at 7:38
3  
I had seen a very funny javascript related questions and legitimate answer was downvoted with -1 for not enough jquery, it is bad that i can't find a link to that at this time. –  Sarfraz Jul 14 '10 at 13:20
5  
@Sarfraz I remember that, but iirc, that was not down voted - it was just commented. -1 for not enough jQuery is just a meme I guess. –  Amarghosh Jul 14 '10 at 13:39
9  
IP banning is not a solution as you risk unfairly penalising innocent users who happen to use ISPs who issue dynamic IPs or those that share a proxy server. –  Dan Diplo Aug 7 '10 at 13:58
3  
@creitve email us if ever becomes an actual versus a hypothetical problem for you –  Jeff Atwood Aug 21 '10 at 1:59
3  
This seems reasonable, as long as it doesn't eventually extend to "nuke any user who doesn't perfectly format and put a minimum of a day's work into solving their problem first". –  RCIX Nov 2 '10 at 22:27

34 Answers 34

Maybe you should add a throttle for new questions based on the average of votes of the user's past questions*.

For example, if the average scoring of questions asked in the last 30 days is -0.5 or less (and the user posted at least 2 or 3 questions), then throttle the account to a maximum of 1 question per day.

If the average is -3 or less, throttle the account to a maximum of 1 per week. If the average is -5 or less then don't let the user post questions for 30 days. The numbers obviously can be tweaked.

This should include deleted questions. Maybe in calculating the average cap the minimum score to -6, so users won't get suspended for a very long time if they post a single exceptionally bad question.


*Everything should be both account based and IP based.

share|improve this answer

New users are laying waste to the collective pool; strike them down at first sight with extreme prejudice.

If one of a new users' first three posts (within their first couple of days/first week) are deleted as spam/offensive, they should be automatically put on suspension at the very least.

Better still, if they're down two and out, just delete all their posts and save the moderators a good deal of time from ploughing that muck.

There's no way a new user to the system posting junk from the outset wants to learn or will bother with constructive posts.

If they racket up quick the spam/offensive deleted posts, just destroy them already. They're a lost cause.

Auto-suspension and/or auto-Windexing will save moderators time in setting the suspension period and blasting caps and allow for the true community model of keeping the place clean.

Remember, together we can all put out these fires.

share|improve this answer
1  
Modding SU shows that new users will very much patiently wait the 20 minutes between posting their spam before they can be dealt with by the propped up authorities. If they're not nipped, they do continue. @det –  random Jul 13 '10 at 1:09
1  
@detly: yes, absolutely -- see waffles's stats for a litany of such problem users. –  Ether Jul 13 '10 at 4:24

The first few things I can think of are possibly some sort of grammar filter, or even "community enacted restrictions"?

If we know many of the characteristics of annoying questions, possibly make a filter for them - e.g. a question with only one paragraph and/or 50 characters, a question that has only one paragraph with over 500 characters, any question that has more than five question marks in less than two paragraphs of text etc.

The next one seems a bit overkill, but if you have taken the time to make your question, I am guessing that you are annoyed at the situation and do not mind thinking slightly outside the box / making a bigger system for the greater good.

The idea is an additional button next to the link/flag/edit buttons - a "Bad Question" button - What this button can do is for starters is display all questions with over 5 markings on an additional section e.g. stackoverflow.com/bad_qs and allow people to suggest new grammar filters based on past questions. (Community Bayesian filter? :S ).

Or if you do not like the above, perhaps even as well as that, the button for "Bad Question" could be linked to the users account as well.

If the user gets 2 or more for a single question, the next time they ask a question, it can go to an FAQ about writing nicely formatted questions.

If the user gets 2 or more on a second question, it then makes a proper warning saying that restrictions may follow.

If the user gets reports on another question, they are then throttled by whatever you decide.

As most likely the users will only be posting one question every few days (unless I have underestimated the bad question problem), I am not sure what the limits can be - but I think the actual warning would be all that is needed in most cases.

I am not a fan of IP based restrictions on this sort of thing because I think there may be many companies where multiple users will use the system, although, it may work to disallow new users from an IP that has had excessive "Bad Answer" type warnings?

I am not the best at English/Grammar, so I am not really the best to come up with rules, but I hope you get the basic idea and like them!

share|improve this answer
6  
Please edit your post and change the button name from Possible spam to Bad question. Spam means indiscriminately bulk advertising a commercial product. That word should not be used for bad questions that contain no advertisements. –  Andreas Bonini Jul 13 '10 at 1:07
3  
spam is anything I don't like, right? :p –  Randolpho Jul 13 '10 at 3:02

A question delayer.

Start by giving new users the benefit of the doubt: a first-offense bad question is permissible. Someone will fix the question and if the user just missed something he'll see the edits and hopefully understand what he did wrong. Smart users are smart, even if they look dumb when they show up.

Subsequently, the user will ask more questions. Using some metric based on what already exists (votes, accept rate, number of closes/deletes/flags, never answers/votes), determine if the user sucks and if so how much. The metric is open to elaboration by the commenters of this answer and will need to be studied by examining the data to see if this idea correlates with bad users.

Having definitive information that the user sucks, delay his question from appearing in the list of questions. Maybe just an hour, maybe a day, maybe a week depending on how bad the user is. Presumably the user is lazy and wants an answer quickly. Stack Overflow is good at getting answers quickly. Circumvent this for users taking advantage of it.

Obviously, if the user had more time he might take that time to edit his question to be better. Or he might take the time to do research on his own while he waits for the question to appear.

Look, the point here is that Stack Overflow answers questions fast and this really fosters help vampirism. Adding a delay from the time the user enters the question that scales with the badness metric of that user and making it obvious to the user why this is happening will either drive the user away (no loss) or make him realize that he needs to put more effort into his questions (no loss).

Either way it's a win for the community.

So how's that sound?

share|improve this answer
1  
The problem with this is that after the bad user researched his question and found an answer, the question (badly written) is posted anyway. This annoys the user, but the fact that the questions get posted anyway is still a problem. –  Andreas Bonini Jul 13 '10 at 1:38
8  
@Kop: It's a deterrent, not a filter, yeah. Maybe you could get a special queue (visible to 10k+?) for this kind of question and anyone who wants to can delete them before they bother the general public. I think the root cause of the problem is the speed at which bad questions get answered: users are rewarded quickly for asking poor questions because they get answers. Slow that down and I think things will get a lot better. –  XMLbog Jul 13 '10 at 1:43
4  
Combine this with the "visible to the user[s IP address] and no one else" idea, and I think it's a winner. If a user does nothing, his crap question eventually shows up, gets down-voted, and the delay continues; if he re-posts, then that'll get delayed and down-voted as well; if he sits down and edits his question into shape, then it doesn't get the down-votes and the delay goes away; if he deletes it and goes away, then the problem leaves with him. –  Shog9 Jul 13 '10 at 2:18
4  
Help vamprism is a very good point. Users find a collective of smart people and throw questions at the wall in the hope that some of the smart will rub off. SO runs really fast, so it's a tempting target. SF gets some of these, but it runs slow enough they can be moved to the right site easy enough and hasn't reached the threshold SO has. –  sysadmin1138 Jul 13 '10 at 5:13
1  
@weblog: ip's do change, and multiple people, especially students and users of library computers, share ip's... there may be a lot of collateral damage.... –  eruciform Jul 13 '10 at 18:37
4  
@eruciform: Users who are this bad at asking questions are probably not smart enough to circumvent something like this. At the very least, they'll need to ask a question on Super User about how to do it. –  XMLbog Jul 13 '10 at 23:11

You're forgetting that most of the good questions have already been asked as SO. I've seen the same thing happening at Experts Exchange, where several topics started to become satiated with the hard questions and answers, thus anyone with some common sense would just find his answer in an existing question with no need to post a new one. But there will always be people who ask low-quality questions, who are new here and who don't understand the system. But while the high-quality questions start to slow down because the system is satiated, the low-quality questions will never run out. SO is at a point where it's hard to ask a high-quality question that hasn't been asked before. But there's still plenty of room for low-grade questions. So I fear that we just have to accept that as the popularity of SO rises, the quality of the questions will just drop a bit.

share|improve this answer
4  
I disagree strongly with your initial premise. In most cases, the bad questions referred to here have a good question trying to get out. –  Joel Coehoorn Jul 13 '10 at 1:35

You requested stats sir, well here you go:

  1. Shoot and run, newbies posting bad questions
  2. The worst question askers

I think there are 2 patterns that need addressing:

  1. People who consistently ask crappy questions. (this sets the tone of SO in a bad way)
  2. What I like to call "shoot and run" people who discover the site and just post a handful of crap questions.

Some things that we could possibly do (still needs a bit thought):

  1. Reduce the visibility of newbie questions until they get an upvote. Perhaps add a new section for "newbie questions", once they get N votes allow them into the normal sort orders.

  2. Tax people who ask tons of crap questions, (−N rep for a new question beyond a certain point, if and only if on average your questions are crap)

  3. Require a rep threshold for asking questions beyond the first N questions.

  4. Look at better ways to give high rep users better visibility of these questions as they happen. (perhaps some pages for high rep users to browse through new user questions or something along that line)

  5. Get rid of crap questions, close and delete any questions that provide no value.

  6. Provide an incentive to repair old poorly phrased questions and posts (either badges or rep)

Longer term

  • Think out better way to provide an incentive for users to step in and improve some of these crappy questions that are actually good questions poorly phrased due to English being a second language or lack of experience.
share|improve this answer
2  
Wasn't there a "ghost town of one" idea where the bad poster was the only one to see their posts? –  random Jul 13 '10 at 1:36
9  
@random, I NEEDZ MY code WHY Nobody ANSwr? –  waffles Jul 13 '10 at 1:37
3  
@random, that's called a hellban, and I think it's even older than the web! –  Aarobot Jul 13 '10 at 1:41
6  
I think reducing the visibility of newbie questions is always a bad idea. You were new once. Tax won't work because taking rep from bad users won't stop them from asking questions. A rep threshold might be nice, but what about good newbies asking obscure questions that are good but don't gain a lot of hits? I'd be worried about falsely identifying good users as bad. –  XMLbog Jul 13 '10 at 1:46
5  
I think the second query would be better if limited to users with more than one negatively-scored question. It looks to my bad SQL sense that you could eliminate the condition on Reputation if you did that. –  John Saunders Jul 13 '10 at 1:50
1  
Ok, I'm blind, which column flags the user as registered/unregistered? Would be interesting to see the registered/unregistered ratio. –  Kev Jul 13 '10 at 2:06
1  
Also, a rep threshold would just encourage people to register new accounts, wouldn't it? –  James McNellis Jul 13 '10 at 2:10
4  
Somewhat orthogonal, but all of the proposed changes in this thread is really going to force the issue that we need better duplicate user detection. –  Ether Jul 13 '10 at 3:29
6  
I see a lot of Indian names in that second list.. is it the language? –  Amarghosh Jul 13 '10 at 4:28
2  
@Amarghosh: You tell us. Much of what I've been told by consultants here who are originally from India and/or spend a lot of time working with offshore teams there say that the strained politeness and rapid-fire information foraging is more of a cultural norm there than it is in North America or Western Europe. If it's not accepted, it's at least tolerated. Is there any truth to that? –  Aarobot Jul 13 '10 at 15:11
1  
@Amarghosh: I wasn't stating (or even implying, I thought) that incompetence is the norm - at least not any more so than any other country. Rather, the ones who are incompetent, I'm told, are able to get away with a lot more, particularly in the area of trying to get somebody else to do their work for them. It's very common in students here but corporate culture doesn't generally tolerate it (at least in my experience). Maybe it's frowned upon there, too, and the prevalence is simply a result of these folks' relative invisibility in the monolithic offshoring shops. I can only guess. –  Aarobot Jul 14 '10 at 13:46

I think you do need help from the Community on this. I think you need some UI to gather feedback from appropriate high-rep users about these questions. A "flag as bad question" is a minimum feature.

Similarly, I think you need a "flag as not an answer", since I'm seeing a whole lot of those. I currently flag for moderator attention and say "not an answer", but I think you might want a separate category, if only so that you can track how many of these are happening, and from which users.

And I think you may need more than this. I think you also aren't getting certain things across to some users. I'm seeing a lot of users who don't understand tags, for instance. Also, a large number who don't bother learning how to format code. I actually saw two cases of [code][/code] tags in the last week. Similarly, there are a large number who create some really crap titles. My favorite of the day is "Data not returned".

I know that one of the goals of the site was to have Google as our front page. That's great for people who are searching for answers - it should be fast and easy to do so. However, I think we need to slow down some of the new users who come here and just dump their crap questions on us (pun intended).

If you had a valid email address from these users, I'd suggest you send them a one-page "read this or else" document, but without that I'm not quite sure what to do. Maybe accept their question, but before actually posting it to the site, display a page saying something like, "Are you sure this isn't crap?", and giving a little checklist:

  • Did you use a good title? It should tell enough about your question to draw experts in to read your question and answer it? link to more about titles
  • Did you use good tags? Tags are meant to categorize your question. link to more about tags
  • Is your English clear enough? If not, consider getting help in making it clear. If you want answers, then someone has to be able to read the question. link to tips on English grammar and/or sentence structure
share|improve this answer
2  
We definitely need to make moderation easier on SO, at the moment we are swamped with stuff, we need better ways to organise our flags and such. I prefer flag as bad question to the downvote, cause the downvote is often followed by the sympathy upvote. One thing we could do which may be awesome. –  waffles Jul 13 '10 at 1:49
3  
@Weblog: My new flag would be for higher-rep users only. I don't want every Tom, Dick and Harry flagging questions as "bad". Instead, I want to see what set of questions would be flagged as bad by high-rep users. I'm thinking at least 3,000; maybe 5,000 rep before being able to use this flag. I would then look at the set of questions so flagged and see if we could characterize the questions that have been flagged this way. –  John Saunders Jul 13 '10 at 1:56
2  
I can get behind that. –  XMLbog Jul 13 '10 at 1:58
2  
If you don't have their email, you could always have a banner message: "Click here to read your personal welcome message from the stackoverflow team" :) –  Benjol Jul 13 '10 at 10:55

Idea 1:

There are language models and classifiers. I bet that a little NLP would identify a significant fraction of the drek.

However, I do not propose to auto-reject drek. Just to force moderation. Questions that flunked the NLP 'remotely like a question we've ever liked' test would go into limbo, and emerge only of rescued by someone with some rep. I would pop up on the post button, with a message like, 'Your question does not appear to be made up of conventional English sentences. Do you want to edit further? If not, it will go into a moderation queue.'

Idea 2: Give up on 'no registration.' OK, you won't like this one.

Idea 3: Give many more people more tools to resolve questions by attaching them to extant, better, questions. One of the results of success is that many new questions, even non-drekky, are not so new. This has been thrashed in other threads, but I continue to believe that 'sit for a while accumulating answers while 5 close votes pile up, then wait for an overworked diamond to work the magic merge machinery.' e.g., allow OPs to agree with a proposed duplicate and trigger a merge, or allow non-diamonds to work the merge machine.

share|improve this answer
2  
+1 for allowing ops to agree to a proposed "close as dupe" to skip some of the other votes, but I think that's a separate topic. Those users at least care enough to improve over time. –  Joel Coehoorn Jul 13 '10 at 2:28

My train of thought on the matter goes something like this:

  1. Users who consistently ask bad questions are not just bad communicators. They also don't care. If they cared, they'd be in there at least adding information and thus brining it to the attention of someone who can edit it, and the quality would improve over time both through edits and their own practice of at least trying to get an answer.

  2. Users who don't care deserve to get banned. Most won't even care (it's kinda their thing). The exception is those who'll try to make a stink about it, but we can deal with them. I think the one real requirement for participation is that you care about your content.

  3. Users who don't care about their questions also won't care about creating new accounts. This means pretty much anything we do will require going quickly for the IP-level restriction. Otherwise, we've just made the problem harder to track and sent the users a little further underground.

  4. Unfortunately, IP-level restrictions for this behavior seem ... dangerous in terms of friendly fire. Even if we don't catch any active users this way, we'll certainly catch some new users (which is probably worse, as an active user can complain and get re-instated, and they tend to provide more answers than questions anyway).

  5. I think keeping the site "low friction" is an important key to it's continued success over time. Some day Jon Skeet and other high-level contributors will all decide to move on (hopefully not on the same day). We need to be careful not to turn away tomorrow's top user today. I think anything that adds friction to the asking process is dangerous. A user with 1 rep should always be able to post their first question, even if they share an IP with an imbecile.

This all means that as tempting as it is, I don't think restricting or limiting low rep accounts from asking new questions is a good idea. I come at the problem from the other direction — increase our ability to moderate these questions. Don't reduce the rep required to gain access to abilities, but do increase the power of those abilities or reduce the number of people required to participate. With that in mind, here are some suggestions:

  • Reduce the number of votes required to close a question asked by a user with <10 rep from 5 down to 3 (or even 2).
  • Require at least 100 rep to upvote questions with a negative score (to help prevent these users from escaping "probation" on pity votes).
  • Make it possible to vote to delete questions by users with <10 rep directly, even if they're still open.
  • Questions by users with <10 that are deleted should just disappear, rather then leaving behind a locked question visible to users with 10K as is currently the case.
  • Add a 10K tool to highlight questions by users with 1 rep. There are enough of these you might want to create some heuristic to score them and show the most likely to be bad — the anti-hot list.
  • Add more incenctives (badges) for continued (not just first or even 10th) use of the 10K tools.
  • Double (or more) the daily close and flag limits at 10K.
share|improve this answer
4  
Concerning suggestion 3, what if there was just a threshold where if a question garnered, say, -5 votes (maybe with no upvotes or within a certain time of the question being asked) it just gets deleted automatically? I can't imagine there being many useful questions that rapidly attract downvotes... –  James McNellis Jul 13 '10 at 2:13
3  
I think escalating responses are gonna be the life-saver here... And if one user manages to post enough crap, fast enough, to get their whole office / school / 3rd-world sweatshop/country banned, then arousing the anger of their compatriots might work in our favor. –  Shog9 Jul 13 '10 at 2:14
10  
#1 and #2 are good ideas, but it might be good to give 10K users more close votes as well; otherwise you'll have the same problem we have now, that the people who close a lot of bad questions run out of close votes regularly. –  James McNellis Jul 13 '10 at 2:16
11  
@Shog9 you want to ban me just because some idiot from my company (that contains 1000s of employees) whom I don't even know, posted crappy questions? I don't have any control over him to begin with - we just happen to use the same proxy to access the web, that's all. Afaict, IP ban is a no-no –  Amarghosh Jul 13 '10 at 4:11
4  
@Shog9 on a second thought, IP based ban/restricted-question-frequency for unregistered users would be a good idea. Wikipedia has this feature, doesn't it? –  Amarghosh Jul 13 '10 at 5:00
1  
I think there is a problem hiding in 5; throwaway accounts will always start at rep 1 –  Marc Gravell Jul 13 '10 at 7:42

I think there are several things you can do, and they should all be taken as parts of a whole: They should be implemented together for full effect.

  1. Extend "Vote to Close" voting limits for duplicates. In other words, don't rate limit voting to close duplicates.

    Effect: This will solve a common problem that these types of questions have, They've been asked before. It will also allow the question to be closed, merged, and then editors can clean up the question (and we generally do).

  2. Create a flag reason called "Flag for editing". Add this to the existing flag reasons. I can't be everywhere; but I can guarantee if I saw posts that were flagged for editing I'd give those my attention (they should accumulate like other flags so that I can tell which ones need it most).

    Editors like editing questions. I happen to love trying to find clearer ways to help the asker ask his question. It's a fun and interesting challenge for me.

  3. Give out a badge for users that edit questions that were at negative score and then received 5 upvotes after their edit. This is only for the editor that changed the most percentage from the original post, and only the first editor to do so before it accumulated the votes (or items that make it work better).

  4. Incentivise finding duplicates.

  5. If a user has an excessive (3?) posts flagged 6 or more times as his post needing editing, he's rate limited in the next question he can ask. That's the built in time-delay system.

share|improve this answer
8  
+1 for suggestion #3. –  Joel Coehoorn Jul 13 '10 at 2:34
20  
1&2 sound great. Not wild about more badges though. #5 is key. –  Shog9 Jul 13 '10 at 2:48
1  
"Optimizer" for the hypothetical badge name? –  Niall C. Jul 13 '10 at 4:07
5  
I propose that #2 be renamed to "Flag as shithouse question" –  Mark Henderson Jul 13 '10 at 5:43
2  
Addendum to #2 - I wouldn't be bothered "flagging for editing" - I would just go in and edit it myself. What would be better is keep the "flag for edit" (for those who don't have edit priv), but also add a checkbox when you ARE editing for "Edited for language" (or something similar). –  Mark Henderson Jul 13 '10 at 5:49
13  
+1 for #4. ---- –  Pëkka Jul 13 '10 at 6:07
22  
Rate-limiting on posting new questions, tied to the quality of previous questions, is the best idea ever. It should ratchet up to ever-longer delays the more the user keeps posting bad questions. After the third post, any user who is capable of learning will have learned already; after that we can safely dismiss them as "not a useful member of the Stack Overflow community". –  Ether Jul 13 '10 at 14:58
11  
"Editors like editing questions" - i don't, i just can't bear the looks of those chaotic, badly formatted posts. –  Georg Fritzsche Jul 13 '10 at 18:18
5  
so if I may summarize.. reasons 1-4 pet axe grinding about personal aversion to any atom of duplication in the omniverse, reason 5 the one everyone thinks will actually work :) –  Jeff Atwood Jul 13 '10 at 22:19
11  
@Jeff Atwood if by pet axe grinding you mean Issues that many SO users have brought up and want you to pay attention to, then yes. If Joel and you had pitched Stack Overflow differently, no one would ever bring up duplicates. The fact is, it was pitched one way and is being maintained a different way. –  George Stocker Jul 13 '10 at 22:30
1  
@Jeff also, if people were only upvoting this for #5, then maybe Welbog should have all those votes instead? My answer was written after his. Maybe people really think that most of these suggestions will help stem the tide? Just sayin'. –  George Stocker Jul 13 '10 at 23:08
1  
@Weblog I've voted for your feature requests and will continue to do so. I'm just surprised that @jeff atwood says it's just a pet axe when there have been dozens of feature requests with 10+ votes regarding duplicates. I'd love it if we had sources for all these feature requests (and we probably do with search), and I don't want people to think that I thought all these things up, but it seems like Jeff wants to dismiss it by painting it that way, instead of legitimate ways to improve moderation on Stack Overflow. –  George Stocker Jul 14 '10 at 1:04
5  
For the record, I think all five of these points are excellent and should be explored. –  Ether Jul 14 '10 at 16:17
1  
All these ideas are brilliant, and spot on. –  Matt Joiner Mar 15 '11 at 12:53

How about a "StackOverflow induction"?

The induction would basically be a 5-minute test of the user's understanding of basic SO concepts and skills such as:

  • The difference between Questions, Answers, and Comments,
  • What the Title and Tags are for (and what they're not for),
  • How to apply code formatting.

I imagine it would be interactive, and styled to feel like SO. For instance the user could be given a list of titles, a list of question texts, and a list of tags, and would be required to pick the best title, best question body, and best tags, and enter them as if they were asking a real SO question. There would be some source code in the body and the user would need to apply appropriate code formatting to it. There could also be some test of his ability to comment/vote/select an answer.

Now, users who have, let's say less than 250 rep can be "recommended for induction" by other SO users. (A nice place to put the "recommend for induction" button/checkbox would be on the Edit page. Or perhaps if it's noticed that an edit substantially changes a question - particularly its title, tags and formatting - the editor has the "recommend for induction" option.)

After getting such a recommendation, the user receives a prominent message suggesting they take the induction. If a user gets say 3 such recommendations, he will be prevented from asking any more questions until he completes the induction.

The worst offenders will probably be too lazy to do this and good riddance to them. The others will at least know that code formatting exists...

share|improve this answer
2  
Yes, a detention button would help consistent bad posters from getting worse. It won't do anything for one-offs who toss a question with minimal information at the wall and wait for whatever help shows up. –  sysadmin1138 Jul 13 '10 at 5:23
1  
Hmm... this idea has merit. +1 –  Randolpho Jul 13 '10 at 12:45
3  
I don't think that this would be very effective. Remember that these kinds of people don't respect other people's time; if they did, they would have read the FAQ already or at least tried to form coherent sentences. If forced to go through this "induction", I suspect that many if not most of them would simply create another account. This would only work if it were required for every 1-rep account with 0 live questions - and that probably raises the barrier to entry more than Jeff & the team are willing to accept. –  Aarobot Jul 13 '10 at 13:54
1  
FWIW, this reminds me a lot of the Stack Overflow game. –  Pops Jul 13 '10 at 16:18
1  
I don't think this is a good idea. People who more or less understand the system will get > 100 rep in no time anyway, so your test basically ends up the same as "users with > 100 rep". –  mafutrct Jul 19 '10 at 13:04

My concern with any system is what effect it will have on legitimate new users. Assuming anyone with under 20 rep is in some way suspicious or in need of a kiddy pool means that the initial experience of new people is one of negative trust. As it sits right now, anyone can ask any question, so the barrier to entry is pretty low. We should probably keep that rather than relegate new users to an even less trusted state. Everyone starts with neutral reputation, I think that's key.

It does mean that one-shotters, who discover the site and think

Gee, there are a lot of smart people here. I wonder if they can explain what a pointer is.

And ask it. It's like they don't trust the search engine.

Once they've been here, then we can start assigning negative reputation to them.

  • Suspend their question-asking ability for a certain time. The time-out will probably weed out those with a small investment in the site. I think we're already doing that.
  • Use an IP block with an open-ID override. Users with good rep coming from a blocked IP (the twit in the cube next door is an annoying twit) can still log in and do stuff, but those under a certain rep-level just can't. This should probably be timed.

Language is hard, since we do have a lot of ESL users. Much as I'd like to see some kind of question-filter that pings on a post contains no capital letters, using a shift-key is not strictly required to communicate meaning. This is the kind of thing that might go into a, 'might need editing' queue of some kind rather that not get posted at all.

share|improve this answer
2  
Nobody has proposed that only low rep should be a criterion. It's low rep and crap questions. –  John Saunders Jul 13 '10 at 6:13
1  
right -- and note @john 's use of questionS, plural! –  Jeff Atwood Jul 13 '10 at 7:37

I would suggest the following combination:

  • Incentivizing the finding and closing of duplicates is a must. Stack Overflow's question base is already being swamped with too many dupes. This is going to become a problem.

  • While Stack Overflow in general has very few barriers, I feel clear feedback and timed suspensions would be the better approach for this problem. How about introducing a "bad question" flag as suggested by John Saunders. Five flags close the question. The user would get a clear message along the lines of

Substandard question

The question is entirely unintelligible, or is part of a series of questions viewed as substandard by the community. While Stack Overflow welcomes users new to programming and questions of all levels of experience, some effort when asking a question is expected.

From my own experience with users with patterns of bad questions, I'd say five to seven questions closed this way would have to result in a 24 hour suspension; twenty flags in a deletion of the account.

That is, if it is made sure that really only the crappy questions get flagged. You and I can tell what a crappy question is when we see them, but they are hard to define. A "bad" flag mustn't be misused by users who don't like a question, e.g. in open/close wars. Maybe the flags could be made contestable in that you can appeal to a moderator to have it "un-badded". I'm pretty sure most users who ask bad questions and are unwilling to improve them - except maybe for the few real trolls - would not make use of the possibility. An addition to the message above could be:

If you feel your question was flagged unjustly, you can flag the question for review by a moderator.

share|improve this answer

This may sound very naive (since I'm no professional programmer), but have you considered teaching a neuronal network (or other kind of AI) your decisions to assist (not replace!) moderation?

share|improve this answer
2  
Jon Skeets thinks your post is suspicious of triviality. Please stand by for a moderator's approval –  Tobias Kienzler Jul 13 '10 at 8:09
1  
Might as well teach it to answer questions. –  Kobi Jul 13 '10 at 9:13
1  
@Kobi: thisAlgorithmBecomingSkynetCost shouldn't be lowered too much... –  Tobias Kienzler Jul 13 '10 at 9:53
2  
Maybe not a neural network, but something somewhat more efficient: I think a machine learning/training approach could really help. There must be patterns to bad questions that can be spotted by a computer, even if a fraction will always get through. –  James Mar 7 '11 at 18:38

I've suggested this before, but I'll suggest it again - when a question is closed, even temporarily, the people who voted to close it get their close votes back. This would allow repeated bad questions from the same user, which is a common phenomenon to be closed without draining the available close-vote pool.

I'd also like to see more stress put on the notion that closing is a good thing to do - currently I think many people see it as something only done by evil bastards like me. This could be achieved by use of badges, blog/meta postings encouraging it, and more involvement in the process by those who should know better.

share|improve this answer
13  
Step 1 is to perma-ban any user caught posting a comment containing both the words "close" and "nazi" or "fascist" (except this one of course). –  Aarobot Jul 13 '10 at 13:57
5  
@Aarobot You forgot "police". –  nb69307 Jul 13 '10 at 14:17
2  
I really want this to work BEFORE the questions have to be closed. Closing is work too, unnecessary work for these terrible questions which should never even make it this far. –  Jeff Atwood Aug 10 '10 at 7:49
3  
@Jeff @nb69307 - I see part of the problem being that down voting costs a person some of their reputation. While I understand the rationale for it, does it make sense to give people back their reputation if a question they down-voted later gets closed for some reason? –  Nick Oct 4 '10 at 14:45

Lurking and posting answers is a good way to learn the system and what's expected of you as a Stack Overflow/Super User/... participant.

So allow users to post their first question "free of charge" - after all they've come here because they've got a problem.

Then don't allow them to post again until they've done some/all of the following:

  • registered
  • filled out their profile
  • accepted an answer on their first question (assuming it had answers of course)
  • responded to comments by editing their question
  • answered some questions
  • received up-votes for their first question
  • received up-votes for some of their answers
  • ...

There would need to be IP logging, e-mail checking etc. to prevent users just posting using a new ID & probably some other stuff I've not thought of.

While it won't totally prevent bad questions from turning up it should slow down the rate at which they do to a manageable level.

share|improve this answer
1  
Hmm... I like the idea, but the IP logging and email check seems to go against the original intent of the site, which was to let anonymous folk ask their questions and get their answers. Perhaps Stack Overflow has grown to the point where that's no longer possible.... but we don't want to be come the expert sex change folk, and requiring any form of such a check moves us one step in that direction. IP tracking simply won't work; too many people are going to be behind a gateway with the same IP, and too many other people will simply have a new IP address every time they log in. –  Randolpho Jul 13 '10 at 12:41

Part of the problem here is that we don't really know anything about these problem users. We are making assumptions about their newbie-ness, motivations, laziness, language skills etc., but we don't really know. I think this makes finding a solution more difficult.

Because if they're newbies, the solution is education. If they're lazy, the solution is making their life more difficult. If their problem is language skills, maybe we need a 'excusemyenglish' tag that they add themselves. etc.

I think flagging for editing isn't a bad idea, though I agree that in most cases it's easier just to go in and edit the question yourself (if you have the rep).

share|improve this answer

One thing I would like to add; 12 close votes per day is not enough; I would prefer you to increase the limits to 30.

I make a habit of only closing questions that have 4 close votes, and there was not a singlee day when 12 close votes was enough; there are just too many lousy questions, inappropriate questions, duplicated questions, belongs-on-superuser/serverfault questions that require merciless closing.

In addition, to create incentive to close questions, you should really, really award badges for users who already close a certain number of questions. Better yet, this set of badges should be able to be awarded numerous times.

share|improve this answer

Allow to sort newest questions by score.

Then bad questions won't get too much attention. They won't attract users who upvote answers, therefore they won't attract bad answerers whoring some quick rep.

Next, allow these "bad" old questions to be closed as duplicates if a new, better worded ones, when they are asked. Currently it's impossible to close an old question as a dupe of a new one (at least I recently couldn't).

share|improve this answer
1  
I'd like to see them as they are currently displayed. Just because a question doesn't gert many votes does make it "bad". –  nb69307 Jul 13 '10 at 20:14

make down-votes on questions free.

share|improve this answer
19  
too destabilizing, I think –  Jeff Atwood Jul 13 '10 at 22:02
6  
A man clearly ahead of our times –  random Aug 27 '11 at 22:31

I don't think it's the luser n00bs who are the problem. It's regulars who see bad behaviour and endorse it, by providing answers to duplicate questions.

share|improve this answer

How about allowing bounties against a question? That is, I give someone a bit of reputation if they capture dead or alive close or delete duplicate or otherwise inappropriate questions.

I know there's flagging, but that hardly ever seems to work.

share|improve this answer

This seems to me too obvious to be plausible, but I'll post it anyhow

if rep_of_asker < 100 and rep_of_closer > 10000:
    treat_closer_as_diamond_moderator() # one vote is enough, no limit

This would allow a large group of people to note and quarantine really bad questions. And also to reopen them for business if someone edits them into some useful shape.

share|improve this answer
8  
but only if the closer wishes the vote to be final –  Ian Ringrose Sep 27 '10 at 21:52

I've noticed this a lot, and it's certainly impacting my willingness to provide answers on SO.

A lot of the bad questions aren't duplicates - they are simply bad questions: Badly written, badly formed, with insufficient information. "What wrong with my codes?" followed by 200 lines of atrocious C++ that's inevitably unrelated to the real problem...

I think questions seem to usually come from 'fresh' users (<10 rep, joined within last 48 hrs), often with a poor grasp of spelling and/or grammar, let alone programming.

I've got two suggestions:-

a: "Cooling off" period: No anonymous questions, and you can only ask a question ?48 hours AFTER registering for the site or when your rep reaches ?50. Users with 1K rep could 'recommend' 1 new user per week - with the new user automatically getting the required 50 rep.

2: "Captcha". Captcha is designed to differentiate computers from people - I'm thinking of a kind of captcha that can distinguish "professional and enthusiast programmers" from, well, everone else. You'd get the "captcha" before your question was accepted by the site if your rep was lower than say, 50. The captcha question would be a simple multiple choice selected possibly according to the tags specified on your question. Failing the captcha locks your out from asking for 24 hours. Lets say you ask a question tagged C++. You could get asked: "How many states does a bool have? 0/1/2/3/4/5"

Dumb example, but you get the idea...

share|improve this answer

I would like to be able to single-handedly delete the really gnarly questions that come by the 10K tools. Questions that are pure unredeemable noise. E.g., http://stackoverflow.com/questions/3288720/can-no-one-answer-this-question-o-closed.

Another thing is that questions have something like a 2 day minimum before they can be killed. Not all questions deserve this honor.

One actionable suggestion I have is this:

  • If a question has a vote of <= -5, 10K users can immediately delete it.

In respect to problem users, I would suggest this actionable suggestion:

  • if the average of the last Q questions is under some negative number N the user has an automatic Y-day suspension. Perhaps (7, -4, 14) for the parameters.

The reasoning here is that Bad Users are Bad. They aren't going to Be Good and Ask Good Questions(If they asked good questions, they wouldn't be bad).

I am a fairly prolific question-asker (in relation to my answering), and it doesn't look like I have a single negative question(at the moment... looks around for vengeful meta users).

share|improve this answer

How about allowing users to filter questions based on the asker's reputation? If "bad" questions offend you, don't look at any until the asker has amassed a few hundred reputation.

share|improve this answer
3  
IMHO, "bad" or duplicate questions do harm the quality of the content, they do make SO less relevant and hiding the problem, even temporarily, is not a solution. Such questions should be improved if possible or removed. But the current system doesn't encourage improving or hunting them so this doesn't happen (and if it doesn't happen when a question enters the system, I don't think it will happen some time after). –  Pascal Thivent Jul 20 '10 at 22:11
up vote 61 down vote accepted

Based on the feedback from this post, we have now implemented a form of screening during the question ask period.

Questions from IP addresses or accounts with a history of extremely poor questions will no longer be accepted. This is intended to weed out the worst quality questions.

(hint hint, question votes matter, so please continue to vote the best questions up and vote the worst questions down.)

Based on our queries and a random audit sample of affected accounts / IP addresses, it seems effective, but we'll have to see now that it's deployed and perhaps tweak further.

Note, the /ask page error text is of the form:

Sorry, we are no longer accepting questions from this account.

Additionally, see How does Stack Overflow attempt to prevent low-quality questions and answers? for several other measures we now take to assist in keeping quality high.

share|improve this answer
9  
What's the criteria for a question to be judged "extremely poor"? –  nb69307 Aug 10 '10 at 11:08
28  
out of curiosity, are the initially banned IP addresses primarily from non-english-speaking regions? –  Kip Aug 10 '10 at 15:26
1  
@Robert Yes, but how many? Remember downvoting is pretty rare, and the thrust of this original question was how to filter out the cruft. If it is to be done with downvotes alone, you would have to make pretty few needed. –  nb69307 Aug 10 '10 at 16:08
4  
Is there anything in place to help bad questioners reform themselves, or are they essentially doomed once they get the metaphorical Heisman? Short of posting from a different IP address, could they email the team and get a temporary reprieve so that they might have the chance to prove they can ask questions properly? Or perhaps the banning is timed? –  gnostradamus Aug 10 '10 at 16:19
17  
How will this affect offices that have internal networks? As in, if I work in an office (or a school) with a different user who is terrible, will my account also be locked out? –  devinb Aug 10 '10 at 19:07
3  
Are deleted questions taken into account? I can think of at least a few users with a history of posting flamebait and/or downright awful questions, but they usually get closed and deleted after picking up 5 or 10 downvotes. –  Aarobot Aug 16 '10 at 16:32
1  
@Jeff, I think your system is a little too strict. This user was banned from asking, but his questions don't seem to fall into the category - "a history of extremely poor questions" –  jjnguy Aug 18 '10 at 13:12
10  
It would be a good thing if the "Sorry" page contained more information -i.e. what things likely contributed to the decision - what is a "bad question", links to how to ask questions the smart way, etc etc. Also a reminder that multiple accounts per user are grounds for suspension/deletion (or they'll just create a new account to bypass the ban). –  Ether Aug 19 '10 at 19:18
3  
@jjnguy they do, you just can't see all his deleted questions like we can –  Jeff Atwood Aug 24 '10 at 22:02
2  
Jeff, have the developers of SO thought about GUI controls for establishing a reputation threshold when viewing questions? For example, a text field under "/questions" that would work as "Only show questions from users between N points of reputation". –  user151803 Sep 28 '10 at 18:08
4  
@jeff I myself have been banned after deleting a couple of my own questions (they were all related to a specific wordpress plugin and I concluded they weren't particularly useful). You can see my user profile here: stackoverflow.com/users/405484/tks and here is the meta question I asked describing my banning more thoroughly: meta.stackexchange.com/questions/96102/…. Up until now I've greatly enjoyed using SO as a resource. Would greatly appreciate an explanation of how I violated the site's ToS –  tks Jun 23 '11 at 2:17
2  
What happens if a registered user with high reputation happens to post from a blocked IP address (e.g. using a public access point)? Will the high reputation trump the IP address blocking, or will he be blocked anyway? –  celtschk Aug 23 '12 at 7:58
2  
@warren If it happens to you, email us. It's quite rare in practice. Generally when it tends to happen it is at certain school IPs in certain countries. Never heard of it happening to an individual. –  Jeff Atwood Jan 21 '13 at 4:03
1  
I just heard this from a colleague in the Delphi-using-community on Stack Overflow that this happened to him recently [user:29290]. He was knowledgeable enough to just disconnect his DSL and reconnect, and the problem goes away. It's this kind of speed-bump that blocks out the young kids and the non-tech-savvy that is why I think this is such a mis-feature. IP blocking is only effective against the littlest "kiddies", and the collateral damage is also against those who have the least resources to deal with it. –  Warren in Toronto Jan 21 '13 at 15:30
2  
Wait. What about anonymizers? Like tor? They can work around IP bans this way. –  Cole Johnson Mar 31 '13 at 0:50

Sorry, I know I'm coming to this discussion late, but I think there's an outside-the-box type of solution: provide localized SO sites. I know I'm beating a dead horse because you and Joel talked this to death on one of the podcasts, but I really think a lot of the bad questions are people who can barely communicate in English. It's not their fault that the best site ever made for programmers requires English.

share|improve this answer
2  
please leave the horse alone. There are laws against that in most places. –  John Saunders Aug 10 '10 at 19:09

The How to ask page desperately needs examples. For example:

If you ask a vague question, you'll get a vague answer. But if you give us details and context, we can provide a useful answer.

is vague about the definition of the word "vague"! What are details and context? These are spoken like the reader knows what we mean by them.

I'd prefer to see some good examples & bad examples, and what makes them good and bad, to give the user who really does want to ask some toehold. Maybe like:

BAD: My php page is not showing the mysql resutls when i click "OK" but i get a Internal Error WTF?

GOOD: I'm using PHP and MySQL and I submit a form, and I'm getting a 500 Internal Error. Here is the query I'm trying to run:

select * from users where....

This may well be the very first time the user has posted to a site like this, and has never considered these questions. I don't mind having an EULA, but let's make it useful for the user, too.

More guides along the right on the submission form might help. There's a link to the FAQ but the FAQ doesn't cover what a user posting a question wants to know. A user asking a question doesn't need to worry about how reputation works.

SO is also treating the "how to ask a question" from the point of view of "Here's how we want you to ask the question." It feels like Question Police. The focus should instead be "Here's how to get more people to give better answers to your question" or "How to get the best answer to your question" or "How to get your question answered faster." All of those are why we guide the users to ask questions better, but we don't tell them that!

share|improve this answer

Place first posts in quarantine. The first post of new user wouldn't be visible publicly on the site until it is approved by an experienced user.

Right now, the First Posts review queue is empty, so it would only add a small delay to their publication.

Another benefit is that it might well reduce the Close vote queue.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .