Handling "Problematic" Stack Overflow participants

Question

What do I mean by "problematic Stack Overflow participants"? Here are the criteria:

1. Asks dozens to hundreds of questions over a period of 3-6 months (or more).

2. All questions are consistently of low quality and hard to understand, requiring major editing.

3. Extremely low answer accept rate (below 10%).

4. No increase in quality (effort expended to make questions good, clear, and useful) over time between oldest questions and newest questions.

5. Extremely low answer count (0-10). User does not "give back" by occasionally trying to help other programmers answer their questions, only "takes" by asking questions.

These are users who have, over a period of 3-6 months, demonstrated an utter and complete lack of interest in ...

a) putting their own effort into the questions they ask of the Stack Overflow community
b) learning from the Stack Overflow community
c) giving anything back to the Stack Overflow community

As of today, we have started to place "problematic" Stack Overflow accounts on suspension, with an email warning.

Fortunately, this is a tiny minority of users, but for these rare exceptional cases, I believe it's something we need to deal with directly.

Answer 1

Response to balpha's answer:

The more the better. Each additional question on SO adds to the knowledge base (if the question gets useful answers) or just sits there to be forgotten—with 48 gigs of memory in your DB server, the latter shouldn't be much of a problem. Real crap can be deleted, anyway.

Not really. Most of them are a bunch of duplicate, vague, unanswerable, basically crap questions. Your assumption that all it takes is RAM is not true. Community resources are spent to edit and try making them clear, answer those questions the best way possible, finding the duplicate and voting to close, etc. but even then, it doesn't provide much added value since the question is really fundamentally vague and not useful. Beside that, the more crap you have in the system, the harder it becomes to search and find good stuff.

So? These questions either turn into something good in the long run, because (with or without help from the asker) they are edited for the better and possibly get good answers, or they get downvoted and drown in the sands of time (or get plain deleted).

They become better, yes. They get downvoted. They get closed. They get deleted. All true. But it's at the cost of community resources. It's a simple cost-benefit analysis. The equal amount of effort can be put to much better questions. Particularly, considering the fact that there are very few users that post really crap. I believe they are less than 10 users but the overall impact is not very low.

Each potential answerer sees the acceptance rate and can base his decision (whether to answer or not) on this. If they provide a good answer and get votes from that, that should be okay–rep-wise because the vote rep outweighs the possible 15 for the accept tick, and knowledge-wise because the best answer (by community choice) floats to the top.

The problem is not whether the OP is going to accept your answer or not. It's the fact that the OP doesn't give a damn about the community. Not accepting answers is just a side effect of this lack of interest. They don't care to learn from the previous edits made to their questions. They don't care about the community at all. They are just a waste for community resources, providing little or no benefit.

The only possible thing to do here is to disallow asking, be it temporarily or forever. This, however, means that none of the questions that otherwise might have been posted will ever have the chance to grow into something better and in the end actually add to the site. Also, if you want to keep up with the "no registration neccessary" rule, the way to circumvent this suspension would be to just clear your cookies. These people usually don't care about their account, let alone their rep.

There might be ways to detect those to some extent (IP address). It becomes harder for the user to post questions. At least, they'll have to care about what they post a little more. And they'll understand that they are not wanted at the community. Currently, they have no ill feelings about what they do.

Beside that, this is a technical issue worthy of a separate discussion. Assuming that we can ban them from SO, will we want to do it or not. This is the question.

Answer 2

Design a metric (apart from rep) like Really Bad Question Rate (RBQR), or whatever name you choose, which takes into account the above criteria. This doesn't need to be published, although the user may see it and be warned about it.

Example:

Only actions in the last 30 days (or last 30 questions, regardless of date?) are considered.

• Question with negative votes and not accepted answer: +2**v* RBQR (v == - Question Votes)
• Question with negative votes and accepted answer: +1**v* RBQR
• Question is closed, additional +5 RBQR
• No tags, additional +1 RBQR
• Mitigating actions: Question with positive votes: -0.5 RBQR, Answer with positive votes: -0.5 RBQR
• Latest 5 questions have double RBQR contribution, for good or bad.

Policy:

For users with RBQR over a certain level, forbid asking new questions (explain why) until 7 days pass. Then they can ask one new question. After seven days they will be able to ask another. They are in limited status now. They may answer questions, though.

The previous metric will keep being recalculated. If data improves, they will go back to their normal status. If things get even worse (i.e. if they use they 7-limit permission only to keep asking yet more negatively voted questions) they will go to an even more restricted status in which they cannot ask anything, say, until 3 months have passed.

Answer 3

Two Three Alternatives

1. Ignore Them - from the description, it sounds like a handful of freeloaders and a few hundred 'bad' questions. One can expend an awful lot of effort to expunge the weeds, but they'll always be more to come. Don't expend significant effort to address a minor issue. And perhaps the poor questions get poor answers, which is enough to satisfy the poor users ;-)

2. Target the Questions, Not the Users - perhaps the questions should be closed as 'low quality' instead of edited or answered. That would discourage more of same, and should be much more frustrating discouraging to Low-Grade Users than suspension or the inability to post questions at all.

3. as @Diago explained, I can see how the volume thing would be annoying - could it be that rewriting a question to 'help' is actually doing the OP's work for them, and thus they have no incentive to improve? Perhaps it would make more sense to "nip it in the bud" so to speak, i.e. notify the OP that their question has been quarantined as 'bad' and until it is rewritten - by the OP - to be sensible it won't appear in the Q list and the OP will not be allowed to post any more questions. That would -theoretically - stop the one or two 'bad question' posters from posting more bad questions and encourage them to improve. Which may end up being exactly what Jeff was proposing all along (I've lost track), but perhaps not - instead of tracking new metrics to find bad users, just put a stop to bad questions and their OPs immediately. Or on the second bad Q, as you mods see fit. ;-)

Answer 4

My 2 cents:

• Increase the rep penalty against down voted authors to -5 instead of -2.

• Limit users to 2 questions per day. Questions with a score of 2 or more adds an additional question per day.

• Once you have 1k rep in the system, remove the question restriction.

That will make it harder to game the system by spamming questions. I really think this behavior is a symptom of a larger issue which is that rep != trust like its supposed to. Is Jon Skeet really nearly twice as trustworthy as Bill the Lizard?

To really take care of this once and for all I think we need to split the current rep system into 2 values; 1 for knowledge and 1 for citizenship.

The citizenship value starts at 100 and gains a little from up-voted answers, more from up-voted comments and more still from up-voted questions. It loses points for questions being substantially edited (tools are already in place to measure this, like for community wiki authorship), down-voted, flagged for moderation or voted to be closed.

The knowledge score is based heavily on answer up-votes, answer-acceptance and very, very little on question up votes.

Thats a bit heavy handed and a huge change to the system that might not be worth the effort. In the meantime, I think the above measures and waving the ban-hammer should be sufficient for now.

I think that system trust could be based on an aggregate of the two values, a minimum of each must be obtained before you get anything.

Answer 5

I totally agree with this move, since it's possible for a tiny minority of users to generate a great deal of noise and (possibly) wasted effort.

That said, however, in the interests of fairness you might want to point out in the FAQ (or some other prominent location) that certain behaviors are expected of regular participants, i.e.:

If you participate regularly, you are expected to:

• put at least a minimum of effort into the questions you ask of the Stack Overflow community
• show that you are trying to learn from the Stack Overflow community
• give back to the Stack Overflow community through helping others

Answer 6

Design an algorithm for detecting them, and then take automated punitive actions incrementally:

• Warning messages when they try to go to the Ask Question page. ("We really want you to play well with others".)
• Place their questions in a penalty box (hours days?) wherein you notify them that this is the case, it is meant to encourage good behaviors, and how they can correct it.
• Throttle their question asking ability, like comments
• Automated deletion of questions with score < 0 with progressively more inclusive logic. E.g.: If they ask a question and it gets more than 5 down votes in one day, then poof. If they keep at it, then drop the threshold to 4 votes. Then 3. Then...
• Add a reputation leakage mechanism, say a percentage of all points, or maybe a freeze on earning new reputation. Or both.

I guess a risk to this is that they use SO without a user account, and then SO can't associate these bum questions with the bum users.

Answer 7

One step would be to encourage closing the questions. Usually, the closest I can come to a close reason is "Not a real question". Lots of the bad questions I've seen are real questions, but they are hard to read and don't have nearly enough information. Come up with a good description of a useless question, stick it in as a close reason, and everybody with 3K rep can vote to close with a good conscience.

The problem with banning accounts is that they're basically free. If all I was interested in was asking questions casually and hoping to get lucky with an answer, I wouldn't care about the account I was using, as long as I could get another one. Closing and deleting bad questions would be more of a discouragement, and would keep the question base cleaner.

Answer 8

Invent another flag available to the top 1% of users, say 'violates community spirit'. If a particular poster accumulates 10 of these, his/her questions/answers/comments/edits do not become visible for 30 minutes after posting for one week. After 50 such flags, the user is suspended for a week. After 100 such flags, the user's existing rep is cleared permanently.

Answer 9

I'm in favour of Daniel's suggestion. However, I'd like to request as part of this: Provide users affected by the Really Bad Question Rate (RBQR) with a channel of communication to the moderators.

RBQR or an equivalent measure, if and when instituted, will form the harshest punishment yet on these sites, and it can be a bitter pill to swallow.

Please provide a simple way for the person to query or give their side of the story. This is good for everyone, even if it is only for the moderators to explain that their questions have been regularly flagged etc. You will get the occasional user who's been wrongly targeted. A human touch is always helpful; lack of communication tends to exacerbate issues. If the number of such users is low, then the communication will be manageable. Also, not all affected users will use this feature, only those who feel wrongly done by.

You could provide a simple text box for the user to enter what they wish, and get their replies through a notification system similar to comments etc.

Answer 10

Oops, some of the things you mention fit my own Stack Overflow profile. Not that I think I will be a victim of this cut, but some of the criterions sounds harsh.

Do not get me wrong on this one. I feel your pain and I think it is a valid and important question. But maybe these two criterions are a little bit harshly written:

All questions are consistently of low quality and hard to understand, requiring major editing.

I am Swedish and I spell bad in my own language, so I guess I am probably will need some editing help.

Extremely low answer count (0-10). User does not "give back" by occasionally trying to help other programmers answer their questions, only "takes" by asking questions.

I think that the beginners should be able to have period of time where they do not contribute that much. I personally got the theoretical background, but almost no real experience and even I think it is kind of hard to find unanswered questions that I can help people with. If you are a beginner in every way you really have to search for questions to answer.

Answer 11

I have to disagree with the others here. I don't think there should be much done about this.

‍1. Asks dozens to hundreds of questions over a period of 3-6 months (or more).

The more the better. Each additional question on SO adds to the knowledge base (if the question gets useful answers) or just sits there to be forgotten—with 48 gigs of memory in your DB server, the latter shouldn't be much of a problem. Real crap can be deleted, anyway.

‍2. All questions are consistently of low quality and hard to understand, requiring major editing.

So? These questions either turn into something good in the long run, because (with or without help from the asker) they are edited for the better and possibly get good answers, or they get downvoted and drown in the sands of time (or get plain deleted).

‍3. Extremely low answer accept rate (below 10%).

Each potential answerer sees the acceptance rate and can base his decision (whether to answer or not) on this. If they provide a good answer and get votes from that, that should be okay–rep-wise because the vote rep outweighs the possible 15 for the accept tick, and knowledge-wise because the best answer (by community choice) floats to the top.

If anything is to be done at all, it should rather be something like the often asked-for "community-accepted answer" (which I'm not a big fan of myself), or maybe a badge for an answer with many upvotes on a question that has no accepted answer but is older than, say, 30 days.

‍4. No increase in quality (effort expended to make questions good, clear, and useful) over time between oldest questions and newest questions.

The only possible thing to do here is to disallow asking, be it temporarily or forever. This, however, means that none of the questions that otherwise might have been posted will ever have the chance to grow into something better and in the end actually add to the site. Also, if you want to keep up with the "no registration neccessary" rule, the way to circumvent this suspension would be to just clear your cookies. These people usually don't care about their account, let alone their rep.

‍5. Extremely low answer count (0-10). User does not "give back" by occasionally trying to help other programmers answer their questions, only "takes" by asking questions.

Do you really want these people to answer questions?

These are users who have, over a period of 3-6 months, demonstrated an utter and complete lack of interest in ...

a) putting their own effort into the questions they ask of the Stack Overflow community

It's hard to imagine that there's a way to teach them different. So I say let's embrace them, try to nudge them into the right direction once in a while with a "You might want to..." comment, and if they post a question that in the end leads to more knowledge on SO, the deed is done. Maybe one day they will themselves realize what a great community this is and improve their own behavior. Maybe not. Cutting them out doesn't do anything good–unless, as I said above, you want to start requiring registration.

b) learning from the Stack Overflow community

If they keep coming back, I guess they have found that on SO, they get answers that help them. If just one out of a hundred takes more from that than just copy & paste the code from the answer–great!

c) giving anything back to the Stack Overflow community

Not directly, true. But if any of their questions turn out to actually add to the site, they have given to SO, which, in the long run, helps the community as well.

---

Long rant. Let me make this clear: I don't like these people's behavior, either. But, as explained above, I don't think there should be any harsh measures taken. The more 3k+ or 10k+ users SO gets over time, the faster the noise is dealt with. As far as I see it, this has worked pretty well so far.

---

Update: Mehrdad has posted a (disagreeing) answer to this one which makes some valid points.

Answer 12

The only caution that I would have would be to be very patient with non-English speakers. I'm sure it would take me more than 3-6 months to become understandable in a new language. I'd do the warning first, then if you don't get feedback (or a change in behavior) that the person is making an effort put the account into suspension.

Answer 13

So? These questions either turn into something good in the long run, because (with or without help from the asker) they are edited for the better and possibly get good answers, or they get downvoted and drown in the sands of time (or get plain deleted).

What actually happens is this:

Someone posts an incredibly easy question that a very large % of users could answer (see this question that I answered). A lot of people rush to get the free rep, and in the process a lot of users view the question. What follows is that there is a higher likelihood for upvotes on the question.

Note that the user who asked this question has their account suspended, and also note that I downvoted this question (after looking at the account and seeing so many questions), even though I answered it. Assuming I am the only downvoter, the asker still gained 28 rep from it, since the question still has a positive 2 votes.

A quick look through the askers profile will show lots of questions which have answers that link to the same exact page of documentation. Frankly it looks like this particular user goes through the documentation in jQuery and makes up "reverse questions" that are likely to get a lot of attention/answering.

For some reason, a question that a lot of people know the answer to is more likely to get upvotes than a really hard question that few people understand. I'm guessing this is due to the fact that more people will open it in the first place based on the title.

Even if you buy that these users not just farming rep, they are definitely not making any attempt whatsoever to learn anything.

Answer 14

I was talking with two other SO users that I know and this idea came up...

The entire reputation system should be changed. Rather than reputation showing a score related to up/down votes, reputation should be a score of your total participation on SO. For example, the number of up/down votes you give, receive, number of your answers that are accepted, number of answers you accept, percentage of questions that you ask that have an accepted answer, number of comments, votes on comments, number (and reason) of closed questions, profile completion, badges (?), and maybe other things like flagging posts or having a post flagged will all be factored into a single number, and that number is your reputation.

This will show a lot of the "bad behavior" on SO:

• Not accepting answers
• Not upvoting helpful answers
• Not posting answers that are helpful (ie - answers that receive 0 upvotes or receive downvotes)
• Not commenting to try to improve post quality
• Not posting helpful comments (helpful comments = upvoted)

Answer 15

Late entry...

560 questions, 65 answers, 6.5k rep

And his profile comment:

Stop asking me to 'accept' more answers. I asked many question and a significant amount of questions are

1. not correctly answered which i no longer need to know the correct answer and do not know it myself
2. Subjective enough to not know which answer is the most correct.
3. Is a question i don't remember what the solution was (typically older)
4. A no answer or wiki question. These dont affect ratio but i can imagine people asking.

Answer 16

I found out after a quick revision of some problematic users that even if many questions are bad, there are some (relatively) good questions with very good answers.

EDIT: The crucial point is moderators' work. I had missed that. Yes, punish the lousy users hard.

Answer 17

One thing I've noticed is a lot of the really bad questions come from (unknown)yahoo or (unknown)google accounts with under 50 rep.

Perhaps require filling out the profile fields if you want to ask questions and have under 100 rep? That is, provide a hassle bar high enough that the casual noise-maker goes away?

Answer 18

While it's a very valid point, I disagree with point c.

These users are (indirectly) giving back to the the community; answers to their questions get upvoted. I've received several Good Answer badges from these kind of questions.

Answer 19

Make downvoting questions free to the voter and increase the penalty on the poster to maybe 4 or 5. Unlike answers, I don't see any incentive to downvote questions except the legitimate, intended purpose - it's a crappy question.

A little bit of noise from crappy questions is not quite as much of a problem as leech users getting enough system trust to actually become annoying. I like to think more downvotes on a question will encourage most people to not answer/move on, and if leeches don't get answers, they will stop!

Answer 20

How about adding a Leach badge for these types of people?

Answer 21

This reminds me of how I used to handle certain people on an old vbulletin forum I ran. You didn't want to ban them because they would just come right back anyway. There was a forum plugin called "miserable users", and it basically did the following to marked users on every page load:

n = rand(1, 8)

if n == 1
    sleep(rand(10, 30))
else if n == 2
    // issue some random http error code
else if n == 3
    // show blank page
else
    // show the page normal

It sure was fun(and very effective). lol.

Answer 22

I've seen in polish business social network Goldenline the flag ignore this user, which makes for the clicker posts of given user invisible in foras (there are a lot of spam from companies trying to sell something). Additionaly, user that has many ignores will get moderator attention.

I think for StackOverflow a modified version of this mechanism could be applied. I could ignore given user (temporarily or permanently) or trust given user. This would affect the direct search result. Additionally, on the most active panel posts from trusted users (and users trusted for my trusted) would be higher, and the posts of the users ignored by my trusted - lower. If possible to affect without visible performance impact, it could solve the issue.

This option should be possible to activate/desactivate in user's settings. If someone wants to see low quality posts and answer them, it is his choice. Note that many of this posts comes from users who simply have poor English level - they may understand English, but writing an understandable for the others sentence is too hard for them. Banning them for it would be unfair, since English is practically enforced for programmer to learn (lack of translated documentation etc.)

Answer 23

I think I'm missing something here.

What is exactly the problem with the "problematic participants"?

If they are a minority, and basically what they are doing is ... nothing... well what's the problem with that?

Are they consuming resources?

Wasn't this the whole rationale of reputation and lately of the answer rating?

If questions are hard to understand or the user doesn't care in making it better, I think we should simply ignore the question in first place!!, There is not an obligation in editing everything is asked and transform an horrible question into a great question ( that I know )

Probably the problem could be our SO addictio, because, we have this "need" of answer what is unanswerable. I see some guys suffering because a question have a punctuation flaw and is just like they can't live if it is not fixed.

Shouldn't the downvotes, the low reputation and ultimately the tools users with +3k have ( close, edit etc ) do the work?

Now, really is not I'm criticizing this effort, I genuinely don't quite get what the problem is? What am I missing here?

Isn't this a "perfecionisn" problem is it?