Turbocharging the Roomba: solutions for premature deletion

Question

A couple weeks ago, I asked for help: How can we stop premature deletion?

And I got some fantastic responses. A big thanks to everyone who participated in that discussion - you're the reason I don't completely hate Meta Stack Overflow.

The big takeaways from that discussion were:

• Show users their deleted content. By far the most common response, and understandably so: without this, it's always a trade-off between cleaning up trash and letting the asker know what they did wrong.

• Delay deletion. There were a few different takes on this, including "pending deletion" votes that would only take effect after a period of time had passed, and straight-up blocking normal deletion votes for a period of time with spam/offensive flags and moderators able to take care of the worst stuff in the interim.

• Improved oversight for deletion.

That last item gets into an area we'll have to discuss in depth later on, namely: stuff that shouldn't be deleted at all. It's important - but it'll have to wait a little bit longer.

Joel, Jaydles, Robert & I met last week to discuss the first two and "quick deletion" in general. Here are three short-term changes we'd like to make:

1. Authors should see their deleted questions (if they have a link)

This doesn't go quite as far as this old feature request, but it does solve the most fundamental problem: if I have my question open when it's deleted and refresh the page, I'm not presented with a 404. It would also make support easier in some cases - for instance, automatic deletion of abandoned questions.

Note that the primary rationale given for declining the old feature-request - authors able to undelete their own content - was fixed a while ago; the concerns over making deleted stuff permanently visible in your profile are still somewhat valid, however.

2. Restrict quick deletions on anything asked "in good faith"

This wouldn't even have to be a system-enforced restriction, although that's probably a good idea. The idea is simple: don't delete any question that isn't blatantly abusive.

• Spam? Kill it.
• Gibberish? Erase it.
• Rants? Banish them.
• A too-localized, vague or overly-broad programming question? Meh. Just close it and move on.

If we did have to restrict deletion, it helps that there are already flags that cover most of the really bad stuff... Flags that automatically delete once a sufficient number accumulate on a post. But that still leaves a lot of rubbish laying around, so we'd also want to...

3. Automatically delete closed and abandoned questions after a short period of time

It's hard to ask folks who already feel they're drowning in an ocean of filth to hold their breath a little longer. I think mog said it best:

I'm getting burnt out on playing janitor

I've felt that way. There's not much of a reward in helping to moderate Stack Overflow. No one compliments you when you close an off-topic question, or even notices when you delete a terrible one. You hold out some hope that if you keep at it, the site will keep being something you enjoy - and maybe that the folks whose writing you enjoy will keep enjoying it as well. But you know in your heart that it's a treadmill to oblivion; with the volume of questions Stack Overflow gets every day, even closing the bad ones is hard - keeping up with deletion is proving to be impossible:

Now, it's a bit unrealistic to say that every closed question should be deleted, even when that's the recommendation; still, there are currently 48 thousand closed questions on Stack Overflow that have no up-voted or accepted answers, score <= 0, haven't been edited since they were closed, and don't have any pending re-open votes...

If we deleted those automatically, that would cover 70% of the questions deleted manually in the last month, and 73% of questions deleted within 2 days of being closed. Forget delayed deletion - for the majority of questions, no one would need to bother deleting them at all.

Automatic deletion already accounts for the vast bulk of all questions deleted. Why not just expand that to handle the stuff we can be fairly certain won't ever be revived? Abandoned questions, already evaluated by the community and closed, with no recent activity to indicate any interest in re-opening them.

Right now, automatic deletion only kicks in for questions over 30 days old. I think we could drop that considerably - say, to 7 days - and get rid of more cruft, faster and with less effort than we're doing now by hand.

We haven't settled on a concrete criteria for what "closed and abandoned" should mean yet. Here are some ideas - look 'em over & see what you think:

• Closed, no answers, and no activity for at least a week (32K)

• Closed, no up-voted/accepted answers and no pending reopen votes (61K)

• Closed, no upvoted or accepted answers, scoring <= 0, no reopen votes, no edits since closed (48K)

The goal is to capture anything that's contributing nothing to the site now, and shows no signs of any effort by anyone to correct the deficiencies that are preventing it from being a useful question. I think if we get this right, then we can afford to look at manual deletion - by moderators or high-rep users - as more of an exception-handling task and stop mixing the time-sensitive deletion of overtly abusive posts with the convenience of just voting to delete as soon as a question - any question - is closed.

Answer 1

The following changes are live now:

• Deleted questions will be visible to their authors, regardless of those authors' reputation. They won't be linked to anywhere^† that they're not already linked, but if someone knows where to find their question and it's been deleted, they should always be able to view it.

• Delete-voting within two days of closing is disabled on questions scoring -2 or above, regardless of the voter's reputation.

• Automatic deletion of closed, abandoned questions for questions meeting the following criteria:

  • Closed more than 9 days ago
  • Not closed as a duplicate
  • Score <= 0
  • Not locked
  • No answers with a score > 0
  • No accepted answer
  • No pending reopen votes
  • No edits in the past 9 days

For recent examples of the sorts of questions that will be deleted, you can view results from SEDE.

^† Linking from the profile was added in June 2013, under some conditions.

Answer 2

I think you should tighten the automatic deletion, using all three bullet points you outlined above, and show people their deleted questions (if they have a link).

I've long thought that automatic deletion could help immensely with this problem if the time frames were shortened, and keeping the deletions out of the user's profile (until they get to 10K rep) would alleviate most of the "whining" that Jeff Atwood so eloquently talks about.

It would also give the mods true "comment and delete" ability, instead of this "almost, but not quite" ability that they have now.

Answer 3

If system-assisted early deletion is the tradeoff for removing the early delete votes of 20k users, I can get behind that. I'd probably be most comfortable with the third of your options, although I honestly couldn't find anything as an example worth saving even in your second option's query.

I'm a big fan of the change in closed question wording to emphasize editing questions into shape and better explanations for why questions were closed, but we have to help that out by making sure that questions which could possibly be redeemed live long to have a chance. Removing 20k early deletion votes is one way to keep some of these alive longer, and you can feel safe that bad questions you forget to return to can be caught by the safety net. Really terrible stuff can still be flagged (like it usually is today) and moderators can remove those exceptional cases earlier.

One request I would have is to change the wording on the page people get when visiting a deleted question. Currently, that page reads (to <10k users):

This question was removed from Stack Overflow for reasons of moderation. Please refer to the FAQ for possible explanations why a question might be removed.

This makes it seem like every deleted question was removed by moderators. This question is an example of the kind of misunderstanding that can occur as a result. Now, in that case the ability for someone to see their own deleted question would have prevented that part of the problem. Still, I think the wording here could be improved for other outside visitors (or those who answered a now-deleted question), particularly if the system is going to be deleting more of these for us.

Answer 4

As a mod, more automatic deleting greatly lessens two of my biggest concerns as a mod:

Getting rid of crap

Having closed, clearly low quality content on the front page is embarrassing. So is having it on the first page of /questions. But when do I manually delete it? A day is too soon. After two days I'm likely to forget. But at the moment, that crap's unlikely to be automatically deleted unless it's answerless, downvoted and closed. Closed I can almost always count on, downvoted, usually, answerless not as often. But the 30 day wait is killer.

Having the community do deletions

This is practically impossible on certain early-mid beta sites or recently graduated sites where there just aren't many users with the delete privilege. I feel like I should let the community make these decisions moreso than me, but it just...doesn't happen. Sometimes it can't happen. But if we don't require high levels of coordination between vanishingly rare top-rep-tier users, this problem goes away.

I'd feel a lot better knowing this junk will, for the most part, handle itself. There's clear paths to keeping the content as well (just edit/fix/reopen/upvote it) so I'm not concerned with the baby being thrown out with the bathwater. At least not a baby no one upvoted, who wants that baby. No one.

Now aside from that, I have one concern: I'm not sure an accepted, unupvoted answer should keep a post around. Off topic questions often get an off topic but correct answer or a "you actually answered my question! I give you this checkmark" accept. Keeping those around doesn't help anyone and it only takes the OP to prevent a deletion. In my opinion, someone else should have to indicate that the answer is useful before we start keeping around the question forever.

Answer 5

The plan of attack you outline sounds great. I assume the work you have done recently to mark duplicates differently than regular closed questions will factor into the auto-deletion routine, but make sure duplicates closed under the old system get distinctive treatment too :)

The most applicable bit I see for smaller sites is the ability for OP's to see their own questions if they have a link. This is particularly meaningful in that they will get an inbox ping or two with comments from a mod if something is manually deleted. Auto-deletion might be silent and they might not have a link to follow, but if there is something they need to read then they would have the link.

Let that Roomba roll!

Answer 6

I'm not a big fan of automatic deletions because there's no oversight, and no warning to the user that their post went missing.

But if you did increase the automatic deletion window to items older than 7 days instead of 30, I'd like to see the posts be:

• closed
  • as something other than duplicate
  • over 7 days ago so the user has time to see the closure reason
• no answers
  • something answered is much more likely to have the OP coming back to the post to re-read the answer
  • the answerer may come looking for their answer so they can copy parts of it into another answer (I do this all the time)
• no upvotes
  • if a post got rep, users are likely to come looking on meta for their missing rep when the post gets deleted, thus increasing moderator workload instead of decreasing it
• no reopen votes in the past 14 or 30 days
  • if someone made an attempt to reopen something, it was probably valuable to them in some way, so give the post a larger window of to stay visible before deleting it
• no activity for at least a week
  • no one is trying to fix or reopen the post

The whole point of automatic deletions is to reduce the moderator workload, and it wouldn't make sense to have the process trigger a lot of "Where did my question/reputation/answer go" posts appearing on meta as a result of the change. You want to be sure you only get rid of posts that are no longer useful to both the community and the OP.

With that said, there are many posts which are not useful to the community and could probably be deleted, but that are useful to the OP (mostly localized posts). If you had some way for the OP to see their deleted posts from either their profile or from some non-publicized url path, then I would be much more in favor of looser automatic deletion requirements.

I've personally had multiple occasions where I've gone to look for an answer I got in the past, only to discover I'm unable to find the post in my profile anywhere because it's been deleted. Sure the question wasn't useful to the community and I could understand the deletion, but the answers that were posted were still very useful to me.

And last of all, on your point of

Restrict quick deletions on anything asked "in good faith"

Here's my problem with that: not every user sees a post the same way.

• One person's version of "spam" may be another person's version of "answering questions", particularly when someone is trying to publicize their personal blog.

• One person's version of "gibberish" may just be another non-english speaker's version of "trying to get answers"

• One person's version of "rant" is another person's version of "asking a question"

In all cases, deleting content as soon as it's closed does not teach the user anything.

You have a whole army of community users who are happy to help out with site maintenance, however most of the time they are looking at recently active posts, and not older posts that have been closed for a bit.

If you provided this large group of users with a way to say "delete this post in X days unless it gets significantly edited or reopened", then you will have many more users stepping up to help out with site maintenance, thus reducing the moderator workload.

Answer 7

I like this. On sites like Physics, the community doesn't participate much in deletions (and we don't have enough active high reps anyway). As a mod, I could do it, but none of us have really felt the need for it. This seems to solve the problem.

However, come to think of it, do we really need to delete the posts? The point is to avoid the broken windows problem. Instead of deleting them (making them unsearchable and unreadable), here's a radical idea:

Just make them hard to get to.

Basically, any 7 day old closed post will be hard to get to unless you specifically search for it. If I search for something, I should get a little link at the top that more or less says "There are 30 old, closed questions that have been excluded from this search. [Include all closed questions]". Such questions should also disappear from all question lists (like the "highest votes" list). This cleanly avoids the broken windows problem. A visitor won't ever come across these, and neither will a regular user unless they explicitly search for it. Deleted questions sometimes have good answers. In other cases there are meta discussions about these. There are many times when I want to see a q on SO, and can't -- because the deletion army got there first. Giving them the silent treatment instead of deleting them avoids such annoyances, while still solving the broken windows problem.

I'm not entirely sure of this; but I'd like to put it out there :)

Answer 8

• Closed, no answers, and no activity for at least a week (32K)
• Closed, no up-voted/accepted answers and no pending reopen votes (61K)
• Closed, no upvoted or accepted answers, scoring <= 0, no reopen votes, no edits since closed (48K)

I have a general aversion to any automatic deletion of content. That is, actual information that could be useful to someone. So if a closed question does have an answer, I think a human being needs to be involved in deciding whether it should go away.

Perhaps we should have tiers. The first tier is what you mention first: no answers, no activity for a week, and closed; such posts are a priori contentless (also, not closed as duplicates). The second tier is the second thing you mentioned, and it kicks in after, say, 3 weeks of inactivity with no pending reopen votes (ie: the answer(s) were never accepted or upvoted).

That should cover most of the cases, while still making sure that someone has given some minimal review to the utility of the content. That is, people saw it and didn't upvote anything.

---

We may want to consider not showing closed questions on the main page, unless they have been edited since they were closed. That will at least deal with the issue of "broken windows" on the main page.

---

If we allow users to see their deleted questions, we also need to make sure that users are informed that their question was deleted via an inbox message. Otherwise, how would they get such a link?

Answer 9

I really love the idea that the early deletion for 20k users can only apply to certain topics like

• Spam
• Hate-Speech
• ...

Kill that stuff immediately. (That is what early delete votes are for IMHO)

But what shall be done with the other posts?
Currently the early delete votes are used on VLQ so they don't have to come back 2 days later.

So I suggest, that you can vote to delete, but this vote will take effect until the 2 days are over.
In this time the OP or some other person can try to salvage the question.

So what should happen with with the votes before the question was edited

1. Discard the old votes?
2. Keep it and hope some people vote for reopen, do not delete?

I prefer the 2. way because this won't lead the OP from editing his question with please don't delete it.

So I suggest that you have to give a reason (multiple choice) to do a early deletion vote.
If the reason is not good enough for the early deletion, count it as "delete it later".

Answer 10

Danger, Will Robinson, Danger!

It's very dangerous to get into more auto-deleting. For now I'd just ask to please make sure and exclude Closed As Duplicates; let the stubs hangout and help searches.