How can we stop premature deletion?

Question

Update: discussion on specific changes to how deletion works is here:

Turbocharging the Roomba: solutions for premature deletion

This needs to stop: https://stackoverflow.com/posts/15348333/revisions

It doesn't help to improve close descriptions or encourage fixing and reopening if closed questions are going to be deleted within a few hours or minutes of being closed.

We get fairly regular complaints - via support emails, here on MSO, even on Twitter - about questions that've just disappeared with the asker left none the wiser. I don't have much sympathy for folks who ask a question and then leave for the weekend, but if you're responsive to criticism then you really should be able to step away for a cup of coffee before returning to the question. I think the 20K "instant delete vote" privilege is being misused in some cases; I'd prefer to not throw the baby out with the bathwater by taking it away entirely, but some amount of discretion needs to be applied if it's to remain.

Background

Originally - as laid out in The Stack Overflow Question Lifecycle - there was a fairly generous review period after a question got closed:

5. Once closed, the question can be reopened by voting to open in the same manner. If the question garners five (5) votes to reopen, the process starts over at step #3.

6. If question remains closed for 48 full hours, it is now eligible for deletion.

If a question didn't get fixed and reopened in two days, then it seemed a pretty good bet that it wouldn't be fixed - so, fair game for folks looking to clean up the clutter a bit. I know I certainly made good use of this as a 10k user. Then, this happened:

Remove the 2 day limit on voting to delete closed questions.

20K users should have enough about them to know when a question should be deleted rather than leaving it closed.

Note that the request was for the removal of the waiting period to vote - but the implementation went ahead and removed the waiting period for actual deletion. So what was intended as an expedient way for savvy folks looking at an unsalvageable question to save themselves the trouble of revisiting it later became a tool that breaks the feedback loop between askers and the community.

How big of a problem is this?

It's not huge, but it's significant:

• 1043 questions asked in the past 30 days have been deleted "early" by voters. Some of them are ridiculously bad - exactly the sort of rubbish that ChrisF had in mind when proposing this feature. Many are just run-of-the-mill poor questions.
• 221 had at least one up-vote,
• 112 had a score >=0 after being closed.
• 52 were asked by folks with at least 1000 reputation on SO.
• 395 had at least one answer.
• 186 had at least one up-voted answer.

You can review all the questions here if you're interested (and have at least 10K on SO): http://jsbin.com/isobux/1

What should be done?

Well, if you're reading this, have over 20K reputation on SO and you're in the habit of casting delete votes on questions when you close them... Stop deleting stuff early if it's not egregiously bad. In particular, if you're voting to close it's really not hard to go back through your close votes and vote to delete after a couple of days - this wasn't possible when the 20k privilege was introduced.

Beyond that, I'm open to suggestions:

• Review queues (how would this work?)
• The original "delayed deletion" idea
• Get rid of early deletion entirely in favor of the auto-deletion that goes with spam and offensive votes.
• ?

Answer 1

I'm here because of a personal email sent by Shog9 to the "top users of this particular feature on Stack Overflow" (I guess that means I delete stuff a lot), otherwise I've been avoiding SO and Meta lately due to frustration with the volume of low quality content on SO and the tools for managing it, including deletion.

The problem is not with deletion itself.

What is the point of a closed question? That's a broad topic but the end goal is to keep the quality of the content high. Too many posts about the exact same topic, crap questions that make no sense, or flame wars like Shark VS Gorilla and the site starts to suck.

So what's the problem? That posts are getting deleted? Or simply getting deleted too fast?

We get fairly regular complaints - via support emails, here on MSO, even on Twitter - about questions that've just disappeared with the asker left none the wiser.

Well that's the problem - the utter lack of feedback from the website itself. They asked a question and it vanished without a trace. Is it a bug? What happened? Should I ask it again?

I've addressed this issue here: Can the OP see their deleted question? If not, they should be able to. and here: https://meta.stackexchange.com/a/168222/159834

The fact that users can't see their own deleted questions has the following effects on overall question quality:

• It affects my moderation behavior (and others I've talked to) by causing me to withhold delete votes, which means more should-be-deleted posts in search results and the "Related" sidebar.
• It can lead to duplicate questions, since the OP has no clue what happened
• It does not help the author understand what they did wrong, and what can be done to prevent it in the future (so, more low quality questions). How is the author expected to learn from the mistake?

There should at least be a warning, or inbox notification when your own question is deleted, especially when it's an active question posted only minutes ago. Even allowing a limited view of the post would be a big improvement (maybe only allow viewing for a certain amount of time, not forever).

Maybe the OP should be able to view the question for a specific amount of time, number of views, or until they click "OK" or something - just let them be able to see the votes, comments, answers, and closure reason, at least once, otherwise they won't learn anything. This goes for all questions, no matter if they are well-intentioned horrible crap or borderline cases like the one you're concerned about.

Delayed close votes seem clever - but then you have this new visible status of "pending delete in 48 hours" or something, and have to introduce "early undelete" votes or some crap to compensate.

it's really not hard to go back through your close votes and vote to delete after a couple of days

Sure it's not hard, but it's also not easy to remember and also takes extra time. Delete-on-sight is much easier, I'm not likely to change my mind in two days. With the amount of stuff coming in that needs to be pruned, this is a weak strategy.

Mistakes can be made sometimes, but I think the system's working fine as it is, and to be honest I don't think nearly enough questions get deleted. Don't hinder our ability to keep the site free from junk - it's one of the reasons SO is so successful. Instead, provide more feedback to the users who are posting it.

Answer 2

I think one major issue is that deletions can't be properly reviewed, mostly because of the incredibly poor interface. The 10K tools are well hidden, even more so after the introduction of the new review queues. And even if you manage to find them, the list of recent deletions is extremely limited and uninformative, you'll have to actually visit each question to get the full story.

Adding just the question's score in the list, for example, would be awesome, I wouldn't have to waste time reviewing extremely low scored questions and could concentrate on deleted questions that might have a chance of getting undeleted.

I don't know if a recently deleted review queue would be the better approach in general, but it would certainly be a far better solution than the current. The review queues have their problems, but the interface alone is reason enough to adopt them for recent deletions and move away from the dumb lists in the 10K tools.

Another thing that might help would be a couple of warnings when someone is voting to delete. The examples that immediately come to mind is whether the question has been significantly edited since it was closed or whether there are pending re-open votes. I'm not so sure delayed deletions would be a good thing (in the long run), but a simple warning that the question had some activity towards getting re-opened seems quite easy to implement and might just do the trick.

Answer 3

This would only be a sticking plaster, but let people see their deleted questions if only for the first 48 hours after deletion. The links to them would still remain invisible to everyone else (except moderators).

This coupled with notification of comments should give people enough time to correct the post (if it is salvageable) while still having it off the site.

Once it's been edited it gets moved to the "undelete" queue for review and hopeful reemergence onto the site.

Of course, if the question has been deleted due to spam or offensive flags then it should not be made visible to the owner. There's no point with these "questions".

Answer 4

I don't think a "stop doing this" post on meta is something that will work on a site the size of Stack Overflow. There are a lot of people who don't participate in meta and given the ease with which some tags generate rep, it's not hard for someone to amass 20k without learning the site's culture. However, I don't advocate taking away the privileges either, as it is indeed useful. Here are a few suggestions:

• The delete link is very tempting and invites a 20k user to vote to delete it (after all, how bad is "very bad" is subjective). A simple solution would be to hide it behind a "tools" link (per post), like the "mod" link for mods. That way, the thought won't cross someone's mind unless they explicitly go through a couple of hoops to get to the link. (Perhaps also lump protect in there, so that the per-post tools link is more meaningful).

• Allow delayed deletion. Just like mods can lock a post for X hrs, allow 20k users to delete something after X hrs (say, 12, 24, 36 and 48 hrs) until 2 days have passed. After that, it is immediate.

• A deletion review queue. Many users are over zealous with their delete votes because of the belief (possibly justified) that if they miss this chance, it'll never get deleted. If you implement a review queue, people will be more content with checking the queue at their own pace and allowing the closed question some more breathing time.

• An un-deletion queue, to review deletions. Right now, if something is deleted, it's gone for good unless the user makes a hue and cry over it on meta.

• Filtering deletion queues by tag (like in the close queue). While one can easily gauge the worthlessness of a terrible question in random tag X, it's harder to gauge the worth of a borderline question in tag X if you are not familiar with X. Allowing users to focus on their favourite tags might reduce arbitrary deletions. I would even suggest that filtering by the user's top 3 tags be made the default rather than a hidden option, so that if you're voting to close/delete willy nilly in random tags, you explicitly chose to do that.

Answer 5

This is both a technical and a social problem.

It is not a new problem. You wrote at the time:

20K users don't need to wait, but... In most cases, they probably should. Hopefully, they've learned a bit of discretion in the time it took them to amass that.

As if.

On the technical side, giving 20k-rep users the ability to delete questions instantly gives them the impression that this is something normal. There are actually very few questions that really need to be deleted instantly, and all of them can be dealt with by any user with 15 rep by flagging. Flag as spam and offensive flags if applicable. If not, flag as “other” and make a case to a moderator that the question is so bad it must be removed from the site.

Socially, users need to be educated to the difference between a question whose presence actively hurts the site, and a question whose presence merely dilutes other, more worthy content. A horrible, unanswerable, irremediable question does not seriously hurt the site when it already has a score of -22. It can safely wait 48 hours for deletion.

(The questions that hurt the site the most are the ones that are bad, but highly upvoted, and get framed as SO's best instead of migrated to Quora. But that's another debate that I'm resigned to having lost.)

Users in need of education includes several SO moderators. Being an SO moderator involves cleaning up a lot of crap, that doesn't mean you should throw away the dog with it, even if it's wet and smelly. Even people who post bad questions should be able to read the feedback on them and attempt to improve them. When a question is deleted by a moderator, it's the end, the question disappears into the ether and no one can do anything about it, least of all the asker. It's important in particular not to delete a question (unless it is actively harmful, not if it's merely crap) until the asker has had time to read the feedback on it. Yes, SO mods, this goes for you too.

Fortunately, you are in a unique position to solve both problems.

• Remove the 20k ability to delete questions early, or else make the delete votes take effect only at the end of the 48-hour period after closure. The delete votes should probably be canceled (with the opportunity to re-cast them) if the post is edited: while this is open to abuse, it is better than the alternative which is to let improved posts be deleted. Possibly the delete votes should only be canceled if there is at least one reopen vote, but waiting for 5 votes, even with the reopen queues, is asking too much.
• Educate the moderators to make the difference between clearly unsalvageable questions and the rest, instead of immediately deleting any bad question.

---

A large part of the problem is that when a question is deleted, it is not visible to its asker. If people could see their deleted questions, they would be able to:

• know what happened to the question, that it didn't vanish due to a technical glitch;
• read and understand the close reason;
• read the feedback provided to them in comments;
• be notified of comments;
• edit the question and flag for undeletion and reopening;
• copy the text and ask a new, improved question;
• copy the text and ask in a more appropriate venue; …

Answer 6

We get fairly regular complaints - via support emails, here on MSO, even on Twitter - about questions that've just disappeared with the asker left none the wiser. I don't have much sympathy for folks who ask a question and then leave for the weekend

I do.

No matter how bad or trashy or whatever your question was, if your question is deleted, you deserve to know about it. It shouldn't simply vanish into the aether. At least then you can review the question and see the reason why it was originally closed, including any comments that may have been added. Because without that feedback, nobody can improve.

Answer 7

I vote for don't change a thing, delete at will. Votes don't matter. Delete delete delete. Do not leave the broken window on the site for any period of time. Get rid of it.

Change what's visible to the asker. You should be able to see your own deleted content in your profile. Fix it, if you are so bothered, and flag it.

Edit: To be clear, I am not arguing against users exercising discretion, I am arguing against eliminating the instant delete feature for privileged users.

Answer 8

I would go for the "delayed deletion" idea, where votes can be cast right away, but the deletion doesn't happen for 48 hours. If the question gets edited then it can go into a queue for undelete votes.

If we have the delay, then we should be able to cast undelete votes for the question in that 48 hour timeframe. You might also look into delaying deletion if there are reopen votes cast for the question.

Answer 9

My issue with delayed deletion is that >20k users are often the first to respond to questions that are spam or otherwise utter rubbish (like the one Shog9 links to above). While such questions will be eventually handled by moderators, they shouldn't remain on the site any longer than they have to.

Instead, what if the privilege to vote to delete early for 20k users applies only for questions with a negative score (similar to voting to delete answers?) This would let 20k users handle the spam or rubbish posts, which almost always have a negative score by the time they are closed, and questions that have had some time to be fixed but haven't can still be deleted after two days have passed.

One could alternatively set the threshold to a score of -2 or lower, which would prevent a user from deleting a question at score 0 by hitting downvote before hitting delete.

Answer 10

I'm in favor of both showing a user their deleted posts in their profile, and allowing users to cast a delete vote that will vote to delete the question in X days.

Users won't learn what they did wrong if we delete their content before they have a chance to see that it got closed and why it was closed.

A closed question is not going to harm the site if it stays around for a few days before getting deleted, and it may even be beneficial since it will educate other users who come across the post about what not to do.

From what I've read on meta, most users vote to delete when they vote to close because its easier than remembering to come back later to cast the delete vote. A way for users to say "Delete this in X days assuming no substantial edits are made to the post" would likely fix this problem.

In addition, I've seen many meta posts about Question Bans, where users don't understand why they've been banned because their profile doesn't show any really terrible post scores, and I'm sure you get many support tickets about those too. The usual response is to undelete the user's deleted posts and tell them to go "fix them up", but by having the deleted posts already listed, users can already see the most likely cause of their ban, and they can "fix" their posts without getting a moderator involved to undelete the posts.

Answer 11

My suggestion is as follows:

• Allow high reputation users to normally vote to delete a post
• Do not delete the post (just yet).
• If it has been more than N(=24?) hours, and the question was not edited/received sufficient reopen votes, delete it.

However, this requires a shift in the thinking of closed questions as well. Make them more visible and more easily reopenable once they get edited. Because right now, even with the help of the [cv-ring] reopening a question is extremely difficult.

Answer 12

This is especially a problem with "Subjective Questions." Not all subjective questions are bad, or not allowed on this site. Most questions will have some subjectivity, but the problem is if the question is closed and deleted immediately, there is no option for healthy discussion as to whether the question is subjective enough to close, or how it could be improved.

Answer 13

I have no data, but I see this far more with what looks like single-handed mod deletion (which is actually just response too flafs) than with the build-up of community votes. That's in c++, at least.

Answer 14

Aytime a redeemable question is deleted before it's had a reasonable chance at being edited and fixed, that's genuinely bad, and we should strive to have that happen as little as possible.

I oppose any change that makes how the system of close/delete votes work more complicated and less transparent. We already have tremendous communication problems surrounding these issues. Do we really want to add layers of "delete votes take effect after X hours, unless the Q has 1 reopen vote within the first Y hours, blah blah blah"? It's already complicated enough that it necessitated an entire blog post explaining the life cycle of a question, and still most people don't fully understand.

Efforts to minimize unwarranted deletions should add as little friction to the system as possible. IMHO we should do only the following two things:

1. Show deleted content (or at least just questions) to everyone, if you wrote it. (For the love of God, why has this not happened yet?!)

2. For all users, when they pass 10k rep, the first N delete votes trigger a short alert:

   Please be careful not to delete content prematurely! Is this post salvageable? Has enough time passed that the author has had a chance to edit and improve it? For more detailed information on how to use your delete votes responsibly, see here [link]. (This message will appear N-1 more times.)

Personally, I think N = 5 is sufficient, and you could roll it out such that everyone above 10k rep sees it for 5 delete votes, and then only to people just crossing the 10k threshold.

My rationale here is: who's behavior is going to be easier to nudge? It might be mean to say this, but I think that the vast majority of people writing questions that end up being deleted are unwilling/unable to provide good questions. Some are, but the numbers are against us. On the other hand, I think the overwhelming majority of 10k+ rep SO users have already shown that they have an understanding of how the site works, and a desire to contribute good content. Some 10k+ users are bad apples, but I think the proportions are reversed compared to the askers of deleted question.

So nudge where you're likely to get the most bang for your buck: the communities' "regular" moderators.