What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

Expanding the criteria for auto-protection has been discussed in the past: Should we automatically protect all questions with more than N answers?

But I think that discussion addresses the wrong problem:

  • Protecting a question with a large number of answers doesn't do anything to fix the problems caused by having a large number of answers. In most cases, the question should still be either cleaned up or closed.

  • Large numbers of answers does not necessarily indicate a honeypot for misguided newbies. While a fair number of these do fall into bikeshed territory, that's not universally true - and blocking outsiders from answering a question just because a lot of "insiders" have already gotten to it doesn't solve anything.

  • The occasional drive-by answer on an old, answered question isn't that much of a problem and we have existing tools for identifying and handling them.

Generally, the existing criteria for auto-protect are sufficient for perpetually-popular posts. However, there is a scenario that isn't addressed by them: a question that sees a sudden, extreme increase in popularity and as a result attracts a large number of non-answers that must then be deleted. The classic case is a post that rises to the top of Reddit for a while, although we've seen similar effects from other sources (and even generated internally from links on our own Hot List or meta posts).

Let me be clear: these 15 minutes of fame aren't inherently bad: a good question that attracts a lot of attention can also attract some really great answers. But there's always that temptation to just participate even when you've nothing of value to contribute - and moderating a sudden influx of such answers can be troublesome, particularly on smaller sites:

  • If no moderator is around, it falls on trusted members of the community to remove non-answers. We're working on distributing the load here, but this will still be labor-intensive on smaller sites.

  • When your first experience on a site is having your post deleted, it doesn't exactly endear you to the community there. Yes, you may well have Learned A Valuable Lesson from it, but we could be a bit more gentle about teaching that.

  • The communities on these sites, knowing full-well how troublesome such answers can be, may try to proactively Protect questions that start to garner a lot of attention - potentially locking out good answers on questions that wouldn't actually be problematic if left to themselves.

That last one in particular concerns me. I think most users use their privilege responsibly, but the notion that "hot" questions need preemptive protection has proved stronger than common sense here - and I don't think we should fight it. Instead, let's...

Expand the existing heuristics to protect questions that are demonstrating problematic tendencies, but haven't yet been proven problematic

As explained in the title, this means protecting questions that've gotten more than N answers (deleted or otherwise) in a 24-hour period from users who've earned less than 10 reputation on the site (excluding any association bonuses). This would be a check made when the answer is posted - so if N is 3, the third answer from a new user would block further answers.

On most sites, N can be fairly large - a threshold of 5 would protect questions like these on Stack Overflow:

On certain sites, we might want to set the threshold significantly lower. A threshold of 4 would protect questions like these on The Workplace:

Coupled with this change, I think it also makes sense to expand the existing Protect privilege to allow users to unprotect questions that were automatically protected, thus allowing them to override the system if it becomes overzealous.

Related:

share|improve this question
    
what threshold is supposed to be set for Programmers.SE? –  gnat Mar 24 at 17:44
    
please consider clarifying whether deleted answers from new users would count towards triggering protection or not –  gnat Mar 24 at 17:54
1  
Clarified, @gnat. And the threshold would be 5 everywhere unless a given site demonstrates a clear need for a different (higher / lower) threshold. As a rule of thumb, sites that have a lower threshold for automatic Community Wiki status on a question would probably want a lower threshold for auto-protect. –  Shog9 Mar 24 at 18:07
    
Shog, don't know if you remember or not, but Programmers have the same CW-threshold (15 answers, half of a typical 30) as Workplace. How strong is this rule of thumb? –  gnat Mar 24 at 18:17
    
I know. That's why I mentioned it... –  Shog9 Mar 24 at 18:23
    
understood, thanks! when can we expect this to roll out? I would be very interested to see how it works (or breaks:) –  gnat Mar 24 at 18:27
1  
I can see the revised heuristics also match some troublesome questions we occasionally see on SO when popular services go down and 1-rep users decide to use the site to post "down for me, too!" answers in questions complaining about it. –  Brad Larson Mar 24 at 18:36
    
Yup. There's a fine line to walk there on SO, @Brad - you want to lock these down when they're just attracting a lot of noise, but not before someone can get an answer in that provides a fix / work-around / explanation if one exists - from what I've seen so far, 5 is pretty safe in that regard. –  Shog9 Mar 24 at 18:43
5  
This is already happening on Math Educators in the 2 days its been public. It's very similar to The Workplace, because every single person on Stack Exchange took math in middle school/high school and has their own opinion. Could we get some of the other features The Workplace has as well? (someone mentioned 15 answers -> CW). –  Brian Rushton Mar 26 at 18:04
    
Mightn't it be safter to wait until the new user answers get a couple of downvotes first? –  ɥʇǝS Mar 28 at 0:45
    
You can also check view velocity for signs of bikeshedding. –  Robert Harvey Apr 2 at 18:07
1  
Outliers are not that difficult to detect: i.stack.imgur.com/4QjST.png –  MichaelT Apr 4 at 14:03
2  
@gnat - both are at 3. –  Oded Apr 8 at 14:15
1  
@gnat - In the original question it was suggested that 15k+ can vote to release an auto-protect as part of this change. I'm curious to know if that went in or not. –  GlenH7 Apr 8 at 14:18
2  
@GlenH7 - it is in. –  Oded Apr 8 at 14:23

3 Answers 3

up vote 15 down vote accepted
+100

This has been implemented pretty much as described in the question and is now deployed across the network.

Default threshold is 5 answers in 24 hours (by new users i.e. users who earned less than 10 reputation on the site, excluding any association bonuses), at the point of which the question gets protected.

The check is run after an answer has been posted on a question - so if an answer by such a user in the last 24 hours has gained an upvote, it will get excluded from the count.

This has been adjusted on several sites:

  • Programmers: 3
  • Workplace: 3
  • English Language Learners: 3
  • Programming Puzzles & Code Golf: 20

Users with the Protect Questions can now unprotect them as well, so such protection does not have to be permanent and unprotecting does not have to involve a moderator.

share|improve this answer
    
Can we also unprotect a question that was protected manually, not automatically? –  Shadow Wizard Apr 8 at 14:37
    
Doesn't that already happen @ShadowWizard? –  Yannis Apr 8 at 14:37
1  
@Sha - yes, going forward. @Yannis - for users that were not mods, it didn`t - only if they were the one that protected in the first place. –  Oded Apr 8 at 14:39
    
@Oded looks like backward too, I was able to unprotect a question that was protected over two years ago. :-) –  Shadow Wizard Apr 8 at 14:42
    
@ShadowWizard - well, yeah. The privilege was changed. We may restrict it in the future (if we see protect/unprotect wars) –  Oded Apr 8 at 14:43
    
@Oded cheers, the "going forward" confused me. I can't see any reason for such wars, but agree we should keep an open eye. –  Shadow Wizard Apr 8 at 14:44
    
I should really grind to 15 krep already... –  Jan Dvorak Apr 8 at 19:55
    
@Oded is past unprotect intended to "block" this feature? I ask because it looks like we've stumbled upon this case recently at Programmers: Why didn't auto-protection kick in in this question? –  gnat May 12 at 17:00
1  
@gnat - a past unprotect shouldn't block it. If it has, then there's a bug. Need to look closer into the post to see what exactly happened. –  Oded May 12 at 17:04

As a 20k+ user on Workplace I spend way too much time moderating and protecting our site from the... fun that is caused by the hot questions list on our site. This nearly always results in lots of opinions, low quality answers, repeat answers, and other noise.

It's hard enough to maintain site quality for more subjective sites without dealing with the deluge of crap which comes in through the trending/hot questions list.

A simple improvement to what you are suggesting would be to make any auto protection automatically be removed after 7 days. This makes the system-generated protection primarily focused on removing noise associated with the "hot questions" list.


Just a note:

  • This is much more important for sites which are prone to "drive by" opinion answers. Some sites are considerably easier for someone to see in the "hot questions" list and go "hey here are my thoughts" than others.
share|improve this answer
    
case in point, about 2 weeks ago, at Water Cooler chat: '@enderland oh, while you're here, could you please protect this question from lemming answerers? "What a hot question! I cant stop myself from answering it..."' –  gnat Mar 24 at 17:42
3  
(for any non 10k+ people, the question @gnat linked has 7 deleted answers) –  enderland Mar 24 at 17:43

Having watched a number of hot questions unfold, I think this suggested change would go a long way to tamping down some of the ... unbridled enthusiasm that those questions can generate.

The auto-protect thresholds you suggested sound about right. I haven't really seen much meaningful content show up after the first N answers roll in.

I also think the idea of expanding community privileges to unprotect a question makes a lot of sense.


Some additional considerations:

  • If you use a higher threshold than the ones you suggested (5 & 4), consider allowing 15k users to be able to protect a question sooner than the current time window. I don't think this will be abused by community members, but I appreciate the concern.

  • Consider having the system auto protect release after some number of days after the surge of popularity has faded. Doing so would better enable the random expert in the field to provide a response.

share|improve this answer
1  
16 days to release auto-protect would likely be safe enough, according to data collected here –  gnat Mar 24 at 17:47
2  
Timed protection is a reasonable solution to this if it becomes too commonplace. –  Shog9 Mar 24 at 18:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .