10

I noticed something today on Software Engineering when dealing with an autogenerated moderator flag regarding "More than 10 answers posted to this question in the past 7 days". Mainly - this flag isn't helpful. Sometimes, it raises a question with a lot of answers where a good chunk of those answers are very low quality and can be deleted. However, often, I end up just dismissing this flag. So I'd like to see some general quality-of-life improvements related to how many answers a question gets.

First, I'd like to see some kind of prompt before this flag can possibly be raised. At some point, the answer box for the question goes away and is replaced by a button that says "Answer This Question". Clicking on this button brings up a prompt (and not a very pretty one) that indicates that a question already has a lot of answers and that new answers should add value to readers. This should happen before the moderator flag is raised. If 10 answers in 7 days causes a flag, the prompt should start appearing before 10 answers are posted.

Second, raise moderator flags only when it's likely answers are low quality. Some examples would be: a large number of answers with several negatively scored, a number of community-deleted answers, answers that keep getting posted even with the prompt in place, lots of answers in a very short timespan (hopefully the prompt slows this down).

Something else to consider - it would be nice if all of these thresholds were able to be configured on a per-site basis. I think we can come up with some good numbers that work very well across the board, but some communities may need some tweaks.

I don't know fully what this would look like, but ideally, it should inform users of what the expected behavior is (adding only new and relevant content) and not adding unnecessary moderator flags to the queue. Perhaps even a review queue can be leveraged to put questions like this before the community and votes (down votes, delete votes, etc) can trigger one of the other moderator flags I mentioned above.

Improve this question
30
  • I would love to see that prompt. Most of the time when we get to that point it's because of people jumping on the bandwagon to give yet another answer that's saying the same thing as the top answer but devoting only a quarter of the effort.
    – doppelgreener
    Jan 8, 2018 at 18:53
  • @doppelgreener It looks like the prompt appears after either 14 or 15 answers have been posted. I'm guessing it should be much lower on many sites. Maybe after 5-7 answers, but I'm not sure.
    – Thomas Owens
    Jan 8, 2018 at 18:54
  • 8
    I agree. On IPS, I see these flags daily and wonder, "Why are you telling me about them before ever telling the people creating the answers?" It's putting a lot of work in the moderator's laps when, often, there's not much to do... at least, not by a moderator... if this is concerning, could we not let the users review them... perhaps some of these could get shunted into a review queue?
    – Catija
    Jan 8, 2018 at 18:55
  • @Thomas Personal experience from memory on Role-playing Games is that in most circumstances we've covered all the basic possibilities after 5 answers, and answers after that point are repeating things others are saying, perhaps combining them in new ways, and their usefulness is predicated on them bringing something significantly better than what has already come before.
    – doppelgreener
    Jan 8, 2018 at 18:56
  • @doppelgreener Thanks. Honestly, just thinking about it, it seems like allowing 5 answers and then prompting on the 6th would be nice for SESE, too. But 2 sites is probably insufficient to make a good decision.
    – Thomas Owens
    Jan 8, 2018 at 18:57
  • 1
    @Catija A large portion of these answers are going into Late Answers. The main problem is that too many reviewers say, "oh, sure, that answer isn't technically incorrect, have an upvote" when someone just repeats what 5 other answers have said, but worse, so it makes the issue worse, rather than better, because now these bad answers get upvotes.
    – Servy
    Jan 8, 2018 at 19:03
  • 2
    @Servy No, they're not. Not on IPS. We hit the HNQ list and that question will get 10+ answers in a day... nothing "late" about it.
    – Catija
    Jan 8, 2018 at 19:04
  • @Catija I said "A large portion", not "all". The point is that when these posts are sent to a review queue they do more harm than good, sending all of them, rather than just some, wouldn't make the problem better, it'd just make it worse.
    – Servy
    Jan 8, 2018 at 19:05
  • 1
    @Servy On SESE, the problem is not late answers. SESE is in a similar boat to IPS. The answers come in very quickly (within a week) and trip the moderator flag before tripping user-facing warnings or queues. Moderators shouldn't be summoned as quickly as we are to these problems.
    – Thomas Owens
    Jan 8, 2018 at 19:06
  • @ThomasOwens My point was simply that sending these to a review queue wouldn't solve the problem, because many, not all, but many, of these types of problematic answers are already sent to review queue, and when they are, the reviews often don't do a good job of solving the problem, so we shouldn't consider, "send even more of these types of posts to a review queue" to be a good solution.
    – Servy
    Jan 8, 2018 at 19:09
  • 4
    @Servy The site's only 6 months old... there's very little going into that queue at all at this point. Fewer than 125 items in that list and 144 questions with 10+ answers... 40 of which have 15+ and those numbers don't include deleted answers. Please, don't tell me that this isn't a problem on my site.
    – Catija
    Jan 8, 2018 at 19:10
  • 1
    @Servy Many of these problematic answers are not sent to a review queue on SESE, though. We don't have a problem with late answers. We have a problem with a large number of low quality answers very, very quickly and there isn't a review queue for that. Moderator flags are being raised without the posting user being informed of correct behavior or the community being made aware of a problem before the moderator flag is raised.
    – Thomas Owens
    Jan 8, 2018 at 19:10
  • @Catija How is the number of posts going into that queue at the moment relevant? And where did I say that this wasn't a problem? I said that review queues aren't a good solution (which, pretty much by definition, means that I'm not disputing that there's a problem to solve).
    – Servy
    Jan 8, 2018 at 19:13
  • 4
    @Servy You repeated more than once that "a large portion of these answers are going into late answers". Showing you that there are fewer than 125 items ever reviewed in that queue is an attempt to show you that the data doesn't support that assertion... very few of the answers this question is attempting to address are going into that queue.
    – Catija
    Jan 8, 2018 at 19:16
  • 2
    @Helmar Questions are not auto-protected after 10+ answers in 7 days. All that happens is a moderator flag is raised. After 14 or 15 answers are posted, users posting a new answer have to click through a prompt to add a new one. Protection must be done manually.
    – Thomas Owens
    Jan 8, 2018 at 19:31

2 Answers 2

Reset to default
5
+50

As Laurel notes, this essentially already exists - but instead of flags, the automated response is Protected status. There are both time-based and all-time thresholds for that, which would appear to cover most problematic situations save for one: a question that has attracted a lot of answers that are low-quality but which the system lacks the necessary heuristics to identify as such.

...This is why there are automated flags triggered purely on answer quantity and velocity: those are themselves strong indicators of a question that is quite possibly encouraging problematic answers.

Of course, this depends heavily on the community and the topic, which is why the thresholds are configurable per-site for all of the following:

  • number of answers per question before the system starts warning new answerers.
  • number of answers from new users per day before the question is auto-protected
  • number of answers from new users deleted (ever) before the question is auto-protected
  • number of answers (ever) before the question is flagged
  • number of answers (per time period - also configurable) before the question is flagged

Each of these was adjusted for sites that had unusual patterns when the functionality was originally rolled out, but that doesn't mean we got it right for all-time - if you're encountering too many flags that require no action, or believe one of the other systems should be kicking in sooner... Let us know and we'll alter the thresholds for the site as needed.

Improve this answer
6
  • wonder what would be a preferable way for moderators to communicate desire to adjust flag thresholds, per-site meta or TL or something else?
    – gnat
    Jan 11, 2018 at 9:57
  • 1
    Could you provide these numbers for Software Engineering? I think the warning happens at 15 and the number of answers per time period is 7 in 24 hours. The middle three I'm not sure of.
    – Thomas Owens
    Jan 11, 2018 at 9:57
  • 1
    @gnat On a per-site Meta. I'm already looking at historical question data for Software Engineering and hope to post something by the end of the weekend for everyone to review. I also gave our mods a heads up.
    – Thomas Owens
    Jan 11, 2018 at 9:58
  • Shog, while I have your attention - would you mind taking a look at this recent bug with close banner (quite an annoying issue)? If you prefer to wait until it gets a bounty, no problem I'll do :)
    – gnat
    Jan 11, 2018 at 21:49
  • 1
    Already looking at it, @gnat; I wrote the spec for that 4 years ago, but, uh... Did a lousy job of it. So not only do I not know what regressed, I also am not entirely sure how it was supposed to work to begin with. Will post something later.
    – Shog9
    Jan 11, 2018 at 21:53
  • 1
    Ah! I see what happened; I was looking at the meta instead of main. Chrome autocomplete got me again... @ThomasOwens, the SESE values are (in order of bullets): 15, 3, 3, 30, 10 (7)
    – Shog9
    Jan 11, 2018 at 21:56
4

Some of the features you are requesting already exist.

I'd like to see some kind of prompt before this flag can possibly be raised.

There are two customizations that can be made on a per-site basis to do this:

  • Change x in the "This question has more than x answers already" popup. According to the post announcing the feature, the default value for x is 30. The threshold is already customized on Software Engineering (at 15 answers), so there's no reason it can't be changed again. (In addition to the sites listed in the linked post, TWP is also customized in this respect, although it is only an unintended consequence of a different change.)

  • Change x in the moderator flag "More than x answers posted to this question in the past 7 days". (The flag triggered by 30 total answers can also be changed.) The relevant meta post can be found here. Worldbuilding is the one site I know of where this number has been changed.

As the comments mention, auto-protection is a helpful feature. There appears to be some customization that can be done with this too on a site-by-site basis. From the FAQ:

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .