17

"Very low quality" flags are quite ambiguous as to their purpose, without a whole lot of agreement. More relevantly, they're marked helpful by an edit—any edit, even correcting a typo‐often before a moderator gets around to declining them.

Occasionally, these show up as review audits, as they did in First Answers Review Suspension because of plagiarized answers*. This is unhelpful, unless we seriously expect people to be able to catch, say, plagiarism (which would be a huge waste of reviewer time—a far better solution would be for the company to integrate with a plagiarism-detection system and flag this stuff automatically).

Further, it frustrates reviewers trying to do the right thing and drives them away from reviewing, when we desperately need more reviewers.

Deleted answers should only be used as audits if there's a helpful "Spam" or "Not an answer" flag on them, indicating (in most situations) that a moderator believed that the answer was NAA†. "Very low quality" flags are problematic, as these flags are frequently incorrectly marked helpful by edits or used for reasons other than "this answer needs to be deleted."

* In this case, the VLQ flag was cast due to moderators using the flags to downvote a series of plagiarized posts without the votes being reversed. We will probably stop doing that now...but it'd be nice if we could...
† There are edge cases here, such as an answer that was NAA, but has been edited since. Ideally, it would only select answers with no edits since the flag was cast.

Improve this question
10
  • 1
    Of the many bad audits that surface on MSO do you have a rough estimate of what percentage comes from mod deleted posts?
    – bad_coder
    Jun 23, 2022 at 2:35
  • @bad_coder Nope, but also not everything surfaces on MSO. I haven't tried to more comprehensively survey bad audits, especially since the tooling doesn't make that super easy.
    – Ryan M
    Jun 23, 2022 at 2:38
  • 3
    Last I recalled, only deleted posts with helpful flags are eligible to be review audits. Both of the posts in the referenced question had helpful VLQ flags which is what made them eligible.
    – animuson StaffMod
    Jun 23, 2022 at 2:39
  • For reference disputed-review-audits tag.
    – bad_coder
    Jun 23, 2022 at 2:41
  • 2
    @animuson Well, that's something at least...but that would be true of a great many posts deleted for plagiarism as well, as they'd often be custom flagged for that. In this case, they were my VLQ flags, as I flag recent plagiarized posts as VLQ to get the auto-downvote. I suppose I probably shouldn't do that, given this issue. Perhaps just using NAA flags for audits would be best?
    – Ryan M
    Jun 23, 2022 at 2:45
  • 2
    (Re: how many bad audits are plagiarism related) Since May 1, 2022, there are 9 (undeleted) disputed audits on MSO with 3 being plagiarism related. The one linked in this question and two others Q1, Q2. The frequency of plagiarism handling has increased recently so that's likely a transient increase, it's hard to say how those numbers will end up leveling out over time.
    – Henry Ecker
    Jun 23, 2022 at 2:47
  • 2
    According to the FAQ it's just spam, NAA, and VLQ flags that work. Which sounds about right. If you think VLQ shouldn't be invluded in that, maybe you could refocus your post to emphasize more on why not than just potentially problematic? Trying to use a flag for a purpose it wasn't designed for doesn't seem like a great reason yo change the functionality.
    – animuson StaffMod
    Jun 23, 2022 at 2:50
  • @animuson Yeah, that's reasonable. I've edited the post to focus on that.
    – Ryan M
    Jun 23, 2022 at 3:05
  • It might also be possible that helpful automatic "low quality (auto)" flags cause review audits as well. That flag type is disabled on SO, though.
    – Sonic the Anonymous Hedgehog
    Jun 23, 2022 at 3:24
  • 1
    The problem with the algorithm for review audits is that there are (false) assumptions that posts upvoted >= 5 times, must be good posts, while anyone around for any length of time, knows that upvote count hardly correlates with quality.
    – amWhy
    Jun 28, 2022 at 16:47

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .