16

I recently started filtering to duplicates in the close votes queue and I've noticed that every review will always have a Duplicate tab allowing you to tab between the question and proposed duplicate.

Whenever presented with an audit, I know immediately because the duplicate tab disappears.

Question/Duplicate tabs

Upon searching for information regarding audit filtering I found Audits bug in the filtered review queue which told me that audits are not filtered but the tags are faked such that a user could not just watch the tags and only pay attention if their filtered tags do not show up, this doesn't help with this situation though.

As far as I can tell, you can currently robo-review the close votes queue indefinitely without ever reading a question:

enter image description here

Assuming I am not mistaken, should something be done about this?

Related (outdated?) but not duplicate: There's no review audits for duplicate

Improve this question
10
  • It would be rather hard to concoct audits for duplicates, if that's what you're asking for.
    – Undo
    Dec 17, 2013 at 23:27
  • @UndotheSnowman, you're right, since there are no overly obvious duplicates (since any duplicate has a max of 5 duplicate votes), and there certainly aren't any undeniably unique questions. I'm not particularly asking for anything, more of an observation. Is it a problem that people can "boost" their close-vote reviews? Perhaps. Is it a problem we care about solving? Perhaps not.
    – OGHaza
    Dec 18, 2013 at 9:31
  • 2
    Come on! This is how I spot audits and I'm not a robo reviewer. The deal is that a person who looks at ANYTHING other than whether the Close button has not lit up yet isn't robo reviewing. Is there any evidence that once a question gets in the close queue a bunch of badge grinders all mark it as a dupe even when it's not?
    – Kate Gregory
    Dec 18, 2013 at 16:18
  • 1
    That is NOT a freehand circle! That is a Paint spraycan circle! Not acceptable.
    – James Webster
    Dec 18, 2013 at 16:24
  • @KateGregory, I'm probably(/almost certainly) being slow but I honestly can't tell if you're being ironic. -JamesWebster, if it helps my case it did take about 10 freehand spray can circles to achieve the correct amount of emphasis.
    – OGHaza
    Dec 18, 2013 at 16:31
  • @OGHaza not ironic. I review only dupes, spot audits this way, and don't want it taken from me even though I am not a robo reviewer. I don't believe making it easy to spot audits hurts, since robo reviewers don't spot the ones we have now.
    – Kate Gregory
    Dec 18, 2013 at 16:57
  • @KateGregory then in regard to "a person who looks at ANYTHING other than whether the Close button has not lit up yet isn't robo reviewing", if from now on I use my 40 reviews a day to repeatedly 'Leave Open', and 'Skip' any time the duplicate tab isn't present, I'm not a robo-reviewer?
    – OGHaza
    Dec 18, 2013 at 17:00
  • 1
    @OGHaza the thing is, there is no evidence of such people existing. People who mindlessly click the same thing all the time but manage to spot the easy-to-spot audits. Nobody cares that the audits are easy to spot. Adding these audits has reduced robo reviewing dramatically. You can propose that someone would robo review and spot audits, you can even threaten to do so yourself, but there's nothing to support a theory that this actually happens. Therefore, why make them harder to spot?
    – Kate Gregory
    Dec 18, 2013 at 17:02
  • And that is certainly a completely fair argument. There is no way for me to tell who is pressing "Leave Open" so I wouldn't know whether or not there was any evidence but I'm sure the powers above (perhaps yourself) do know, and I don't dispute it.
    – OGHaza
    Dec 18, 2013 at 17:06
  • I also review only duplicates, largely because I feel those are the most valuable close items to review (it's far more valuable to a future searcher to add a useful pointer to a question, than to close a question). I think the audits are sufficient as they stand; yes they're obvious, but you have to think for at least a half second, which is enough to stop robo reviewers.
    – Joe
    Dec 18, 2013 at 17:32

3 Answers 3

Reset to default
6

It is a pity that SE dev team doesn't invest effort into these audits because per my understanding, designing reliable and realistic audits for dupe closure is not really difficult.

For example, an audit where user is expected to Close to pass can simulate a fairly routine case of a migrated cross-post for an otherwise good question:

  1. Take any "known good" audit item for close queue - it will fake a "master" question
  2. For a question to be closed, take the very same audit item and fake it a little to make it look like a typical migrated cross-post: shift posting date for half an hour and attach fake migration notice

...that's all! This way leaves no reason for a responsible reviewer to vote Leave Open.

Similarly, for audits requiring Leave Open to pass, another real-life scenario makes a good fit: when an inexperienced low rep user, maybe just testing "how things work", blindly flags an otherwise good question as a dupe of a crappy one:

  1. Take any "known bad" audit item for close queue - it will fake a "master" question
  2. For a question to be closed, take any "known good" audit item

...voila, no reasonable reviewer should vote Close in cases like that.

The only limitation of above is that of audits composition/selection without a “human factor”, but that's another (sad) story.

Improve this answer
2

It would be hard to fake one of these questions. I suppose it could be done by just showing a mediocre post similar to the "this should be left open" post to vote on the way the low quality or opinion based queue audit occurs. However, I don't think that the duplicate filter is being a haven for robo reviewers. As seen from this statistic (copied from the all powerful Shog)

You can see that the duplicate closures are not a majority of the closures, not even second place. Okay, so they are third (of 5 though). But still, it is also very obvious if a question is closed as a duplicate and they do not match. That should be overturned in the reopen queue rather quickly (especially since the reopen queue is often empty it should get reviewed right away).

All in all, I don't think this is a grave concern and it just doesn't seem like the data would support that either.

Improve this answer
1
  • 1
    In regard to it being very obvious if a post has been incorrectly marked as a duplicate - true, but I'd be marking them as "Looks Good" anyway because it takes less effort (just like all the robo-reviewers in suggested edits) - which would also invalidate any indication from that chart (not that I'm suggesting there is mass abuse). All I'm saying is that if I felt so inclined, I could go into the review queue tomorrow and perform all my reviews without reading any questions (except for those marked as audits - although can I just skip those?) and the system would not pick up on it.
    – OGHaza
    Dec 18, 2013 at 1:28
2

If I really wanted to make fake dupes I would just choose two highly upvoted questions (since these are likely to be correctly tagged and titled) that have no tags in common and were not asked in the same month. Statistically these will not be dupes. Done.

But take a look at how little effort goes into making fake bad edits. I have suggested far better ways to make fakes that actually test reviewers skills, but there's no interest in it because if you read anything other than the big action buttons (and that includes noticing the absence of a Duplicate tab) then you're not a robo reviewer.

Improve this answer
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .