12

This question showed up in the close review queue for C++:

enter image description here

I was leaving a comment as to the inappropriateness of the c++ tag for this question, when I went to the question and found this:

enter image description here

The question does not have the c++ tag, and has never been revised. Surely any ninja revision would be over a week old, far too old to influence the close queue filtering.

Diamond mod Oded was able to confirm that the honeypot messes with tags sometimes.

That's bad, because it messes with the quality of the question and you can no longer make a judgement about the user's reaction when you show them an alternate version of the question. In this case I was leaving a negative comment using the review queue, and it did show up on the real question! Of course I deleted it because I realized that it wasn't deserved, but if I was only paying enough attention to decide the question was ok (I do know enough Java for that), then I might not have visited the question.

If you're going to make a fake version of a question to show the reviewer, then review actions including commenting need to be disassociated from the real question. In order to not discard useful comments, you may need to drop the facade if the reviewer tries to leave a comment, and send them to the real question to decide whether it still applies.

Related but different: Close vote review queue tag filter broken? (different because there the system recognized that the tag wasn't present, and passed it through the filter anyway)

Edit: Neither of the linked questions, or answers, address comments left on faked answers via the review audit system. There's a solitary comment that mentions the issue, but nothing is resolved, since it isn't answered. This bug report is separate from those two.

|improve this question
  • 1
    First time you got a review audit? – Oded Mar 2 '14 at 20:08
  • @Oded: That's what I meant by honeypot. When I dealt with the review, though, it didn't say it was an audit. Please un-close this, you have a real bug. – Ben Voigt Mar 2 '14 at 20:11
  • @Ben - we do sometimes add tags on review audits, in particular if the chosen audit question comes up on a filtered queue and doesn't have the requisite tag. – Oded Mar 2 '14 at 20:13
  • @Oded: Fine, I'm going to edit this question a bit to highlight what I feel is a real misfeature associated with doing this in review audit. – Ben Voigt Mar 2 '14 at 20:14
  • 1
    Confused the hell out of me the first time I saw this, too. I spent several minutes looking over the post's history and other details. – jscs Mar 2 '14 at 20:17
  • @hichris123 The commenting part seems like the most serious issue here. – Mark Amery Mar 2 '14 at 20:24
  • @hichris123: No, that's a duplicate of the question I linked at the end of mine as "related but different". It has nothing to do with pretending the question has a tag it does not and never did. It also has nothing to do with leaving comments on audit questions – Ben Voigt Mar 2 '14 at 20:25
  • @BenVoigt Look at m0sa's answer: While reviewing the faked posts appears like it matches your filters:. Except for the commenting part... but it explains why, just for reference. – hichris123 Mar 2 '14 at 20:26
  • @hichris123: Actually: David's comment "I was so confused by the tags that I went to the real question and almost left a snarky comment about learning to read the tag wiki excerpts when tagging a question. That could have even embarrassing :) " over there appears to be a duplicate. But it wasn't ever properly addressed. – Ben Voigt Mar 2 '14 at 20:29
  • 2
    @BenVoigt Too bad we can't mark as duplicates of comments. :P Yeah, you're right, not really a dupe. – hichris123 Mar 2 '14 at 20:29
  • 2
    Thanks to all close voters for simply piling on without taking a minute to understand the issue and that neither of the linker questions resolve the issue of comments left on faked questions during audit. In fact, I would say if this were an audit, all five of you failed. – Ben Voigt Mar 3 '14 at 6:13
4

This is not a bug.

It is a test. To see if you are paying attention... which you were.

We need these, as not everyone is diligent in the pursuit of badges - robo reviewers have been and still are a problem that audits are there to help with.

We will sometimes add a tag to a review audit on a filtered queue - to ensure that the tag is there (and not tip our hands too early that it is an audit), as selection of a good audit question doesn't guarantee it will fit the filter.

|improve this answer
  • When I dealt with the review (with Skip), though, it didn't say it was an audit. Please un-close this, you have a real bug. – Ben Voigt Mar 2 '14 at 20:12
  • @BenVoigt Skip does nothing with the review, so no message is ever shown. – hichris123 Mar 2 '14 at 20:12
  • 7
    Also, it allowed me to leave a permanent comment (which I've since deleted, when it became apparent the system lied to me) using the review interface. That's not a good behavior for review audits! – Ben Voigt Mar 2 '14 at 20:12
  • 1
    I'd say that being able to add a comment when on the queue is a bug. If I recall correctly, comment adding when on the queue should be disabled. – Oded Mar 2 '14 at 20:35
  • @Oded they were turned off back in October, but re-enabled – psubsee2003 Mar 2 '14 at 20:37
  • @psubsee2003 - huh. Wondering now what the consensus is... should they be on or off? – Oded Mar 2 '14 at 20:38
  • @Oded IIRC from that time, the community's opinion was they should be on, for what that's worth. Personally, I think they ability needs to be there. – psubsee2003 Mar 2 '14 at 20:40
  • 1
    @psubsee2003: I suspect there may not be a clear consensus. The problem is that, ideally, they should be on except for audits... but that has some obvious issues of its own. – Ilmari Karonen Mar 2 '14 at 20:44
  • 4
    @IlmariKaronen or comments during audits don't pass through to the original post. Block that from happening and you solve the problem – psubsee2003 Mar 2 '14 at 20:45
  • @psubsee2003 - or possibly, comment actions on audits will pass a known bad audit but not a known good audit. – Oded Mar 2 '14 at 20:47
  • @Oded assuming that it is possible to have a known good audit with the tags modified to fit my filter, that's going to trigger a failure for a case where the reviewer is trying to comment on the tag issue such as the OP here. – psubsee2003 Mar 2 '14 at 20:49
  • @psubsee2003 - yeah. Wasn't thinking through all the cases. – Oded Mar 2 '14 at 20:51
  • @psubsee2003: "or comments during audits don't pass through to the original post." That's pretty much guaranteed to sooner or later piss off someone who spends ten minutes composing a really good comment, only to be told that it was an audit. But even so, I still think it might be the best solution so far. – Ilmari Karonen Mar 2 '14 at 20:54
  • @Oded ok - have another thought - disable commenting in the close review queue and provide the optional ability to leave a comment when closing - maybe a field in the CV dialog to leave a comment. So you don't run into this issue and still provide the needed comment functionality in the close vote queue to leave a comment as to why you are closing the post. – psubsee2003 Mar 2 '14 at 20:59
  • 1
    @IlmariKaronen: I believe my question already points that out: "review actions including commenting need to be disassociated from the real question. In order to not discard useful comments, you may need to drop the facade if the reviewer tries to leave a comment, and send them to the real question to decide whether it still applies." – Ben Voigt Mar 2 '14 at 21:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .