What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

I'm talking about

Improving the Quality of Reviews: Project Honeypot

I find it too easy to spot which posts are fake ones from the horrible quality of the edit, including adding things like "eating" randomly. However, this seems to be the intention since "they are created so that they are either objectively wrong".

On the other side, when there's an edit on a question/answer, this element to be edited shows a (1) besides edit (1). This is not the case with the honeypot posts, they don't show an (1) besides the edit, so it could be really easy, when in doubt, to check if it's a honeypot.

Furthermore, I think this could easily be automated to vote thousands of times and get the two badges real-fast.

Disclaimer: I don't have any intention to exploit the system in the way exposed above. However, someone might, so I'm just pointing where I think there're some flaws hoping that someone fixes them.

TL;DR

Problem: The edit anchor for honeypot questions/answers shows whether the review is a honeypot or not.

Solution: display edit(1) instead of edit on the original post of the honeypot if the user being tested access it.

share|improve this question
13  
If you've gone to the question itself for more detail then you have already passed the test –  Richard Tingle Dec 13 '13 at 15:06
    
I take care to properly choose what to select, sometimes going to the question, but this time I just went to check whether the question itself existed or not (since it clear that it was a honeypot). It existed, but found the bug I described. A script could easily go to the question itself to see if there was any text besides the edit or not. –  Francisco Presencia Dec 13 '13 at 15:08
3  
changing edit to edit(1) seems a trivial enough fix, given that it is script-exploitable. –  Ben Barden Dec 13 '13 at 15:08
5  
@BenBarden He's talking about on the post itself, not on the review page though. That would mean that, for example, if someone else was looking at the post and didn't have 2k rep they'd think they couldn't edit the post (due to a pending review) or that another user might not try to edit the post knowing that there's a pending review. They could also notice the pending review and click "edit", with intentions of voting on the pending edit, and be very confused to find none. –  Servy Dec 13 '13 at 15:11
    
@Servy Ah. I had misunderstood. Yeah, that's not necessary. –  Ben Barden Dec 13 '13 at 15:12
    
Can I ask why? It's still script-exploitable and I don't think it'd be so hard to implement –  Francisco Presencia Dec 13 '13 at 15:13
2  
@FranciscoPresencia Well, first off, it wouldn't be trivial to implement, and second, it would have significant negative repercussions for non-abusive reviewers, and so shouldn't be implemented. –  Servy Dec 13 '13 at 15:14
    
@Servy that's why I added the conditional on the proposed solution, display the (1) only if the visitor is the user being tested –  Francisco Presencia Dec 13 '13 at 15:14
2  
@FranciscoPresencia Then a script can exploit it by looking at the page from the point of view of another user; still exploitable. Where does it end? –  Servy Dec 13 '13 at 15:14
    
@Servy okay. I see now it's not so trivial. Still it is a small exploitable bug. –  Francisco Presencia Dec 13 '13 at 15:15
1  
@FranciscoPresencia A bug implies it's not intentional. In my eyes it's by design. –  Servy Dec 13 '13 at 15:16
1  
@BenBarden there being exploits arising from seeing the question itself is the least thing that is to worry about people able to script their way around. Hint: check what the server already sends –  Jan Dvorak Dec 13 '13 at 15:17
    
I understand now, @Servy , sorry for being so insistent and thank you for explaining it. –  Francisco Presencia Dec 13 '13 at 15:19
3  
The point is that robo-reviewers don't pay attention. If you are paying enough attention to go visit the question itself you are no longer a roboreviewer. I don't buy the argument that roboreviewers will use a script to check for that edit(1) link. I'm sure the developers can in that case detect the pattern of requests from such a cheat instead. –  Martijn Pieters Dec 13 '13 at 15:20
    
@MartijnPieters I think that waiting is the least the autoreview script can do –  Jan Dvorak Dec 13 '13 at 15:21
show 4 more comments

1 Answer 1

up vote 11 down vote accepted

There are a number of ways of identifying audit questions, some of them, sadly, aren't based on actually evaluating the quality of the post.

That said, the point of the audits is to make sure that people are at least paying a bit of attention. Noticing details like this indicate that. The audits aren't designed to go into more depth than that.

There are also a number of cases (this among them) in which you can cause problems for non-abusive reviewers by trying to make audits harder to spot. Because of this, a number of identifying factors of audits are intentionally not fixed simply because fixing them would cause more problems than leaving them.

The audits have a successful track record as a reasonable deterrent of people clicking through reviews without paying any attention. If a user is truly interested in making improper reviews at all costs, and are willing to put a lot of time and effort into getting past the audits, they're likely to succeed. Fortunately the review system is capable of dealing with a small percentage of abusive reviwers, by simply requiring enough review actions per item. By having audits successfully deterring most abusive reviewers it allows the other remaining defensive mechanisms to deal with what's left.

share|improve this answer
    
After your comments and slowly reading this full answer (not native), I understand what you mean now. I'll mark it as correct. –  Francisco Presencia Dec 13 '13 at 15:18
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .