What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

Possible Duplicate:
The current review system encourages fake reviews; some people upvote everything rather than actually fixing problems
“Revolving door” reviews are getting out of control

Closely related:

I found myself in an interesting situation earlier today. I had just voted to close this post, when a suggested edit to it was made. I reviewed it and decided it was incorrect - a couple changes were helpful, like fixing the spelling of "parameter", but the rest were pointless or introduced grammatical errors.

I voted to reject it, because IMO it made the post worse. To my consternation, it was approved within seconds by other people, apparently grinding away in the edits queue. It seemed clear to me that this was wrong - so I rolled back the edit.

A minute later, the same user made exactly the same suggested edit. Again I voted to reject it and again it was almost instantly approved by other people. I rolled back again, and made my own edit, fixing the mispelling etc. so that hopefully the suggester would be satisfied and leave.

At this point it occurred to me that something had gone seriously wrong with our editing and review process. My previous impression of it as a self-correcting, community-based process had been replaced by an image of a misguided, gamification hamster wheel that nondeterministically introduced gibberish into the system with community approval.

[wanted to post screenshot of hamster wheel from Incredible Machine 3, couldn't find one]

Troubled, I flagged the post for moderator attention, saying: "Our editing process is seriously broken. Please review what is happening here." I figured a moderator would see the edit history and sympathize with my point of view. Unfortunately the responding mod declined the flag, saying: "Please don't rollback useful edits. Expand them, add to them, but don't roll them back." He then rolled back my rollback and locked the post. It seemed I had flagged myself for inappropriate behavior.

I flagged again, saying: "RE declined flag by George Stocker: I think you may have misunderstood. The edit I rolled back was incorrect. Please read the edit history closely." After a while this flag was reviewed, but no further action was taken. Luckily, it was marked "helpful" which completely resolves the matter since I'm a robot whose actions are only motivated by points and shiny decorations. Beep boop, must get Marshal badge.


In all seriousness: I'm wondering how I could've approached this situation differently. What would be the best course of action in this scenario, where I was systematically in the minority, but genuinely felt like the other people weren't thinking as much about their actions?

More broadly, I just want to add to the calls for some serious reflection on the current edit and review process. I know that's already taking place in the above linked posts, so this need not become another debate about what they are already covering. But it's important to recognize that this isn't just a matter of fake reviews, but some really careless ones too. Let's not become hamsters.

share|improve this question

marked as duplicate by slugster, Toon Krijthe, Rory, Martijn Pieters, ChrisF Nov 20 '12 at 10:29

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2  
What is this? I don't even. –  BoltClock's a Unicorn Nov 20 '12 at 5:27
8  
OK I fixed it. The suggested edit is complete nonsense - it literally disembowels the first sentence, for one thing. How, pray tell, does destroying a sentence make a post more understandable? –  BoltClock's a Unicorn Nov 20 '12 at 5:30
9  
Hm... There is only one answer to this: review is broken. And that's something you already know. That said, "Our editing process is seriously broken. Please review what is happening here." is not really a helpful mod attention flag, "Our editing process is seriously broken." doesn't really help communicate exactly what the problem is, and "Please review what is happening here." is redundant (it's a flag, that's what flags are about). A (perhaps) better one would be "Horrible edit that superficially looks good is getting approved, halp!". –  Yannis Nov 20 '12 at 5:36
    
@YannisRizos Yeah you're right about that - I think it became too meta (in the general sense) of a moment for me and I didn't communicate well about what was actually going on. –  Paul Bellora Nov 20 '12 at 5:42
2  
Now that I've sorted that out, thanks for making me want to play the original TIM again. –  BoltClock's a Unicorn Nov 20 '12 at 5:50
    
@YannisRizos If you could please post some of your comment as an answer and expand a little, that's more of what I wanted to discuss and have feedback on (when you get a chance). –  Paul Bellora Nov 20 '12 at 6:08
2  
    
@Yannis That's absolutely spot on. Make every word in a custom flag count. –  George Stocker Nov 26 '12 at 3:34
add comment

2 Answers 2

The best course of action is for you to hit that Improve button, then add your improvements to the suggested edit. The person who suggested the edit gets the 2 rep points*, you get an edit action that counts towards a shiny badge, StackOverflow is tidier, and everyone is happy.

This is a well advertised issue at the moment - the review system itself is good, the problem is the people trying to rort it. Some people may get the hint once a few of their suggestions get rejected (if enough are rejected they are prevented from making further suggestions for a period of time) or they don't gain their 2 rep for the suggestion.

Personally I believe the approval privilege is set too low - raising it won't totally fix the problem but it would eliminate some of it immediately.

*Of course while editing the suggestion you could uncheck the Suggested edit was helpful checkbox if it was a poor suggestion, then the person who suggested won't get the rep.

share|improve this answer
    
+1 Wise words, and I will certainly do this in the future, especially now that I'm conscious of racing the inevitable approval clock. At the time, a reject vote made the most sense to me. –  Paul Bellora Nov 20 '12 at 6:04
14  
The only problem with this is the edit getting accepted while you improve it. –  Yannis Nov 20 '12 at 6:05
    
About the review system issues - thanks for tolerating the noise and for pointing out the other posts. I'm just glad this is getting serious attention. –  Paul Bellora Nov 20 '12 at 6:06
    
I regularly uncheck the Suggested edit was helpful box because of all the extremely minor and/or useless edits I see going through the queue. –  Nightfirecat Nov 20 '12 at 6:19
7  
I think it's time to really discuss a practical mutual exclusion system for edits. There is a lot more content that needs to be churned through /review and this is becoming a nuisance. Very short lived 'locks' would really help here. –  Tim Post Nov 20 '12 at 7:15
    
@Tim, a Feature Request I made for something very similar a month ago is located here. –  Barak Nov 20 '12 at 7:51
2  
@YannisRizos the issue you mentioned kills 1/3 or 1/4 of times when I try to break out of bad review using Improve. "people keep approving crap while I'm looking at it". Technically, this can be avoided quite "cheaply", using exclusion mechanism mentioned by Tim, eg as described here in details –  gnat Nov 20 '12 at 9:33
add comment

I've previously made this point in my answer to *A way to “downvote” a suggested edit and affect both the edit suggester and reviewers*.

While bad reviews are a large part of the problem, the issue of someone continuously resubmitting edits is a part of the problem I discussed in my answer above.

That issue is other than a short sentence when rejecting an edit or commenting on a rollback. There is limited ability to actually provide constructive feedback to the editor on why his edit was incorrect or invalid. Most editors don't know, all they see is they didn't get there two points, or that their edit was rolled back. And understanding where to look (the edit history, or the details on the suggested edit) are not well publicized.

I've had five edits rejected before I hit the 2K threshold, and because I was involved with other things on the site at the same time, (and because I really didn't care about the +2 rep points), I only rarely noticed the lack of the reputation points or knew those tools existed until I brought up a question on Meta Stack Overflow (Suggested Edit overwritten by OP edit), because I was trying to get some feedback on a possible bug when a specific edit was rejected by the Community.

I know Jeff Atwood has not supported telling users when they do something wrong in the notification bar, but we are going to allow anonymous users and one reputation point users to make suggested edits, there needs to be some what to provide some guidance on how to edit well, and to give them feedback when their edit was incorrect. I, for one, would have appreciated it because I was only trying to help and if I was given any feedback on the rejections, it would have helped me (although, seeing how some people do overreact to the slightest criticism, I do see why the notification bar may not be the best per Jeff Atwood's answer I linked).

Raising the edit approval bar might help (although I would hate it since I would lose that permission), but I'm wondering how much. Right now, the review queues are a problem because of badge hunters. Raising the threshold won't eliminate them, but it will reduce their numbers, slowing down the approval process, and hopefully allow good reviewers more time to review. But if you raise the approval threshold too high, you end up putting too much burden on fewer users and the queue ends up becoming too large.

I think the right answer is probably not necessarily tied to reputation points, but tied to some other mechanism to rank reviews, reviewers, and the approval process. Gnat proposed something very similar here and I think it makes some sense.

Currently the "flag weight" still exists and is used to rank flaggers in the moderators' queue for reviewing flags. If we could determine a way to quantify a reviewer's performance (and the editor's performance), you could place more limits on the reviewer (how many they could review or who's suggested edits end up in their queue) or on the editor (also limit how many edits based on performance or temporarily ban them (from making edits or approving edits in the case of the reviewer) if they have too many controversial edits (currently they are banned only when having too many rejected edits but not poor edits that still get approved by a 3-2 score).

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged .