32

As a reviewer noted nicely:

User is editing too much post today. Obviously not interested in content but in reputation

Not exactly a valid rejection reason in general, but said user suggested many edits indeed, and while a few were ok, many merely placed keywords in backticks (which btw should not be done for non-code, e.g. for the reasons listed here) and should have been rejected (a shockingly large amount of these were however approved). This suggestion behaviour is most certainly not what the system is intended for, so I wonder if there is a limit on suggestions in place? Note I'm not referring to the due to rejections for seven days but to a general limit of, say, 20 suggestions per day.

As many of you have noted, reviewers actually accepting such crappy suggestions are a problem. But they are not the problem, but rather a consequence. Limiting the amount of edit suggestions1 limits the source of all evil bad edits, while the serious reviewers are rather the victims who loose the battle against are slower than robo-reviewers and thus more likely to reject a suggestion when it has already been approved. We should educate the editors to "make it count", which to my knowledge is already the desired behaviour, but not in a vague "minimum character count" - one missing backtick edited in may severely improve a post, while a second one to emphasize a non-code keyword is bad.


1 And as discussed in comments, 20/day is not necessarily a good measure, something like max. 5 unapproved edits in the last 24 hours where approval a) make another edit available and b) increases that limit with time might be better - the suggestion here is about the consideration of such a limit, not about its implementation details

23
  • 13
    "and should have been rejected" <---- there's your problem.
    – Bart
    Commented Aug 21, 2013 at 9:40
  • 19
    I think we should educate reviewers that putting random keywords in backtick is not okay. For example, we can start by introducing such incorrect edits to the review audits. Commented Aug 21, 2013 at 9:43
  • 1
    @Bart There's my other problem - that would trigger the correct edit-ban, but the sheer amount of suggested edits by said user caused ca. 7 out of my toady's reviews to be by him; and to be fair not all suggestions were bad. Implementing a daily limit independently of approval/rejection would encourage users to think twice whether their suggestions is really worth being suggested Commented Aug 21, 2013 at 9:43
  • 3
    @OldCheckmark In fact I started haunting such reviewers with an On an unrelated note, please don't [approve suggested edits](link_to_suggestion_here) using backticks for emphasis, but reject or improve them - see e.g. [here](http://meta.gaming.stackexchange.com/q/7437/88) why comment on one of their posts (unfortunately the only way to call out to them). It's not exactly model behaviour by my side I assume, but fortunately users are understanding Commented Aug 21, 2013 at 9:44
  • 1
    @CRUSADER Well reviews on suggested edits are limited to 20 per day as well, and reviewers don't even get rep (yeah, a crappy badge, but meh). I'm talking about limiting editors, not reviewers Commented Aug 21, 2013 at 9:55
  • 5
    On the same subject: Is this OK? The user edited many posts with a single tag - and even reached the reputation cap with edits only :P
    – Kobi
    Commented Aug 21, 2013 at 10:01
  • 2
    @Kobi That guy shouldn't do that. I like that fact that he linked to a meta post as an excuse to farm reputation.
    – FalconC
    Commented Aug 21, 2013 at 10:28
  • 1
    How about adding a -1 reputation penalty for a rejected edit? Making editor think before submitting an edit Commented Aug 21, 2013 at 11:03
  • 3
    What problem do you hope to solve? Bad suggestions are bad and good suggestions are good, regardless of the rate at which they're made. Limiting the number doesn't do anything to improve the proportion of good suggestions. Commented Aug 21, 2013 at 11:34
  • 1
    @Gilles not necessarily - most good suggestions come from users who don't suggest that many edits, therefore by limiting suggestions those who over-suggest mostly bad edits become less in relation to the total amount of edits. Plus, a bad editors is more likely to become edit-banned by rejection before having flooded the review queue Commented Aug 21, 2013 at 11:42
  • 1
    @TobiasKienzler “most good suggestions come from users who don't suggest that many edits” I'd want stats for that. In my stint as a reviewer I came to recognize both good suggestors who made a ton of improvements and bad suggestors who only ever butchered the formatting. Good point about letting the edit-ban come earlier. That, and on sites other than SO not flooding the front page, are good reasons to rate-limit suggested edits, but at a finer granularity than 1 day. Perhaps limit to N pending edits? Commented Aug 21, 2013 at 11:51
  • 1
    @Krishnabhadra As long as the robo-reviewers keep approving the majority of bad edits, I don't think that will stop the serial editors. Commented Aug 21, 2013 at 20:44
  • 1
    @S.L.Barth I agree we need to educate reviewers. But they are our second line of defense against bad edits. We should educate new editors to make substantial edits from the start, and an initial -1 reputation penalty and a personal comment (just like the comment when flag is declined) telling the reason for rejection is a good way to pass the message through. Commented Aug 22, 2013 at 12:18
  • 1
    @TobiasKienzler I feel this should be a "feature-request", instead of "discussion". Commented Aug 26, 2013 at 9:45
  • 1
    As related evidence, this guy stackoverflow.com/users/466082/arthur?tab=reputation has hit rep-cap a couple of days in a row purely on edits
    – tacaswell
    Commented Sep 1, 2013 at 4:40

3 Answers 3

21
+400

As Bart stated in the comments, the main problem is that suggested edits are accepted too easily. Which bringst us to another point of attention: what suggested edits should be accepted? In my opinion these suggestions should be accepted:

What are the solutions to reputation farming by editing?

Educate the reviewers

An edit should make the post more readable or clearer. Nothing more. What our culprit does is just adding backticks, __and shouldn't be accepted in the first place__. That is the easiest change. Emphasize that trivial changes shouldn't be accepted.

It still doesn't stop reviewers who review just to get badges. In my opinion, suggested edits are accepted too fast. The reviewers sometimes don't even bother to change obvious typos.

Add a suggestion limit

Should we use a suggestion limit? A suggestion limit of 20 seems too strict. I can suggest twenty good edits. But the culprit has _six_ pages of edits from today, so approximately 90 suggestions.

I propose to use a suggestion limit of 40 a day. If you have a higher number of accepted edits, the limit should be lifted. This would only work if reviewers aren't afraid to reject trivial changes.

Discourage trivial changes

What if (new) users continue to add backticks, even when there aren't needed. Well, notify them that these edits shouldn't be suggested. And with notify, I really mean notify them by a notification, and every time when the user clicks on "edit" a blue popup balloon or something like that comes on. The automatically kicks in, because the suggestions were rejected.

Use a rollback script

[In the comments Kobi has linked this person][4]. He has multiple edits that only change one thing: adding the tag. I believe this can easily be detected by a script running every day, like the [vote fraud script][5]. Adding tags is a very easy way to farm reputation, and the reviewers, _again_, should reject it. In the case of repeated tag-adding, the pattern could be detected by a script, and the reputation should be taken away from the editor.

Summary

Emphasize (to the editor and reviewer) that trivial changes are unacceptable. Add a suggestion limit of 40 for new users. Lift the limit when the editor behaves well.
11
  • Good points there - maybe the limit should only count suggestions that have not been approved, i.e. either rejected or not yet reviewed. And as you said, the limit should increase for better editors. But really one of the most important points is feedback, which should be more prominent both on rejection and approval Commented Aug 21, 2013 at 11:56
  • I had no idea that there was anything wrong with adding backticks to relevant text in text blocks (personally I do so whenever I mention method names in questions). Consider me educated!
    – Mansfield
    Commented Aug 21, 2013 at 13:03
  • 1
    @Mansfield Are you referring to this? In my opinion it is okay, because HttpApplication.LogRequest is very unpleasant to read without the backticks. What this guy did was just placing random backticks. The edit should be rejected.
    – FalconC
    Commented Aug 21, 2013 at 14:20
  • @FalconC I wasn't referring to that post in particular, but yes, that kind of thing. I do that in my own questions as well...
    – Mansfield
    Commented Aug 21, 2013 at 14:34
  • 2
    I think the visual studio super tagging deserves a separate question Commented Aug 21, 2013 at 16:31
  • Playing devil's advocate ( for myself :) ). How can reviewers decide what is a super tag and what is not to reject those edits? Is visual-studio a super tag of visual-studio-2010 or not? What about python vs django? Etc. There are just too many edge cases. The only solution I can see would be a community consensus on a tag hierarchy, which has been proposed and rejected. The same goes for the rollback script. How can it differentiate useful retags from non useful ones? Commented Aug 21, 2013 at 18:05
  • @cirosantilli I have taken the (counter)arguments into account, and a hierarchy just for version numbering might be a bit unnecessary. Since you can wildcard tags, super tags for versioning shouldn't be suggested. OP should add them, but please don't suggest any retags. The rollback script does something else. It looks for patterns in (suggested) edits. When a user suggest the same edit for a large number of posts, the reputation is too easily given out. Repo shouldn't be too easy to earn, and the script takes away the repo.
    – FalconC
    Commented Aug 21, 2013 at 19:13
  • 1
    +1 for Educate the Reviewers. There is nothing more frustrating that having spent time on an edit and then see it being apparently randomly reviewed. But it goes both ways - there are poor reviewers that reject too much as well, but that is less visible.
    – user213634
    Commented Aug 29, 2013 at 7:58
  • Ooh, ooh, me, me! Implicit tagging hierarchy Commented Aug 29, 2013 at 8:36
  • 1
    A rollback script that cancels rep sounds good. Regarding the daily limit of edit suggestions, I wouldn't want to make that dependent on the number of accepted/rejected edit suggestions. Not until the robo-reviewer problem is solved. Commented Aug 29, 2013 at 12:49
  • I think 20 edit suggestions per day would be more than enough for serious editors. And as pointed out before, I don't want the limit to change based on reviews until the robo-reviewer problem is solved. But I find your answer the best and most elaborate, so I've awarded the bounty to you. Commented Sep 2, 2013 at 7:45
6

I think that this more of a review problem than an editing problem.

If we solve the robo-reviewing problem, the mass editing problem should resolve itself. If enough of these edits are rejected, the editor gets the boot, and the message is sent that suggesting a lot of minor edits isn't really what we're looking for from editors.

Setting a daily limit would slow down the mass editors, but I doubt it would really solve the problem. Rather than seeing 100 minors edits in a single day we would just see 20 per day for 5 days.

If you really want to go after sloppy mass editing, and solving the robo-review problem is really that insurmountable, wouldn't a rate limit be more effective than a daily limit?

If the editor is forced to wait for a full two or three minutes between edits, the chances that the editor will fix more than a single issue increases, if not the chances of the reviewers catching up increases, and hopefully if the reviewers can catch up, the chances of edit rejections putting a stop to the sloppy edits before they've gone too far also increases.

7
  • 1
    Robo-reviews are another problem closely related to this - the vast amount of silly edit suggestions encourages robo-reviewing while serious reviews are severely slowed down. Sure enough the daily review limit mitigates that a bit, but by sending editors the message "make it count" (and not only by that flawed minimum character count, adding a missing backtick is a valid edit while fixing the spelling of "thakns in adavnce" should not) would help educating the source of the bad edits instead of punishing the honest reviewers Commented Aug 29, 2013 at 7:35
  • @TobiasKienzler edited to correct
    – apaul
    Commented Aug 29, 2013 at 12:51
  • 2
    @TobiasKienzler I think part of the education could start here on Meta, when we see people asking about spelling or re-tag campaigns we should remind them to check for other issues while they're at it. meta.stackexchange.com/a/188665/217863
    – apaul
    Commented Aug 29, 2013 at 13:04
  • 1
    "Rather than seeing 100 minors edits in a single day we would just see 20 per day for 5 days." I think we'd see 100 minor edits in 5 days, instead of 500 minor edits in 5 days. Commented Aug 29, 2013 at 13:10
  • @S.L.Barth true, but if we get the review issue worked out they should get bounced out on day 1.
    – apaul
    Commented Aug 29, 2013 at 13:47
  • @apaul34208 If we had the review issue worked out, we probably wouldn't have this discussion.... so in that respect you are right. But I don't think that problem will be solved any time soon, so another approach is necessary. Commented Aug 29, 2013 at 15:03
  • @S.L.Barth You may be right, but I still think that poor reviews are more of a problem, at least a more common problem, than mass editing. Mass editors come through once in a while, but poor reviews happen every day. If we take a more serious approach to cracking down on robo-reviewers we would effectively kill two birds with one stone.
    – apaul
    Commented Aug 29, 2013 at 15:33
3

Ok, based on the comment discussions here's a hopefully more sensible limit suggestions:

Grant users a certain amount of N "unapproved edits"-tokens per (rolling) 24 hours, i.e. once they suggest an edit, they have N-1 suggestions left. If their suggestion is accepted, they get that token back, if it's rejected or not reviewed within 24 hours (how likely is that on SO? But I digress...) the token is only returned 24 hours after the suggestion was posted. Once the approved to unapproved ratio (long-time or during the last 24 hours, both should be somehow considered) exceeds (or goes well under) a specific amount, the user gets their N permanently increased (decreased).

To add some numbers:

  • Anonymous/unregistered users and [registered ones with < 20 local rep (association bonus doesn't count, just like on protected questions)] start with N=2, which is enough to point out severe flaws while reducing their ability to spam/vandalize/thankyouverymuch
  • Registered users with 20-100 rep earned at the respective site start with N=5
  • 101-1999: N=10
4
  • In addition, the lower a user's rep the larger the edit should be to make it onto the queue. Hence users with low rep (like me) could only make significant edits.
    – andy256
    Commented Aug 29, 2013 at 9:36
  • 1
    I'd like to see a limit to the number of edit suggestions one can make per day. But I'd rather not make it dependent on how many of a user's edit suggestions have been accepted/rejected. As long as the robo-reviewers are a problem, that is not a meaningful variable IMO. Commented Aug 29, 2013 at 10:56
  • 1
    @andy256 I don't like that - I've seen plenty of minor tab->space replacements or even renamed variables just to circumvent the minimum character count. And as mentioned, fixing a missing backtick is IMHO a severe enough reason to edit, while an edit adding "I have this problem too plz sent soluzion to [email protected]" is ''long enough'' but sucks nonetheless Commented Aug 29, 2013 at 11:00
  • @S.L.Barth Good point, though I somehow still hope that the suggestion-limit itself will lower the relative amount of robo-approvals. Also note that there probably also is a significant amount of robo- rejections. Were I to cheat the system, I'd at least toss a coin every now and then ;) Commented Aug 29, 2013 at 11:02

Not the answer you're looking for? Browse other questions tagged .