We need a hard rate limit on suggested edits, to limit the damage someone can do with blasting the review queue with minor edit suggestions.

Case in point: this user peppered the site with a whopping 133 tag-only edit suggestions in 2 hours and 15 minutes. That's one edit every minute on average, but towards the end he was entering a new suggestion every 20 seconds. He only stopped because a call to action here on meta drew enough rejections to have him banned from entering more.

If there was a limit to the number of suggested edits you can enter in a time period we could have stopped this guy earlier. No <2k user needs to edit hundreds of tags across that many questions in 2 hours time.

Considerations

  • There are cases where a retag drive or tag burninate justifies editing a larger number of posts, but I'd be fine if such tasks were limited to people with 2k+ reputation. A retag or burninate job requires a minimal engagement with the community to know when they are justified anyway.

  • The suggested edit review process, by itself, was clearly not enough to stop this barrage. It cannot be seen as a rate limiting measure, at least not as a sufficiently efficient one.

    The vast majority of the edits made in the above example were too minor, leaving spelling, grammar and formatting mistakes in the post untouched. The user was going too fast to pay attention to such details anyway. Yet this user has 98 approved edit suggestions, but all but 6 or 7 I'd have rejected as too minor, so the review process itself is broken.

What should be limited

  • The number of review votes reviewers have is already limited; I burned through 20 reviews in short order just reviewing the suggestions from just this one user. Certainly the number of unreviewed suggestions should be limited to let reviewers catch up with a user like this. I'd say having 20 outstanding suggested edits pending review is plenty.

  • Suggested edits should be made with some consideration; we don't want minor edits, so if you are suggesting more than 20 in an hour (that's 1 every 3 minutes) you are not improving posts anymore, you are gaming the system to gain reputation.

  • 6
    Why not have a daily rep limit of 10 for edits? Cf. 200 upvote daily rep limit. – Bathsheba Dec 6 '13 at 14:43
  • 16
    Why not give the editor -1 rep for a rejected edit? – Bathsheba Dec 6 '13 at 14:44
  • 2
    That would not necessarily stop such a barrage. It would take away an important incentive, sure. – Martijn Pieters Dec 6 '13 at 14:44
  • 1
    There are other, related, feature requests that suggest a penalty for rejected suggested edits. – Martijn Pieters Dec 6 '13 at 14:45
  • 1
    Hmmm, almost two years ago a similar question popped up. meta.stackexchange.com/questions/116636/… No definitive answers though or a status-declined – rene Dec 6 '13 at 14:45
  • 11
    @Bathsheba I don't think we should punish users for attempting to help in good faith. – Stijn Dec 6 '13 at 14:46
  • 3
    Indeed I'm sure this is not a new phenomenon but being able to acquire 1000 reputation just by adding tags etc. doesn't sit well. – Bathsheba Dec 6 '13 at 14:46
  • 1
    Were the proposed edits appropriate except for their size? Or do you think the additional tags didn't belong there? – Peter Alfvin Dec 6 '13 at 14:46
  • @Stijn: you mean the -1 for a rejected edit? The rate limit is not a punishment, just a natural break in how fast they do this. – Martijn Pieters Dec 6 '13 at 14:47
  • @PeterAlfvin: See the other post for the specific details on that user. I am only addressing the frequency here. – Martijn Pieters Dec 6 '13 at 14:47
  • @MartijnPieters Yeah I meant the -1 for a rejected edit. (I've actually upvoted this question) – Stijn Dec 6 '13 at 14:49
  • @Stijn neither to I. We are relying on the integrity of the reviewers. – Bathsheba Dec 6 '13 at 14:54
  • 1
    Actually, it might be beneficial to ratelimit >2ks too. Tag wikis should not be edited lightly. – Manishearth Dec 17 '13 at 9:49
  • 1
  • 1
    @S.L.Barth you can always flag an edited post and ask a moderator to suspend the user. – Shadow Wizard May 18 '16 at 12:06
up vote 57 down vote accepted

As of a few seconds ago, this limit is now active network-wide:

  • at-most 20 pending suggested edits per editor on beta sites
  • at-most 5 pending suggested edits per editor on graduated sites

These limits are based on my analysis of the speed at which edits are generally approved and the number of edits (and editors) that would have been affected over the past few months. They can be adjusted if it becomes necessary. For future reference, here's a query that estimates who would be blocked at a given threshold and another that calculates the median time to review edits for a given site, in minutes.

Kudos to Michael Stum for implementing this!

Please report any bugs here on meta.

  • 2
    What happens if you try to edit the wiki and excerpt on three tag wikis? The 5th and 6th edits would be submitted with the same click. – yellowantphil Jul 13 '16 at 3:51
  • 3
    I haven't done a thorough review, but I believe it would go through without complaint (and then you'd be blocked on the next wiki you tried to edit), @yellowantphil - the check runs before you're shown the editor, not as the edit is submitted (because that'd be cruel and frustrating). – Shog9 Jul 13 '16 at 3:55
  • 4
    Michael Stum is on the Stack Overflow team? Sweet. – BoltClock's a Unicorn Jul 13 '16 at 4:13
  • 4
    Couple of questions: first, is there any daily limit? Or can users still submit 200 suggested edits in a day? Second, what about anonymous suggested edits? – Shadow Wizard Jul 13 '16 at 7:31
  • @ShadowWizard Going by the information available, I would say that it's a window, not a hard limit. So, no, limits per-volume aren't implemented. – Braiam Jul 13 '16 at 13:03
  • @ShadowWizard: given the current median time of 28 minutes on SO, on that site you should be able to have about 250 suggestions approved per 24 hours, if you time them just right (e.g. submit a new suggestion every 5 minutes and 36 seconds). In practice the time can vary greatly of course, so my money is on mass editors giving up earlier than that. – Martijn Pieters Jul 13 '16 at 13:09
  • @MartijnPieters true, no doubt the new limit is good and welcome, just wonder if it's accompanied by any other limits. – Shadow Wizard Jul 13 '16 at 14:20
  • Keep in mind, the goal here is not to say anything akin to, "> X edits / period is bad and you should feel bad" - it's merely to make folks aware that, while suggesting, they're using a shared resource... And they can't use it all. – Shog9 Jul 13 '16 at 18:30
  • at-most 5 pending suggested edits per editor on graduated sites I hope no Suggested Edits queue has thousands of suggestions waiting just like SO's close votes... – dorukayhan Jul 13 '16 at 19:34
  • 3
    No. There's a hard limit of 200 pending edits, at which point no one can suggest any more, @dorukayhan. Currently SO is at 168. – Shog9 Jul 13 '16 at 19:37
  • The pending edit queue size for SO was experimentally increased from 200 to 500 in June. meta.stackoverflow.com/a/351728/149428 – Taylor Edmiston Aug 27 '17 at 19:04
  • Yes, that's temporary @TaylorEdmiston - we're currently working on a better solution: meta.stackoverflow.com/questions/355233/… – Shog9 Aug 28 '17 at 16:03
  • Actually, the check does happen when submitting the edit - I tested this by opening the editor on more than five posts (which the system allowed me to do since I had zero pending edits), then submitting the edit on five of them. On trying to submit the sixth I was prevented from doing so. – Sonic the Inclusive Werehog Dec 10 '17 at 7:42
  • @MartijnPieters 28 minutes for tag edits? Unbelievable: last year, my tag edit suggestions were always in the queue for 2-10 days on SO. – Cœur May 13 at 2:01
  • 1
    @Cœur: right, see this updated query; tag wiki edit suggestions have a median time to approval of 1054 minutes, so 17.5 hours (!!!). – Martijn Pieters May 13 at 10:48
up vote 20 down vote
+125

There are 2 main problems that I see that would be resolved by rate-limiting of some fashion:

  1. As mentioned, this would prevent someone flooding the review queue
  2. The second is someone repeatedly making the same minor edit for a large number of posts.

As Martijn mentions in the question, limiting the number of pending suggested edits would go a long way to resolve #1, but it would only partially address #2 as it would allow someone to resume editing as soon as an edit is reviewed (whether it be accepted or rejected), and in sites with fast moving review queues (i.e. Stack Overflow), this could be almost immediately.

We know that an edit ban kicks in when enough of the users edits in the last 7 days have been rejected, but this is sometimes not enough when someone is editing en masse. They can usually garner enough accepted edits to raise the threshold for an edit ban. If you are able to maintain a 3-to-1 accept to reject ratio, you'll never trigger the threshold. Enough so-so looking edits mixed in with a handful of robo reviewers and you'll be able to maintain that threshold.

To avoid this, I think further rating limiting based on edit rejections would be a fair solution. The exact numbers can be fleshed out with further discussion, but the idea would work like this:

  • User is rate limited to 20 pending suggested edits
  • This limit is reduced by 5 for every rejected edit in the last hour (the reason for the time limit is to prevent this from being a complete ban and just a way to slow them down until the community can handle the situation).
  • So if a user has 20 pending suggested edits and gets a rejected edit, they are now limited to 15 pending suggested edits. This means they can't propose any new edits until they have fewer than 15 pending edits. And this number would be further reduced if they pick up any additional rejects in the preceding 60 minutes.

A bonus of this is 4 reject edits in any 60 minute period effectively bans the user for the remainder of the hour. It would be a very temporary ban, but should be a sufficient period of time to allow the community to catch up and/or leave him a message explaining why their edits are not acceptable.

This could have an additional benefit on sites that have very strict tag burnination processes (Stack Overflow, I am looking at you). It's not uncommon to end up finding some "helpful" soul trying to clean up a tag that hasn't been completely resolved (or even if it has been resolved, just starting a mass tag-only editing spree). By both limiting edits in the queue and further limiting by rejections, it helps the community to limit the damage and allow the user to be educated on the process and proper editing.

  • +1 for gradually reducing the available edit slots. This way, we can educate the editors in an early stage. – S.L. Barth Dec 24 '13 at 9:56
  • Good point about the tag burninations. But the editor who made me raise the bounty was not just retagging. (If only they were!) – S.L. Barth May 18 '16 at 12:05
  • I was curious as to how much of an effect this would have, so I put together a little query to help guess at it. – Shog9 Jun 24 '16 at 1:27

To counter the answer given by Lance Roberts:

At the onset I dislike the idea of rate limiting the number of edits we can suggest in a day. I have been a prolific editor myself and it was the privilege of suggesting edits which encouraged me to increase my participation on sites in the Stack Exchange Network. Any rate-limitation of the likes that I can only suggest x edits in a day (where x is too small) would have irked me for sure. I have easily crossed 20 edit-suggestions-a-day on numerous occasions - at times the similar number of my edits were pending review.

Moreover, an anonymous user is also able to suggest edits - how would you deal with it? I don't consider this as spamming if done in good faith, so using the IP level ban/restriction doesn't seem appropriate to me if it's not spam.

However, what Martijn Pieters proposes:

Suggested edits should be made with some consideration; we don't want minor edits, so if you are suggesting more than 20 in an hour (that's 1 every 3 minutes) you are not improving posts anymore, you are gaming the system to gain reputation.

should be fine in my opinion and provides a good balance for rate-limitation and quality of suggested edits.

I am sort of neutral on his other suggestion:

The number of review votes reviewers have is already limited; I burned to 20 reviews short order just reviewing the suggestions from just this one user. Certainly the number of unreviewed suggestions should be limited to let reviewers catch up with a user like this. I'd say having 20 outstanding suggested edits pending review is plenty.

As I mentioned in my answer earlier, there have been times when there have been more than 20 of my edit suggestions pending in review queue. However, once I reach there I am sure I won't feel bad that I can't review more for the time being - as I know that I can edit more once the community catches up and starts trusting me more. So, I am neutral on that suggestion.

  • 2
    Note that my other suggestion limits your unreviewed suggestions. On site with an actively managed suggested edits review queue, having 20 reviews still outstanding is going to be almost impossible. But on a quieter site, pushing 20 reviews into the queue would slow you down in proportion to the reviewing capacity of the site. – Martijn Pieters Dec 19 '13 at 19:26

Another option is to limit the rep mining in these cases -

  • Option 1: Allow users to edit as fast as they want but +2 rep eligibility is replenished only once a minute or more
  • Option 2: Allow users to edit as much and as fast as they want, but limit the rep from edits to overall 20 a day (or any other value). To sweeten that a bit you can add a badge saying the user have reached X daily edits on Y days.
  • No, a badge for a number of edits is only going to lead to more gaming. I think that not handing out rep for edits is going to lead to a lot of confusion, complicating the rules too much. – Martijn Pieters Dec 22 '13 at 14:10
  • 1
    @MartijnPieters, handing out almost unlimited rep is just as bad. Ok scratch the badge, why not limit the rep rate? if someone invested 5 seconds in an edit (like the guy above who probably didn't even read the questions), does he deserve the rep for it? – Leeor Dec 22 '13 at 14:26
  • 1
    There is already a cap on suggested edit reputation. You cannot earn more than 1000 points that way. – Martijn Pieters Dec 22 '13 at 14:53
  • I vote for option #2. – Neolisk Jun 12 '14 at 14:00
  • @MartijnPieters Huh, I didn't know about that. Why is that? It seems odd that no matter how many good edits you have, you'll never earn the ability to skip the suggested edit approval process at 2k. (It would take a pretty long time to get 1000 good edits, but still...) – jrh May 11 '17 at 14:48
  • @jrh: at 2 points per edit, you'd only need 500 edits to get there. We want you to do other things besides editing to gain reputation. – Martijn Pieters May 11 '17 at 15:00

The limit for approving suggested edits is 20 a day, so I would think that would be a good limit for suggesting those edits.

  • 9
    I don't agree, actually. There is more than one reviewer on any given site. This is why I proposed a limit to the number of unreviewed suggestions you can push into the queue. 20 in a day is very low, really. That is about 1 every half hour, in a 10 hour period. – Martijn Pieters Dec 19 '13 at 19:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .