9

I submitted this edit suggestion today on Stack Overflow. You can see that it has been approved by three reviewers. What you can't see is that in the first draft of the suggestion, I made a mistake that introduced a significant error into the problem statement.*

I caught the mistake right after submitting the edit, and was able to correct it within the grace period. However, the first reviewer had already approved my edit, and this approval remained in place after I changed the edit suggestion. That strikes me as questionable behavior because there is no history of what content the first reviewer approved.

The ability of users to change their edit suggestions during the review process has been brought up before on MSE (1, 2, 3). This answer in the latter discussion says that the grace period for such updates extends indefinitely until all required reviews have been completed. This is in contrast to the grace period for updating submissions or edits not requiring review, which is limited to 5 minutes.

This means that users can suggest a reasonable edit to a question and submit an "update" that changes the edit to something completely different following some number of votes to accept; this new suggestion would only need to be accepted by a single user.

This could happen purely by chance, but it could also be exploited for mischief; e.g.:

  1. User suggests a reasonable edit on Stack Overflow.
  2. Two users vote to accept the suggestion.
  3. User updates the suggestion, adding obscene ASCII art.

This vandalism will pass through the queue if any one of the next three reviewers votes to accept it. It gets two "free" votes from the previous reviewers, whose names remain associated with the review item, despite never seeing the vandalism and having no record of the revision that they actually reviewed. If we're only saving the active revision in the edit history, then we have no way of telling what the previous reviewers actually saw. Maybe it was a simple grammatical error; maybe it was a small error that was nonetheless important; maybe it was a huge difference. Since we're not saving the revision history for the suggestion itself, we don't know, and we should just throw away those votes to make sure the active revision gets a thorough review and avoid misinterpreting the votes of the previous reviewers.

The possibility of a feature request was brought up here but I can find no record of anyone having actually made the request, so here it is: Only votes cast on the active edit suggestion should count toward approval or rejection. If the active suggestion changes while being reviewed, any existing votes should be invalidated or discarded.

It would also be prudent to prevent reviewers who have an open tab with the former revision from casting a vote on that revision; if someone tries to cast a vote to review a revision that's no longer active, the vote should not be recorded. Ideally, the page would be refreshed with a warning that the content has changed.

* The mistake was that, in lazily copying the problem statement over from Project Euler, I had changed "5th power" to "fourth power"

  • 3
    Couldn't someone abuse this as well? If they see one or two quick rejections, they could make a trivial change, and hope for more lax reviewers. Not as bad as the situation you're describing, but still somewhat exploitable. – resueman Dec 18 '14 at 21:47
  • 1
    @resueman Yes, I imagine so. There's also the avenue of disallowing updates once votes have been cast, if that's a concern. – Air Dec 18 '14 at 21:53

You must log in to answer this question.

Browse other questions tagged .