22

In August, we released a new feature, significant edits, that changed how posts enter the Reopen queue. As promised, we conducted a data analysis to determine whether or not users are abusing the new feature. We determined that abuse is minimal, so we won't be implementing additional measures to limit its use.

To summarize the Significant edit feature, it is a checkbox that appears when users are editing a closed question. It lets users indicate that their edits resolve the original close reason. When the significant edit box is checked,  we send the question to the Reopen queue (sometimes by way of the Suggested Edit queue if the user hasn't earned the Edit privilege). 

Users can use the significant edit feature to send a question to the Reopen queue multiple times, so our analysis centered around how often this is occurring.

Results of data analysis

We analyzed the period between Aug. 26 - Oct. 14, 2021.

  • Posts entering the Reopen queue due to significant edits: For Stack Overflow, we found that 69.9% of posts entering the Reopen queue were due to significant edits vs. users casting reopen votes. Across the entire Stack Exchange network (excluding Stack Overflow), 61.3% of posts were due to significant edits.

  • Posts re-entering the Reopen queue due to multiple significant edits: For Stack Overflow, we found that only 4.5% of posts with significant edits went on to subsequently re-enter the Reopen queue due to multiple significant edits. Across the entire Stack Exchange network (excluding Stack Overflow), 9.9% of significant edit posts re-entered the Reopen queue.

  • Average, median and maximum times that posts entered the Reopen queue due to significant edits: For Stack Overflow, we found that both the average and median number of times that a post entered the Reopen queue due to a significant edit was 1 time, with the maximum number of times being 4. Across the entire Stack Exchange network (excluding Stack Overflow), the average and median number of times that a post entered the Reopen queue due to a significant edit was also 1 time. The average  maximum number of times across all sites was 3 times, and the maximum on any site was 5 times.

  • Original posters vs. other editors: For Stack Overflow, the original poster made 92.6% of the significant edits, while across the Stack Exchange network (excluding Stack Overflow), the median was 94.6%.

3
  • 4
    Will be there another release about the impact of these on the reopening of questions? Are significant edits more likely to get a question reopened than plain reopen votes? Did it improve the reopening rates vs previous behavior?
    – Braiam
    Oct 19 '21 at 18:33
  • 2
    I was especially interested (as probably many readers) to see the difference in total number of items entering the queue due to an edit before and after addition of the "significant edit" option. I think this was the communities main concern. Especially for SO the reopen queue went from a constant size of 1k to around 200. Since the problem was most pronounced on SO overall, can that specific graph please be included - perhaps as an answer? (Total items due to reopen vote/edit entering the queue before and after.)
    – bad_coder
    Oct 20 '21 at 7:22
  • What was the percentage of successful reopens for a question that entered the queue via a "significant edit"? That seems to be the most important data point to me to determine whether the feature is effective, compared to the percentage of successful reopens for Qs that entered the queue from any edit over the same duration from before the feature was introduced.
    – TylerH
    Oct 20 '21 at 14:14
15
+200

Thank you for sharing the research! I was anticipating this a bit (therefore, I wrote a SEDE query a couple of weeks ago); I've noticed that quite a lot of Reopen Votes review items here on Meta Stack Exchange used to be caused by 'active reading' of otherwise unsalvageable posts and I'm glad that does not happen anymore (irrespective of the high quality of the edits). My hypothesis was that the number of questions reopened via the queue would remain stable, but the total number of review items would decrease. To my surprise, that has not happened (yet), but we did see an increase in the number of reopened questions in September:

enter image description here

I'm curious to see how this will evolve in the next months; only time will tell ...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .