5

Recently, with the changes to the top bar, the review queue got new functionality. Instead of displaying numbers when there were tasks, the queues were given trigger points and a red indicator would pop up any time there were as many or more items in the queue than that trigger point. You can read more about it in this related question: How does the review-needed indicator work exactly?

In my answer to that question, I noted that I was concerned that a trigger of fifty items in the close vote queue was something unattainable by most small sites and that if there weren't users checking the queues without being prompted to do so, some close-worthy questions would fail to be closed. This number was reduced to twenty but even that seems high for some of the sites I use.

I noticed recently that I have ten close votes/and several close reviews that aged away on Cooking since the change to the top bar and it made me wonder whether I was simply really crappy at close voting or whether this was a symptom of not sufficiently notifying users when there were items in the close vote queue... or something else, like low reviewing behavior on the site.

An absolute comparison of review tasks before and after the change is probably going to be a good indicator of whether the review queues are getting the same amount of attention after the indicator change than before it.

So, a way to see how many close vote reviews were done by users compared to how many items were put into the close review queues would probably be helpful in this.


This may be tangential to the question relating to the review indicator but it is additional information that I would find helpful in looking into this data.

Knowing how many close votes have aged away over time may help with this, though I understand that this data may not be able to attribute this to whether the question was fixed, wasn't really close-worthy, or other variables. That said, I think that if a question received three or more votes to be closed over time (including some aging away in the interim), that would be a good sign that it was likely a question that should have been closed.

Is there a way to see which questions attracted more than three close votes and then ended up not getting closed due to the votes aging away?

5

Some of the data for this can be found publicly from SEDE. I've written a couple of queries that might address this to some extent:

  • Aged Away Reviews on Potentially Bad Questions
    This is a pretty narrow search, so it may not be broad enough for your purposes. Technically, it finds reviews that have had more than MinReviewTasks close reviews submitted (by default 3), that have since aged away.

  • Aged Away Reviews
    This is a slightly broader search: it finds any review task that has aged away. This may include invalidations that were not due to aging (there are a few other reasons that review tasks can be invalidated), as well as reviews that weren't Close reviews.

As for reviews before and after the review indicator was added, here's a graph of reviews per week over time. Change the value of PeriodLength to average over a month or any other time period instead.

  • When you say "reviews" that is limited to people who actually clicked "close" in a review queue? Does that mean it excludes the person who originally voted to close on the question itself and anyone else who did so? – Catija Dec 7 '17 at 0:01
  • @Catija Correct on both counts - actual close votes that aged away (as opposed to close reviews, which is what this queries) aren't available in SEDE. – ArtOfCode Dec 7 '17 at 0:07
  • Note: tasks will be invalidated if the question is deleted, or if the question is closed outside of review (for close reviews) - so use PostHistory and/or PostsWithDeleted to eliminate those possibilities. Example – Shog9 Dec 7 '17 at 0:09
  • Hmmm, it also seems to include aged votes on questions that were closed as duplicates... though I suppose that may have been done out of review, too. – Catija Dec 7 '17 at 0:18
  • Are you sure that reviews per week over time is correct if you change the period length? This looks really odd when I set the period for 30 days. – Catija Dec 7 '17 at 0:55
  • @Catija No, not entirely... I've made a change which should at least mostly fix it, but it still looks a bit odd. Will keep looking. – ArtOfCode Dec 7 '17 at 1:02
1

This doesn't address aged-away votes, but it does address the underlying "are people using the queue, and was the answer affected by review-needed indicator changes?" question: number of close queue review task results by date.

Examples:

On some sites, there's a qualitative change around October 18 (I believe when indicator changes launched) where there start to be higher spikes than there ever were before, and I think more total task results. But not on all sites! It doesn't look like overall number of task results fell anywhere. So: seems to have been a change that increased activity some places and left it flat-ish elsewhere.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .