28

So, I just came across a review in the Late Answers review queue.

A late late answer

The link to the review itself: https://softwareengineering.stackexchange.com/review/late-answers/47599

This wasn't an audit (the dates on it struck me - I went out to double check that the vote count matched). Its a late answer from two years ago (on a three year old question) showing up in the late answer queue.

Is there something amiss from scripts? or is this intended functionality that old, unreviewed, first posts occasionally pop up in the queues?

I blame caching.

  • 1
    It seems right to me...the question was asked back in late '10, and the answer was provided back in early '11. There were six months between the question and answer. That qualifies as a late answer to me. – Makoto Dec 21 '13 at 4:20
  • 1
    I think a bug was solved or the algorithm was changed -- first posts had about 240 posts at 3AM UTC yesterday. – Qantas 94 Heavy Dec 21 '13 at 7:33
25

This is, while unusual, an expected result of the system.

The way the late answers queue works is by checking two things.

  • Was the answer posted at least 30 days after the question was asked?
  • Was the answer posted by a user who has 10 50 or less reputation (disregarding suspension)

If both of these are true, it is entered into the review queue until it eventually gets reviewed. In most cases, this will catch things as they come in and so you will only see new posts, so we don't spend any resources checking time.

However, on rare occasion, someone will actually drop below 10 reputation, while having some past answer that does qualify. That's what happened in your example - the user received a dunk in reputation from 15 down to 5 on December 21st, 2013, and so this ancient post of theirs that has not previously been reviewed, is now put into review that same day.

  • 1
    Whatever the algorithmic explanations, a question posted over two years ago is hardly by a "new user". OK, "new user" two years ago. But not a new user now. The opportunity to label the poster as a "new user" has now been lost. – Nick Gammon Oct 8 '15 at 6:54
6

This has been a regular source of confusion (and occasionally, excessive amounts of work) for years. After a thorough analysis, I'm unable to find a compelling justification for this behavior; "late" late answers are infrequently reviewed, and even more rarely actioned; the exceptions are primarily on sites where the queue piles up for weeks and then gets cleared out, mostly negating the value of having a queue in the first place: giving feedback to a new user weeks after they've posted has little utility, while tools like the (10K-only) "New Answers to Questions More Than 30 Days Old" are much faster to scan and more thorough in their criteria for inclusion if you're just looking for non-answers.

So with that in mind, today we're gonna start excluding these answers from the queue once they're more than 7 days old. This is how it has worked on Stack Overflow for the past 6 years already, so there's a side bonus of consistency here... However, this threshold can be adjusted on a per-site basis if necessary.

While I was reviewing the data for this, I also took a look at the First Posts queue: posts here were dropped after they were 30 days old, but most actioned reviews happened much more quickly; here the exceptions were limited to sites where the majority of review tasks were invalidated, either because the post was closed or deleted from outside the queue, or because it simply got too old. Again, not much value in letting these pile up: it becomes discouraging for reviewers and provides progressively less value to either the author or the site. So I'm reducing the threshold here to 14 days, which covers the vast majority of actionable reviews.

...I also managed to break Late Answer syncing for a solid hour, so... Sorry about that.

  • Quick, somebody fabricate a screenshot of this very answer in a late answer queue … – Wrzlprmft Feb 6 at 18:36
  • Heh... It won't enter the LA queue because I have too much rep, @Wrzlprmft... But it does show up top of the list in "New Answers to Questions More Than 30 Days Old", linked above. – Shog9 Feb 6 at 18:40
  • Meta doesn’t even have that queue. I just wanted to mess with you, making you think that you totally botched implementing this. – Wrzlprmft Feb 6 at 18:41
  • 1
    I may've spent 20 minutes staring at the logs for a certain site, waiting to see a big pile of tasks invalidated... Before realizing that I'd selected the meta site instead. – Shog9 Feb 6 at 18:42

You must log in to answer this question.