187

I would like to raise the bar for late answers to enter the Late Answers Review Queue to 50 rep, which is the threshold after which users gain the ability to comment. Here are some answers that were posted by users with between 10 to 50 rep (yes, there's a little selection bias to be sure. I suspect most/all of these will be 10k links only by the time you're reading this. Note, all of these are from the last 5 hours.

These are all from Stack Overflow, where I have the most experience, but since Late Answers have a much higher probability to be bad across all stacks, I think this applies generally. Also, I have bolded all the users references to the inability to comment, that were not in their original answers:

  • https://stackoverflow.com/a/32702471/1768232 (23 rep)

    Solution provide by @brandonyang works for me too! (I can't add comments in a response 'cause my low reputation, sorry).

  • https://stackoverflow.com/a/32703040/1768232 (47 rep)

    I tried your answer but it doesn't work in office 365 outlook online. Do you have any suggestions?

  • https://stackoverflow.com/a/32701373/1768232 (26 rep)

    My question is somehow related to the same issue. I am fairly new to C# and have been doing JAVA for quite a long time. When I am coding in C# with .Net framework 4.5.1 using visual studio 2013, I get errors for using certain libraries but I donot know which dll should I reference in my project How do C# people figure that out? For example I wanted to code to draw an ellipse and needed system.Windows.Media and System.Windows.Controls both of which were giving me errors, after googling for quite sometime I got to know that we need to reference PresentationFramework.dll, is this the only way to figure it out?

    PS >> I am unable to post this as a comment hence putting it as answer.

    Thanks.

  • https://stackoverflow.com/a/32699722/1768232 (19 rep)

    i like this video. It explains the differences between int and Integer and cautions of using Integer

  • https://stackoverflow.com/a/32698255/1768232 (21 rep)

    I have a question, i am actually trying to copy the database from Prod server to test server ( SQL server 2008R2). I am wondering would it copy the users logins, roles and permission as well ?


I know I suggested 50 rep above, but while I was compiling that, I found one from a 61 rep user. Perhaps it should be 100 rep, not 50. I am less confident about this, but I thought I'd add it as an addendum for discussion.

The answer posted here is the simplest solution that you are looking for to solve your problem, so take a look ;)

  • 50
    50 rep is okay. Don't raise it too much or the queue will get flooded. – Patrick Hofman Sep 21 '15 at 19:57
  • 3
    @PatrickHofman I definitely want 50, because of the comment privilege threshold. 100 I'm less sure about. – durron597 Sep 21 '15 at 20:02
  • 12
    @pat The late answer queue, at least on SO, doesn't get many reviews at all. I think it only takes 1 review too to complete the review. So definitely don't see a problem increasing reviews there. I also have a feature-request to increase reviews of late answers. – CRABOLO Sep 21 '15 at 20:29
  • 12
    @CRABOLO That's a good request. I also have a request to increase the number of eyeballs needed to dismiss a post as no-action needed – durron597 Sep 21 '15 at 20:38
  • 8
    @CRABOLO: On ELL, RPG, and SX the late answers queue is one of the hardest to ever find any reviews in at all (second only to reopen). I think we can confidently state that LA is not presently at risk of being flooded on most (all?) sites, even with a substantial bump in volume. – Nathan Tuggy Sep 21 '15 at 21:55
  • 2
    @inɒzɘmɒЯ.A.M: My entire first page of rep dropdown is downvotes I just cast on late answers. It's great. – Nathan Tuggy Sep 29 '15 at 20:16
  • 2
    @Adel: Judging by the inflation in most sites' LQP queues that I've seen, it seems there was a fair need for this. That plus the up/down-votes is a pretty good justification. The SF/SU review flooding was a surprise, though. – Nathan Tuggy Sep 30 '15 at 2:12
  • 4
    It is getting tiresome that the standard solution to quality problems on the site(s) is always "more reviews". Instead of demanding quality from new users before their post is even uploaded to the site, people keep insisting on reviewing existing posts and look for crap. Block the crap from entering the site in the first place! For example, block every short answer that contains a question mark at the end of a sentence. – Lundin Sep 30 '15 at 8:43
  • 5
    People are posting comments as answers, because they are not allowed to comment. Instead of allowing them to comment, we're now creating extra review work and still the users are not able to comment, although some of the example comments were helpful. How about fixing the problem instead of the symptoms? – Chris Sep 30 '15 at 9:58
  • 4
    "I can't add comments in a response 'cause my low reputation, sorry" Oh man I fscking HATE that. What neurological catastrophe has led these people to think that bypassing the restriction by doing a bad thing is appropriate/okay/acceptable??! Do they not spend even a microsecond determining why the reputation limit was there in the first place? It's not a leap from that to the conclusion "oh, okay, I shouldn't post this." saiduhfaklsiudfa – Lightness Races in Orbit Sep 30 '15 at 10:52
  • 3
    @Chris many of the sort of NAAs this captures also wouldn't make good comments either, such as the first example. – durron597 Sep 30 '15 at 12:17
  • 7
    @Lighness To me as a layman the current rules appear as a privilege inversion. The more serious and consequential action -- answering -- requires less reputation than the rather inconsequential and easy-to-delete or -ignore comment. Of course the users contribute (in their opinion: help) in the way that's open to them. Not the users are illogical, but the rules are. Perhaps new users should gain reputation from upvoted comments before they are allowed to answer... or at least be able to comment right away as well. – Peter A. Schneider Sep 30 '15 at 12:30
  • 1
    @LightnessRacesinOrbit Could be because there's not really any sound rationale why someone would be allowed to post an answer but not a comment, while we at the same time have much higher expectations of the quality of an answer than on the quality of a comment. I believe the only reason behind the 50 rep rule was to dodge outright spammers. So it actually has nothing to do with "only experienced users can be trusted with the privilege to comment". – Lundin Sep 30 '15 at 12:58
  • 1
    @Lundin: Perhaps. Though arguably the subsequent behaviour of these users really only shows that they can't be trusted with the privilege to answer, either. – Lightness Races in Orbit Sep 30 '15 at 13:23
  • 4
    @PeterSchneider: I know it looks backwards on the surface, but I think requiring 50 reputation to comment has saved us from having to deal with all sorts of trouble that other sites have with their comments. It's also critical that people learn right from the start that participation on the sites requires asking and answering, not commenting. That said, I'm actively working on a system to let new users comment if they have a mentor to help them learn to use the site. A goal of this queue is to help people earn the reputation necessary to comment. – Jon Ericson Sep 30 '15 at 16:56
121

TL;DR: As of today (September 29, 2015) the maximum reputation for having an author's late answer enqueued has been raised from 10 to 50.

As for further increase to 100:

Queue growth

Review queues lose effectiveness if posts aren't regularly (and accurately) cleared. A queue such as close votes on Stack Overflow that never seems to get to 0 fails to provide time-sensitive feedback that aids learning. So if we raise the bar for late answers which increases the number of tasks going into the queue, we risk making the queue less useful for people answering questions late.

As I write, the Late Answer queue on Stack Overflow is empty. In the last 24 hours the number (non-audit) review tasks of each type added to the queues:

Review Tasks Type           
------------ ----------------- 
2467         Triage            
2428         First Post        
2333         Close Votes       
1846         Suggested Edit    
 883         Low Quality Posts 
 619         Late Answer       
 594         Helper            
 141         Reopen Vote    

These two factors suggest that there's room to add more tasks to the Late Answer queue. Assuming we raised the reputation limit to 100, we'd add about 300 tasks:

Rep Bin Tasks 
------- ----- 
 10       417   
 20       160   
 30        85    
 40        76    
 50        44    
 60        41    
 70        37    
 80        21    
 90        20    
100        20   

You might expect the number of tasks associated with the 1 to 10 reputation bin would equal the number of late answer tasks in the last day. The discrepancy stems from reputation earned since answers were added to the queue. But the takeaway is that raising the reputation limit to 100 would add half again as much work to the queue. That doesn't seem particularly onerous.

Not surprisingly, Stack Overflow would see the most added Late Answer reviews by far. Other large sites, such as Ubuntu, Server Fault, Super User and Mathematics, would get 10 or so extra reviews a day.

Justifying 100 rather than 50 reputation

Discovering answers as comments certainly would suggest 50 as a good limit for this queue. But this queue is as much about granting exposure to potentially upvote-worthy answers as it is about finding problems. Especially on large sites, it's pretty easy for a good answer to a very old question to be lost in the noise. As a strategy for new users answering old questions that already have good, but not excellent, answers seems productive if not that few people will notice those answers. At 10 reputation, a knowledgeable new user has one shot at earning upvotes from excellent answers to aged questions.

A potential complication

The one concern I have is that answers are dequeued once the author has reached 100 reputation. If we raise the maximum reputation for going into the queue, we should also raise the minimum for coming out of it. Unfortunately, I don't have an easy way to gather numbers to set an appropriate level. One of the reasons I'm only looking back a day in the numbers I showed above is that it minimizes the effect of gained reputation. It's possible, but messy to estimate reputation at the time a user answered a question. So while my instinct would be to raise the deque minimum to 200, but I can't really predict if that's appropriate or not.

Resolution

Raising the bar to 100 would require changing two variables at once. Since that would make it harder to evaluate each individual change, I raised the reputation for getting a Late Answer enqueued to 50. I will reconsider upping it to 100 at some point in the future. (Hopefully we can do it in a less jarring manner, however.)

  • 15
    Are you sure you want thousands of old answers to enter the queue now? Are you sure that all sites profit from an increased threshold? – yo' Sep 29 '15 at 19:27
  • 8
    @JonEricson: One on C.SE from 2013 and one on PPCG from 2011. A month, eh? – El'endia Starman Sep 29 '15 at 19:34
  • 7
    @JonEricson We've got hundreds of posts with "active: 2 years ago". Are you should about "answers should not be older than a month"? I smell a bug here... :D – yo' Sep 29 '15 at 19:34
  • 7
    @El'endiaStarman: Looks like this is the second time in as many weeks as you've caught me making bad assumptions about how things work around here. ;-) Well, now's the time to earn that Late Answer Reviewer badge you've always wanted. – Jon Ericson Sep 29 '15 at 19:39
  • 8
    Please roll back this change immediately. We have now a 3.2k entries long review queue on ServerFault. That's total crap. – Sven Sep 29 '15 at 19:44
  • 4
    @muru: I'm sorry about that. Thank goodness I didn't raise the limit to 100. As a reminder, it's easier to clear a task from this queue than many others. A single review dequeues. – Jon Ericson Sep 29 '15 at 20:03
  • 22
    @Sven: Reverting won't help, I'm afraid. The damage has been done. I have gone over to pitch in on Server Fault as an act of penitence. – Jon Ericson Sep 29 '15 at 20:04
  • 2
    This as also got Arqade queue to grow a lot for the site's size, 1.5k there will be a long queue to empty. Is there any way to help with this by giving members that can review a higher review count for some time? – Jonathan Drapeau Sep 29 '15 at 20:26
  • 8
    @Sven It really won't take that long, I'm sure it will be gone in less than 2 weeks naturally. – durron597 Sep 29 '15 at 20:36
  • 5
    @JonathanDrapeau: I just did a bunch of reviews on Server Fault. My most frequent action on the Late Answer queue turns out to be voting (up or down). Unfortunately, I hit the vote limit. So even if we gave more reviews temporarily, there are other limits that will slow you down. At the point, the best bet is to chip away at it for a few weeks. I apologize for the annoyance. :-( – Jon Ericson Sep 29 '15 at 21:22
  • 6
    @NickGammon: Yeah. That was unplanned. However, the other way to think of it is that these users are not new in the "fresh" or "recent" sense, but in the "unseasoned" or "untried" sense. Alternatively, my excessive use of quotes might be seen as an example of "spin". ;-) – Jon Ericson Sep 29 '15 at 23:01
  • 8
    @AndrewLeach: Since the queue on ELU has already dropped from 900 to 800 in a half day, I expect the emergency will be over before such a change could be rolled out. – Jon Ericson Sep 30 '15 at 0:25
  • 8
    Right now I expect SF's "long" review queue to be cleared in a few days, not a few weeks. – Michael Hampton Sep 30 '15 at 2:14
  • 8
    @ByteCommander After 24 hours you've reviewed almost 1000 of them. At that rate they will be gone before next Monday. It was a mistake but I don't really see the problem. – durron597 Sep 30 '15 at 21:57
  • 3
    The queue on Server Fault is cleared. As I said, days not weeks. – Michael Hampton Oct 8 '15 at 16:55
37

I think this is a good idea. I spend a lot of time on the New Answers to Questions More Than 30 Days Old list on Stack Overflow (10k link), and I would agree that many of the questions I flag as not an answer are from new users who want to comment, find they don't have the privilege, and leave an answer instead.

The main concern I would have about the proposal would be whether this would overload the Late Answers Queue. To analyze that for Stack Overflow, I've looked through the 200 most recent late answers on the site. Here is the breakdown by reputation of the poster:

  • 45 (23%) were from users with reputation 1-10
  • 46 (23%) were from users with reputation 11-50
  • 10 (5%) were from users with reputation 51-100
  • 99 (50%) were from users with reputation 101+

From this limited dataset, it seems like we might see roughly a doubling of the number of answers added to the Late Answers Queue by increasing the reputation limit to 50. Increasing to 100 would not add too many more late answers to the queue. Given that the Late Answers Queue is often empty on Stack Overflow, I don't think this would be too problematic.

To get a sense of the users who are creating problematic late answers, I reviewed my most recent 200 helpful flags on late answers on Stack Overflow. Here is the breakdown by reputation:

  • 132 (66%) were from users with reputation 1-10
  • 39 (20%) were from users with reputation 11-50
  • 10 (5%) were from users with reputation 51-100
  • 19 (10%) were from users with reputation 101+

This suggests that users with reputation 1-10 are generating much more than their fair share of problematic late answers, users with reputation 101+ are generating much less than their fair share, and users with reputation 11-100 are generating about their fair share. I think this could mean that reviewing late answers for users with up to 100 reputation could make sense, but there would be diminishing returns raising the bar above 100 reputation.

It would be helpful to see this analysis repeated on other sites to see if the results are qualitatively similar (it sounds like the relatively small size of the Late Answers Queue may hold across other sites).

  • Bear in mind that a lot of people use SEDE to look for questions worth flagging and low-rep users are an obvious place to start any query. This may introduce significant bias when considering helpful flags by rep range – Basic Sep 29 '15 at 22:30
  • 3
    @Basic For sure! In this answer I limited to my most recent 200 helpful flags that I had generated from the "New Answers to Questions More than 30 Days Old" list on Stack Overflow, so the results I quote about helpful flags aren't biased in the way you describe. Of course others might flag different questions than I would when reading the list, so that part of my answer is meant more to give a sense of the number of problematic late answers generated by different rep groups than to be an authoritative breakdown. – josliber Sep 29 '15 at 22:35
2

Perhaps letting people comment sooner would also be a way to solve this problem. Note that many of the cases cited here are people w/o the rep to comment. Creating a situation where they are thus "forced" to post an answer that isn't an answer is actually not a good thing. It violates the "truth in packaging" idea; makes the user (presumably usually new) feel bad about the whole site, and invites others to criticize their posting an answer that is not an answer, thus cycling the whole thing downwards... being a new user on SO is not easy (and I happen to be a PhD who had about 15 years of industry experience when I started using!)

  • 3
    People who want to comment, but can't have to post their two-cents as answers? How is that a good thing? – JosephDoggie Sep 30 '15 at 14:10
  • 4
    Anyone can ask or answer a question. Commenting is a privilege gained by users who have proven that they are able to contribute, and I think it should remain that way. Did you feel bad about the whole site? Did you fall into a spiral of despair when you realized you could not comment? I didn't when I was new. I was ecstatic about the fake internet points I was gaining by helping people with Java questions. – Rainbolt Sep 30 '15 at 14:40
  • 1
    I've also commented when asking for clarification on someone else's question/answer. In this case, it would be inappropriate to ask an entirely new question, but that is the only option for someone without enough rep to comment. – ChicagoRedSox Sep 30 '15 at 16:11
  • 2
    @ChicagoRedSox, there are actually three options: (1) leave a comment as an answer (this is wrong), (2) do nothing, (3) get the reputation properly by either answering or asking a question and then comment... – Matthew Peters Sep 30 '15 at 16:48
  • 2
    @JosephDoggie: I'd recommend posting a new question with a fuller proposal for allowing comments at lower rep without being flooded by users commenting that have no idea what they're supposed to use those for. This is really just for handling problematic answers, and simply allowing comments won't fully solve those. – Nathan Tuggy Sep 30 '15 at 17:10
  • I think it's actually exactly the opposite, and is "truth in packaging" in a good way. The new user learns that this isn't a regular chit-chatty web forum and that the expected patterns of discussion fit into specific conventions. – mattdm Sep 30 '15 at 20:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .