112

Consider:

  1. User posts an answer.

  2. Community autoflags it as "low quality (auto)." (This is frequently a false positive.)

  3. User edits answer after 5-minute grace period. (Or perhaps someone else edits the answer.)

  4. Community sees the edit and obediently marks the "low quality (auto)" flag as helpful.

  5. Community applies an automatic downvote to the post, triggered by said edit.

Users are, therefore, penalized for improving their answers. This is wrong, wrong, wrong.

If a user posts an answer which gets autoflagged and someone else edits the answer to fix a typo, it absolutely should not come with a downvote. This makes no sense at all.

Possible solutions, in increasing order of preference:

  • Implement something like this, where the downvote is revoked if the post is later upvoted (this is what happens with "very low quality" flags, but not "low quality (auto)" flags). This is... not ideal, since the downvote shouldn't be there in the first place.

  • Do not apply an autodownvote if the flag was cleared by an edit. This is also not perfect, as it's an ugly special case and requires extra complexity. Which leads to...

  • Do not clear any type of flag on edits. This doesn't really make any sense in the first place. Edits are not necessarily guaranteed to improve a post; indeed, sometimes edits make a post worse. And using edits as a trigger to clear flags can be actively harmful, too, because perhaps the edit did not remove the content that the answer was flagged for, but the auto-flag-clearing prevents a moderator from seeing it.

  • Do not apply automatic downvotes when "low quality (auto)" flags are marked helpful. In theory, it's a good idea, but I often reflexively mark any Community ♦ flag as helpful just because it's less clicks and it doesn't make a difference... except in this case. It's hidden, confusing behavior, and I can't count how many times I've accidentally "downvoted" perfectly valid answers (indeed, it took me quite a while to finally catch on to what was happening). This problem is exacerbated by the fact that this specific flag has a very high false positive rate (on the sites I moderate, at any rate).

  • Both of the above (don't clear flags on edits, and don't autodownvote when flags are cleared as helpful, since neither are at all intuitive).

  • "The system will now revoke the automatic downvote if, after the flag is marked Helpful, the post later gets an upvote. Remember, the automatic downvote essentially exists because folks use this flag in cases where they should be downvoting. As such, it serves an important role... But should still be considered weak compared to actually downvoting the post in cases where there is conflicting information..." (Community downvotes posts that were just fixed) – gnat Feb 29 '16 at 12:42
  • 6
    @gnat I'm talking about the "low quality (auto)" automatic system flag (as I mentioned repeatedly in my question), not the "very low quality" flag. – Doorknob Feb 29 '16 at 13:19
  • 1
    I think same reasoning should apply to auto-flags: upvote to post forces Community user revoke downvote. If this doesn't happen, I would say that's a design mistake and linked post explains why the other way is correct – gnat Feb 29 '16 at 13:57
  • Wait, auto VLQ flags are still a thing? I haven't seen one in.. over a year? – ɥʇǝS Mar 3 '16 at 20:29
  • @ɥʇǝS We get them nearly daily on PPCG (last one was Low answer quality score [80] just yesterday, in fact). – Doorknob Mar 3 '16 at 20:36
  • @Doorknob I'm looking at this request - are you seeing this more on PPCG or Puzzling? Asking because you're a mod on both. – Taryn Mar 3 '16 at 21:08
  • 1
    @bluefeet You mean the undesirable behavior? It happens on both. Or do you mean auto low quality flags in general? They're very common on both sites as well (probably just because short answers are common due to how Puzzling and PPCG work). – Doorknob Mar 3 '16 at 21:45
  • @Doorknob Both. I'm going to pull some numbers on it so I wanted to know where to look initially. :) – Taryn Mar 3 '16 at 21:46
  • 11
    Holy ****. Is this why I see downvotes on crappy answers almost immediately after I edit them? I kept thinking it might be coincidence, but it's happened so often... – muru Mar 4 '16 at 14:07
  • 1
    Adding the [status-deferred] tag to this since it's been written up internally for discussion and for the devs. – Taryn Jul 7 '17 at 16:46
29

Update: August 31, 2017

Internally, we discussed several ways to implement this, but in the end we made them work the same as normal VLQ flags. That means, if the flag was marked helpful, but later receives an upvote the automatic downvote is revoked. This is available as of the latest build. Thanks to the devs for implementing this change, and making the behavior the same across both VLQ flag types.


I agree that this probably shouldn't happen, if someone edits a post to improve it, punishing the OP after a post was edited isn't the best experience.

I was curious how often this happens so I pulled some numbers for the last 365 days. Stack Overflow is excluded because it doesn't get low quality (auto) flags, the data below is for all non-meta sites that had at least one helpful flag:

Total Flags Total Helpful Total Helpful By Edit Pct Validated By Edit Site Name 
----------- ------------- --------------------- --------------------- --------- 
5252        4582          441                   9.62                  Super User
1678        1229          91                    7.40                  Server Fault
18          18            0                     0.00                  Stack Apps
898         721           42                    5.83                  Cooking   
855         801           52                    6.49                  Home Improvement
407         369           34                    9.21                  Game Developers
5331        4955          377                   7.61                  Gaming    
923         884           56                    6.33                  GIS       
349         280           36                    12.86                 Mathematics
363         348           25                    7.18                  Photography
701         658           44                    6.69                  Statistical Analysis
1221        1195          91                    7.62                  Web Apps  
320         287           23                    8.01                  Webmasters
4548        4086          205                   5.02                  Apple     
25          24            2                     8.33                  Theoretical Computer Science
3541        2664          407                   15.28                 English Language and Usage
262         258           5                     1.94                  Personal Finance and Money
172         157           16                    10.19                 Role-playing Games
879         796           135                   16.96                 TeX - LaTeX
393         314           40                    12.74                 Ubuntu    
2101        1735          444                   25.59                 Unix and Linux
1227        1144          111                   9.70                  WordPress 
218         211           17                    8.06                  Bicycles  
136         135           9                     6.67                  Programmers
4766        4595          272                   5.92                  Android Enthusiasts
125         109           9                     8.26                  Board and Card Games
1287        1197          81                    6.77                  Physics   
50          37            3                     8.11                  Homebrew  
545         532           34                    6.39                  IT Security
96          89            5                     5.62                  Writers   
1244        1218          78                    6.40                  Electronics and Robotics
799         640           43                    6.72                  Graphic Design
719         645           77                    11.94                 Database Administrators
789         659           89                    13.51                 Science Fiction
1           1             0                     0.00                  Area 51 Discussions
341         331           28                    8.46                  Code Review
229         166           78                    46.99                 Code Golf 
76          68            8                     11.76                 Quantitative Finance
28          26            0                     0.00                  Project Management
197         197           3                     1.52                  Skeptics  
849         735           75                    10.20                 Drupal Answers
103         101           10                    9.90                  Fitness and Nutrition
426         385           26                    6.75                  Motor Vehicle Maintenance and Repair
173         173           3                     1.73                  Parenting 
1063        959           62                    6.47                  SharePoint
240         217           9                     4.15                  Musical Practice and Performance
118         112           17                    15.18                 Software Quality Assurance and Testing
148         128           22                    17.19                 Jewish Life and Learning
74          73            2                     2.74                  Astronomy 
163         126           30                    23.81                 German Language and Usage
108         63            11                    17.46                 Japanese Language and Usage
136         127           12                    9.45                  Gardening and Landscaping
97          96            9                     9.38                  Philosophy
48          46            4                     8.70                  Personal Productivity
633         584           28                    4.79                  Travel    
88          86            6                     6.98                  Cryptography
107         107           14                    13.08                 Signal Processing
136         100           22                    22.00                 French Language and Usage
308         308           12                    3.90                  Christianity
255         249           13                    5.22                  Bitcoin   
54          53            3                     5.66                  Linguistics
120         120           2                     1.67                  Biblical Hermeneutics
32          31            4                     12.90                 Economics 
166         159           7                     4.40                  History   
110         95            8                     8.42                  LEGO®     
151         142           22                    15.49                 Spanish Language and Usage
42          38            2                     5.26                  Computational Science
909         889           66                    7.42                  Movies    
235         123           16                    13.01                 Chinese Language and Usage
222         220           7                     3.18                  Biology   
40          38            4                     10.53                 Poker     
165         149           23                    15.44                 Mathematica
55          54            2                     3.70                  Cognitive Sciences
67          59            3                     5.08                  The Great Outdoors
32          32            0                     0.00                  Martial Arts
136         135           5                     3.70                  Sports    
269         255           15                    5.88                  Academia  
132         122           13                    10.66                 Computer Science
121         119           7                     5.88                  The Workplace
551         524           12                    2.29                  Windows Phone
547         538           34                    6.32                  Chemistry 
132         108           19                    17.59                 Chess     
520         422           23                    5.45                  Raspberry Pi
104         77            5                     6.49                  Russian Language and Usage
363         361           11                    3.05                  Islam     
770         648           56                    8.64                  Salesforce
55          49            0                     0.00                  Patents   
236         231           15                    6.49                  User Experience
28          28            4                     14.29                 Genealogy and Family History
41          41            0                     0.00                  Robotics  
68          63            3                     4.76                  ExpressionEngine
48          46            2                     4.35                  Politics  
97          86            5                     5.81                  Audio-Video Production
731         659           81                    12.29                 Anime and Manga
1255        1067          80                    7.50                  Magento   
793         677           81                    11.96                 English Language Learners
22          22            0                     0.00                  Sustainable Living
17          17            1                     5.88                  Tridion Stack Exchange
51          41            6                     14.63                 Reverse Engineering Stack Exchange
262         210           23                    10.95                 Arduino Stack Exchange
214         198           6                     3.03                  Network Engineering Stack Exchange
42          40            4                     10.00                 Open Data Stack Exchange
22          22            2                     9.09                  Freelancing Stack Exchange
384         353           30                    8.50                  Blender Stack Exchange
193         178           8                     4.49                  MathOverflow
91          90            4                     4.44                  Space Exploration Stack Exchange
112         104           5                     4.81                  Sound Design Stack Exchange
124         117           5                     4.27                  Tor Stack Exchange
139         138           2                     1.45                  Pets Stack Exchange
23          19            0                     0.00                  Amateur Radio Stack Exchange
30          26            3                     11.54                 Italian Language Stack Exchange
872         750           102                   13.60                 Stack Overflow em Português
197         181           9                     4.97                  Aviation Stack Exchange
32          31            2                     6.45                  Ebooks Stack Exchange
43          40            0                     0.00                  Beer Stack Exchange
189         186           15                    8.06                  Software Recommendations Stack Exchange
68          65            2                     3.08                  Expatriates Stack Exchange
20          16            0                     0.00                  Mathematics Educators Stack Exchange
49          47            1                     2.13                  Earth Science Stack Exchange
127         119           13                    10.92                 Joomla Stack Exchange
41          40            4                     10.00                 Data Science Stack Exchange
435         366           107                   29.23                 Puzzling Stack Exchange
54          43            5                     11.63                 Craft CMS Stack Exchange
59          57            11                    19.30                 Buddhism Stack Exchange
206         204           5                     2.45                  Hinduism Stack Exchange
6           6             0                     0.00                  Moderators Stack Exchange
31          28            2                     7.14                  Startups Stack Exchange
207         201           15                    7.46                  Worldbuilding Stack Exchange
256         165           24                    14.55                 スタック・オーバーフロー
47          37            10                    27.03                 Emacs Stack Exchange
20          19            1                     5.26                  History of Science and Mathematics Stack Exchange
239         222           23                    10.36                 Lifehacks Stack Exchange
55          52            5                     9.62                  Engineering Stack Exchange
27          22            4                     18.18                 Coffee Stack Exchange
25          20            4                     20.00                 Vi and Vim Stack Exchange
62          55            8                     14.55                 Music Fans Stack Exchange
32          24            3                     12.50                 Woodworking Stack Exchange
66          50            4                     8.00                  CiviCRM Stack Exchange
36          36            0                     0.00                  Health Stack Exchange
2659        1844          602                   32.65                 Stack Overflow на русском
453         249           139                   55.82                 Русский язык
11          10            0                     0.00                  Mythology Stack Exchange
32          31            0                     0.00                  Law Stack Exchange
6           6             0                     0.00                  Open Source Stack Exchange
165         154           22                    14.29                 elementary OS Stack Exchange
29          23            7                     30.43                 Portuguese Language Stack Exchange
3           2             0                     0.00                  Computer Graphics Stack Exchange
10          10            3                     30.00                 Hardware Recommendations Stack Exchange
47          44            13                    29.55                 Stack Overflow en español

Besides the Russian Language site, Code Golf has one of the highest percentages so I understand why you noticed this.

I'm answering because I'm looking at it and will try to figure out the best way this should be implemented. To be clear, any change will only impact the auto flag, it will not impact VLQ flags placed by users because those have been seen by a human and that means a higher signal of a problem.

  • 15
    it will not impact VLQ flags placed by users because those have been seen by a human and that means a higher signal of a problem. That sounds like the downvote should be added immediately, like spam flags -- not like the user should be punished when they fix that problem. – Matthew Read Mar 16 '16 at 4:44
  • Stack Overflow на русском has the biggest impact on that, seems legit... – EKons Aug 3 '16 at 8:03
  • 2
    @ΈρικΚωνσταντόπουλος Perhaps Cyrillic isn't being treated as text by LQ(a) bot. – wizzwizz4 Dec 25 '16 at 19:32
  • In the meantime, is there anything I can do to undo the effects of the downvote after I've mistakenly marked the flag helpful? I can upvote myself but that's not ideal. – Ganesh Sittampalam Jan 30 '17 at 10:50
  • @GaneshSittampalam Unfortunately, no. You'd have to vote on the post.. – Taryn Jan 30 '17 at 14:59
  • 9
    It's been a year of looking into this. I understand that this likely doesn't have a very high priority, but could we get some form of update? – Martin Ender May 17 '17 at 8:06
  • 2
    @MartinEnder I threw it up internally for discussion. Hoping to move it along. – Taryn Jun 13 '17 at 2:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .