Timeline for Are 'reasonably bad' reviewers in any way being detected or 'punished'?
Current License: CC BY-SA 3.0
21 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 3, 2020 at 13:30 | history | edited | CommunityBot |
Commonmark migration
|
|
| May 23, 2017 at 12:35 | history | edited | CommunityBot |
replaced http://stackoverflow.com/ with https://stackoverflow.com/
|
|
| May 13, 2013 at 5:43 | comment | added | Flexo - Save the data dump Mod | I'm not convinced counting the outlier votes (1 vs 3) reviews will work - it's not that unusual for the 1 to be the better outcome than the 3. | |
| May 13, 2013 at 0:40 | history | edited | nickhar | CC BY-SA 3.0 |
Edited to reflect updated comments
|
| May 13, 2013 at 0:13 | comment | added | user200500 | In the grand scheme of things, it doesn't matter if a correct suggested edit was approved by a "genuine reviewer" or one of these lazy layabouts that keep snapping up your easy reviews. As with answering "low hanging fruit" questions, the motivations of the user do not matter, as long as there is a benefit to the site. | |
| May 13, 2013 at 0:10 | comment | added | slugster | @Asad And before you ask "but then how do people learn?", you learn by doing, not by avoiding. It comes down to people's motivations for being in the queue - is it just to get a badge, or is it to make a difference and help with the quality of the site? | |
| May 13, 2013 at 0:08 | comment | added | slugster | @Asad while those edits do have to be reviewed like any other, if you want the shiny badges you have to do the work, don't just pick off the easy stuff. Not everyone has the technical expertise to make judgement calls on some questions, but when that lack of expertise or motivation stretches to most questions then you have to ask why the person is even in the review queue. We could argue this all day, but i dislike lazy and robo reviewers - if people are only targeting the easy questions then they should get out and let genuine reviewers do it. | |
| May 13, 2013 at 0:01 | comment | added | user200500 | @slugster What is wrong with reviewing the "low hanging fruit", as you call it? | |
| May 12, 2013 at 23:33 | comment | added | Niels Keurentjes | I tried constructing some queries on the data subsite but it would appear all possibly sensitive data such as reviewing behaviour is not published, pity but understandable. I'm very curious if a developer could produce some real world statistics on common accept/skip/reject ratios, and whether it's possible to easily query for users whose reviewing decisions 'frequently disagree' with the final result of a review. | |
| May 12, 2013 at 23:23 | comment | added | slugster | @Niels I think the system already tracks Skip so that you don't get presented with the same review again, but I agree with you that they need to be aggregated as part of the stats. Personally I only use Skip for duplicates or wiki edits where I don't have the technical expertise to make a decision, something like that should be tracked and used as part of the decision process for whether people should be review banned or not. | |
| May 12, 2013 at 22:56 | comment | added | Niels Keurentjes | Come to think of it, I'm pretty sure that 'skips' must also be tracked, since otherwise the system would eventually keep confronting you with the same questions to vote on. So obviously it will have to keep track of the ones you skipped as well. | |
| May 12, 2013 at 22:34 | comment | added | nickhar | @slugster Nitpicking is good! Yes, agree that they can be averaged over time. What I mean by skip is 'not within my field of experience' rather than avoiding decisions, but I still like to see them looking for complete clangers which need to be pulled up - but I didn't make this clear enough; I'm remarkably happy negatively reviewing edits/posts and so should others. I completely agree on the low-hanging fruit too. | |
| May 12, 2013 at 22:26 | comment | added | slugster | I want to nitpick a couple of your points: 1. approve/reject rates can be averaged over time. 2. People who avoid making hard or negative reviews by using the Skip button are bad reviewers - negative reviews need to be made, if they don't/can't make the call they should get out of the review queue. Sure it's okay to use Skip if you cannot gauge the technical accuracy of an edit (I use it myself) but if someone is using it too often then they shouldn't be reviewing. People should not be reviewing only the low hanging fruit! | |
| May 12, 2013 at 22:06 | comment | added | nickhar | @AaronBertrand Indeed. Seen it a few times too. | |
| May 12, 2013 at 22:01 | comment | added | Aaron Bertrand Staff | Just something I've noticed on multiple occasions - sometimes the exception is the right call, while the three that you're deeming were "right" were just dog-piling. | |
| May 12, 2013 at 21:48 | comment | added | Niels Keurentjes | me neither, and if they aren't it means the reject ratio is probably indeed a bad candidate for initiating automated peer reviews. Still means the other part of my suggestion (peer reviewing people that frequently cast votes that do not comply with eventual consensus) is possible. | |
| May 12, 2013 at 21:46 | comment | added | nickhar | @Niels Honestly, I think I've hit skip far more than any other button in any queue... I don't know if those clicks are tracked!? | |
| May 12, 2013 at 21:44 | history | edited | nickhar | CC BY-SA 3.0 |
added 19 characters in body
|
| May 12, 2013 at 21:40 | comment | added | nickhar | @Niels It could be an influencing factor, but I like the general idea! :) | |
| May 12, 2013 at 21:39 | comment | added | Niels Keurentjes | The reject ratio mentioned is a rough indication from what I've observed on a small subset of reviews 'backchecked' to see whether my own votes agreed with eventual consensus, I have no idea whether the 25-50% bandwidth really makes sense, it might also be 15-60% if you look at 99% of moderators. It's the other 1% that reject 2% or 95% that warrant looking at. | |
| May 12, 2013 at 21:33 | history | answered | nickhar | CC BY-SA 3.0 |