7

I propose to add a requirement for the or to help reviewers "Learn to love that Skip button".

I believe that ignoring "Skip" in current badge requirements tends to encourage sort of the blind spot about using this option in reviews ("Avoid! No progress on my badge!").

As the popular opinion is that There is no shame in using "Skip", it would be nice if some of the badge requirements reflected this, and rewarded users for making the correct choice when they are not sure on the review action to take.

I think Skip should be used to make a badge harder to achieve and teach reviewer a good habit along the way, for example:

awarded for 1000 reviews and no less than 25% skip actions

or

awarded for 250 reviews and no less than 25% skip actions

Meaning that, in the case of the Steward badge, one has to perform at least 1000 reviews (just like it is now) and, additionally, fewer than 25% skips would block receiving a badge.

I personally was incentivised by the prospect of earning badges to pay the review queues some attention. I think that this proposed change would teach users like me (who don't set out to abuse the system) the importance of using "Skip" to review correctly.

This would work in a similar manner to the currently works:

Voted on 600 questions and 25% or more of total votes are on questions.

Electorate encourages voting on questions, to promote good ones worthy of attention, helping negate the bias of just voting on answers which helped you personally.

Hopefully adding this similar requirement to the Steward or Reviewer badge would help teach users to be less reluctant to press "Skip".

Improve this question
37
  • 3
    Keep in mind, this wouldn't be quite as applicable to smaller sites. I do happen to skip a lot of reviews on ELL, but out of laziness, not out of lack of knowledge. I'm also a little concerned over just making the metric a flat number to skip, rather than making the criteria a litle finer-grained. As implemented, it wouldn't change robo-reviewing much. One would merely have to modify the workflow to skip 1 in 4, which would be utterly trivial. (Obviously skipping exactly 1/4 would be noticeable, but you get what I'm saying.)
    – Jonathan Garber
    Commented May 22, 2014 at 13:04
  • 3
    @JonathanGarber ...as for robo-reviewers, my understanding is these are expected to be stopped by audits, the proposed feature is to better educate responsible reviewers and help them learn better habits
    – gnat
    Commented May 22, 2014 at 13:15
  • 1
    @gnat: But that's a good point about the audits. Totally forgot about those since we don't have them on beta sites.
    – Jonathan Garber
    Commented May 22, 2014 at 13:16
  • 1
    @JonathanGarber the goal here, here as I see it anyway, is to educate users who aren't deliberately trying to abuse the system on how to responsibly review.
    – Amicable
    Commented May 22, 2014 at 13:35
  • 2
    One other concern: with proper usage of filters, skipping can become unnecessary. 25% may therefore be too high
    – mhlester
    Commented May 22, 2014 at 13:39
  • 2
    @gnat: I guess behavior shaping with badges isn't terribly unusual, so fair point.
    – Jonathan Garber
    Commented May 22, 2014 at 13:41
  • 7
    That would be incredibly annoying. Anyone can just click Skip a bunch of times until they hit 25%. That would be a completely pointless modification that would change nothing.
    – animuson StaffMod
    Commented May 22, 2014 at 13:45
  • 3
    @animuson I think you are confusing the desirable thought of preventing robo-reviewing from being possible at all, with the goal of this change which is putting emphasis on the importance of using "Skip" in proper reviewing.
    – Amicable
    Commented May 22, 2014 at 13:49
  • 8
    Robo-reviewers don't care though. They will just do whatever to reach 1,000 and then start clicking skip to meet the requirements. All you're doing is inhibiting the real reviewers who may have a legitimately low skip rate.
    – animuson StaffMod
    Commented May 22, 2014 at 13:51
  • 4
    I'd support a skip requirement for the bronze level badge, but not for the gold level badge. At that point, you should already know that skipping is a good thing.
    – fbueckert
    Commented May 22, 2014 at 14:02
  • 1
    +1 for turning the negative experience of yesterday's post into the positive one of today.
    – mhlester
    Commented May 22, 2014 at 14:06
  • 2
    @fbueckert "just skip everything" isn't what this feature request asks about. Skips alone won't make any badge, just like it is now, no amount of skips would - 1000 reviews would still be required to get a badge
    – gnat
    Commented May 22, 2014 at 15:00
  • 3
    @gnat "awarded for 1000 reviews and no less than 25% skip actions". Skips are being classified as a review. Which means 1000 skips = gold badge. If that's not what is meant, I think it should be clarified.
    – fbueckert
    Commented May 22, 2014 at 15:25
  • 2
    This would be about 500 times better of a question if it were a discussion on how to encourage use of the 'Skip' button more. Desired behavior is great from where I sit, the details of execution seem to be attracting downvotes though.
    – jmac
    Commented May 23, 2014 at 9:29
  • 5
    funny to see how a senseless comment "That would be incredibly annoying" from a user having quite a limited experience (mere 4K reviews, all at at a single site, less than half of what I have in just one of 15 queues) gathers upvotes from mindless readers only seeking for something to justify their "gut reaction" to vote down. Oh well, meta as usual
    – gnat
    Commented May 23, 2014 at 13:24

2 Answers 2

Reset to default
29
+150

I wasn't going to go out on a limb but since a comment has been made...

That would be incredibly annoying...

...I would like to address that.

I performed over 20,000 reviews total in about 20 queues at 4 different sites. I am well above Steward badge limits in 4 queues and continue reviewing in these (primarily because it goes so easy), and in any other queue at all.

  • And all of my experience has been that "Skip" makes a wonderful grease, allowing one to avoid getting burned out when doing large amounts of reviews.

I aim to act on review when I can clearly explain to self what the post is about and what review action is desirable for it. Apparently, this is not always the case (typical example is when post is out of my area of technical expertise).

  • What else but Skip can let one get out of such a deadlock? clicking on a randomly guessed action, like robo-reviewers do?

But getting out of total deadlocks is not the only case where Skip is incredibly useful, I'd say not even the most frequent one.

It often happens to me that evaluation takes more time and effort that is comfortable, review queues contain substantial amount of posts that aren't really clear cut for the action. One option in cases like that is to keep trying to grok it, frying my brain until I am ready to act.

How long can it take, 5, 10, 20 minutes? what's the use of it if after such a brainstorm I am tired and unable to do quality reviews anymore and just looking at review page makes me sick? how much is it better than total deadlock described above, does a single extra review makes it better if I can't go on anymore...

  • ...is it worth it? Of course not, and here, Skip is the way to go, I bet all 20K reviews I've done so far.

"But wait!" I hear you say. "That post you skipped, it's out there in the wild and is not taken care of, orphaned and forgotten." This is, of course, nonsense.

In active review queues, there are always other reviewers to pick it after you, either better qualified or simply less worn out, maybe even because you cleared the road for them by quickly working out more items off the queue.

In low traffic queues, the post will eventually return back to you again... but (and this is a very very BIG BUT) it ain't gonna be like first time, trust me.

First time opening a post for review, I am a little bit anxious, because I don't know what's there. But when there's post I already seen, it's a relief ("hello buddy I know you!") and this really helps a lot in getting your brain ready for a harder work. Also, if you think of it, the very break between encounters increases your chances to get to decision, compared to permanently banging your head against that post.

Combined with lowered anxiety, this break does wonders to my mind, I often find myself wondering what made me stuck at prior time, as things look really clear and simple. And, in rare cases when it still doesn't work out well, the way to go is, as usual, Skip.

  • Skip is what lets me do reviews instead of frying my brain. Only filtering 1, 2, 3 has more impact on productivity than using Skip.

Skip is... "incredibly annoying"? give me a break. The only reason I can see for someone to honestly think like that is Dunning–Kruger effect.

Improve this answer
7
  • I fail to see how this makes adding skip to a badge a good thing in any sense. All you're saying is, "it helps me prevent burnout". If you're doing that many reviews, stop reviewing.
    – fbueckert
    Commented May 24, 2014 at 3:17
  • 2
    @fbueckert I am doing many reviews and that doesn't cause burn out, and using Skip helps in that. If you look at the data, you'll see that many (way too many) reviewers burn out at really small amounts of review. I support adding skip to badge requirement because I hope it will help others learn to review productively
    – gnat
    Commented May 24, 2014 at 4:32
  • 2
    Perhaps it would be better (if there's a need to do anything) to add a bronze badge for first skip (per site, not per queue). I remember trying to avoid skipping questions when first reviewing - I don't remember what guidance the site gave if any, and it aught to be considered the default action.
    – AD7six
    Commented Jun 23, 2014 at 8:27
  • @AD7six dedicated badge would certainly help reviewers recognize an importance of Skip and is probably easier to implement. I'd still prefer the way suggested here though as I believe it would better help them build a habit of regularly using Skip (this is mostly based on my own experience with Electorate badge which has somewhat similar additional requirement to incentivise one to regularly vote on questions)
    – gnat
    Commented Jun 23, 2014 at 8:44
  • I've read your answer a couple of times and tbh apart from not liking the mentioned comment I'm not sure what your stance is (I agree with everything written, but it's not addressing the "question"). You are for awarded for 1000 reviews and no less than 25% skip actions? You are pro "you must skip x reviews to get this badge"? skip % is going to be something that varies wildly from person to person. IMO there's nothing necessarily wrong with a reviewer having a skip% of 90% or 5% if they are using, or not, the button as intended. The only value that could be considered wrong to me is 0%.
    – AD7six
    Commented Jun 23, 2014 at 8:57
  • 1
    @AD7six as I wrote, my answer mostly focuses on stressing importance of Skip action because that was challenged in comments. As for concrete implementation of the badge, the one suggested in feature request looks okay to me; per my reading it means one needs to do same amount reviews (actual reviews, not Skips) as now, plus some substantial amount of Skip actions. I do not have strong opinion on what amount to consider substantial, I'd only want it to be large enough to help a (responsible) reviewer to get a habit of using Skip without hesitation
    – gnat
    Commented Jun 23, 2014 at 9:07
  • related support request: Looking for stats on how frequency of skip actions correlates with amount of reviews
    – gnat
    Commented Jun 26, 2017 at 9:29
2
+100

25% seems steep. Do we honestly expect people to not know what to do that often?

It also means that if you are currently on your 990th review without having skipped too often, when this is enacted, you will now have to make yourself ignorant of a higher percentage of review tasks to balance it out with your previous track record. I.e. a person coming in fresh would be able to get this badge at 1000 reviews if they reasonably use the skip option, but if you already have a significant number of reviews without so much skipping (which could happen even with responsible reviewing) then you will now have to be proactively useless and skip questions to which you know what to do in order to fill out your skip quota. Holding at 990 reviews, with lets say 50 skips, you would then need to have 280 consecutive skips to bring it up to 25%. You are then promoting inaction.

I think it would be more reasonable to pick a (low) number, not a percentage, in order that the skip-quota could be filled without having to refrain from addressing edits that I do know how to deal with. 75 skips seems reasonable to me, or even 50.

Another alternative would be to make a number of skips part of the bronze badge requirement, and then make the bronze badge a prereq of the silver/gold badge, and don't make it retroactive for the bronze badge. This will at least work for new users, without penalizing those who have already done significant reviewing.

On further thought, though, I think that the problem should be looked at differently. Robo-reviewers will be robo-reviewers. You can't make skipping count towards the badges, because then they will become robo-skippers. However, those who are genuinely reviewing (and haven't gotten the badge yet) may be disinclined to skip because of their mission to get the badge. Maybe instead of encouraging skipping, we wan't to remove the discouragement of skipping. So, maybe a certain amount of skips could count towards the total - don't require skipping, but allow it to an extent towards the badge. I.e. let's say that the silver badge could have 25 skips included and the gold could have 100. The robo-reviewers won't be changing because they don't care which option they select. Those who review responsibly and know what to do won't change, because you haven't imposed skipping on them. Those who are trying to review responsibly but their badge hunger gets the better of them will now be able to have some reasonable leeway to skip without "missing out" on their badge-quest. Granted, you won't "teach" the value of skipping, but at least you remove the motivation to avoid skipping.

Improve this answer
9
  • 2
    one of suggested options is adding skip requirement for silver, not a golden badge ("awarded for 250 reviews and no less than 25%"), what's your take on this?
    – gnat
    Commented Jun 29, 2014 at 21:12
  • @gnat same equation with different numbers. If you have 240 reviews with 20 skips, you now have to do 60 consecutive skips.
    – Y     e     z
    Commented Jun 29, 2014 at 21:16
  • I understand that, 25% looks too steep to you in silver badge either, correct? what amount would make better sense then? or, you strongly prefer to have skip req in golden badge, these very 50-75 skips you mention?
    – gnat
    Commented Jun 29, 2014 at 21:23
  • 1
    @gnat My main point is that a percentage will require over-compensating (meaning reducing things other than skipping in order that the skipping will become a significant percentage). A set number would not have that problem.
    – Y     e     z
    Commented Jun 29, 2014 at 21:34
  • understood, you're maybe right. "I do not have strong opinion on what amount to consider substantial, I'd only want it to be large enough to help a (responsible) reviewer to get a habit of using Skip without hesitation" (quoting self:)
    – gnat
    Commented Jun 29, 2014 at 21:38
  • @gnat upon further consideration, see edit.
    – Y     e     z
    Commented Jun 29, 2014 at 21:43
  • I see. Consider that "Allow “Skip” to count towards Steward badge progress" hasn't been well received
    – gnat
    Commented Jun 29, 2014 at 21:46
  • 1
    @gnat I suppose the one caveat that I added was to cap it. Not to make it the same as approve/reject, but to allow a small amount to count. But point taken.
    – Y     e     z
    Commented Jun 29, 2014 at 21:55
  • 1
    ahh cap... I see now - that makes a difference! Rephrasing that unfortunate old request, "Allow a limited amount of “Skip” to count..." That would be an elegant solution, and I can't find flaws in it
    – gnat
    Commented Jun 30, 2014 at 7:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .