I failed this audit, and I'm not sure why. I hit "no action needed" because it seemed like a fine answer, but I was told that I failed the audit. I wasn't able to copy the exact reason why I failed, but I remember it saying that I should have upvoted the answer. I didn't have enough knowledge on the subject to decide whether it deserved an upvote or not, so what should I do in this situation in the future?
17wait, we are supposed to upvote anything that the community might love? This is especially bad if we are to fend off those fixmycodez questions.– John DvorakSep 29, 2013 at 12:49
@JanDvorak Downvotes on audits don't count (you can't vote on deleted content, and downvote audits are usually deleted), not sure what happens with upvotes.– Rob WSep 29, 2013 at 12:51
@interjay I'd be more willing to believe that I misclicked if it hadn't told me to upvote the answer.– fvrghlSep 29, 2013 at 16:01
1If we can fail an audit by not upvoting, should we also (in addition to upvoting) start flagging good answers that don't have enough upvotes yet? :)– Joshua TaylorSep 29, 2013 at 20:20
This isn't a rant, I'm just passionate about the reviews being a bit of a dark art with no solid info as to how to manage them.
Votes are for a positive or negative opinion on the answer or question, and votes shouldn't be included in how reviews are carried out or how audits are assessed.
When you lack the skills to assess a post, you should click on the "Skip" button.
I agree, but our lack of knowledge should not include whether to up or down vote. Knowing precisely when to vote and if up or down would require knowledge of the system's algorithms.
Voting is personal opinion based on a potential of hundreds of things a script cannot take into consideration.
Skipping a review because you're not sure whether to upvote or not is nonsense. We should not need to vote in a review for it to be perceived as done correctly.
It being a requirement is of course is reflected in this being an audit and fvrghl failed for not voting.
How are we to know when to vote, and when to up or down? In this case, because 13 others have? I cannot see the vote counts. What if I could, and there were 50 down and 63 up? In this case I'm surely fine to downvote?
Who knows. I don't, it's quite confusing as there's no decent info about it all.
Why was there a need to upvote?
And before anyone replies with "just skip then", that's counter productive.
One could answer all questions like this. "I can't get this script to work it returns errors"... " ah well, put it aside, leave it, do something else..."
Should fvrghl perhaps have left a comment because 20 other users did?
What about "it has enough upvotes already"?
What about, as voting is a personal opinion, "fvrghl disagreed and downvoted"?
How are we to know what is right and what is not?
If there's decent info/help topics somewhere, please link to it. I've searched a great deal and found barely any info about reviewing.
Why fvrghl was logically correct in their actions
Reviews should be about reviewing questions/answers to help out Stack and see if they have any nasties - spam, bad code, code not in code tags, self promotion, no code, wrong answer, something else.
fvrghl found this answer was:
- Not spam, self promotion or other nasty;
- Well constructed;
- Posted by a user with 92k rep;
- Code in the right place and neatly indented;
- Had 13 upvotes and was an accepted answer;
Whether fvrghl knows the scripting language and so can assess also whether the answer was perfect or not is not necessary here as we have 13 upvotes and an accepted answer. While 13 people can be wrong, on this site it's quite unlikely.
The above bullet points dictate the state of this answer perfectly and there is only one perfectly logical action to take:
No action needed to review this as it's a great answer and I wish more were like it on SO
and if we want to vote or not that's surely our prerogative, up or down?
By voting "No Action Needed" you were indicating the answer was of sufficient usefulness not to downvote, but was not good enough to upvote.
Eh? Logically, and without info on reviewing, it can also be:
"By voting "No Action Needed" you were indicating the answer was not bad enough to downvote"
And in this particular case it is quite true to state:
"By voting "No Action Needed" you were indicating the answer had enough upvotes (13)"
Either way, if what you say is indeed the case, then we most certainly need more info about reviewing, as I do not see reviews are to be used for voting, nor do I know how to work out what the system expects of me.
Besides, by the very fact the system failed fvrghl for NOT upvoting meant it was indeed a fine answer and so fvrghl was correct in their evaluation that this review (audit) was a good answer and so "no action needed".
I didn't have enough knowledge on the subject to decide whether it deserved an upvote or not
When you lack the skills to assess a post, you should click on the "Skip" button.
1Are we supposed to upvote anything that the community might love? This is especially bad if we are to fend off those fixmycodez questions. Sep 29, 2013 at 12:49
17Personally I think a vote can be a satisfactory way to review a post, but it should never be compulsory for the review - IOW you shouldn't have to up vote a new post to pass a review.– slugsterSep 29, 2013 at 12:53
9Isn't determining if an answer is valid is separate from determining if it is good? For instance, if I don't know anything about language Z, but someone posts a detailed answer to a question about language Z, I can at least asses that it worthy of being on SO. I should have to pass on it because I don't know about language Z, and I shouldn't have to vote on it.– fvrghlSep 29, 2013 at 13:24
1@fvrghl I'm sure that most will agree that the answer in your case looks acceptable. In general, however, when you're unfamiliar with the topic, it's better to skip the review and pass it to the next reviewer who might be able to make the answer even better. (Personally, I wouldn't have upvoted the answer either, because I'm not familiar with the topic and it already had 13 upvotes; I would have either clicked "No action needed"or "Skip").– Rob WSep 29, 2013 at 13:40
1The purpose of the reviews is not to judge the correctness of answers. Just as moderators can handle "not-an-answer" flags without knowledge of the specific topic, reviewers can judge if a post is too low quality. If reviews are intended check for low quality in this sense, than you shouldn't be expected to judge the posts correctness. Sep 30, 2013 at 0:47
This is an unfortunate situation, but your failure was correct.
First and foremost:
I didn't have enough knowledge on the subject to decide whether it deserved an upvote or not, so what should I do in this situation in the future?
The emphasis is mine and is the key point here. This means you should skip, skip, skip and skip some more.
When judging questions, you don't always need to be well versed in the topic to decide whether or not to upvote as voting on questions in part is based on the apparent quality, effort, research, and conformance with the question asking guidelines. But when reviewing answers, it is much more difficult to vote appropriately when you don't understand the topic. How do you know the answers the question or contains technically accurate information?
From your comment on RobW's answer:
Isn't determining if an answer is valid is separate from determining if it is good? For instance, if I don't know anything about language Z, but someone posts a detailed answer to a question about language Z, I can at least asses that it worthy of being on SO. I should have to pass on it because I don't know about language Z, and I shouldn't have to vote on it
The tooltips on the voting buttons I think answer this.
The tooltip on the question voting buttons say:
- Upvote: This question shows research effort; it is useful and clear
- Downvote: This question does not shows any research effort; it is unclear or not useful
Whereas the tooltop on the answer voting buttons say:
- Upvote: This answer is useful
- Downvote: This answer is not useful
Just based on the tooltips, there is a significant difference in the voting guidelines for questions and answers. For questions, you are encouraged to judge the quality on clarity and research effort, in addition to usefulness. However, for answers, you are encouraged to vote on usefulness only.
From this idea, you should not be voting on apparent quality when reviewing answers, nor are you supposed to be judging research effort or clarity. You aren't assessing appropriateness for SO, especially in the First Posts and Late Answers queue. You are assessing the quality of the answer based on the usefulness. And how is a detailed but factually incorrect answer useful?
Now if the post was clearly inappropriate (spam, offensive, a "Thanks" or "Me too" post, or another question), then you should flag it for moderator attention, but just because it is not inappropriate doesn't mean you should take no action.
By voting "No Action Needed" you were indicating the answer was of sufficient usefulness not to downvote, but was not good enough to upvote. Or if you encounter a decent quality post that you don't think deserves more upvotes. Neither situation appears1 to be the case here (the reason for the latter is if it were a review, you probably would have seen the post with just a 0/+1 score and not it's real score.
1 - I can't say that the post was actually useful myself, but judging by the quantity of upvotes, it would appear to be quite useful.
9While I agree with your main point; that the OP should have skipped, I think it is fundamentally wrong to require an upvote in an audit. If the premise for that part of your answer holds, we shouldn't have
No Action Needed. 1) Your vote is your own. 2) When reviewing we are primarily looking for material that should go, which means that negatively we have the option of editing, downvoting and flagging. Positively we only have upvote or
NANfor neutral. To genuinely decide whether an answer like this is usuful, you'll need to know the subject matter well or run and compile it.– user213634Sep 29, 2013 at 14:33
NANwas actually created as an alternative to robo-upvoters. By following links in this question, it is obvious that the overall take on whether to vote from queues or not has not exactly been fixed. They do encourage it in late answers, but I still think auditing for it is excessive.– user213634Sep 29, 2013 at 14:36
3@AndersUP I don't necessarily disagree that requiring an upvote to pass a review is a fundemental problem, the problem is how do you judge the review if NAN is considered passing. If NAN is neutral and is considered passing for a good post, then it should be passing for both good and bad posts. But I have trouble declaring "NAN" a positive review so it is considered a positive action for "high quality" review posts. It is more a fundamental issue with the review process and auto-selecting random posts for reviews. Reviews should be obvious to someone reading the whole post, not judgements Sep 29, 2013 at 14:59
@AndersUP to clarify since I don't think my objection to NAN being a positive review was apparent after rereading this - if NAN is a valid positive review then you are giving robo-reviewers a free pass on most positive posts in the FP and LA queues since they normally just hit NAN anyway. Sep 29, 2013 at 17:51
4And that is the inherent problem with
NAN- it's a free pass. And I think I understand why they would include 'upvote' posts for audits, but requiring a vote is against the usual preaching that your vote is your own - and a vote is subjective: What is a useful answer for you may not be a useful answer for me even if it is objectively correct. This is also one of the reasons the system allows multiple answers to the same question.– user213634Sep 29, 2013 at 18:42
1But essentially, I agree with you. Perhaps the solution is to suggest that
NANis removed again. But then they might run into a lot of headless upvotes, which will bias some (potentially mediocre) answers to be heavily upvoted.– user213634Sep 29, 2013 at 18:43
1@AndersUP Definately. I think the current implementation is perhaps the least of the possible evils. We occasionally hit an "upvote is required" review and we don't skew votes with robo reviewers Sep 29, 2013 at 19:39
How is NAN any more a free pass than just upvoting the post? Doesn't the current behavior make "upvote" the free pass? (And upvoting is the potentially more harmful action compared to NAN) Sep 30, 2013 at 0:35
@sth from an audit stand point requiring an upvote is a bad idea, but allowing "NAN" as an acceptable review is also a bad idea in my opinion. The point to audits is to trap robo reviewers and the NAN feature was added to give them an action that was not harmful to the site. If you make NAN an acceptable review then you just let those users get a pass on the audit without trying. As I mentioned in an earlier comment, neither situation is good, but it seems that rejecting NAN as a valid positive review will trap more robo reviewers. Sep 30, 2013 at 0:56
@psubsee2003: Currently "upvote" gives them a pass on the audit without trying. Is that better? And is it good that the audit teaches them to upvote instead of clicking NAN? Sep 30, 2013 at 1:00
@sth Blindly upvoting will cause you to fail the negative audits. Incidentally, they would be the same where
NANfails. Or, so I thought until this thread, where
NANcan also cause you to fail the positive audits, making votes the safer choice for robo-reviewers.– user213634Sep 30, 2013 at 9:26