15

Someone suggested this edit to one of my posts. It fixed an actual problem in the code; I had used the wrong variable in the loop, which meant my code wouldn't work. It was just intended to be a quick skeleton of an example, but still, it's nice for what's there to be correct.

Three people reviewed this and rejected it as "too minor"; one approved it. This seems to be a pretty bad decision; the review in question did fix a substantial problem in the code. In a code example, a single wrong variable can be a substantial problem; it's not minor at all.

It seems that the way the review system is set up, it encourages lots of people to do reviews (in order to get a variety of badges), but has no way to provide accountability to ensure those reviews are good. People can do reviews once they've hit a particular reputation barrier, and when you do reviews, it isn't linked at all to tags that you're active in. Now, for reviewing formatting and textual edits, that's probably fine; people with enough reputation presumably know how to use StackOverflow reasonably well and can review potential changes well. But for code changes, it generally requires someone who's familiar with the language or API in question to be able to do a good job evaluating if the fix is actually correct or not.

For answers, these problems are solved by voting and tag filtering. People generally only answer questions that they know something about, and can use tag filters to narrow in on those questions; and if they answer poorly, they are downvoted.

By emphasizing quantity of reviews with no accountability, and giving no mechanism like tag filtering so reviewers can focus on fixes in areas that they have expertise in, I feel like we're going to get a lot of bad reviews.

It looks like there's a "review audit" feature that is designed to deal with poor reviews. I can't see how I can audit this particular review, though; and again, if the people auditing a review don't have expertise in a code-change review like this, they may not be able to answer well either.

|improve this question
  • 11
    In general, it's better to leave a comment on a post explaining the code error and how to correct it, than it is to fix the code with an edit directly. You don't always know the poster's exact intent, and if the code is in a question, you're basically invalidating the question. The reject reason was incorrect; the correct rejection reason should have been that it "changes the meaning of the post." – Robert Harvey Dec 18 '12 at 19:33
  • @RobertHarvey In this case, the code was in an answer, so my answer was incorrect. The way the review was phrased, it probably should have been a comment (as the review text seemed to be asking if the edit was correct), but it was still a valid fix, and it would have been better to apply the change than reject it. – Brian Campbell Dec 18 '12 at 19:35
  • 1
    Quis custodiet ipsos custodes? – MikeTheLiar Dec 18 '12 at 19:35
  • 5
    Note that I deliberately didn't mention the validity of the code change. – Robert Harvey Dec 18 '12 at 19:36
  • 1
    Related: meta.stackexchange.com/a/153279/186381 – Servy Dec 18 '12 at 19:38
  • @mikeTheLiar It's true, you can only go so far on watching the watchers. I guess my point is that while we do have a way of watching the watchers, none of it emphasizes being familiar with the content of the question or answer at hand. I know that I've gone through the review queue, and had to skip many edits that I just didn't know enough about to determine if the edit was correct. – Brian Campbell Dec 18 '12 at 19:39
  • A review of review has been suggested many times before. Fixing /review is the topic of the month (and has been for a few months now). You haven't addressed any of the criticisms of the suggestion here, such as that any problems with reviews will still exist in a review of reviews for exactly the same reasons. – Servy Dec 18 '12 at 19:41
  • @Servy: While Shog9 says this is bunk, he also says that you have to be sure, and if you can't be bothered to check your facts and explain your change, it's better to hit the skip button. – Robert Harvey Dec 18 '12 at 19:41
  • @RobertHarvey Yep. I've read the post. His point is that the validity of the change isn't moot; its quite relevant, and since the change is indeed correct in this case (as admitted by the post's author) it shouldn't be rejected for the reason you are proposing. – Servy Dec 18 '12 at 19:43
  • 1
    @Servy: One of the fundamental principles of moderation is that you shouldn't have to know the technical details of a post to determine its veracity. That's what votes and comments are for. – Robert Harvey Dec 18 '12 at 19:45
  • @RobertHarvey That's what I asserted in the linked post, Shog, and the other voters in that thread, felt otherwise. – Servy Dec 18 '12 at 19:46
  • @Servy I think that you could do better with direct votes or comments on reviews (so that in this case, I could give feedback to the reviewers themselves), and with some kind of tag filtering of reviews (maybe have it only apply to edits which are marked as "content changing" or which edit actual code). – Brian Campbell Dec 18 '12 at 19:47
  • @BrianCampbell Those are also things that have been suggested a number of times since the recent review changes. I suggest you spend some time reading up on the recent discussions on that topic before making your own proposals, to avoid even more duplication. This is the primary thread on the topic. – Servy Dec 18 '12 at 19:49
  • 2
    @Servy: Shog9 writes that "Editing mistakes in code is no different than editing mistakes in grammar or spelling in this regard." Well, he's entitled to his opinion, but I don't consider that at all self-evident. It's one thing to fix spelling and grammatical errors; it's quite another to modify code that has specific meaning to a computing machine. – Robert Harvey Dec 18 '12 at 19:50
  • @Servy Thanks for the pointer. I'll have a read-through of that thread. – Brian Campbell Dec 18 '12 at 19:52
8

The way you review a review is how you are doing it right now.

If a review is arguably bad, post a question on Meta so that we can discuss it. Checking up on things like this is the bread and butter of Meta.

|improve this answer
  • 1
    Well, sure, but there's no way to know if the reviewers will actually see this. Posting on meta is the catch-all; it might be nice to provide direct feedback. Again, you can get a little too meta; feedback on a review of an edit on an answer to a question. I was just wondering if there was anything better. – Brian Campbell Dec 18 '12 at 19:42
  • 1
    You can leave a @user comment to the reviewer in the post that was reviewed, pointing them to the Meta post. – Robert Harvey Dec 18 '12 at 19:46
  • 2
    Do @user comments work for users who haven't ever participated in the thread? I didn't realize that; I thought they only worked for people who had been directly involved in that question somehow. – Brian Campbell Dec 18 '12 at 19:48
  • 1
    @BrianCampbell @ comments do notify any users who have edited a post or suggested an edit to the post, it just won't auto-complete the name for you. – Servy Dec 18 '12 at 19:50
  • @Servy Does that include users who have reviewed (and rejected) an edit? That's who I'd be trying to communicate with. And you can only notify one person per comment, right? So I'd have to post one comment per reviewer? – Brian Campbell Dec 18 '12 at 19:54
  • @BrianCampbell No, users who have reviewed an edit are not notified through @ replies. – Servy Dec 18 '12 at 19:55
  • 1
    You'd have to find a post from each of the reviewers, and @notify them there. – Robert Harvey Dec 18 '12 at 19:56
2

I agree.

Why not introduce a second layer of reviews like this to help catch garbage reviews that are creating so much frustration in the community?

Review audits are a problem of their own, and they're traps designed to catch offenders who do garbage reviews all the time. They do not help address specific edits, fix them, and inform the original reviewer about what they went wrong.

I don't buy into the "yeah but who reviews the reviews of the reviews" meme. The vast majority of reviewers do decent work. Establishing a second layer of reviewing will vastly improve quality simply because the likelihood of an item having to crappy reviewers decreases. Also, you're likely to have a very keen eye when asked to review somebody else's decision.

|improve this answer
  • All that said, I'm sure there must be a bigger discussion about this very subject somewhere that I haven't found, no? If there is, please point me to it. – Pekka supports GoFundMonica Feb 12 '13 at 20:56
  • According to Servy in the comments above, the bigger thread is here. – Brian Campbell Feb 12 '13 at 21:14
  • @Brian thanks. That thread is arguably a mess though – Pekka supports GoFundMonica Feb 12 '13 at 21:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .