7

Put an accepted answer into production code for a commercial website, which was then subjected to pre-production testing. A significant, easily-reproducible bug was found, tracked down and fixed. The fix was edited into the accepted solution, which was rejected during peer review. It seems unlikely that the peer review involved any testing of the validity of the edits. The reason given for rejection appears to have been purely pro-forma.

What now?

This is the q/a:
https://stackoverflow.com/posts/9220653/revisions

13
  • 2
    Links please.,~
    – chown
    Feb 10, 2012 at 15:01
  • 35
    You seriously put untested sample code into production? Now I know there's no rule or such that prevents anyone from doing so, but, you know, we programmers have our own breed of what others call "common sense"... Feb 10, 2012 at 15:02
  • Can you show a link to the answer in question?
    – Pekka
    Feb 10, 2012 at 15:03
  • stackoverflow.com/posts/9220653/revisions is this what your refering to?
    – chown
    Feb 10, 2012 at 15:03
  • 2
    How exactly is this a bug?
    – jrg
    Feb 10, 2012 at 15:05
  • 2
    @BoltClock'saUnicorn note OP wrote bug was discovered at "pre-production testing" not in production
    – gnat
    Feb 10, 2012 at 15:10
  • 3
    @gnat: Makes this even more perplexing. How can production code be subjected to pre-production testing? Feb 10, 2012 at 15:11
  • @BoltClock'saUnicorn agree the wording sounds slippery indeed. Since pre-prod testing is mentioned, most likely OP refers to "production candidate code" but I am not a mind reader to guarantee that
    – gnat
    Feb 10, 2012 at 15:16
  • 6
    Who cares whether it's pre-post-or almost-production. The question is fair, isn't it?
    – Pekka
    Feb 10, 2012 at 15:25
  • 7
    This really isn't that confusing...he's trying to emphasize that the bug was found by professional testers, and explaining why rigorous testing was involved by mentioning that the code will be going into production for a commercial website (obviously it hadn't yet -- most sites don't wait until after the code is actually on the production servers to do the testing) Feb 10, 2012 at 15:26
  • 9
    I'll bite - I am pretty sure I voted to reject the edit - it amounted to a pretty cryptic screed in a comment inside a code block about how this answer was wrong + modifications to some of code. It looks to my eyes like an invalid edit. It should have been posted as a separate answer and a comment left on the offending answer pointing out what was perceived to be wrong with the answer.
    – talonmies
    Feb 10, 2012 at 15:52
  • 4
    @talonmies yeah, rejecting the edit was perfectly fine IMO.
    – Pekka
    Feb 10, 2012 at 15:54
  • I sure hope you get your money back. Feb 14, 2012 at 19:45

3 Answers 3

33

Post your own complete solution pointing out the bug in the accepted answer, then post a comment on the accepted answer referring back to your corrected answer. If the correction is valid, then it should either be edited into the accepted answer, or the new answer should be accepted instead.

1
  • 10
    This is the correct course of action. Also be aware that answers aren't guaranteed to be bug-free - this is why comments and new answers are allowed. Peer review didn't necessarily fail in this case - any significant change to an accepted answer with lots of comments indicating that the answer as-is worked, suggests that changes should be made with care. In this particular case a comment or new answer was more appropriate.
    – Pollyanna
    Feb 10, 2012 at 15:53
12

The fix was edited into the accepted solution, which was rejected during peer review. It seems unlikely that the peer review involved any testing of the validity of the edits. The reason given for rejection appears to have been purely pro-forma.

I saw, but did not take any action on, your second suggested edit. My opinion was "well, this might well all be true, but I don't think much of the tone. I'll let this be someone else's problem[1]".

As to what did eventually happen to your suggested edits, as the faq says,

What is reputation?

Reputation is a rough measurement of how much the community trusts you; it is earned by convincing your peers that you know what you’re talking about

When you come along with three days of membership and no more than the starting amount of rep, it's not particularly likely that peer review of your suggested edits will necessarily involve rigorous testing of the change; and (as might be learned from experience of suggesting edits), nearly all rejections of suggested edits will be with reasons that are "purely pro-forma", because the volume of suggested edits is such that it's impractical to do much more.

None of the answers here are offered with any warranty or guarantee as to fitness; therefore it's not really surprising that no major fire alarm was instituted when mistake was reported to have been discovered in one. There's no SLA here, or response time guarantees, or anything like that: just a bunch of volunteers who, like any group of humans, sometimes make mistakes.


[1] I'm allowed to think like this, because I'm a volunteer

11

I always reject edits that attempt to fix technical errors in the answer. If the answer includes code that doesn't check for something, and I see a suggested edit that adds the check, I reject the edit as

This edit is incorrect or an attempt to reply to or comment on the existing post.

Comment on the answer if you have the rep for it, or add your own answer if you don't. (Your own answer should not say just "the answer by Bob has a horrible flaw"; it should present the correct answer with the flaw fixed.) Edit-reviewers can't tell if you're right that it might leak memory or delete a good file or whatever the horrible problem is, so editing the accepted answer is not the right mechanism for fixing the Internet in this case.

5
  • 1
    Problem is nobody ever rigorously tested the original code. I did test it. Hopefully, the end goal is to produce correct answers, not to follow some proforma rules. An intelligent peer review, as practiced by academic journals for example, is based on the sense of the material being reviewed, not on abstract, automatic rules.
    – kcrossen
    Feb 10, 2012 at 16:19
  • 7
    I am not arguing with your review. Only with your choice of "suggested edit" as the way to submit your results. Would you use a tweet to fix a scientific paper? And expect it to work? Feb 10, 2012 at 16:25
  • 6
    @kcrossen: you are missing the point. Stackoverflow is the ultimate meritocracy - answers are posted and voted on according to their accuracy and efficacy. If you find an answer to be deficient vote it down. That is what voting is for. If you have what you believe is a better or more correct answer, post it as your own answer and let the meritocracy do its thing. That is what passes as peer review around here, not what you did.
    – talonmies
    Feb 10, 2012 at 16:28
  • 1
    @kcrossen in fact you did just what you say no-one did. You tested the code. Yay! We're all volunteers and you volunteered. I'm glad. Then (because our mechanisms are sometimes opaque) you didn't submit your results the right way. No harm done, you know the right way now. Please keep on reviewing and fixing. Feb 10, 2012 at 16:34
  • 1
    I tried many different approaches (other than reading a LOT of faq, that is). I tried to comment on the original solution, not possible, and so on, you probably know better than I. So I asked what to do, got my answer, and amended my answer accordingly. Thanks.
    – kcrossen
    Feb 10, 2012 at 16:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .