What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

It was already discussed before with some suggestions (even from me) but here is a final and simple suggestion that in my opinion will hunt down the badge hunters.

Now that the number of reviewers was raised to at least 3 on Stack Overflow, when there is a suggested edit rejected by three and approved by one user we can assume for almost certain that the one user was utterly wrong; either badge hunting, random vote just for fun or whatever.

I suggest that same way that users are blocked from suggesting further edits for X days after Y suggestions are rejected, users who were in 1-3 minority in the review for A times in a row, will be blocked from reviewing any further suggested edits for B hours, instead getting a friendly message.

Sensible numbers are 3 for A and 24 for B, but it doesn't really matter as long as something is done with those users.

Can't do without example so here they are. Same user, several really bad approvals. Exhibit #1 and Exhibit #2. Don't think I need to explain why those two edits are invalid.

Note: unlike other past suggestions, I'm not asking to check approve/reject rate of the user; it's totally fine to only approve suggestions without ever rejecting, as long as all the approvals are valid.

share|improve this question
11  
Probably a separate feature request, but I would love to see my own track record of reviewing to see where I am approving or rejecting edits that the majority agree or disagree, so i can keep tabs on my own review performance. –  psubsee2003 Oct 25 '12 at 13:06
    
@psubsee2003 you mean this? –  Shadow Wizard Oct 25 '12 at 13:08
1  
Not specifically, I do check that as I go now, but I was thinking of something more along the lines of a running total. Something like a quick 2 or 3 line statistic that shows 25% of rejected suggestions approved and 12% of accepted suggestions rejected. Or even more details so I can see how many I am the only accept vote or the only reject vote. –  psubsee2003 Oct 25 '12 at 13:19
6  
I know I said I wouldn't comment on this issue anymore, but... is that really TheTXI? (It sure looks a hell of a lot like it.) –  BoltClock's a Unicorn Oct 25 '12 at 13:20
2  
By the way, one of my old posts caught the eyes of a couple of rogue reviewers. Let it be known to the lovable meta-monsters here that I have now set up a little hit list of users who make bad reviews on my posts, starting with the two users who approved that edit. No word on publicity yet ;) –  BoltClock's a Unicorn Oct 25 '12 at 13:32
1  
@psubsee2003 interesting, but probably "too much". You can probably construct a query in here to find those exotic details. :) –  Shadow Wizard Oct 25 '12 at 13:38
2  
@ShaWizDowArd probably right, but would be useful for new 2K users who really want to see if they are doing things right. I didn't think of using DE, but since the data is from June and I didn't get to 2K until Sept, probably not going to help me too much initially. –  psubsee2003 Oct 25 '12 at 13:48
1  
@psu just sift through some of the reviews.. looking at 20 of those should give you good enough prospective. –  Shadow Wizard Oct 25 '12 at 13:51
1  
I think you need another variable - C would be the timeframe. "Users who were in 1-3 minority in the review for A times within C timeframe, will be blocked from reviewing any further suggested edits for B hours". Also, I think percentages would be better than hard numbers, since if you go through a very high number of reviews a day you are much more likely to hit the "ban" threshold than if you only go through a few a day. –  Rachel Oct 25 '12 at 15:22
20  
This has been suggested, in various froms, by several people in the main "how to fix reviews" question. When you assume that the majority was correct and the minority was wrong the system fall flat on it's face when there are a significant number of bad reviewers, often enough to even outnumber the good reviewers (at least some of the time). I can't count the number of times I made a good review and several others rubber-stamped the wrong review. This system would punish me for that. This system only works if there are very few bad reviewers. –  Servy Oct 25 '12 at 16:16
3  
It seems that so few people have noticed that these problems became much more pronounced with the new /review changes. Two of the big changes are adding badges, and a leaderboard to show who has done the most reviews. This has caused a lot of people who never cared to do reviews for reviews sake to start doing them just for the acknowledgement. Take away the acknowledgement and those reviewers will go away again. Yes, these problems all existed before those changes, but they were much less pronounced. –  Servy Oct 25 '12 at 16:40
2  
@Servy It's not just rewards, but oddly ease of use of the new review system. Previously, it took more effort to review. I wonder how visibility of both review items and their result comes in to play with the new review system. –  Jason Sturges Oct 25 '12 at 17:03
1  
@gnat good point; having that will indeed render this request pretty much obsolete. :) –  Shadow Wizard Oct 25 '12 at 19:49
2  
@Servy indeed, my request heavily relies on the majority of reviewers to be educated. So maybe wait a little until such thing is achieved.. :/ –  Shadow Wizard Oct 25 '12 at 20:05
1  
-1 If I get unlucky and have 3 morons doing a bad review 3 times in a row, I should get banned from reviewing? –  Stijn Aug 12 '13 at 7:14

5 Answers 5

This is all a probability game, right? Hypothetically, if 1% of reviewers are bad, then the chance of this working would be

99% * 99% * 1% * 3 = 2.94% (two correct reviews and one incorrect review)

while the chance of a false positive would be

1% * 1% * 99% * 3 = 0.03% (two incorrect reviewers overruling a correct reviewer)

The rest of the time, you get agreement, in which case the review doesn't count against anyone. Okay, not super-effective, but it would probably help some in the long run. If you increase the number of bad reviewers to 20%, the numbers become

80% * 80% * 80% = 51.2% (good result: correct review)
80% * 80% * 20% * 3 = 38.4% (best result: correct review and bad reviewer caught)
20% * 20% * 80% * 3 = 9.6% (worst result: wrong review and good reviewer caught)
20% * 20% * 20% = 0.8% (bad result: incorrect review)

How often do we get bad reviews in practice? Remember, the figure has to include not only idiot badge grinders but also well-meaning users who just aren't good at reviewing and occasional bad choices from good reviewers. Twenty percent doesn't seem unreasonable, and at that point, this feature would ding a good reviewer once for every four times it gets a bad one.

share|improve this answer
    
Sorry for being heavy, read your answer twice and still not sure I understand if you think it's a good idea or bad idea? –  Shadow Wizard Oct 25 '12 at 17:12
3  
I presented some numbers. It's up to you to decide whether a 1:4 ratio is acceptable or not (or whether 20% was an accurate estimate). If you really want my opinion: I think the proposal is a bad idea. –  Pops Oct 25 '12 at 17:19
    
OK valid point; now that I changed to "users who were in 1-3 minority in the review for A times in a row" the numbers greatly change; User who repeatedly act against the majority decision isn't likely to be innocent victim. –  Shadow Wizard Oct 25 '12 at 20:07
2  
@ShaWizDowArd That depends on what you fix all of the numbers to, and how accurate they are. If the number of poor reviewers is sufficiently high, or A is set to just a few items, you'll still get lots of false positives. As A goes up it will reduce false positives, but also increase false negatives. –  Servy Oct 25 '12 at 20:09

So what is the underlying problem here, given that we agree that there are poor quality reviewers in the queue frequently performing the incorrect review action?

Is the problem that posts are being approved when they should be rejected, or rejected when they should be approved? That's what I would assert is the main problem.

Another problem that some people seem to have is that people are getting shiny badges when they didn't actually do the work the badge is designed to recognize. Personally, I don't much care about this. I'd be willing to give everyone the gold reviewer badge for free if it meant perfect reviews (unfortunately it won't).

If we build a system by which we assume that the majority is correct, and therefore punish those not voting with the majority, then we're saying that all, or almost all, of the items going through the current review queue are being properly handled. Some people, here or there, are performing the wrong action, but it's never enough to actually result in the majority action being incorrect. That is the only situation in which it would be appropriate to punish the minority voter.

If that's the situation we're in then we don't even have a problem. The correct actions are being taken, so there is nothing to worry about. A few people might get an undeserved badge here or there, but the review process is still solving it's actual duty of improving the content on the site.

If we still have a problem then the problem is a result of there being enough poor reviewers that they actually make a majority decision to perform the wrong action, and they do it frequently enough to make a significant difference. This means that if we have a problem at all that this proposed solution won't actually help solve it. It would mean that "correct" reviewers would be punished and incorrect reviewers would not.

So if there is a problem it doesn't help solve it (or even makes it worse) and if there is no problem then well...there's no problem and we don't need to add a solution.

There is no case where implementing this proposed change would result in improving the site.

share|improve this answer

This does not work. The problem is in the fact that you are choosing to do this based on it happening three times in a row. This assumes that such a bad reviewer does most of his or her reviewer incorrectly.

However, when an edit is approved, the poor reviewer will have reviewed correctly and as such, the counter will be reset according to this plan. As such, you would need three consecutive rejected edits in order to be able to catch anyone in this proposed system.

In my experience the large majority of edits is approved, and as such it will rarely happen that three consecutive edits are rejected. This means this system does not work.


I decided to do some number crunching. I said in the comments I wasn't sure about how to do the math, but at that time I was trying to do some rules I was taught that I both didn't know too well and were hard (if at all possible) to apply to this problem. As it turned out, I just needed to step back and use some general math knowledge instead.

Basically, this involves three cases. First off, you might just have had two rejects last. In this case, you have either the chance p that it's done by the next edit review (p * 1) and the inverse chance that we're going to do have an approved edit and have to start over. (p is the part of the edits that get approved). If we have only just had our first reject in a row, then we have a p chance we reject another review and from there we go to the two rejects we discussed above, and otherwise we once again have an approved edit and have to start over. Finally if we start without any streak of rejected edits, have a p chance to advance to the previous case of having a streak of one, and otherwise we do a review and end up right where we started: without a streak.

Now I know that may not have been too clear, but maybe it'll be clearer if I put it in formulas:

M(n): the number of moves we'll expect before three consecutive rejected reviews
      when we currently have a running streak of n rejected reviews
M(2): p + (1 - p) * (M(0) + 1)
M(1): p * (M(2) + 1) + (1 - p) * (M(0) + 1)
M(0): p * (M(1) + 1) + (1 - p) * (M(0) + 1)

Substitution gives us:

M(0) = p*(p*(p + (1 - p)*(S0 + 1) + 1) + (1 - p)*(S0 + 1) + 1) + (1 - p)*(S0 + 1)

Which can be rewritten as (I renamed M(0) to M here as we're only looking at the whole thing from the beginning now):

M = p^2 + p + 1 + (-p^3 + 1) * M

The first thing I was going to calculate was how long it takes when we only assume that 50% of all edits is rejected. Filling in p = 0.5 gives you M = 14, which is exactly what the web has to say about the topic (just google consecutive coin flips). From there, let's add a bit more realism to our model.

Let's say that 45% of edits are unanimously accepted, 45% are unanimously rejected and 10% of all edits are disputed by true reviewers. Let's say that on those question, 50% of all valid reviewers votes to reject and 50% votes to accept. Now let's see what that means for our badge hunter.

A question that our badge hunter sees has him in it. The undisputed questions are simple, but the disputed questions are a little harder. The idea is that whenever he ends up voting to accept a disputed edit, we'll need the first three other reviewers that are reviewing the question to reject it, or otherwise it won't be a 3v1 and won't count against his streak. Like we did silently before, let's assume that there are no other malicious reviewers for now.

The chance of having three consecutive reject votes on a disputed question are 0.5 * 0.5 * 0.5 = 0.125. As such, 87.5% of all disputed questions will be combo breakers. We had 10% disputed questions, so that brings us to 8.75%. So our combo breakers are now 0.45 + 0.0875 = 0.5375, which means that p drops to 0.4625. Throwing p into our function gives us: M = 16.945, we're almost down to triggering the system only once a day.

Now let's add other badge hunters into the mix. If we meet another badge hunter on any question that would otherwise be a 3v1, it becomes a combo breaker, as he'll also accept it. Assuming one percent of the reviewers are bad apples, the chance of that happening is: 0.01 * 1 * 1 * 3 = 0.03 As such we have to decrease our 3v1 losses by 0.4625 * 0.03 = 0.013875. Now p = 0.448725. Fill it in and we get M = 18.2626.

With the maximum number of reviews per day being 20, this means on average each bad reviewer will get banned only once per day. This means that a single day ban won't do any real good for the system - I believe it may mean a decrease of badge hunter reviews by somewhere between 25% - 50%, but I don't feel like doing the math on that as well (nor do I feel like recombining this result with my previous results and take this into account for the number of other badge hunters encountered). Of course, this system could be manipulated by punishing harder on multiple day-bans in a short period of time, but I think the situation is actually quite a bit worse than what I described here, as the numbers I used are pretty generous.

So can we just up the number of days you are review-banned for? Let's take a look at the false positives for that:

On a disputed question, you have 50% chance to vote to accept. If you do and the first three other people to vote on it are either badge hunters or happen to vote to reject. The chances of this happening on any disputed question are: 0.5 * (0.01 + 0.99 * 0.5) * (0.01 + 0.99 * 0.5) * (0.01 + 0.99 * 0.5) = 0.06439. On a clear reject, the chances of a false positive are 0.01 * 0.01 * 0.01 = 0.000001. The total chance of a false positive on any question our real reviewer reviews is thus 0.06439 * 0.1 + 0.000001 * 0.45 = 0.0064394. Usiong p = 0.0064394 we get M = 3769375. A reviewer doing 20 reviews each day would on average be banned for more than a day each 3769375 / 20 = 188468 days. If he does his 20 reviews every single day of the year, that's once every 3769375 / 365 = 10327 years. This sounds acceptable enough to me. Let's take a look at this from one more angle. Going back to M = 3769375, and let's say this time that a 1000 valid reviews are done every day, giving us an average 3769375 / 1000 = 3769 days before any legitimate reviewer is banned, which still over 10 years, which again is acceptable in my opinion.

So it looks like a ban longer than a single day is needed for enough of a punishment on a three streak of 3v1s. However, one should note that I believe the numbers I used are quite optimistic and as such, I believe you would need a ban of quite a bit more than a day to make this effective. It also looks like the false positive won't be too much of a problem. However, here we have the same problems with the numbers we used and on top of that we're not dealing with fact that some people accept more easily than others, meaning that someone who may not be too good a reviewer but isn't a badge hunter may well be banned by this system. The question is how long a ban you're willing to give this person.

Before we wrap this up, I want to look at one more thing: varying the length of the streak required for a ban. Let's start off with a streak of 2. Here the problem is false positives, so let's look at that. Our function becomes M = p + 1 (-p^2 + 1) * M Filling in p = 0.06439 from above, we get M = 155, meaning that for every eight days a legitimate reviewer does his 20 reviews, he'll get banned once (on average of course). This might be acceptable if you realize that most people spending much time on this website will have a lot of experience so might well have too much experience to be affected by this. However, if we add to the mix that my numbers were optimistic estimates and that some people are more inclined to accept than others without being badge hunters, and I don't think this system holds up. So how about making the streak size 4 then? In that case, our function becomes M = p^3 + p^2 + p + 1 + (-p^4 + 1) * M. Here the problem is how long it takes to catch our bad guys, so we input p = 0.448725 from above. We get M = 42.9275, which is over 2 days and is already getting into dangerous territory. A quick look also show that if the real chance is 0.1 lower, this has the effect of increasing the time to catch a bad reviewer to over 5 days, whereas it would still be about one and a half day with a streak of three. This gets out of hand pretty quickly and I'd say that this is not in any way effective (and we're getting into territory where a ban has to be so long that a single false positive is unacceptable) unless my estimates are actually quite accurate (or the difference is on the other side of what I thought them to be). In brief, I don't think using different streak lengths is a possibility.

In the end, the number crunching provided nothing surprising (to me anyway) but I hope it provides the numbers to back up my original claims that this system doesn't work.

And for good measure, here is all the assumptions I made in my calculations:

45% of all edits are rejected by all good reviewers
45% of all edits are accepted by all good reviewers
10% of all edits are disputed
A good reviewer will accept 50% and reject 50% of all contested questions
(There is no difference between contested questions.)
1% of all reviewers are bad
bad reviewers accept in 100% of cases
Each day, an average of 1000 times someone votes to reject or accept an edit
(For the last one goes that I have only used it in one calculation, which
 wasn't too exact anyway)

As per request, here's the math for 25% bad apples:

Once again, we'll add other badge hunters into the mix. The chance of any question that was otherwise going to be a 3v1 having another badge hunter is: 0.25 * 1 * 1 * 3 = 0.75 As such we have to decrease our 3v1 losses by 0.4625 * 0.75 = 0.346875. Now p = 0.115625. Fill it in and we get M = 730.359. Basically we won't catch people legitimately.

The chance of a false positive becomes 0.1 * 0.5 * (0.25 + 0.75 * 0.5) * (0.25 + 0.75 * 0.5) * (0.25 + 0.75 * 0.5) + 0.45 * 0.25 * 0.25 * 0.25 = 0.01923 filling it in we get that it takes 143381 edits on average to get a false positive, which is still sort of acceptable until we start adding poor sincere reviewers and the dynamics of the real world.

share|improve this answer
    
Actually, I find that close to 50% of suggested edits, on average, should be rejected. At least, that's my historical voting record. I tend to be a bit more strict than others, so maybe a 35-40% rejection rate would be a more conservative estimate, but that's still quite high, really. –  Servy Nov 14 '12 at 16:52
    
@Servy: you shouldn't look at what should be rejected, but at what is rejected. The proposed system is based on majority vote, not on "what should be done". As for the math on 50% votes, my gut says it still takes a long time to get to three consecutive rejected answers, but I'm having some trouble coming up with the exact math. –  Jasper Nov 14 '12 at 17:17
    
If 50% of the reviews should actually be rejected then the odds of getting three in a row are (.5)^3, which is 0.125, not all that low. As Pop.'s answer shows, if the percentage of bad reviewers isn't small the false positive rate can get not so small as well. –  Servy Nov 14 '12 at 17:21
    
@Servy That's not what I was talking about. I was talking about how long it would take before you get three consecutive rejected edits. It's not all that relevant, though. –  Jasper Nov 15 '12 at 0:50
    
@Servy: I added the math now. –  Jasper Nov 15 '12 at 15:25
2  
"1% of all reviewers are bad" I reject that assumption. I would use something much, much higher than that. Somewhere in the ballpark of 25% at least. –  Servy Nov 15 '12 at 15:27
    
@Servy The point is that it will only make things worse on both the side of catching legitimate badge hunters and the side of false positives. However, I've added the math as per your request. –  Jasper Nov 15 '12 at 16:07

I do think a quality review system based on statistics is appropriate, but using the pure number of disagreements with the crowd rather than relative number of times they disagree is a bad call. Especially when you consider that out of the two examples you hand-selected to demonstrate the need to punish minority reviewers, one of them is wrong! #Exhibit 1 is a perfectly legitimate edit that should have been accepted! The reviewer removed extraneous code that wasn't being referenced anywhere and therefore wasn't germane to the question in a generalized sense. We WANT people to do this type of cleanup, but three reviewers looked at it, didn't bother to understand why lines of code were being removed and said "Nope!"... it's the majority who are wrong here, and this happens A LOT.

With some non-trivial frequency, the obvious review is not the correct review, so we need to be very very careful with any filter that blindly presumes that the popular vote is the correct vote.

EDIT

I want to clarify that just because our friend TheTXI was correct and the majority was wrong in Exhibit 1, it does not make TheTXI a good reviewer. His approval on Exhibit 2 is basically unforgivable: Approving an incorrect edit to the code of a four-year old accepted answer to a core question with 95 upvotes, and which is likely a core resource for many programmers is downright dangerous. I suspect that his reasonable approval in case #1 was incidental rather than intentional and there should be a system for hitting his rep or suspending his review privileges. It just shouldn't rely on a simple and often-wrong heuristic like disagreeing with the majority. There's a huge amount of subjectivity in some of the review process (definitions of "trivial" vary by reviewer; some people prefer that certain types of edits be referred back to the original author as a comment rather than making the correction themselves; the term "off topic" has as many definitions as there are reviewers, etc etc).

We clearly need to do something to improve the review process, but basing it on whether you agree with the majority is not it.

share|improve this answer
1  
I have seen many edits like that which were rejected for being too trivial (a very easy reason to use), the situation is unlikely to improve unless the review eligibility also takes tag participation into account. –  prusswan Oct 25 '12 at 16:10
    
Ben is correct here, the answer in Example 1 will actually fail as those columns don't exist –  Nathan Koop Oct 25 '12 at 16:10
3  
Instead of editing the question, perhaps that should have been pointed out to the OP for the OP to edit. Maybe it contributes to the problem, and now the context is lost. Certainly it decouples the answers which included that code. This should have been addressed in the answer. –  Jason Sturges Oct 25 '12 at 16:18
3  
@JasonSturges -If the reviewers had bothered to look at the original question (and noted that the question had been edited to a way that left the answers incorrect) it would have been obvious that the subsequent suggested edits to Lews Therin's answer were appropriate. I simply see this as lazy reviewing (and another reason why the review system should show more context) –  Ben D Oct 25 '12 at 16:23
1  
@BenD Saloni attempted to alter both question and answer, and the only comment given is "code changes". I think the entire review system is broken. –  Jason Sturges Oct 25 '12 at 16:30
2  
@JasonSturges - I'm starting to agree. Abuse is rampant and actively discourages thoughtful review. There are "obvious" reviews (incomprehensible posts, link-only answers, three word questions, etc) but must reviews should take time, and I think we're starting to see the fastest gun in the west problem emerging in the review queue. –  Ben D Oct 25 '12 at 16:39
1  
@BenD Yep, and the result is that either good reviewers become bad reviewers so that they can actually get credit for their reviews, or they just stop reviewing (from the queues) entirely (I fall into this bucket) both of which increase the ratio of bad reviewers to good reviewers, thus feeding the cycle and making the problem worse. –  Servy Oct 25 '12 at 16:44
1  
I feel your pain. I've moved into (almost) only reviewing close votes so I actually have time to read through it. And even then you sometimes miss your window of opportunity when you actually try to vote. –  Ben D Oct 25 '12 at 16:48
5  
Totally disagree about "perfectly legitimate edit". Removing or changing code in posts is sure recipe for errors and disasters. Think that something is wrong/not needed? Post comment and let the author change it. I will always reject those suggestions. –  Shadow Wizard Oct 25 '12 at 17:15
    
@ShaWizDowArd - Why? Perhaps Saloni should not have edited the question in the first place, but why request an edit rather than just making the edit when it's obvious and makes a no-longer-accurate answer correct? Why would you reject an edit that is both substantive and makes the answer more applicable and accurate? How could this possibly constitute a "really bad approval[]"? –  Ben D Oct 25 '12 at 18:10
    
@ShaWizDowArd Consider the case where the person did post a comment an the OP said, "sure, you're right, that should be removed" but didn't edit the actual question. (I see things like this a lot where new users don't know they can edit their content.) I will then go in and edit the code/post/whatever to reflect the proposed changes in comments. –  Servy Oct 25 '12 at 19:29
3  
@Servy and Ben - I don't argue that sometimes such edit might be valid. But removing lines of code just because they are not vital part of the answer/question is not a valid edit in my opinion. I won't rollback or start edit combat in such case but also won't approve it. –  Shadow Wizard Oct 25 '12 at 19:56
    
@ShaWizDowArd - this might sometimes be the case, but extraneous code makes answers less clear, and in this case might confuse future visitors (especially ones who don't know to look at edit histories). Why not just have it be concise and correct. Regardless, this whole discussion is exactly why an automatic review-lock based on majority opinion won't work: subjectivity. We disagree on what constitutes a reasonable edit but I don't think either of us is doing a disservice to the review process when we vote... so long as we're being thoughtful and thorough. –  Ben D Oct 25 '12 at 20:06
    
I just want it to be consistent with the current auto block system for suggested edits. User who keep suggesting code changes might be successful sometimes, but enough rejections and he'll be blocked. –  Shadow Wizard Oct 25 '12 at 20:10

In the name of fun and games, why not have the minority reviewer(s) "donate" rep to the other reviewers? This will "encourage" reviewers to carefully consider the actions of their peers and affirm the majority decision.

share|improve this answer
    
That's not what I meant. So far there are no "fines" and I don't want this to be the first. –  Shadow Wizard Oct 25 '12 at 17:11
    
@Sha I prefer to see it as a wager that overturns the isolated nature of the reviewing process. A bunch of people farming their own gear in a MMO isn't how the game is meant to be played. –  prusswan Oct 25 '12 at 17:14
1  
It's not a game and I'm sorry you see it that way. –  Shadow Wizard Oct 25 '12 at 17:16
1  
Unless you take away the badges and rep and whatnot, it will remain a game. –  prusswan Oct 25 '12 at 17:20
7  
I disagree with this, but it does have an interesting grain to it. Rep is supposed to be a "measurement of how much the community trusts you"... your rep SHOULD suffer if your reviews make you untrustworthy. I vehemently disagree with using simple majorities, but perhaps a "I challenge this review" trigger, which subjects the review to scrutiny, with rep docked from the reviewer if it's found wanting. –  Ben D Oct 25 '12 at 18:17
    
I didn't downvote the answer because I feel that losing rep for a bad review is a bad idea, I docked it because I don't feel that a simple majority is a good measure of the quality of a review. I am often in the majority because I took the time to look in depth, beyond the obvious, or because I took a stand on a borderline suggestion. This would punish me for that. –  Servy Oct 25 '12 at 19:27
    
@Servy in that case the rules of determining the majority should be improved, right now it is too "simple" –  prusswan Oct 26 '12 at 8:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .