13

When your flag weight is shown, and below 500, you can see it move if you flag. But once you hit 500 you can't be sure that your flag quality is still good - if you flag half a dozen badly and another 6 well, your weight stays at 500.

Is there any way, short of keeping a record of each flag and going back later, to check?

Improve this question

3 Answers 3

Reset to default
13

Posted as a feature request:

Flag weight should be calculated in terms of % correct, not as though it were voting

Statistically speaking, the current system is broken.

If I get 51% of my flags correct, ultimately I will reach 500. Shouldn't someone who flags correctly 90% of the time have their flags shown in the list first?

Conversely, if I flag 49% of the time I will reach 0. Shouldn't I be more highly ranked than someone who only flags correctly 10% of the time?

Why is it that someone who flags correctly 51% of the time gets to be at 500, while I'm stuck wallowing in the mud because I'm flagging correctly 49% of the time?

Solution

I'd rather have a system that uses a sliding window of, say, the last hundred flags. How many of those were acted upon? It would easily translate into a percentage, and be very easy for the user to understand how well they are currently flagging.

It's technically more difficult to implement than the current system, so an alternative system would have much the same effect, and should be easy to implement:

The higher the weight a user achieves, the less effect a good flag has on weight, and the more effect a bad flag has on weight, and vice versa.

So at 500 weight, good flags have no effect anyway, but a bad flag might drop the weight by twenty points or more. It would then take 10 good flags at two points each to get back up to five hundred. At the bottom, a single good flag will jump the user up by twenty points, and it would take ten bad flags to get them back to zero.

In the middle, each good and bad flag would be ten points up or down. In this way, users would tend to stay near the middle unless they had a better or worse than average flagging record, and if their average is under ninety percent, they won't be at 500.

It would be easier to implement than a sliding window, and it would still give a useful effect. One couldn't get to five hundred, then simply maintain a fifty percent average and stay at the top of the list.

A simple implementation:

A good flag: currentweight += ( 500 - currentweight ) / 25

A bad flag: currentweight -= currentweight / 25

Rounded up, so at four ninety nine you'll still get one point for a good flag to get you to five hundred, and vice versa.

Improve this answer
17
  • 1
    ah, this is an excellent idea
    – Jeff Atwood
    Mar 7, 2011 at 4:07
  • 5
    Wait, why is this good? Why is someone punished more for a mistaken flag if they have a history of good flags? That seems the opposite of what you'd want
    – Michael Mrozek
    Mar 7, 2011 at 4:43
  • @Michael It depends on the point of the flag weight. It's a system to rank a user according to how likely they are to be flagging correctly. If you flag correctly 99% of the time then you should be ranked better than someone who flags correctly only 60% of the time. But guess what? You're both at 500 with the current system. The current system does not differentiate someone who flags above average from someone who flags 50% once they've both reached the same level. Further, someone who flagged horribly at first will be 0 even if they've improved and are now getting 40-50% correct.
    – Pollyanna
    Mar 7, 2011 at 15:41
  • @Micheal Comment incorporated into answer.
    – Pollyanna
    Mar 7, 2011 at 15:44
  • 5
    I think this makes a terrible assumption that someone can make it to 500 with 51% flag success rate without getting pulled aside for egregious amounts of bad flagging. Remember, moderator dismissal is a human action. There are only so many moderators, and they will notice when there is a trend of bad flags, even if your flag weight might be pretty good. We've paid attention and annotated these kinds of things even before we had the option to dismiss them as valid or invalid. Those speak a lot more, and cause more action, than just the flag weight number will.
    – Grace Note StaffMod
    Mar 7, 2011 at 15:51
  • @Grace That's a separate, though related, issue. And it's fine if that is the preferred way to deal with those that have a bad average for flagging. I used extremes to make my point. Here's another example: If I flag correctly 85% of the time, and someone else flags correctly 99% of the time, shouldn't their flags be given more weight than mine? Right now our weight is usually the same. With a sliding window or the system I'm suggesting above I will have a lower weight than the better user. The fact that it breaks at the 49% to 50% range makes the current weighting useless for ranking.
    – Pollyanna
    Mar 7, 2011 at 16:05
  • 1
    I think flag success ratio is a separate issue, actually. If you flag well 85% of the time, what that means in comparison is not that you are of lower priority than a 99% user (especially if your flags are still excellent). What matters is that you'll get a talking-to for that 15%. There's no use to ranking between the top of the top when it comes to flag weight - the good ones still remain good, and well above the people who don't traditionally flag well. That is what the point of flag weight is. Not some kind of thing to lord about as if exalted.
    – Grace Note StaffMod
    Mar 7, 2011 at 16:10
  • 1
    If you flag well, we validate them and take action. If you flag poorly, then we invalidate them and may contact you for repeated offense. It doesn't particularly matter the ratio of these, nor what your flag weight is when it comes to decision making. Really, because there is feedback for bad flagging, people shouldn't expect to sit at 85%. That should grow to 99% as they stay at 500. If it doesn't, then you're probably not listening to the emails we sent about what you shouldn't flag.
    – Grace Note StaffMod
    Mar 7, 2011 at 16:22
  • @Grace Interesting. It appears you are suggesting that the flag weight really serves little purpose, and thus the fact that it's broken doesn't matter. Regardless, can't we fix flag weight, even if it has no purpose, other than to display a number on our profiles?
    – Pollyanna
    Mar 7, 2011 at 16:33
  • 1
    @Pollyanna I think it's useful for what it actually does mechanically: rank items in the mod queue and provide people with more flags per day. It doesn't feel broken to me in this respect. What I think serves little purpose is getting hung up on the value, which is why I opposed its publication in the first place. As a number on its own, flag weight is often meaningless to the user. Even if you see it go down, you still won't necessarily know why it went down. Success ratio doesn't matter in the end, and that's why it's actually good that it isn't publicized as such.
    – Grace Note StaffMod
    Mar 7, 2011 at 16:36
  • 1
    If someone with a history of a 60% rate manages to reach 500 rating, that probably means they amended their ways and are flagging properly, so they will be improving that ratio. So what if the first almost half of their flags were bad? If they're no longer making those bad flags, that's a 100% victory rate. And that sounds to me like a functional system. It's not easy to reach 500. You need to make good flags, and every bad flag you make is just going to make it harder. It's by not consistently churning out bad flags that lets one reach 500.
    – Grace Note StaffMod
    Mar 7, 2011 at 16:41
  • @Grace Well, let me ask a different question then: Would modifying the flag weight in this manner damage the system, or cause it to fail to meet the intent of the system? I guess what's making me scratch my head about your argument is that you're not arguing that the current system is good, nevermind better. Is my suggestion bad or wrong? Or is this simply an issue that doesn't need to be addressed because the current system is good enough, and we shouldn't frivolously spend development time on it?
    – Pollyanna
    Mar 7, 2011 at 16:43
  • 1
    @Grace Also, you have made many good points in this comment train. Consider submitting an answer to this question.
    – Pollyanna
    Mar 7, 2011 at 16:45
  • I did consider an answer, haha, but it doesn't actually address "How can you tell if you're still flagging correctly", so much as it says "the ratio argument shouldn't really factor in". But, reviewing the asked question, it does remain a valid answer, so I will toss it out.
    – Grace Note StaffMod
    Mar 7, 2011 at 16:48
  • 1
    @Grace @Pollyanna Great discussion. I didn't realise that bad flagging would result in a shot-across-the-bows email. What seems imperfect to me (as a flagger) is that we have a signal that appears to stop working at some point. If we're going to have the weight visible it would be good if it reduced the amount of warning emails that need to be sent. I understand that one shouldn't get hung up on one or two dismissed flags, and the weighting is great as it allows accurate flaggers to help more with janitorial work, but perhaps it needs a tweak.
    – martin clayton
    Mar 7, 2011 at 18:58
10

It's a cumulative score. If you ever see it go below 500 (for more than a few hours), you're flagging incorrectly. I wouldn't expect a 100% hit rate consistently as flags are (often) a matter of interpretation.

90% is more than excellent, and should be sufficient to maintain the maximum flag weight.

If you flag, say, 20 "not an answer" posts, I fully expect to disagree that at least one of them is an answer (though perhaps rather low quality) in any given batch. That's not always the case, just more often than not.

Also, weekends deserve special consideration. I might act on a few flags during the 20 minutes taken to check e-mail or whatever else. It's conceivable that I might dismiss one out of the batch of 20 flags you sent as invalid hours prior to validating the rest. That's quite typical for many of us on Saturdays and Sundays.

Improve this answer
6

This isn't really an issue at 500 anymore than it is an issue at any other point in time. If I have 230 then make a half dozen bad flags followed up by 6 great flags, I will remain at 230 just the same.

Basically, if you're at 500, you can tell you're still flagging correctly by whether or not you've changed your flagging habits. You reached 500 in the first place because of good flagging, so the only way that will change is if you stop doing the kind of flagging that got you to 500.

Think of it this way. If you stall at any point in a climb due to bad flags, that means you're building up a history of bad flags. In addition to hindering your progress, you may receive an email or other contact telling you that, for example, "Hey, stop flagging your posts to announce bounties".

Ideally, you listen to the advice and stop the bad flagging, which paves the way for you to make more good flags, overall improves your weight, and consequently your success rate rises. If you don't listen to this advice, that means you'll continue to make bad flags, and may receive warnings or even a suspension if you're particularly bad about it.

So, when people hit 500 flag weight, they're all pretty much going to be in the 90% or higher range of success rate, which I agree with Tim Post as being pretty stellar. These people either never had a history of bad flags, or they amended that history to become exemplary flaggers. They don't need perfect rates because people are allowed to make occasional mistakes.

What impacts it is going to be when you make a trend of bad flagging. And that'll have a more noticeable effect on your flag weight than the occasional mistake. Even if it is a trend of the occasional insertion, keep in mind that there are humans behind the validation and invalidation of flags. If you're consistent, no matter how spread out, it forms a pattern.

Improve this answer

Not the answer you're looking for? Browse other questions tagged .