Flags in chat are defective by design

Question

Yes, I know the topic has been discussed before. I know that Meta already has threads that complain about how flagging doesn't work. I wanted to give a practical example.

---

Today, a user came in to the JavaScript chat and asked the following question:

Hey guys, as soon as I embed javascript, all my html elements are gone. Even if the script is empty!? ...Can anyone help out? The example is really minimalistic <script type='text/javascript' src="http://ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js"/> <script type='text/javascript' src="script.js"/> These lines make everything vanish. script.js is empty! I use google chrome and the file is on localhost

Some context:

• The JS room encourages people to come and ask questions that are too small/badly put for main.
• The room encourages people to just ask questions rather than ask to ask. In fact we even have a bot that prompts users to just ask if they have a question whenever they join.
• The above user has a good Q to A ratio having 1K rep in the main site and over 80 answers.
• The user has no negative context in the chat. No previous offenses or anything like that.

That message got 7 flags

Had a mod (ThiefMaster) not been present and invalidated those flags in time that user would have probably been suspended from the chat. This is bad, it sends the wrong message and it is very rude.

Now, some other rooms don't allow people to just come and ask questions right away so maybe users from other rooms validated the flag. Maybe users thought he was being rude? I don't know.

I do know that we don't want this sort of negative atmosphere in the JS room (and I'm sure other useful rooms such as PHP don't want it either), we have enough on our hands trying to maintain a community while helping people joining and handing with vamps/trolls.

Possible solutions:

• Drop spam/offensive flags altogether.
• Make spam/offensive flags room specific for big enough (tm) rooms.
• Give room owners just the ability to invalidate spam/offensive flags.

Answer 1

Yeah, they should totally do that

Answer 2

I think it would be a good idea to ban a user from flagging (in chat) if the user has had more than 3+ invalid/useless flags.

I have also had a lot of my comments flagged in the JS Room over the past purely because the user didn't like me.

There is very rarely a case in the javascript room where a message actually deserves to be flagged, 99% of the time when a owner moves an irrelevant message to the trash can, that is good enough.

Also, this may be a bit extreme, but say there was a message in chat that was extremely offensive then the people that flag it are not going to care that everyone knows they flagged it, because it would be the right thing to do.

So making the flags public may make people think twice about invalidly flagging posts, because people would be straight on their case.

Answer 3

I've been thinking about this issue for several weeks now, and I recently got my hands on 120 days worth of actual chat flag data. The data covered all cases where at least two people found a message flag-worthy (including the original flagger) and at least one person thought the flag was not valid. In Stack Overflow's chatrooms, this happened 318 times; in Stack Exchange's, 168. I didn't get data on Meta Stack Overflow chat, although personal experience suggests that flags are rare there.

Here are a few of the cases I looked at:

• Votes from people in a room would have deleted a message on their own, but the addition of votes from people outside of the room caused the message to be "saved"

  This didn't happen a single time in the whole four months.

• The opposite of the above: outside votes helping delete a message when the room alone agreed with the flag but couldn't put together six votes

  This happened seven times on Stack Exchange and 65 on Stack Overflow.

• The room total agreed with the net total

  This happened about half the time on SE and about a third on SO.

I didn't have information on which users cast which votes, so I couldn't do much with ideas based on room owners. However, I did see what would have happened if votes from people in rooms counted twice as much as those from outsiders. On SO, that would have resulted in 14 more deletions and zero saves; on SE, 12 deletions and zero saves. In all cases, the messages that would have been deleted were ones that actually ended up with a net tally of between one and five votes (i.e. not a great message, but not bad enough to delete).

My preliminary results are:

• people do indeed tend to view and judge flags from rooms they're not in
• the number of times "outsiders" change the final disposition of flagged messages is non-trivial, but also not overwhelming
• the only posts that gather lots of "'meh' votes" are the ones not written in English
• giving special powers to rooms as a whole would not have the desired effect

There are a number of problems with relying too heavily on these results, including but not limited to:

• all flags for messages moved to "bin" or "trash" rooms show up as coming from outsiders, so I didn't count them at all
• my numbers didn't account for the fact that the order matters when counting flag votes (five agrees in a row would immediately delete, but three agrees, then four disagrees, then four more agrees wouldn't)
• I could only track agreement or disagreement, not who made the right call
• I couldn't see which votes came from diamond moderators and which came from high-rep non-diamond users

Again, these are only preliminary results. I plan to run some more thorough numbers (feel free to make suggestions) and give more thought to the more qualitative aspects of chat behavior as well (e.g. what to do when something that doesn't offend room regulars does offend an outside observer).

Answer 4

Firstly, regardless of the room, flagging a user asking a question as offensive is not OK. Flags are for offensive things, and if a user is being annoying, call in a moderator.

I agree that there is a problem here, though.

One major stumbling block is that the problem cuts both ways. There have been cases where room regulars take it upon themselves to ban inoffensive users with the flag, and restricting flags to a room causes . There have also been countless cases where lowered standards of etiquette (within bounds) in a room have been ignored by network 10k users leading to unnecessary bans.

On idea I've been toying with is to set a maturity rating level for chat rooms (which is shown with the flags), but that gets a bit annoying, and doesn't take care of issues like the one mentioned in the question.

What if we gave room owners the ability to unban a user that was banned in one of their rooms by non-mod flags?

Another option is to have an escalation system for flags. I feel that room-specific flags can be abused, because a chat ban from a particular room is a chatban everywhere.

Finally, it really would be nice if there was a "kick" option to deal with such things. With a kick, a room owner or mod can exclude a particular user from a room for some period of time. I have a bookmarklet that lets me accomplish this, but it's hackish.