I'm asking if providing this kind of feedback would make sense before developing it into a full request
TL;DR: I'm not sure, but I'm leaning heavily towards 'doesn't make much sense to me yet'.
In a way, it does sound like it would be 'nice' of the system to show users that they are balancing on the edge (of being contacted and suspended by a moderator).
In another way I'm having a hard time finding any other arguments then 'it sounds like it would be friendly':
- It comes with a risk of people demanding to know which of their comments were deleted/arguing over comment deletion/kind vs. unkind even more than they already do.
- People might use this automated feedback to 'stay under the radar'.
- Automated feedback might be pretty hard to time right, given the current thresholds for the automatic moderator flag.
- Automated feedback is going to be a lot more generic than moderator messages, and might not be as helpful for users to 'self-correct' their behaviour.
If it makes sense, how do we balance between being too noisy (so users just ignore it) and being too late (so moderators are probably going to have to intervene anyway)?
Right now, an automatic moderator flag is raised if three comments from a single user have been deleted as rude or unfriendly in the past 7 days. After raising that flag, the clock resets, and the user has another 7 days to 'score' another 3 rude/unfriendly deleted comments.
If we're going with the auto-flag statistics, I'm wondering in how many cases an automatic message can be shown where it wouldn't be too late already. How many times does the automatic flag get raised where it isn't just a single disgruntled user on an f-you spree/a single whisky fueled outburst/a single heated argument in the comments (perhaps over multiple answers to the same post)?
Three rude/unfriendly comments can be made in a really short time in these cases, before the community or a moderator deletes them and they can count toward this 'score'. And after that, we already have the auto-flag, a users comment history to clean up anything the community might've missed and moderator messages/suspensions to correct behaviour.
If you're going to provide automatic feedback, perhaps it's best to have it follow the same kind of pattern as the automatic moderator flag: If you're at 2 comments that were deleted as unkind, show the feedback. As soon as you trip comment nr. 3 and the automatic flag, dismiss the feedback.
This might come with a risk of people never seeing it, but at least this way it won't create unnecessary noise: You're now going to get feedback from a moderator in the form of a moderator message or suspension. Then start the counter again: If you manage to get 2 comments deleted in the next 7 days as rude/unfriendly again, show the automated feedback message again.
I picked 2 deleted comments in the above example, because another point to keep in mind when talking about noise: The automatic flag is raised after three comments that were deleted as rude/unkind within 7 days. You're kind of stuck with those seven days here: 1 comment in seven days can become two, can become three (which triggers the automatic flag). Should 1 unfriendly/rude comment be 'punished' with a week of noise? Perhaps two is better here, though you might then get users that make it a sport to continuously walk that line.
Perhaps show automated feedback for the rest of the day for 1 deleted comment within 7 days and for something like three days for 2 deleted comments within 7 days? That way, you can never be sure if your next rude comment is going to trigger that auto-flag or not, unless you actually care enough to time the automated feedback you get. You're also not stuck with a week of noise for 1 or 2 unkind comments.
Last point here: perhaps the age of comments should be taken into account as well. If someone posted an unkind comment that slipped through the cracks for half a year, are we going to bother them with automated feedback about this, when they're now behaving like an exemplary user?
My goal is to help users self-correct before mods have to intervene.
This is going to be another hard part to handle automatically.
Users might (ab)use automated feedback to consciously stick beneath that threshold of an automatic moderator flag instead of self-correcting posting another rude/unkind comment as soon as they get rid of the 'feedback' that tells them they might be contacted by a moderator soon. This then puts the burden on the community or moderators to flag/recognize these users, and put an end to those shenanigans.
The above doesn't even have to be done deliberately: Just not commenting until you don't see the feedback anymore and then continuing where you left off has the same effect. Especially for those users that get upset from time to time, this might prevent moderators from seeing the signs and realizing the need for contact/suspension. Timing the automated feedback so that it doesn't totally coincide with the thresholds for the automatic moderator flag might help with this, though.
I'm personally more fond of sending moderator messages to those people that seem genuinely stuck with being kind in comments, or upset about something on the site, explaining how they can get across their message without coming across as rude. For those messages, I generally use recently flagged and deleted comments as examples, and point out specifically the points that can be improved upon.
Automated feedback is going to have a hard time giving very specific guidance on how to improve, the kind that you can put in a moderator message, even when you show people which comments were actually deleted as rude/unkind. That only shows people what was considered wrong, but doesn't give them any advice on how to do the same thing right.
