3

TL;DR:

Reviewers of a post should be able to comment on the task in a separate comment section visible to everyone but only actionable by the reviewers, 10k users and moderators.

Communication between reviewers of the same post would lead to better review choices. An experienced user might make a valid reason to close, reopen, leave closed, or leave a question open. If such reasons can be communicated to other reviewers, reviews would do their job much better.

Currently, the best option we have for this is the comments section on the post. These comments would unnecessarily flood the poster's (and any follower's) inbox. They might also be flagged by low-rep users who do not understand the concept of review, creating extra work for moderators. A non-reviewer might leave a review-like comment, potentially fooling the reviewers. Such comments might also be confusing to new users, and might even make them leave the site if a "this post is bad"-esque comment is posted. For all these reasons, "review comments" in the usual comment system might be treated as abuse.

Such comments might also be buried under several other comments or be moved to chat, making it hard for a reviewer to see them. They might also bury other comments if other reviewers agree with and upvote the review comments, going against the purpose of a comment section.

Why would a review-related comment be flagged?

In the Suggested Edit queue, comments on a review task would not be related to the post itself, but to the edit. This would be a major reason for flagging. A "This post is okay because (something)" comment would also be flagged for being outdated when the post receives many upvotes. These flags would create work for mods.

How to solve this?

I am suggesting a feature request: comments on a review task. Any reviewer of a task can comment on that task, and such comments are only accessible to future reviewers who review the task (except as an audit) and 10k users (and moderators). Such users can also post extra comments in the thread or flag or upvote comments. The same user can post multiple comments on the same task. All the comments on the review would be displayed somewhere next to the review task itself. Once a post is fully reviewed, its review comments are saved by the system, but can still be acted upon. Just like normal comments, review comments should be upvote-able and flag-able by reviewers. Review comments would be visible to 10k users and moderators in a separate "review comment archive".

But wait: what about audits?

For that, I suggest that the system stores "audit comments" which are indistinguishable from review comments on a post for each specific type of review. An audit comment can be added by any user who reviews the audit, just like with normal review comments. Audit comments from passing users are shown to the next user who is reviewing the post as an audit. Audit comments from failing users are saved, but not shown to anyone directly. Saved audit comments are visible to 10k users and moderators in the "review comment archive".

Can't meta be used for this purpose?

It can, but no reviewer would bother to check meta!

Policies regarding review comments

  • Review comments should be relevant to the review task in question.
  • Review comments should not be used to say "This is obviously an audit" in any form.
  • Review comments should be polite and not demand anyone to do something.
  • Review comments should not say anything like "please upvote this post" or "please downvote this post" or "please close-vote this question" or "please reopen-vote this question" or "please delete-vote this post" without further explanation why.
  • Most of all, review comments should follow the Code of Conduct and the "Be Nice" policy.

A moderator might go through your old archives of review comments and ban you from commenting on reviews (or even ban you from reviewing), so these rules are enforceable.

Examples:

First questions: "This question isn't bad for the site because the asker has showed context."

First answers: "This answer is bad for the site because it answered a poorly-written question."

Late answers: "The only purpose of this answer was to bump an old thread. This info was clearly in the comments."

Suggested edits: "This edit adds a link which might be treated as spam."

Low quality posts: "This question is short, but it is of very good quality."

Close votes: "This question is off-topic, but has historical significance. Please flag so that it can be locked."

Reopen votes: "This question is subtly different from the duplicate that was used to close it because it deals with automorphisms of undirected graphs, whereas the duplicate deals with automorphisms of directed graphs."

13
  • 8
    I think it is an interesting idea, but the solution to a broken comment system is not to create a new parallel comment system every time we need to use it for something it was never designed to do. I agree that we need some way for reviewers to share information while the post is under review and that abusing the current comment system is not a great solution for that. I think we can figure out a better solution.
    – ColleenV
    Feb 23, 2023 at 17:55
  • @ColleenV This system makes sure that the reviewers see the comments. It is also by far the simplest solution.
    – mathlander
    Feb 23, 2023 at 17:59
  • If the new system is complicated, then the current comment system would be abused.
    – mathlander
    Feb 23, 2023 at 18:02
  • I think it would be better to list the requirements and let the folks that know the code figure out exactly how to do it. We need review comments separated from other sorts of discussion and presented in context. The review comments probably need to be archived after the review task is completed so as not to get confused with any subsequent reviews or different types of review. It might be useful if the author of the post could participate in the review discussion to ask for clarification. I could probably come up with a couple more constraints/requirements if I thought about it.
    – ColleenV
    Feb 23, 2023 at 18:14
  • 1
    @ColleenV I added the archivation and the OP being able to see review comments.
    – mathlander
    Feb 23, 2023 at 18:16
  • Basically, my beef here is the "Let's just take this system we're already abusing and copy the same design to a different spot to make it easier to pretend it meets our requirements" approach. I don't mean that to sound harsh, because we all would rather post a solution than just give our requirements for a solution and let someone else figure it out. That's just human nature. I find myself doing it all the time even though I have been trained in requirements gathering :)
    – ColleenV
    Feb 23, 2023 at 18:17
  • 1
    @ColleenV The design should be slightly different so that the intended use of the system is not treated as abuse.
    – mathlander
    Feb 23, 2023 at 18:19
  • 3
    The design should meet the requirements. Whether that is slightly different or very different is unknown because we probably haven't finished gathering requirements. What if it looks like chat more than it looks like comments? What if reviewers could add items to a checklist of things an author could do to prevent their question from being closed? Are the requirements for the close/reopen queue the same as for the low quality queue? Honestly I don't mean to discourage you... I should be working on my own stuff right now and am typing a bit hastily.
    – ColleenV
    Feb 23, 2023 at 18:23
  • 1
    Reviewers of a post should be able to comment on the post in a separate comment section only visible to the reviewers and the moderators. So the author of the post doesn’t get to see the discussion about their post? When you say “the reviewers” do you mean people with the privilege to review in that queue or just the 3-5 people that saw that particular post in the queue?
    – ColleenV
    Feb 23, 2023 at 23:36
  • @ColleenV It should be specific to the people who reviewed the post (to avoid abuse). Also, the OP can always go in and upvote review comments that support his post, flag the other comments, and abuse the system in other ways, leading to bad reviews. Such comment sections might also be hostile to the post, and if the OP sees the comment section, he might quit the site entirely.
    – mathlander
    Feb 23, 2023 at 23:39
  • 1
    As for your section on Policies regarding review comments, I personally don't see the need to say all that. You're repeating the essence of guidance that we already have for regular comments and any form of communication on the network. I'd just leave it at that.
    – starball
    Feb 24, 2023 at 3:19
  • I think the policies should still be listed for good measure.
    – mathlander
    Feb 24, 2023 at 3:24
  • 1
    @mathlander Tags should only be used on a question if the question is fundamentally about it, not merely if it mentions that topic. The main point of this request is about special review comments; comment abuse is only mentioned tangentially. Also, if no questions exist about a tag's topic, one should hold off on creating that tag until one or more questions have been asked about it. Feb 24, 2023 at 4:39

2 Answers 2

6

TL;DR: If these examples are the kinds of things motivating your feature-request, I think you're trying to solve a problem that doesn't really exist / is already sufficiently solved.

I get that your list of examples (probably) isn't supposed to be exhaustive, but the fact that I can say to basically every one of them that they don't need a comment in a special designated zone says something about the necessity of this proposal I think.

In reponse to your examples:

First questions: "This question isn't bad for the site because the asker has showed context."

(this paragraph outdated by changes to the question post) Then why only show this to reviewers in review queues? The close vote counter is visible to anyone with close-vote privileges inside or outside of the review queue UI. You don't want those people to also see this kind of comment to stave off bandwagon close-votes? (rhetorical question. I think you do). So... basically you'd want it to be a comment that anyone can see anywhere? Hm... I'm getting feature deja vu.

Besides that- if the question is well researched, useful, and clear, you should just be upvoting it.

First answers: "This answer is bad for the site because it answered a poorly-written question."

Just downvote and move on. If you're the kind of person who really wants to send a message, send it with a loudspeaker and do it as a regular comment (and link to a related meta post if available).

Also, if by "poorly written", you mean that there's value and the spelling, grammar, punctuation, formatting, ordering / other presentational aspects just need touching up (i.e. there's good stuff to salvage), then you as a reviewer can and should just edit it. If by "poorly written" you mean "off-topic", then you should just close-vote the question / flag the question for closure.

Late answers: "The only purpose of this answer was to bump an old thread. This info was clearly in the comments."

If it's an answer, it should be an answer and not a comment. If it's not an answer... *whispers* there's a flag for that (see How do I properly use the "Not an Answer" flag?).

Suggested edits: "This edit adds a link which might be treated as spam."

A lot of spam is pretty obvious. I also don't see why in the mind of a spammer new to the SE network, one would want to suggest an edit (which takes time to get reviewed) instead of posting an answer (which is visible instantly). If you're worried about it being seen as spam, then just edit the post to naturally / inconspicuously make it clear why it's there. You can give it an introducer statement that honestly says what it is. Usually saying what it is is enough to explain why it's there. If it has a good reason to be there, and any affiliation is disclosed, and it's not there to sell something (usually that's easy to tell), then it's not spam. If you don't know whether affiliation is needed, guess what you should do? (answer: comment asking the post owner, which gives them a notification, whereas comments in your proposed review-comments section, the post owner is not notified).

Low quality posts: "This question is short, but it is of very good quality."

Concerning answers (which your example isn't about, but I'm giving my two cents anyway, because most of the posts I see in that queue are answer posts (On Stack Overflow, I'm pretty sure it's only answer posts)): Really now? I want to see some examples of this.

Concerning answer and questions: If you think it's good quality... just upvote it :| Voting is a signal to reviewers too. And I think it's a sufficient signal at that.

The upvote button tooltip is clear and speaks for itself: "This answer is useful", and "This question shows research effort; it is useful and clear".

Close votes: "This question is off-topic, but has historical significance. Please flag so that it can be locked."

One mod flag is enough (yes, having more will bump the flag priority, but the system does not absolutely need more than one flag to function). You can thank the mods who work their bums off for being timely in handling flags (in my experience). Even if it gets closed before it gets locked, if it shouldn't be closed, I'd just expect the mod who handles the flag to handle that. If you're really worried that that might happen, then I suppose you could ask a mod to reopen when they handle the lock-request flag if it's closed when they handle it.

Reopen votes: "This question is subtly different from the duplicate that was used to close it."

This should be a comment or an edit (linking to the proposed dup and explaining why it's different). Ideally an edit, but a comment could be a useful signal as well.

(this paragraph outdated by changes to the question post) Same point as stated before: Why would you want this info to only be available to review-queue reviewers? It should be available to everyone.

6
  • We don't want people who don't know much to upvote a stupid review comment (or flag an important review comment).
    – mathlander
    Feb 24, 2023 at 5:39
  • @mathlander if the comment is just "stupid" (whatever that means) and not actually grounds for flagging, then too bad for you. People have a right to vote how they want on comments (except for fraudulent voting involving sock puppets). If someone inappropriately flags an important review comment, 99.9% chance the flag will be declined by mods. If someone gets enough declined flags, they get a temporary flag ban. If it keeps happening, suspension, and possibly in really long-term, bad cases, explusion from the network.
    – starball
    Feb 24, 2023 at 5:43
  • This still doesn't handle the issue of suggested edits, whose comments are irrelevant on the post.
    – mathlander
    Feb 24, 2023 at 5:44
  • @mathlander for suggested edits, my general point applies: each reviewer can make their own decision on the good-ness of the suggested edit. And if meta discussion is needed, then it can happen on meta. See my other answer post.
    – starball
    Feb 24, 2023 at 5:46
  • We also don't want a user flag-banned for flagging review-comment-esque comments.
    – mathlander
    Feb 24, 2023 at 5:49
  • 1
1

Can't meta be used for this purpose?
It can, but no reviewer would bother to check meta!

Agreed and point taken. But as far as I was aware, this is what meta is for: Talking about the main sites. That's why established meta sites have , , , and tags.

(more for the sake of discussion and lateral thinking and not necessarily because I think this is itself a great or better idea...)

Alternate : Add a UI section to the review queue UI similar to the "Linked Questions" section that we have for posts on main sites, but it's "Linked Meta Posts": Meta posts which link to the post / post revision being reviewed. (To be honest, I don't see why limit this to review queues, but that's besides the point here).

The list would be ordered by relevance of tags on the meta site (the only problem there being that tag names are not really consistent across network meta sites). Ex. meta posts linking to that post / post revision that don't have a relevant [specific-X] tag to the type of review item would get sorted lower in the list. Ties would be broken with meta posts with more recent activity sorted higher.

This could be taken a step further by adding a "Ask Meta Question" button to the review queue UI. It could fill in the and [specific-X] tags, and fill in a link to the review item and post in the question body.

A similar feature-request to this has been made here: Showing meta questions in the same manner as linked and related questions


The main problem I can see with this counterproposal (and a problem that your proposal doesn't have) is that it could have the potential to generate a lot of trivial discussion (triviality in the eye of the beholder- to an extent) on meta, and could end up with a lot of canonical duplicatry. But... I don't think that's necessarily a bad thing. I can see how some people could see that as annoying or perhaps a bit noisy, but I can't think of an objective, significant reason off the top of my head right now why more meta discussion and canonical duplicatry on meta sites would be a bad thing.

If canonical duplicatry seems to be happening too much (the same kind of problem is getting reviewed and talked about on meta), then that's an indication that something about the system needs to change. That's also a good thing: the mass canonical duplicatry will indicate needs for system changes, such as modification to close reasons, or UI guidance.

And to balance that out, in my (limited) review experience, I don't think this kind of meta discussion is really needed very often (I feel confident enough to say less than 10% of the time). The current mechnisms of voting, flagging, and close-voting are usually sufficient to handle any problems a post could have.

10
  • Won't this unnecessarily clog up meta?
    – mathlander
    Feb 24, 2023 at 4:25
  • @mathlander I just made another edit. TL;DR: I think not.
    – starball
    Feb 24, 2023 at 4:31
  • Posts are generally expected to be long. Such posts would have to be as short as a comment.
    – mathlander
    Feb 24, 2023 at 4:36
  • Also, don't forget about unnecessary inbox notifications.
    – mathlander
    Feb 24, 2023 at 4:36
  • @mathlander Every time I have asked about a specific review item or something that I could have encountered as a review item, or that I could have pushed into a review queue, I have written something longer than what one would expect as comment-length, and cared to be notified of what other people said about it. Examples: 0, 1, 2, 3.
    – starball
    Feb 24, 2023 at 4:43
  • By "have to be" I mean for a reviewer to read it.
    – mathlander
    Feb 24, 2023 at 4:43
  • 1
    @mathlander this is why I think your post needs more examples. I can't see why a review would "have to" read such meta discussion. As a reviewer, I might want to, but most of the time, I'd be able to form my own evaluation on my own. And that can be a good thing. We want to get diversity of review evaluations. The system is designed to avoid having the same people review a bunch of posts (the randomization mechanisms of review task assignment).
    – starball
    Feb 24, 2023 at 4:51
  • Examples added.
    – mathlander
    Feb 24, 2023 at 4:52
  • What about the privacy reasons I mentioned?
    – mathlander
    Feb 24, 2023 at 5:10
  • 3
    Counterpoint: why should it be private? I like my meta discussions public thanks. I think if anything, it's healthy for askers to be included in meta discussion.
    – starball
    Feb 24, 2023 at 5:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .