Changing the question reopening experience

Question

Now that the Public Platform team is actively working on updating review queue workflows, it’s time we address the feedback and feature requests related to the question reopening experience.

How it works today

A closed question has a single opportunity to enter the Reopen votes queue and be considered for reopening. If a closed question receives any body edit, it will automatically go into the Reopen votes queue. This process does not discern the value of the edit itself. If the edit doesn’t resolve problems with the closed question, reviewers are likely to choose to keep the question closed. Once the question leaves the reopen queue, the post author and other community members have little to no option of getting the post reopened.

Updates

After reviewing the community ideas and weighing the pros and cons of some different approaches, we have landed here with some improvements to both the editing and reopen votes experiences.

1. Adding a “substantial edit” option
All users are still able to submit edits on closed questions. This new feature allows editors to determine whether they’re making a minor edit (i.e. grammar, spelling) or a significant edit. By submitting an edit for review, the editor agrees that their edit attempts to resolve the question’s close reason(s) and should be considered for reopening in the Reopen votes queue.

Once a significant edit has been submitted, editors are still able to return to the edit screen and make further adjustments.

Users who do not have enough reputation to submit an edit outright will have their significant edits go into the Suggested edits queue accompanied by a post notice.

2. Leave closed reasons
If a Reopen votes reviewer elects to leave a question closed, the reviewer will be presented a “Leave closed reason” modal (similar to the close reason modal) that will return feedback to the editors. For example, if a question was originally closed because it lacked details, it may still require further details even after an edit, or new issues with the question might present themselves like asking too many questions at once.

In addition to inbox notifications, the closed question’s post notice will also be updated.

3. Multiple opportunities to send question for review
We understand that it may take multiple attempts to get your question just right. If a significant edit fails to reopen a question, users will now have an opportunity to continue to rework and edit their question and have it reviewed multiple times. Reviewers will not be able to see the same question twice in the Reopen votes queue.

To make sure that the significant-edit option isn't abused, we will monitor overall completed review tasks, the significant edit attempts, and the number of closed/reopened questions.

These changes are planned to occur with the other review queues workflow updates.

Please share your feedback and questions on this post.

Answer 1

Thank you! This seems like a really good feature and I'm very glad you're implementing it!

One thing that worries me is the "Reviewers will not be able to see the same question twice in the Reopen votes queue". I get how that makes sense for the large sites with many reviewers, however I fear it will essentially break review for smaller sites. Some sites have very, very few people with enough rep to see the review queue and even fewer who actually choose to spend their time there. On such sites, I suspect we'll have cases where the edit will languish in the queue indefinitely if we don't show it to the same reviewer.

In addition, some technical sites can have questions that are only understood by the small subset of users who are experts in the particular sub-field. In such cases, it is often only those users who can really assess whether the edit fixes the issues with the question and that means an even smaller pool of possible reviewers.

Perhaps this should be changed for smaller sites with few reviewers? Maybe by allowing users to see the same question when the number of potential reviewers is less than some threshold? Or, alternatively, could there maybe be a user option that lets people choose to be shown the same question if it's been re-edited? Or something else? I just fear this could be a problem on the smaller, niche sites.

Answer 2

Assume that a user who does not have enough reputation to submit an edit outright makes a good edit that improves a question. The editor also checks the "significant edit" box, but the edit is slightly not enough to justify sending the question to the Reopen queue. Then I as a reviewer see the following post notice:

In this case I would be torn, because I don't want to reject the good edit, but I also don't want to send the question to the Reopen queue yet. In this case I would like to have the option to uncheck the "significant edit" box and override the submitter's choice. Would this be possible?

---

As indicated in the comments, the two reviewers might disagree on whether or not the "significant editing" box should be checked. I'd like to provide a few ideas on how to solve this issue:

• Require a third (decisive) review if there is no consensus. However, this might be overkill (we have too few reviewers anyway) and too complicated.
• The last reviewer "wins", i.e. their decision whether to check the box or not is binding. But this seems a bit too nondeterministic.
• The question will not be sent to the Reopen queue if at least one reviewer has unchecked the box. I like this option the most.

Anyway, I'm fine with any solution as long as there is a chance to override the editor's decision.

Answer 3

status-completed

Please make sure that voting to reopen doesn't remove the question from the reopen queue.

Currently, if the editor whose edit puts the question in the reopen queue subsequently votes to reopen, then the question is either not placed in the reopen queue or removed from the reopen queue (ref 1, 2). This is, unfortunately, exactly the opposite of what is expected by users who both edit and vote to reopen.

As part of these changes, please make sure you change the logic surrounding voting to reopen and putting the question in the reopen queue so that questions are not removed from the reopen queue, or prevented from entering the reopen queue, as the result of someone voting to reopen. Such a change should be an expected consequence of the changes for which you're in this discovery process. However, I want to be sure that this issue also gets addressed at the same time as, or before, the changes you're working on are made.

Answer 4

feature-request

Picking reasons why a question should stay closed is nice! But there are more reasons for closing a question (and keeping it closed) than listed here: The community specific close reasons. Can we get fields for those community-specific reasons too, or at least an 'other' field/an option to not have to pick a reason before voting to leave closed?

Right now, I would miss being offered the community specific reasons, and I would end up voting to reopen, because there's no reason to keep this question closed in the list, and I really want to complete my review and get my score up!

If a question was closed for a community specific reason, and the edits haven't improved it to a point where it no longer falls under that community specific reason, there really should be an obvious option for those reasons (ideally one that doesn't require people to type, people may be too lazy for that) in the list.

Or, as pointed out in the comments, can we have an option like "The edit did not resolve the original close reason" when the question entered the queue through an edit, and have that checked by default? That would solve more than just community specific reasons, and could also cover all the other existing ones where the edit just didn't resolve the original issues, instead of requiring people to match the 'remain closed' reasons with the original close reason.

Answer 5

I can imagine a problem with the new "substantial edit" checkbox:

In the example, the bold text says Submit for review. I am afraid that users unfamiliar with the system, but frequent editors on the other hand, might confuse it (and don't bother to read the explanation) with actually sending the edit for review - which is what they expect to happen.

Maybe some different wording like Submit edit for question's reopen review or simply Substantial Edit.

Answer 6

There is one major flaw with the current review logic that needs to be fixed before this can go ahead

TL;DR: The current logic disqualifies questions that have been both edited and voted to reopen by the same user from a reopen review. This behavior should be removed if your proposals are to work in the manner you've indicated.

The current review logic which adds edited questions to the reopen queue is designed to exclude edits from users who've also flagged or voted to delete the question. This is to prevent edits from users who've indicated that they don't wish to see the question reopened from adding the question to the queue.

The trouble is, since close and reopen votes are internally represented as flags in the system, this means that if the question is edited by a user to get it reopened (i.e. to resolve close reason(s) or point out why the closure was incorrect), and is also voted to reopen by the same user, the question won't qualify for a review since as far as the system knows, the edit came from a user who had also flagged the post. This behavior is explained in detail in the answer; in fact, in that specific case, the question had actually made it to review, but the task was later invalidated for this reason.

(Technically speaking, this is avoided if others also voted to reopen, but as the review logic also excludes reopen votes that were cast less than 15 minutes ago, if the votes from others come in too quickly, they won't be counted and the question will still be disqualified - that's what happened in that case. Also, on smaller sites, reopen votes may end up coming in slowly because there aren't many active reviewers.)

The second solution in that answer should be implemented - reopen votes should be excluded when considering if there are flags that should invalidate the review. The answer mentions that this would allow questions to be put through the queue twice, a violation of the current design which only allows them to go through it once, but as you say, the design will be changed to allow that. Even so, it favors that solution, as it's the simplest.

That said, since there's now going to be an explicit option to indicate that an edit makes the question reopenable, it may be time to remove these exclusion filters entirely. As I said, the whole point of them is to try and detect if an editor doesn't want a question reopened - we're now asking the editor directly, so it doesn't make sense to guess if they don't.

Answer 7

Nitpick for the leave-close reasons:

The last two options end with a period. The previous three don't. Please standardize it one way or another.

Answer 8

Make it easier to access an ongoing review item from the post itself

Could you please make it easier to access the review item from the post itself? We can directly vote to reopen from the post, but if we come across a post we think should stay closed, we have to search through review queues or play around with filters until we find the post and can vote to leave closed. The link in the timeline to the review is only added once the review is complete, so during an ongoing review it is hard to find.

Why? I think that users with the review privilege, who organically come across a post, often have more expertise in the subject matter than a random user from the queue and it should at least not be more difficult for them to vote than it is for reviewers .

Answer 9

The Too Broad close reason is not only for multi-part questions, but also for questions that are impossible to answer in a concise manner appropriate for our Q&A sites.

This was a problem with your modifications to the flag/close dialogs.

This close reason is not only about posts that ask multiple questions. It is also about questions that are too broad and which it is unreasonable to expect to be answered in the SE Q&A format. For example:

• How has the USA's foreign relations changed since independence from the UK?
• What steps do you need to take to establish a TLS 1.3 connection?
• What is the Catholic Church's understanding of Mary?

Please don't make the same mistake here! Restore the original meaning of Too Broad!

Answer 10

I appreciate that the default for the "reopen" checkbox appears to be "off", having seen too many well-intentioned minor edits waste a question's chances for reopening.

Would the checkbox default to "on" for an edit by the original author?

Answer 11

There are conflicting messages given to the editor and the reviewers. The editor thinks that they are telling reviewers that they "resolve the original close reason" (which kinds of ignores that a question can be closed for multiple reasons), meanwhile the reviewers are given the message that the editor claims that it "improves and/or resolves problems" with the question.

Rather than talking about the close reason(s), it would be better for the message to the editor to say "this edit makes the question meet the quality guidelines" and for the message to the reviewers to say that the edit "brings the question within the quality guidelines". This would prevent people from itemizing badness and to instead focus on generating good, on topic, specific questions.

Answer 12

First of all, thanks for working on this long standing area of grievance. I think this will make things better for all users, new and old alike, by removing friction from the system. On the one hand new users get better feedback on why their question remains closed, and on the other hand experienced users get less "garbage" in the reopen queue.

There is one major concern I have though.

Currently close notices on posts are designed to show only one close reason. In theory 5 users could have chosen 5 different reasons to close the post however.

We expect people to ask good questions, by reading [/help], [/on-topic], etc. but are proven that this is far from reality most of the time.

If we are now giving people feedback on why their question isn't going to get reopened. We could, in theory, give them 4 consecutive times different feedback on why their post remained closed. Because every time they edited, they fixed one issue, but were unaware of the other issues that were discovered with their question, and were presented a single reason on why their question couldn't be reopened. This could lead to very frustrating experience for the OP. Reviewers are protected against this grievance because they only get to see a question once in the reopen queue.

Then again this might prove to be problematic for smaller sites, where the number of active reviewers is limited. Perhaps all active reviewers have been depleted after the second try of reopening the post, so the post will sit in limbo for a long time, before eventually ageing out of the queue. So depending on what data shows us after an initial period, there might be a need for serving the same question to reviewers twice, after it sat in the queue for 6-8 days.

TLDR:

• Please improve the post notices to include multiple reasons on why a question can't be reopened.
• At the same time also improve the post notices to include the multitude of close reasons that were used to close the post, so users are better educated on what needs to be improved on their post, so their first try in the reopen queue can be more successful.
• Consider serving a reopen review item to users that have seen it before once a post is in the queue for 6-8 days to prevent posts from being in limbo on sites with a limited number of reviewers.

Answer 13

This proposal quite comprehensively resolves the issues I've brought up with the current review architecture:

• Only one edit can add the question to the queue, which means that if someone makes a minor edit only making grammar/spelling changes which adds the question to the queue, the author or anyone else won't be deprived of the ability to have their question reviewed for reopening. I actually previously requested the checkbox in idea 1; see my post for further details.

• Question authors aren't currently notified of their questions being reviewed for reopening, so if the author edits it in an attempt to get it reopened, and reviewers disagree with it, they won't see that anything has happened and will just assume that their effort amounted to nothing. (This is especially applicable if a different close reason applies from the one shown in the close notice - e.g. a question closed as opinion-based that is edited to be objective, but is also off-topic for a specific reason - the review advice says such questions should be left closed, but the author is never told of the other close reason and now doesn't have a chance to fix that.)

• As the review page shows up the edit by default, I've often seen users just review posts with minor edits as Leave Closed without checking the question itself. In many cases, the question itself was closed incorrectly or can be edited to be reopenable. I like how your proposals help limit such edits from even becoming review tasks in the first place, and in the cases where they do end up in the queue, users are prompted for a close reason.

There is one way that step 2 can be improved, though: if someone reviews to Leave Closed, it should open up the full close dialog where someone can choose any close reason, and if all reviewers choose a different close reason from the one shown in the banner, the question's closure will be changed to show that reason instead. Why?

• As I pointed out in my second bullet above, one main use case for that button is if a different close reason applies from the one it was originally closed as. The proposal here is only adding one sentence - the rest of the close banner will still show guidance specific to the earlier close reason and the author won't be directed to the same resources they would had it been closed for the other reason in the first place.

• In the case where the question should be instead be closed as a duplicate (i.e. if the question is closed for a non-duplicate reason, none of the non-duplicate reasons apply to the current revision, and a duplicate exists for the question), the current guidance says to reopen and re-close as a duplicate so that other viewers are directed to an answer. This is inefficient, in my opinion, as the same voters may not come back and re-close the question. It's better to have this close reason change be automatic rather than tediously manual.

This will also address this answer which asks for community-specific reasons to be available choices. This should provide an option to specify that the current close reason still applies, so users don't have to go through the dialog and select it again, which is tedious and also wouldn't work if a question was closed for an old close reason that was removed (but is still enforced).

Answer 14

status-planned

Will the checkbox be viewable in the revision history for the post?

I can imagine arguments for it going both ways, so if somebody does(n’t) check the box while editing a closed question, would their choice be visible in the revision history.

Also: Would whether or not somebody checked the box be visible to moderators?

Answer 15

Once a significant edit has been submitted, editors are still able to return to the edit screen and make further adjustments.

Is there a grace period to the "significant edit" option that allows unchecking it after initial submission by the editor?

I sometimes make an edit only to have a change of mind precisely after hitting the submit button.

Answer 16

If a Reopen votes reviewer elects to leave a question closed, the reviewer will be presented a “Leave closed reason” modal (similar to the close reason modal) that will return feedback to the editors.

What if it didn't enter the queue due to an edit (i.e. through reopen votes)?

• Will it still present the reasons modal, or assume the original reason still holds?
• Will it still update the closed post notice?

Answer 17

What should be done in the case of this feature being abused?

For instance, user X has 2k rep. They edit a question to improve its grammar. They (for whatever reason) check “substantial edit”. What should be done if you view this happening?

Answer 18

feature-request

If I choose to leave a post closed because the edit missed the mark a bit and the question needs more clean-up, but I believe it is salvageable, I might choose to follow the question so that I can see when OP / the editor edits the post again to see if it can then be reopened.

I understand why the queue does not allow the same reviewer to see the post again (or at least, I can see some reasons; I do not know if they were the actual reasons behind this decision).

I suggest an option or a section in the "leave closed" dialog to make it easier to reject the reopen and follow the post. It might be worth adding this to the whole UI, as a post you choose to vote to reopen might get rejected and then you can take a look at it again later (although as far as I remember, you won't be able to vote to reopen it again, so I'm not sure how useful this would be).

This also might make it more clear to users who don't know about the follow feature that they can follow the post to be notified of further changes to it to keep watching it.

Answer 19

I applaud this, especially the part about checking the box to indicate that you believe your edit should send it to the Reopen Votes queue, and the feedback that comes from the queue to the editor. I have seen quite a few edits by a certain user on ! posts (StackOverflow -> Stack Overflow) that unnecessarily throw the question into the queue and basically take up space.

Aside from that, questions:

1. Will the "Improve Question" label be shown to all users, or just to the sub-3K users as it has been in the past?

2. Does the "Reviewers will not be able to see the same question twice in the Reopen votes queue" (another nice touch to avoid adding your opinion 10 times) apply to diamond moderators or other users with more privileges than others?

Answer 20

Can we please get a new reopen queue workflow like the one below?

Answer 21

The "Improve Edit" option (see How do suggested edits work?) should also be available for edits to closed questions that are marked as significant, for the same reasons it exists in general. It would automatically approve the edit, add a new edit, add a reopen or leave-closed vote, and move the question to the "Reopen votes" queue if its reopening decision hasn't gotten enough votes yet. This would result in a total of 5 options (instead of the current 3):

• Approve and reopen
• Improve edit and reopen (added)
• Approve and leave closed
• Improve edit and leave closed (added)
• Reject edit

Answer 22

I was unfamiliar with the community rules, and then I asked a question that was closed, and then prompted that I could post a new one. But there's a catch to this, and that is to delete the old question first. Because when I didn't delete the old question, my new question was closed again. The reason is: Your post has been associated with a similar question.

So I think there should be some tips for newbies, let's delete the old questions before asking new ones.

first edit 2023-3-14

If, as you say, post a new question is a bad idea, then this should also be prompted. Then in your reply I realised that the question was closed, just not answered, but viewable. I don't think this makes sense, as the question should be closed and not viewable.

As the questioner, I want the question to be answered, it doesn't make any sense to me that it can be viewed without being answered. As an answerer, if I see a question that can be viewed but not answered, the question should be skipped. As a general user looking for an answer, it doesn't make sense for me to find the question, because the question doesn't have an answer.

second edit 2023-3-14

@RobertLongson You can check out the link above, which proves that this is not a problem for me alone.

The original words were You can edit the question or post a new one. My English is not good, but or should that mean that the two choices do not have priority. Not that the first one is your preference?

I don't agree with what you said However in general many people do interpret asking a new question to mean asking the identical failed question again. repeating your mistakes. I think the reality is the opposite of what you are saying. Just like you said, repeating your mistakes will not result in a better outcome.I think that's what most people know. I have to emphasise that again the importance of the hint, the reason why the question was closed is very clear. So what most people do, should be to build on the old problem, try to fix the error and then ask a new question. As I said before it is a similar question and not a same question.

third edit 2023-3-19

The following inferences are based on the premise that my guess is correct:

Conclusion first, The current review queue mechanism is a bad experience for posters. I emphasize the present, not the review mechanism itself.

My question has been closed for 4 days and is still in review, I don't get any progress feedback, I guess the reason should be that no one has reviewed it, I think it will probably wait longer for feedback.

The reason I chose to edit the question instead of asking a new one is that this question already has an answer, and if I delete the question, the answer will be deleted as well (my guess).

Now I prefer to simply delete the old question and ask a new one.

I'm surprised that most people are discussing the design of reopening the queue, but no one pays attention to the actual experience of the poster? Based on this first experience, I don't think you can blame most people for opting to delete the question and ask a new one, Even if the question is not modified as you expected.

Lastly, I'm curious can someone tell me how long question that were successfully reopened spend on average in the review queue? I think it's an important indicator.

fourth edit 2023-3-22

Closed 7 days ago.