Closing changes: [on hold], unclear, too broad, opinion-based, off-topic reasons, bye-bye to Too Localized

Question

After a ton of discussion, work and input from the community, we're rolling out the remainder of major closing changes that you've helped us design, as discussed in these prior posts.

They're live, network-wide RIGHT NOW!!! See also: https://blog.stackoverflow.com/2013/06/the-war-of-the-closes/

The close reason rework project:

1. Changes to "close as duplicate" (part deux)
2. Help us make "Off-Topic" close reasons clearer to the OP
3. Help us make "Not Constructive" and "Not a Real Question" closures more effective
4. Every "close" has its thorn: replace "close" with "on hold" for the first five days

NOTE: This post does not address all the details of the interface, etc. for each change. Its purpose is to summarize the changes in one place with enough detail to convey what's happening.

Why are we doing all this? Isn't closing good?

Yes, closing is great. It's one of the main things that stands between us and Yahoo! Answers. And it's critical to how we keep quality up. But over time, we've come to believe that it can be improved in a couple of ways that won't undermine its effectiveness:

• The tone of the current verbiage (both "Closed" and things like "Not Constructive") tended to elicit argument and debate, rather than improvement

• Even in those cases when a poster did improve their question, the odds of it getting re-opened were extremely slim, as there was no natural path to re-opening for improved posts

• Many of the the close reason descriptions aren't specific enough to convey what exactly the OP needs to do to fix the question (I'm talking to you, NARQ and Not Constructive) or what made the question off topic ("It IS about programming...")

So, here's what's changed/changing:

1. Duplicate changes (live since Feb)

• Duplicate question must link to a question with an answer
• Questions closed as dupe show as [duplicate], not [closed]
• The duplicate language is designed to read more like a pointer to your answer, rather than a dead end

2. Questions edited by the OP within five days of closure go into the re-open queue (live since Feb)

• Previously, posters often had to resort to a meta post to get an improved question considered for re-opening
• Now, timely edits by anyone are proactively pushed into a process for re-opening review

We may consider letting later edits trigger addition to the queue at some point.

3. Questions will display as [on hold], rather than [closed] for the first five days after closure. (new)

• The goal is to better convey that questions can be improved and re-opened during the period where they have the best shot at that happening
• Questions put [on hold] will still not accept answers, and will behave identically to [closed] questions
• The language will change to [closed] if the question is not re-opened within five days, to continue to serve as a clear long-term signpost

4. "Not a real question" and "not constructive" are being replaced by the following: (new)

• unclear what you’re asking — Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking.
• too broad — There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.
• primarily opinion-based — Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.

In each case, the language is much more specific about what needs to be changed to be acceptable here

5. Off-topic closures will include feedback on what specifically is off-topic for that site. (new)

• Each site will have a list of its own specific pre-selected “Off-Topic” reasons
• Each closer will either select one of the site's standard reasons from the list (for instance, “Recipe requests are off-topic, although recipe replacements, etc. are allowed”),
  or,
• Closers can enter a free-form reason ("Your question appears to be about 'Cat Grooming', which is off-topic for Stack Overflow.")
  Free-form reasons will be presented as comments, but the close dialogue will refer the reader to the comments for more info
• Free-form reasons picked by closers will be available to subsequent close-voters on that question as one of the selections from the list
• These lists will be determined by the communities, and moderators will be able to update them, subject to review by each other, their community, and the SE team

Reasons will need to be specific enough to make it clear to most readers what is and is not allowed (off-topic reasons of the form "Things that are NOT X" will be discouraged).

This is also the place to address any closing reason that applies to one site but not others (for instance, the "General Reference" close reason on English Language and Usage is moving here).

5(A). "Too Localized" is no longer necessary since the specific off-topic reasons now address its main use case. (new)

Too Localized was, by far, the most misused close reason in our surveys, with both Community Managers and Moderators deeming over 50% of randomly sampled TL closures to not have merited closure (including on SO).

Today, TL is used helpfully on code dump questions on SO, but the new OT reasons are where this is now correctly addressed. SO can use "Large blocks of code with requests for debugging without meaningful supporting info" as a specific OT reason.

Here's what the new list looks like:

Questions closed prior to these changes will not be mapped to the new reasons, as it's not a 1:1 mapping, with the exception of Duplicates and Off-Topics. The others will continue to reflect the reason picked at the time they were closed.

Answer 1

I think the changes are okay but they still don't fill the gap that Too Localised provided earlier. I used Too Localised as a way to close certain questions that could definitely not help anyone and trying to put those under "Off Topic" feels like a stretch to me which makes it seems way too Catch-22 and might not fix the problem you initially wanted to fix. With the inclusion of using "Off Topic" to be more broad, it makes it seem as if the effort put into being more descriptive has failed outside of its changed domains.

Consider the questions that are basically a copy paste of a homework question. With the current system there is no way to actually close it due to its low effort. Too Localised might have done it but closing as Off Topic is a bit of a stretch because it is well within the topic of SO -- a programming question. It definitely isn't Too Broad and it can't really be Unclear because if it was then the assignment was bad to begin with. It can't be a dupe because it's probably only applicable to the user themselves. So what close reason do those get? Unfortunately none outside of the stretched Off Topic catch-22.

It isn't just homework question copy pastes either. It's other low hanging fruits like forgetting a semicolon. That isn't Off Topic, it isn't Too Broad, it isn't Unclear, and it isn't a duplicate. Then there's other questions that show no effort for debugging or attempting to solve the problem themselves. Is this Off Topic? I doubt it. Too Broad? You might be able to stretch it, along with Unclear.

I feel that this is a step in the right direction -- but we still have some holes left in the close reasons that don't allow us to be descriptive with our closes. I feel the solution PeeHaa 埽 suggested with a "Low Effort" close reason might alleviate this problem.

Answer 2

I like most of these suggestions. I think it is well worth making the changes outlined here but there also needs to be some consideration for preserving the community from becoming a job shop environment.

Please do not remove "Too localized". This close reason was spot on. Clearly some of the other close reasons needed work but "Too localized" filled a niche.

I think @Jeff Atwood said it best

how can you have "too broad" without a corresponding "too narrow"?

link to comment

Too localized is what would fill the gap in order to close questions which basically request a feature from the community or (as noted above) are missing a simple syntax correction (such as a semi colon). "Unlikely to help future visitors" might be too direct, but it is entirely accurate.

Without a "Too localized", or similar, reason then under the current list laid out here it will be nearly impossible to close a question which requests specific code for a specific issue without highlighting any issue other than "I was too lazy or could not accomplish this task without learning, please code it for me".

Previously, a question could skate past duplicate because it was original, it was on topic in that it was definitely in the same scope of most of the other questions, it was a real question in that it made actual reference to code requirements and a base approach, an answer would be supported by facts and there would not be any extended circumstances or opinions. The one caveat is that the question ends in "I need this to b and all it does is a right now" and was caught by "Too localized" since it would only help that one user for the community to implement their feature.

With the current changes, that question would be somehow pigeonholed into "Off Topic" for reason of "Too localized". How did "Off Topic" somehow become a parent for every other close reason that one of the exchanges could consider? On-Topic seems to be defined as matching the scope of the exchange and plenty of request for work questions match the scope, show effort, and reference fact.

Answer 3

"Too Localized" is no longer necessary since the specific off-topic reasons now address its main use case.

The problem with "off-topic" & why we need "too localized"

We've talked a lot on Meta about closing and migrating questions and one of the things that sticks out is that a lot of crappy questions (used to) get migrated with the result that the receiving site is not happy about it. I believe this is why ServerFault was removed as a migration destination from SO in the Off-topic list.

I think the consensus was (in fact, I don't even think it was controversial) that if the question is not a good question to begin with, it doesn't deserve to be migrated, it should just be closed. Labeling a question as "off-topic" strongly implies that it would be a good, on-topic question elsewhere. How do I keep my puppy from getting fleas? is off-topic; we want the OP to take that question and post it somewhere like dogforums.com. When I type '1+1' into my calculator it displays '6' is not something we want to encourage the OP to post elsewhere, it's something we want them to reformulate into a question of broader interest.

"Off-topic" should be reserved for "good" questions that would be welcome elsewhere but are just not right for this site. "Elsewhere" doesn't even have to be another SE site, but "off-topic" should not be a dumping ground for crappy questions or questions that do not have answers or questions that are no longer relevant for whatever reason. Remember, the purpose of giving a reason for closing is to guide the OP and others toward appropriate action. When someone asks a a question like What is the max bandwith of LAN and WAN network, we want to give them guidance that both helps them get the help they are seeking and keeps them from being a further burden on SE. Telling someone with a question that no one wants to (or can) answer that it is "off-topic" is not leading them toward improving the question and getting an answer, it's leading them toward finding a different forum in which to ask it and wasting more of our volunteers' time. (In this particular case it got migrated from SO to ServerFault where it had to be closed again.)

"Too localized" should be renamed, but IMHO the correct essence of that label was well described in the previous explanation of it: the answer to this question is unlikely to help anyone else. (Of course, "only relevant to a small geographic area" was kind of bizarre for a programming question, so I'm not saying we shouldn't clear up the description.) This is not a question that would be better asked on another site, this is a question that would be better asked of a teacher, coach, or colleague. It is also a question that can be made better by simply broadening it from a specific question to a general question, which on SO will usually lead to the OP figuring out the problem on their own.

Since we have "too broad" as a close reason, let's replace "too localized" with "too narrow"

There are two types of too narrow that should be covered in the explanation.

One, the question is too specifically related to the OPs particular situation. It is a question not about a general principle or even the specific behavior of a publicly available resource, but rather it is a question about a situation that no one else will find themselves in because of the details that form the basis of the question.

Two, the question is about a situation that does happen to enough people to be of interest, but that question has an answer that everyone else is happy (enough) with, so attempts to further answer the question are futile. (I speak from experience.)

Fixing the old problems with "too localized"

While looking for examples of questions that were "too localized" I was rather dismayed to find that more than 75% of the questions I found with "too localized" votes were, in fact, not "too localized" at all. In other words, I found what the other surveyors found. So I agree we need to do something about that. It seems most of the questions were more along the lines of "too basic" or a highly specific instance of a general class of problems that would be of interest if presented in a general way.

I think that by changing the label to "too narrow" and improving the description of it, we can avoid a lot of the problems.

---

Aside: if you want to go the extra mile, you could implement a list of reasons to choose from when someone votes to "Leave Open". E.g "Basic Questions are Allowed: Questions that are very basic in nature or could be answered with a quick search are allowed unless they are duplicates." or "Illustrates a General Problem: questions that are highly specific examples of problems of general interest are allowed. Even though future users may not have exactly this problem, the answer to this question will likely help others with similar problems." The point of these reasons for leaving open is that it encourages a dialog within the voting community which can help us converge on a consensus and eventually lead to more consistent voting. But please, leave the idea of reasons for voting to leave open out of the discussion of this answer (or this question overall). Head over to the feature request if you really want to get into it.

Answer 4

I think we can simplify even further, at least for English Language &Usage:

Answer 5

Can we keep the migrate to another site in the Stack Exchange network on the first page? Right now, I have to click through two radio buttons and then type the name of the site I want to migrate to.

That's quite a bit much, there's no reason for the extra click, especially since the dialog can expand to fit the site migration box.

If this was accessible only to moderators (and it looks like it is) then that's fine.

Answer 6

The off-topic reasons under "Off-Topic" ought to stand out more. That is, the summary of the off-topic reason should appear in bold. As it is currently, it's hard to pick one without reading them first, since there's no at-a-glance overview. For example, the reasons could appear like this:

• Programming questions are off-topic on Meta Stack Overflow. Please refer to how to ask on Stack Overflow. See also: Why are questions no longer being accepted from my account?

• The problem described here can no longer be reproduced. Changes to the system or to the circumstances affecting the asker have rendered it obsolete. If you encounter a similar problem, please post a new question.

• Other (add a comment explaining what is wrong)

Or they could be shortened:

• This is a programming question, so it's off topic.

(This shouldn't preclude using the previous text as what actually appears in the box next to a closed question.)

Answer 7

Today, TL is used helpfully on code dump questions on SO, but the new OT reasons are where this is now correctly addressed. SO can use "Large blocks of code with requests for debugging without meaningful supporting info" as a specific OT reason.

But that reason isn't even there!

Answer 8

There is one thing that really does not feel good about this redesign, and that's the focus on having essentially everything an "off-topic close reason".

The choices you're essentially given are:

• too broad
• unclear
• subjective
• some already defined off-topic reason
• define a new off-topic reason

I suspect that this will lead to a lot of lawyering about just what the defined off-topic reasons actually cover, and a lot of meta reasons that exist for no other purpose than to cover what isn't already covered.

At that point, a site no longer adjusts its topic definition, or scope, it just tries to turn it into a "safe" environment for moderation - something that doesn't cause unnecessary discussion, complaints and the such. That just isn't about what is and isn't on-topic anymore.

Answer 9

Since the banner now displays close reasons point by point, could we get the ability to VTC as more than one close reason? (if not for everyone, at least for moderators)

It would be quite a bad experience for a new user if s/he sees "please do X to get it reopened" and is later told "No, you need to do Y too" after having done X. It's better to be able to list all post issues at once (which the current system does in the case of community closes where everyone picks something different).

I've seen plenty of fixable questions which are NARQ and NC (UWYA and POB now). Some questions are even a combination of three or four close reasons. (now that TL is no longer there, this may become rarer).

(Not sure if I should post this as a separate feature request)

Answer 10

Good but, but "unclear what you're asking" is worded a bit differently than the rest of the close options. It reads like a private message to the asking user, not a message to the moderator trying to close it.

Right now it reads:

unclear what you're asking. Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking.

Suggest:

unclear. As it's currently written, it’s hard to tell exactly what the question is asking.

If that's too unclear for ya

unclear. As it's currently written, it’s hard to tell exactly what the question is asking. The problem in the question needs to be clarified more specifically, and additional detail should be added to the question to describe exactly what the questioner needs.

I think the latter answer is too long to read at a glance, however.

Answer 11

"primarily opinion-based" doesn't quite fill the void that's left by Not Constructive. What do we do about big-list questions? If you ask "What are some books to learn ", it's not necessarily opinion based. It's just a list question, which the SE engine isn't particularly good at. Till date these have been closed as NC on most sites. Should the mods of each site add it under "off topic"?

(see also: https://meta.stackexchange.com/a/176247/178438)

Answer 12

OK, cool, but you're still lying to the entire world about what I think of, say, this question:

I voted to close as ... well, with these changes, I don't even remember what it's called. But it certainly wasn't duplicate — that duplicate is totally wrong.

Answer 13

On thing regarding the missing "too localized" that I believe is misstated in the question post (or perhaps the system has simply been modified since it was posted):

[...] SO can use "Large blocks of code with requests for debugging without meaningful supporting info" as a specific OT reason.

I wasn't able to find that close reason in the off-topic list of reasons (except that there is a free-text "other reasons" field where I could have entered it).

But on closer inspection I found this off-topic reason:

Although I was shocked at first when I realized that the "too-localized" close reason had disappeared, I actually believe the SSCCE-reason above is a good replacement. Most large chunks of code submitted as requests for debugging do not match the criteria for an SSCCE (because they are not examples of a specific problem at all), and using this as close reason seems appropriate. (I welcome comments telling me that I misinterpret this close reason if that's the case.)

It may be a good idea to include the spelled-out version of SSCCE ("short, self-contained, correct example") in the close reason explicitly though.

As a general comment, I think the new system of close reasons, and the new "on-hold" mechanics behind it are a great step forward. In the past I often found myself adding comments to questions I had voted to close because the official close reason was misleading. This is much better now.

Answer 14

The new hold notice appears to be missing a description of "what happens next."

We tell the user why the question was put on hold:

put on hold as ...The users who voted to close gave these specific reasons:

Then we tell the user what to do:

If this question can be reworded to fit the rules in the help center, please edit the question or leave a comment.

However, we don't explain why they should do these things or what will happen when they do. I suggest we add a line such as:

Your edited question will be placed in a queue for re-evaluation. If reviewers agree that it matches the standards described in the help center, it will be reopened.

Answer 15

After seeing these changes in action for a little while, I just wanted say YES!!! They appear to be working spectacularly well.

The immediate change I've noticed is a significant increase in the number of questions appearing in the "Reopen" review queue, and more importantly, a significant increase in the quality of the edits made to questions that put them in that queue.

In other words, it's working: people who ask poor questions used to see their questions closed and just get annoyed with SO for being elitist. Now they're seeing their questions put on hold and with a useful description of why, and they're responding by editing the questions to include better quality information.

Sure, there are still people who just can't get it right, but this change does seem to have made a big difference.

Well done to the SE dev team.

Answer 16

I think the issues with "Too Localized" and "Too Broad" can probably be condensed into one fundamental problem. Certainly in some cases a broad question can be a good question, and in some cases a very specific and localized question can also be a good question. The common feature, I think, in both cases, however, that makes them a bad question is when they are questions whose answers require implementation of a programming project rather than a solution to a programming problem.

In the case of "Too Broad" the question typically reads like OP is a poorly literate version of management who is throwing out a vague set of specifications for which they would like a software solution. OP in this case generally has very little idea of what they are doing, doesn't know where to start, and is in way over their heads - the SO post is a desperate attempt to have someone do the work for them. The only expected result is "I need codes" and the problem is "I don't have codes".

In the case of "Too Localized" the question is generally of the same nature, but rather than encompassing an entire software solution it generally focuses on the implementation of some esoteric functionality that is of little general use to anyone but, contrasted with the above, is laid out in software engineering terms rather than vague management terms. Like the former case, however, the solution generally constitutes a programming project which OP is out of their depth with and, while the project may be described with great specification as contrasted with the "Too Broad" case, it nevertheless also touches on a myriad of tools and techniques required to accomplish the stated goal. This type of question is often the "large blocks of code filled with errors that won't compile for uncountable reasons and won't work for as many reasons more". The only expected result is "it should work" and the problem is that "it doesn't work".

In both cases, OP fails to successfully break the problem down into its constituent components. Rather than generate a series of general questions, the sum of whose answers would form the foundation of the required elements for implementing their project (and which individually answered would each be of general utility outside the scope of their project) they resort to simply describing the quandary they find themselves in and asking for help.

I'm not sure how this can be useful, of course, but at least it highlights, I think, a type of question that reflects a very specific deficiency on the part of the asker - namely that the question in question has not been sufficiently analyzed and reduced to its essential constituent parts and that OP is lost out of their depth. Whether anything useful can be implemented in the SO interface to address this is an open question. These types of askers need most to learn how to learn but I'm not sure that there is any readily available resource or method that they can be directed to that will help. Perhaps some simply can't be helped and we just close it and be done with it...

I guess to sum it up in a few words :

The question currently describes a programming project and not a programming problem. To adequately answer this question would require solving a large number of programming problems. Try to isolate specific problems and ask them as separate questions.

Answer 17

The page is not written in comprehensible English

I'm sorry, but the wording should be vetted at English SE.

"This question does not appear to be about on-topic, within the scope defined in the FAQ"

About on-topic, seriously? Is this standard English?

As Yannis wrote,

"Unclear what you're asking"

You are barking up the wrong tree, m'lord. "I" am a humble reader who'd like to keep SE relevant. "He/she" is the original poster.

Recently, there's been a spate of re-designs in Stack Exchange, which would have benefited from our community's involvement beyond MSO - User Experience and English SE, first and foremost.

Answer 18

I haven't attempted to close anything in a while, but today, I decided to review a few questions in the close queue. Almost every one of the questions was marked:

off-topic: This question does not appear to be about programming within the scope defined in the help center.

Not having read this meta question, I was baffled how any of the questions fit into that category. They were all clearly about programming and didn't fall under what I would consider "off-topic". In fact, I'd say the most common issue was "not enough research" or "no attempt to solve".

I didn't know that choosing "off-topic" would lead to another list of problems, many of which had nothing to do with the topic. To make matters worse, the "review close votes" screen made no mention of the off-topic sub reason, only the basic (flawed) definition of off-topic.

So, while I like that there is a more specific list of problems, lumping them all under "off-topic" doesn't make sense to me, and the usability of this system from the POV of the user doing the closing could be better.

My suggestion would be: flatten out the hierarchy

I won't try to suggest what the specific items should be, as there is already a ton of discussion on this above, but I would definitely like to see you bring everything that isn't about the "topic" per se up a level so it doesn't confuse the closer and/or the closee.

Answer 19

Some rant from an user.

Often, I search Google when I have a programming question (as many of you might do). I do this more often when it's a language I'm just learning or re-learning. It's my primary documentation tool, and sometimes it leads directly to the official language documentation. I might search for "How to concatenate pathnames with ruby" or "How to delete a key from hash tables on Ruby", I do so expecting a quick overview and ready-to-use sample code, so I typically I want to se a code snippet using File.join and hash.delete() in the first click.

I have noticed that many Google results are of bad quality. There are a lot of blogs that are very verbose on their presentation yet don't deliver the essentials. Usually I have two preferred search results: the official documentation and Stack Overflow. But SO can, for example, suggest third party libraries and warn about common pitfalls, something official docs seldom do. SO is useful as a task-oriented programming documentation.

For SO work as I use it, it needs to have low-hanging questions. To cite some questions in the comments, "How do I create a Linked List in Java?", "How do I declare default parameters in C#" or even "What is the syntax for a literal array in Java" are, together, essential for this site, so I find it sad to see them labeled as 'low-quality'. Many (or most?) programmers are beginners, and SO is most useful for them. It is also common to have learned a language a long time ago, and after a while re-learn the specifics, even basic language constructs. I see the literal array question by itself is kind of dumb, but when picking up Ruby after some years I found myself using Javascript syntax and could as well Google for the actual syntax -- and it's natural to land on SO when googling programming questions. If such questions weren't already asked and provided with quality answers, SO wouldn't be as popular as it is now.

This rant isn't specifically about homework, but I understand a homework question may be an useful beginner question. In this sense, they wouldn't qualify as "too localized", even if lacking in some aspect. Other homework questions may not be useful. Describing a very detailed project and expecting people to work on all its parts is normally not useful for the community at large, and typo questions are useless, so there is that.

Answer 20

The following is inspired by a discussion over custom close reasons at Biology SE.

Many feel the need for a close reason that expresses that the basic premisses of the question are incorrect (factual errors, etc.). Shouldn't there be a need for this close reason on many (most?) SE sites? I first thought that the close reason "primarily opinion-based" could be interpreted to cover this, but it is clearly aimed to discourage general discussion-style questions without a clear answer. At Biology SE this problem often comes up on "controversial" issues such as evolution and parts of human biology, where some questions are ideologically based and built on flawed premises, and where the OP refuses to modify the phrasing of their question to make is answerable. I can imagine that this problem should come up in many science-based SE-sites.

In the discussion I'm referring to above one example/suggestion for the close reason was given (as a starting point for discussion):

The premise of your question relies on factually incorrect information, and thus, your question cannot be answered in its current form.