I'm getting pretty sick of this meme that the title check being run on Stack Overflow (and Super User and Server Fault) is somehow ineffective, obviously broken, etc. etc.

So, I'm going to drop the science hammer on it.

Before we get into the numbers, I'll outline the inspiration and purpose of these title quality checks.

In a nutshell, there are a few phrases ("problem" and "help" being the most obvious) that are really really strong indicators that a post could use some work; so we slap a "no you don't" hurdle in front of users forcing them to rework the titles at least. Titles are really important, both from a "quality of the site" perspective (who wants a homepage full of Help! posts) and from a "getting your question answered" one (if you're hunting for a question to answer, the more detail in a title the better the odds you'll click on it).

To be clear, the desired outcome of the title quality check is to encourage users to add more detail to (and remove useless, low quality indicative, phrases from) their question titles.

I'm looking at the 200 most recently rejected titles that were still eventually posted as questions. Each of these events will be sorted into one of three buckets based on the changes made to the rejected title: actively helpful, benign, and actively harmful.

actively helpful changes are those that result in more detail being added to a question title. This could just be additional technologies being mentioned, or a more thorough re-write (provided it's still the same question, naturally). Examples would be "heap size problem" -> "Android maximum heap size", or "Technology X Question" -> "Technology X Validation".

benign changes are simple deletions of the offending phrase (problem, help, and what have you) or simple substitutions. "Problem with X" -> "Issue with X" or "Heap allocation problem" -> "Can't allocate on heap" would be examples.

actively harmful changes are the "Pr0blems" (or random "s"s, or awkward non-sense that's clearly meant to dodge the filter) that some are insisting happen every time a user trips over this restriction. Also included are "blind idiot deletions", where a user just deletes words until they stumble around the filter, as that's clearly not "trying to make the question better" behavior.

Final outcome?

actively helpful - 37% of all titles gained some detail
benign - 62% of all titles became no worse
actively harmful - 2% (3 total) posts obviously hacked around the filter

We can break down benign even further, into deleted and replaced. Deleted would be posts that removed the offending word (Problem, help, whatever), replaced meaning that something else we don't check for is slapped in there (Issue, confusion, so long winded way of saying Problem, etc.).

Deleting one of these phrases is a net win. They are the salutations of titles, adding nothing. It's not a strong enough win for me to lump them into actively helpful, but it's still a better outcome than posting the original title.

Replacing one of the phrases is a completely wash, GIGO-style. All the filter accomplished in this case is wasting a few moments of someones time (though they arguably deserved it).

Sub outcome?

deleting - 66% of benign modifications removed pointless verbiage
replacing - 33% of benign modifications were just substitutions (though not harmful ones)

Some other quick stats, these 200 posts occurred over a 24 hour and 30 minute period during which 5166 questions were asked (so approximately 4% of all questions tripped this check). Of these 200, there was a single "false positive" (the title really did need "question" in it, in this case); however that post was ultimately deemed off topic and the owner self-deleted it.


Factoring in reputation, I looked at users > 150 rep. The same ratio holds (76% improved in some way overall; 40% actively helpful, 0% harmful), though our sample size is getting awfully small (43 users). For >2k rep users only 4 posts triggered this, so it's hardly conclusive, but 75% of those were actively helped with the last post being a deletion (so 100% improved in some small way). The theory that reputation indicates title quality is lacking in support, although aforementioned sample size concerns (exacerbated by the rarity of high rep users asking questions) do exist.

tl;dr - the question title check is pretty effective, contrary to what has been asserted (rather blindly) elsewhere. ~78% of all post titles that trip it are improved in at least some small way, 37% in an unambiguous way. "Hacking around" the filter is in fact very very rare.

  • 2
    Can you share your data, Kevin? Commented Nov 21, 2011 at 23:39
  • 10
    @Michael - there's no way to anonymize this data (since we're looking at specific titles and titles are now unique ), so no I won't be sharing it. I'm not comfortable publicly shaming 200-ish users. Commented Nov 21, 2011 at 23:52
  • 9
    I get that, Kevin, but without it, this hardly seems like either "science" or a good way to close discussion on this question. I think you're begging the question. Commented Nov 21, 2011 at 23:59
  • 1
    (I wouldn't consider it "shaming" particularly, myself.) Commented Nov 22, 2011 at 0:01
  • 12
    "Shaming users"? I don't particularly see the problem. As easy as that...
    – slhck
    Commented Nov 22, 2011 at 0:02
  • 2
    @slhck: nice, but not the same dataset as what Kevin used. His is only available to devs. Commented Nov 22, 2011 at 0:06
  • 1
    @Michael I know, I just wanted to point out that it's not like you can't easily go ahead and query for posts that will somehow shame a lot of users.
    – slhck
    Commented Nov 22, 2011 at 0:08
  • 10
    Not A Real Question.
    – user1228
    Commented Nov 22, 2011 at 16:04
  • 2
    Wow, I'm surprised how effective that is: I am thus no longer opposed to the word filter. It'd be nice if there were some way around it for the 0.5% that need it, but I understand that not being a high priority.
    – derobert
    Commented Nov 26, 2011 at 11:02
  • I agree - the filter definitely helps the quality of question titles and the only place it's ever been a problem for me is on StackApps where question titles also double as 'one-liners' promoting an app / library. Commented Nov 28, 2011 at 19:34
  • 11
    -1 didn't notice how that science hammer dropped supports the idea that a veteran higher-reputation user can't use the word "problem" or "question"
    – gnat
    Commented Dec 2, 2011 at 17:00
  • 4
    Thanks for the data. But would this not have been better posted as an answer on the current question related to the filter, or as a blog post?
    – razlebe
    Commented Dec 5, 2011 at 9:42
  • 3
    Quoting an error message in the title should be allowed. If not the quality of the title and the chance of the question being answered goes down.
    – Coyote
    Commented Jan 3, 2013 at 23:18
  • 1
    -1 this is not science. Commented Aug 27, 2013 at 2:31
  • 1
    None of this explains why only the singular form of the word is blocked.
    – jscs
    Commented Nov 28, 2014 at 0:33

4 Answers 4


I'm neither upvoting nor downvoting. I have never doubted there is a lot of sub-optimal content that gets improved at least in a small way this way, and I've always admitted as much.

However, the idea that a veteran higher-reputation user can't use the word "problem" or "question" in a post any more - even when it makes perfect sense, or even is the only right term (like in a defined math or programming problem) - remains an insult to our intelligence. (A slight one, but still.) If you have managed to ask a number of good, highly upvoted questions, you are supposed to be trusted by the community to contribute half-way decent content. There is a ton of other things that you get the more trusted you become. Why not a relaxation on these checks as well?

Blocking "problem" and "question" is not a quality filter. It is a filter that uses circumstantial evidence to assert a quality problem with the title. Hence I don't think your argument in the related feature request:

the same quality filters and rules apply to everyone.

really applies.

  • 1
    I was going to post a an answer conceding to Kevin's findings, but adding a comment that would turn out just like your answer. So I'm going to +1 this answer accordingly. Commented Nov 22, 2011 at 0:26
  • 4
    +1 another fine post by Pekka, I beg the question again, exactly when will Pekka be hired?
    – JonH
    Commented Dec 2, 2011 at 20:02
  • @Pekka, in light of Kevin's numbers I think you and I do agree on the moral principle. I wouldn't quite call it an insult, but that's indeed an inconvenience that has bitten me in the past. However, I'm still dubious about how we should implement the thing. You're referring to veteran high-rep users, and that lets me wondering. Commented Dec 9, 2011 at 20:57
  • For instance, I've been participating in SO for little more than a year, so I'm hardly a veteran. Should the problem filter apply to me? On the other hand, I have more than 10K rep, should I be immune to it? What about meta, where I have neither seniority nor rep? Should my veteranism on SO allow me to bypass the filter there? All in all, that still looks to me like a very subjective decision to make. Again, I don't know if rep elitism would actually serve us here. Commented Dec 9, 2011 at 20:57
  • 3
    @Frédéric if you have 10k rep on any SE site, you have shown that you understand the system to some extent. I think rep elitism would help here. (However, shlck has posted a very interesting suggestion that I find even better: Block, but automatically flag "Problem" attempts in the title as "low-quality"
    – Pekka
    Commented Dec 9, 2011 at 21:02
  • @Pekka, so a 10k rep filter override would apply to all the other sites and their metas? (10k being an example, that might be less.) That's... interesting. Also, shouldn't shlck's post be titled Don't block, but automatically flag “Problem” attempts in the title as “low-quality”? I'm confused (but supportive of the request if I understood it correctly). Commented Dec 9, 2011 at 21:12
  • @FrédéricHamidi re the 10k: Yes, that's what I would support. Re the title, you're right!! I'll add a comment.
    – Pekka
    Commented Dec 9, 2011 at 21:16

"Science hammer" huh? Can I get one of those for work? It would be really helpful.

Of course, it's nothing like science, as it's not peer reviewed and there's not enough data for us to evaluate your procedure and results. (As a hint, your rounded percentages don't add up to 100%. If you had provided more raw data, we could calculate the true percentages ourselves.) Plus you've not provided even one poorly-formatted graph. ;-)

But let's concede your analysis: the question filter is super-effective. Now that we have the data, we can begin the debate.

Did you know that wearing a seat belt is super-effective too and that lots of governments have enacted seat-belt laws? But not all of them, because despite saving thousands of lives, seat-belt laws are unpopular. Of course there are some people who don't like seat belts, but even people who do wear them sometimes oppose a law requiring them. Principles such as liberty, personal property and the role of government come into play.

Then there are implementation details. Should Grandpa's Model-T with a top speed of 35 MPH downwind have seat belts retrofitted? In some places, "yes" and in others, "no". Do you need a three-point harness in the front seat or is a simple belt enough? The run-up to the date these laws are enacted is filled with news stories and clarifications about what is and what is not required. People die in car accidents who would have been saved by seat belts in that time period.

There's a (possibly apocryphal) story that when Italy enacted a seat-belt law, it became fashionable for men in Naples to wear white T-shirts with a diagonal line across the chest. To passing Polizie it would look like these men were wearing their seat belts. Do you think showing them statistics that a seat belt will save their life would make them obey the law? Of course not.

You see, there's a subtle difference between changing a line or two of code that affects the users in an unpopular way and changing a legal code. Or rather, it's totally different. And when you move from the technical sphere to the social sphere, the rules of engagement change. It isn't just about truth and numbers and science, it's about how we all are to get along.

I'm all about doing what's right even if it's unpopular. Let's filter the hell out of question titles! Stand firm in your beliefs. Don't back down. But you still gotta treat people well and at least give them the illusion that they have a voice. You gotta at least understand why something is unpopular and wave your hands in the direction of acknowledging them. You gots to stop actin like folks are idiot even when we are.

Look, I'm never going to be affected by this law one way or another and it's none of my business how StackOverflow is governed. But I really hope that some of the smaller StackExchanges I participate on will have the sort of problems that StackOverflow has had almost from the beginning. And if they grow, how can I trust that I and other members of the community will have a say in new laws? Because right now, I see people who are more involved in StackOverflow than I ever was acting seriously pissed off about this law. And it sure sounds like the people in control of that site are not listening to them. And that scares me.

  • By the way, you wouldn't have me bothering you if there were a separate site for people who just want to ask about the SE engine. Think about it! Commented Dec 9, 2011 at 18:13
  • 1
    So you're saying seatbelts shouldn't be the law because they're (supposedly) unpopular? Also, codinghorror.com/blog/2005/05/… is an interesting read.
    – balpha StaffMod
    Commented Dec 9, 2011 at 18:31
  • 11
    +1 TL;DR version: how can I trust that I and other members of the community will have a say in new laws? Because right now, I see people who are more involved in StackOverflow than I ever was acting seriously pissed off about this law. And it sure sounds like the people in control of that site are not listening to them. And that scares me.
    – gnat
    Commented Dec 9, 2011 at 19:13
  • 4
    @gnat if you aren't pissing someone off some of the time, you're not doing anything very useful or interesting. Commented Dec 9, 2011 at 22:24
  • 3
    @Jeff: That sounds like a post hoc ergo propter hoc justification to me. Can you please point me to the place where all of this was discussed beforehand? The closest I could find was this question on "bad titles", but there's not much discussion there. Why ought I trust you? Commented Dec 9, 2011 at 22:45
  • 9
    @Jeff - of course the inverse of your logical proposition—if you are pissing people off, you're doing something interesting or useful—isn't necessarily true. In this case it certainly isn't. Commented Dec 10, 2011 at 3:30
  • 2
    @adam the data doesn't support your position. Commented Dec 10, 2011 at 4:30
  • 3
    @Jeff - I know you hate pluralization bugs, but it should be the data don't support your position :) But whether this is useful is a value judgement. You're forcing some probably mediocre questions to have slightly better titles, while prohibiting users from asking intelligent ones: halting problem select n + 1 problem NP Complete Problem et cetera, ad infinitum Commented Dec 10, 2011 at 4:36
  • 2
    @Jeff, more importantly, the people I've interacted with on SO are some of the smartest people I've ever "met"; it's hard to believe I've learned so much here. If these people are lining in such numbers to decry this policy, I urge you to exercise some epistemic humility and consider the possibility that you might be wrong Commented Dec 10, 2011 at 4:43
  • 2
    @adam I urge you to look at the data. Commented Dec 10, 2011 at 4:56
  • 4
    @Jeff, I've looked at the data. I urge you to consider the possibility that changing 50 mediocre questions here and there to have slightly better titles does not justify limiting what good users can ask on this site. Look at the people who disagree with you. Commented Dec 10, 2011 at 5:06
  • 2
    @adam it's logic. youtube.com/watch?v=Xa6c3OTr6yA also, this "your theoretical restriction of my rights that I will never actually encounter in practice, is absolutely intolerable!" thing gets really old. Commented Dec 10, 2011 at 5:13
  • 6
    @Jeff - I never claimed it was my right to post questions with the word problem in it; that's a straw man you made up. The fact that you are choosing to put yourself on the opposing side of many of your most valuable contributors to this site betokens a disappointing degree of epistemic arrogance on your part. At some point a truly rational actor would consider the possibility that maybe some of his underlying assumptions used to analyze the data might be wrong. Commented Dec 10, 2011 at 5:31
  • 3
    @adam The data tells me this is a non-issue. The internet is a big place; if things are not to your liking here, perhaps some of the millions of other websites might be? Life is too short to be unhappy. Commented Dec 10, 2011 at 5:38
  • 8
    @JeffAtwood if you mean that problem filter is something useful and interesting and that it pissing off someone serves as an indication of that - I agree. I wonder though if you are ready to apply similar reasoning to self? I mean, if say you find out yourself pissed off by guys suggesting to disable the filter for mods / high rep users, would you be ready to consider idea that these guys are doing something useful or interesting?
    – gnat
    Commented Dec 10, 2011 at 7:38

Saying that the title filter is a good thing because it usually results in some marginal improvement of a likely marginal question, notwithstanding the occasional halting problem, is like saying that obscenity filters are good because they usually catch naughty words, notwithstanding the occasional sextant, titular, niggardly, spic and span, etc.

  • 4
    +1 Thank you. The goal has perhaps been lost. We want better questions, making the titles "better" doesn't make a bad question better.
    – MPelletier
    Commented Dec 10, 2011 at 17:11
  • 8
    @problAm - of course, and more importantly, ok questions with bad titles can be edited Commented Dec 10, 2011 at 17:24
  • You are right that low False-Positive Rate is important. (Without defending the current implementation of filter without whitelist). A good title is generally necessary, but obviously not sufficient, for a good question. What is the next big issue with question quality that can be semi-automated? Brevity? Clarity? Specificity? Lack of MRE? Lack of effort/research? Wrong or inadequate tags? 'Offtopic' definitely can't easily be automated. I put a lot of time into reviewing fixing and answering questions in SE network, and I don't see any low-hanging fruit.
    – smci
    Commented Nov 6, 2016 at 11:23

Perhaps there's a genuinely strong case to be made in favor of the filter - it's hard to judge without access to the raw data - but I'm entirely unimpressed by the statistics given here, for three reasons:

  1. You're not providing us with the raw data to look at, and the classification of changes into the 'helpful', 'benign' and 'harmful' categories is sufficiently subjective that I question whether others would agree with your calls.
  2. Your definition of 'helpful' changes seems to take for granted that any substantial rewrite of a title including the word 'problem' is a helpful one, when it's pretty obvious that a substantial rewrite can be helpful, harmful, benign, or a mixture of helpful and harmful.
  3. Most importantly, your definition of 'harmful' doesn't include the single most important harm that everyone posting about this issue cared about: users not being able to use the word 'problem' in situations where it's the only appropriate word, and a synonym will not do. You have not told us how many of those 200 titles included error messages that had to be moved into the question body (or worse, culled completely by a confused new user), nor how many recognizable formal names of problems had to be twisted into an incorrect or substantially less common version, like the slightly embarrassing 'n+1 selects issue'. Saying that your data shows that the good outweighs the harm when you've only attempted to measure trivial harm that nobody cares about is simply begging the question.
  • Another harmful effect is possibly users canceling some of their useful edits because the existing title is now rejected and it's more trouble to come up with a good replacement title than the edit itself was. Of course, these numbers cannot be taken into account in the statistics, since canceled edits are (AFAIK) not recorded.
    – wovano
    Commented Nov 30, 2021 at 7:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .