17

Some of you might not be aware, but there are a few sites that use the StackExchange engine to address non-programming questions. These sites are not called Stack Overflow, but they share this very same meta. They don't get thousands of questions a day and some of them don't even get tens of questions a day. If Stack Overflow is like Amazon, the smaller sites are like Ben and Jerry's. More "niche" sites have (and require) different cultures. Please bear this perspective in mind.

On Biblical Hermeneutics we got a question that asked if there were any contradictions in the Bible. As a question, that's just too broad, but it was also filled with grammatical errors, and useless phrases and all the things you might hope a "quality" filter would catch. After some editing work by others, the question was beaten into shape: success!

However, we got a question on our meta from someone who asked a much better question that would have been right in our wheelhouse, but it got tripped up by the quality filter. If the person had not asked (and known to ask) on our meta, they'd probably still be confused, and I wouldn't blame them for being annoyed enough to give up on the site altogether.

Can you give us any help in communicating to our users that their questions are welcome when the system blocks them? Can we find who got tripped up by the filter, and what questions they were asking? The canonical answer really doesn't meet the need.

Addendum

Notes on the answers so far:

  • Jeff's suggestion that since the question the filter rejected was easily answered by Wikipedia is a red herring unless the filter somehow searches Wikipedia and knows the question is easily answered. It's do doubt true that if a person includes more research, they are less likely to run afoul of the filter. But people who do zero research still stumble past the filter somehow. No filter will be able to determine if the asker is willing or unwilling to learn, which is the primary way I tell if a question is worth answering.

  • Further, and this is another topic altogether, a variety of well formulated answers beats the Wikipedia harmonization of a variety of scholarly views any day. Wikipedia is a great place to start thinking about a question, but it's not the be-all and certainly not the end-all answer to non-technical questions. (I assume everyone agrees at some level, but I want to make this point clear.)

  • An Amazon-style site needs to optimize for crowd control and a Ben and Jerry's site optimizes for customer satisfaction. Biblical Hermeneutics is less likely to attract the homework and "plz email me teh codez" questions than other, broader-interest sites. So filtering that works for one might be a waste of time on the other. My guess (and I am denied any data) is that the filters have a higher false-positive rate on the niche sites.

  • It's really a bit disingenuous to blame the niche sites for being small and suggest that they ought to be hosted on some other network. Each one followed the Area 51 process and was approved. It's not as if we hijacked the system to do something unauthorized. Maybe the site approval process is broken. Maybe the network's goals should change to fit what it can actually deliver.

  • However, I think the niche sites are doing just fine. A month and a half ago, we had 1.4 questions a day. Today we have 1.9. (Yesterday it was 2.0, so the number does shift around quite a bit. Could we get these numbers graphed over time? That would make evaluation of a site's "vital signs" a bit more meaningful.) We are seeing slow growth and we are making the internet a better place. Unless I hear otherwise, I must assume we are meeting our goals.

Improve this question
22
  • 3
    I think in this specific case, it was the great similarity between title and body
    – Pekka
    Jan 11, 2012 at 20:04
  • 2
    This question turned out to be a "General Reference" question. Looks like the system works as intended.
    – user102937
    Jan 11, 2012 at 20:07
  • 3
    Also, the canonical answer you refer to explains why the filter details are kept secret.
    – user102937
    Jan 11, 2012 at 20:10
  • 2
    @Robert Harvey: Perhaps. But we only know of one example from a very persistent user. I can only guess that there might be others we don't know about. Please remember the first paragraph. (And I still think the question was a valuable one even if it would have needed some editing to be a good question.)
    – Jon Ericson
    Jan 11, 2012 at 20:13
  • 7
    @Robert Harvey: The canonical answer is "You did something dumb, but we won't tell you what even if you take the initiative to find out." This is not helpful to the sites that hope to draw less-technical people than SO.
    – Jon Ericson
    Jan 11, 2012 at 20:17
  • 3
    @JonEricson: The details of the filters are not released, in order to prevent gaming of the system.
    – user102937
    Jan 11, 2012 at 20:19
  • 8
    @Robert Harvey: I understand what the answer says--I just happen to disagree with it. It's frustrating to have heavy-handed, big-site solutions to problems we don't have. It's doubly frustrating when the solution is guarded like a state secret. (I have all sorts of problems with the phrase "gaming of the system" in this context too, but comments aren't the best place for that conversation.)
    – Jon Ericson
    Jan 11, 2012 at 20:28
  • 5
    @Wikis: Sadly, your reasonable suggestion has already been rejected for a related, um, "problem".
    – Jon Ericson
    Jan 11, 2012 at 20:40
  • 1
    @Wikis: While I agree in principle, in practice there are a few users with fairly high reputation that still manage to ask low-quality questions; higher rep shouldn't exempt people from asking good quality questions.
    – user102937
    Jan 11, 2012 at 20:42
  • 2
    @RobertHarvey: agreed, but these will be very few that can be picked up by human moderators and the community. There will always be exceptions and grey areas, no algorithm will pick them all up.
    – Reinstate Monica - Goodbye SE
    Jan 11, 2012 at 20:45
  • 1
    I don't have the slightest idea what Cromwell's rule has to do with this. The algorithm is based on identifying catch-phrases, grammar and punctuation problems and other anomalies that have a high correlation with poor quality questions. It is possible that those metrics change when you move away from Stack Overflow, but not likely.
    – user102937
    Jan 11, 2012 at 20:51
  • 4
    @RoberHarvey: Why not just tweak the system for that StackExchange to send all questions to Jon Ericson for editing? If there aren't all that many being posted - even fewer being kicked back - surely Jon would be willing to clean them up and post them for the user? It's for the community, after all, and if the person asking the question gets a good answer, being penalized - from time to time - in terms of question reputation would seem a minor nuisance.
    – Patrick87
    Jan 11, 2012 at 20:58
  • 7
    @Robert Harvey: Cromwell's rule suggests that you actually look at the data on non-SO sites or evaluate your assumptions or resist reflexively defending a feature many consider to be broken. Just about anything besides the same song and dance that always shows up in the comments to this sort of post would be a welcome change.
    – Jon Ericson
    Jan 11, 2012 at 21:00
  • 1
    @Robert Harvey: That's right. And that question walked happily through the filter as if it didn't exist. Some helpful moderators fixed it to be a good question. So you can see my frustration.
    – Jon Ericson
    Jan 12, 2012 at 22:50
  • 1
    @JonEricson Computers are stupid that way; they don't know anything about subjective heuristics. The filters are not designed to catch subjectivity, sarcasm or any other uniquely human idioms.
    – user102937
    Jan 12, 2012 at 22:54

3 Answers 3

Reset to default
17

You're asking the wrong question; there's nothing to "bypass" because the system is working as designed.

  1. User asks very short question with no detail.

  2. System blocks question as, basically, trivial.

  3. Question is "answered" on meta by a link to Wikipedia, proving that the question was indeed trivial to begin with and correctly blocked in the first place.

And the problem here is ...? A hypothetical concern about theoretical injustice to unknown questions?

If you can come back with actual examples of incorrectly blocked questions, perhaps. But know that we spent months looking at blocked questions, and the types of questions we wanted blocked, and the thresholds are pretty generous.

Improve this answer
3
  • 3
    I don't understand how the fact that a Wikipedia link answered the question justifies the system blocking this question. The question was blocked by, "very short question with no detail", not because the answer could be found on Wikipedia.
    – Reinstate Monica - Goodbye SE
    Jan 12, 2012 at 13:03
  • 8
    I think his concern is this is discouraging traffic on low-traffic sites. It makes a LOT of sense for SO/SU/SF, but if a site gets 10s of questions per day they probably have the bandwidth to handhold users who ask these kinds of questions and make them productive.
    – JNK
    Jan 12, 2012 at 13:36
  • 9
    Somebody unhelpfully removed the most important paragraph of the question. Did you, by any chance, read it?
    – Jon Ericson
    Jan 12, 2012 at 22:20
11
+650

Everytime I get anywhere near this damnable topic, I feel trapped in a Catch-22. If you don't know why, I can't explain it to you.

I have no particular hope of breaking through, but let's try one more thing:

Send me a copy of every filtered question on Biblical Hermeneutics from now until the site leaves beta.

I'll sign any NDA you want and swear not to reverse-engineer the filter. I'll even shut up about filters.

Improve this answer
7
  • 4
    How about you invest your time on questions that were properly phrased and worth your time instead?
    – Ivo Flipse
    Jan 13, 2012 at 8:29
  • 16
    People reading this discussion need to bear in mind that Biblical Hermeneutics averages only 1.4 questions per day. They probably do not have the choice of investing their time in more "worthy" questions; it makes sense for them to invest much more effort into retaining and coaching new users than on larger sites. This might not be behaviour we want to see in a site that's launched, but it's very reasonable for a site that's still trying to establish its core group of users. @IvoFlipse
    – Jeremy
    Jan 13, 2012 at 17:44
  • 3
    @JeremyBanks: "People reading this discussion need to bear in mind that Biblical Hermeneutics averages only 1.4 questions per day." Then you need to keep in mind that this is not a viable site. It's not healthy for a site to be doing that poorly; it suggests that it's on the borderline for failure. If many of those questions are poor quality, then that too suggests that the site is dying. Not all sites make it.
    – Nicol Bolas
    Jan 18, 2012 at 5:59
  • 7
    @Nicol Bolas: Almost all of our questions are high quality. In fact, I suspect the consistently high quality of questions, combined with the relative obscurity of the topic, are what keeps the SE people from pulling the plug on us. Personally, I think the site launch shows some problems with the Area 51 process. But I don't have enough information to make any sort of judgement. Time will tell.
    – Jon Ericson
    Jan 18, 2012 at 17:05
  • 1
    Throw a new bounty on this after March 1 - maybe management will give it a more honest look
    – Adam Rackis
    Feb 24, 2012 at 20:38
  • 4
    @Jeremy - exactly. I'd love to see the more obscure SE sites treat their users with some nicer kid gloves than what SO (by necessity) uses.
    – Adam Rackis
    Feb 24, 2012 at 20:39
  • 1
    @JonEricson Which is probably why the bar for sites to get to beta has been raised on Area51, recently.
    – Hod - Monica's Army
    Mar 19, 2012 at 23:28
3

They don't get thousands of questions a day and some of them don't even get tens of questions a day. Unlike busier sites, these sites are more desperate for content than for filtering out bad, duplicate or junky questions. Please bear this perspective in mind.

Very well. So long as you bear this perspective in mind.

Stack Exchange is a piece of software designed to run Q&A sites tailored towards technical questions and answers. It is designed and implemented for that purpose. To the extent that such software can be made to run for less technical pursuits like gaming and the like, then it should be used for that.

But Stack Exchange is not the modern PHP-forum for the Web 2.0 era. It isn't a tool that you customize for your particular needs. The Stack Exchange methodology and software is designed for the purpose to which it is designed. If something outside of that design fits into it, great! If it doesn't, too bad.

My perspective is this: if the quality filters built into SE get too many false positives for you, then one of two things has happened. Either you want to have bad questions on your site (which is bad). Or you have a non-trivial number of users who are asking good questions that the filter picks up because the subject matter conflicts with what the filters were designed around.

If it's the former case, tough; make your site better by getting good questions, not by trying to prod bad ones into good ones. If it's the latter case... then maybe you should switch to different software.

I'm not against the notion of specialized quality filters. I think it's a perfectly legitimate idea for different sites to use different criteria for defining what is likely to be off-topic. But the simple fact is this: SE is meant for technical issues. The farther away from that your site gets, the less likely it is to work correctly with that filter. And if the Powers That Be don't want to change the filters, then there's nothing you can do about it.

Please bear in mind that Biblical Hermeneutics averages only 1.4 questions per day.

Users on new sites like it often do not have the choice of investing their time in more "worthy" questions. It makes sense for them to invest much more effort into retaining and coaching new users than it does on larger sites. This might not be behaviour we want to see in a site that's launched, but it's very reasonable for a site that's still trying to establish its core group of users.

This a big city solution to a big city problem. It shouldn't be applied indiscriminately.

The perhaps these sites need to become web forums, or something else that works in the small-city scale.

Not every municipality is big enough to incorporate, after all. Stack Exchange is a big city solution for big city problems. Web forums tend to work reasonably well for small city problems.

If Biblical Hermeneutics only gets 1.4 questions a day, then maybe it shouldn't be a Stack Exchange site at all.

Improve this answer
16
  • 4
    I'll tell you that english.se questions hit this filter a lot, and it was not a good sign. That's also not so coincidentally why we introduced the unusual and experimental "General Reference" close reason there..
    – Jeff Atwood
    Jan 13, 2012 at 8:13
  • 5
    The software started out for technical questions but now covers a broad range of forums, as can be seen at the bottom of this page (and there are even more in Beta).
    – Reinstate Monica - Goodbye SE
    Jan 13, 2012 at 11:49
  • 1
    @Wikis: SE makes Q&A sites, not forums. They are different things for different purposes. One does not replace the other.
    – Nicol Bolas
    Jan 13, 2012 at 16:17
  • I accept your correction to my terminology, thanks.
    – Reinstate Monica - Goodbye SE
    Jan 13, 2012 at 16:19
  • 4
    On "if the quality filters built into SE get too many false positives for you": that's the problem. I can't know how many false positives there are because the system (or the SE administrators) won't tell me. (By the way, we are getting some traction with Google and are at 1.9 questions a day.)
    – Jon Ericson
    Feb 24, 2012 at 19:05
  • 1
    If it's the latter case... then maybe you should switch to different software - yeah - shutting the whole site down makes much, much more sense than just turning the filter off for this particular SE site. I wonder what the VC funders think of that mentality.... <ping @JonEricson>
    – Adam Rackis
    Feb 24, 2012 at 20:43
  • @AdamRackis: Yes, I'm sure the "VC funders" are really banking on traffic from Biblical Hermeneutics.
    – Nicol Bolas
    Feb 24, 2012 at 20:47
  • 2
    @NicolBolas - I'm sure VC funders won't appreciate any traffic getting scared away for no good reason. A small but growing community might be helped by turning the stupid filter off....and people are saying no?!?! Wow.
    – Adam Rackis
    Feb 24, 2012 at 20:49
  • @AdamRackis: People might be helped by turning it off? That's like saying that more people would be helped by turning it off on SO. Yes, they would. But in so doing, all the experts would flee due to the influx of bad questions. There is a good reason for those questions to be stopped. Some people ask bad questions, and those questions need to be stopped. If some potentially good ones are stopped sometimes, so be it.
    – Nicol Bolas
    Feb 24, 2012 at 20:58
  • 5
    @Nicol - as has been explained repeatedly, SO has the filter to stop the massive influx of crappy questions. Jon's site does not have that problem. Your entire argument is a non sequitur.
    – Adam Rackis
    Feb 24, 2012 at 21:01
  • @AdamRackis: How do you know that it doesn't? You're assuming that every question that hits the filter is a great one. It sounds more like they want to artificially inflate their question count.
    – Nicol Bolas
    Feb 24, 2012 at 21:02
  • 2
    @Nicol - I'm assuming Jon knows more about his site than I do. He says this will likely help his small but growing community, and the only argument I've heard against it is this circular nonsense that the quality filter can't be removed from an SE site because all SE sites have the filter
    – Adam Rackis
    Feb 24, 2012 at 21:05
  • 2
    @Nicol: just this week the Christianity site had a user with homework questions. It wasn't fun to deal with someone looking for answers to get a grade to pass a class that they had no interest in learning from. As far as I know, the filter did nothing. But the person who got tripped up really wanted to learn and the system hassled them for no good reason. Question count be damned! Let us provide excellent service because we want to treat fellow seekers of knowledge as human beings.
    – Jon Ericson
    Feb 24, 2012 at 21:10
  • 2
    @Nicol: Why is this a secret? Why am I not allowed to know? That's the question I'm asking and not getting any decent answers to.
    – Jon Ericson
    Feb 24, 2012 at 21:11
  • 4
    @Jon - I'm guessing there's no built-in system to get you data on everything that gets kicked back from the filter. But for me, I think the fix is simple: turn the thing off until your community decides it's better off with it. I really hope this dogmatic, not-listening-to-others-even-for-simple-silly-things business will improve in 5 days
    – Adam Rackis
    Feb 24, 2012 at 21:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .