Most sites in the network have a specific set of languages that you are allowed to use in posts. For example, on Stack Overflow and most other sites, all posts must be in English. On German, all posts must be in English or German. This applies to most of the other language sites as well, e.g. on Italian, all posts must be in English or Italian. There are a couple of sites that allow other languages, e.g. Judaism which allows Hebrew, I believe.

For Stack Overflow, specifically, there are also language-specific "sister" sites, for example in Portuguese and Russian.

In general, the sets of languages supported on a site are small and well-defined.

I propose to detect posts whose majority of user-authored natural language content is not in one of the site's allowed languages and display a warning message to the poster when they submit the finished post. If there are language-specific sister sites, and the detected language of the post matches one of the languages of the sister sites, the message should additionally inform the poster about the possibility to ask their question there.

Something like (on Stack Overflow):

It looks like you are trying to ask a question in Portuguese. Please note that Stack Overflow is English-only. However, you can ask questions in Portuguese on our sister site Stack Overflow em Português.

This is a part of site curation that could be somewhat automated and thus lessen the burden on the community, the reviewers, and the moderators. It could also improve the experience for the poster, because it will take less time (immediate automated response instead of minutes or hours for the community) to give better feedback (a constructive message instead of downvotes, closure, and deletion).

In particular, the recommended practice of dealing with such questions (close as "Unclear", delete) is technically correct from the site point of view, but may be confusing to the poster, since it is well possible that the question is perfectly clear in their native language.

Non-English questions are a somewhat regular occurrence on Stack Overflow, and they mostly seem to be asked in languages for which a dedicated sister site exists, so they look more like honest mistakes. A feature like this could help those honest askers who simply confused the two sites as well, not just the moderators and the community.

  • Which content should be included in the analysis? Only natural language content authored by the poster. This excludes, for example, code, LaTeX, circuit diagrams, and block quotes as well as URIs. (Questions with code where all identifiers and comments are in a different language and it is thus hard to understand what the code is supposed to do, also exist, but handling those automatically would be too complex.)
  • When should the message be shown? At the very end of the composition process, when the poster submits the post. (For questions, this would mean the "Review" or "Ask" button, for answers the "Answer" button. IFF we want to include comments, it would be the "Add Comment" button or the Enter key.)
  • What about false positives? We should take certain measures to avoid false positives, such as making sure that enough content of the question is actually user-authored natural language content, and not e.g. mostly code. We should also ensure that both the total amount of content as well as each individual block is long enough to give the analyzer a chance to detect the language.
  • What percentage of false positives is acceptable? Ultimately, that is a community decision. In my experience, given a long enough input (see above), language recognizers are fairly accurate these days.
  • Why a warning message and not automatically rejecting / placing in a review queue / migrating? Because false positives.
  • Where/when does that message pop-up? – rene Jun 28 '20 at 8:45
  • What would be an acceptable false positive rate for this language detection? – rene Jun 28 '20 at 8:47
  • 4
    It looks like you are trying to ask a question in Portuguese. Nope, that is what my C++ normally looks like ... – rene Jun 28 '20 at 8:48
  • "Where/when does that message pop-up?" – I have clarified that this should be at the very end of the composition process on final submission. This allows for, e.g. gathering your thoughts in your preferred language first, then translating to the site language. – Jörg W Mittag Jun 28 '20 at 9:06
  • "What would be an acceptable false positive rate for this language detection?" – Ultimately, that would be a community decision. I have added some points about false positives to the question, e.g. ensuring that there is enough content. In my experience, which is admittedly mostly with technical sites, questions are either 100% English or 100% not. Questions on German might contain a mix of German and English, but if they do, the German is typically within block quotes. – Jörg W Mittag Jun 28 '20 at 9:07
  • "Nope, that is what my C++ normally looks like …" – I have clarified that I am only talking about user-authored (i.e. not quoted) natural language content (e.g. excluding code blocks, LaTeX, circuit diagrams, URIs, etc.). – Jörg W Mittag Jun 28 '20 at 9:09
  • Seems like a lot of development time for only a small community gain. – DavidPostill Jun 28 '20 at 9:10
  • @DavidPostill: We already have checks for the ratio of code vs. user-authored content, for example. ("It looks like your question consists mostly of code, please add an explanation.") So, the quality checker already analyzes posts and separates the different kinds of contents. Feeding the user-authored content to a language recognizer should not be more complex than analyzing the ratio of user-authored content vs. code. – Jörg W Mittag Jun 28 '20 at 9:17
  • 3
    I've said elsewhere in a comment that SE doesn't have a great track record of "detecting" things and then handle it appropriately. Maybe that changed with their comment classifier. The thing here is that it would need to handle lots of posts, so the CPU/network load need to be accounted for and then there needs to be a buy/build decision and even they buy the classifier that still needs plumbing for a problem that can also be handled by the community. Maybe not as timely as you envision but time is not a factor IMO. – rene Jun 28 '20 at 9:20
  • 3
    Most effective would be if that OP didn't even start writing their post and then we are back on the on-boarding experience for new users. – rene Jun 28 '20 at 9:20

Yeah, they tried this already on Stack Overflow, for question titles. It seems like it’s no longer active, though you may get a warning that similar questions have been downvoted instead.

What I think we learned from that is that it’s not easy to make something that accurately detects what language something is in. It’s not only about knowing that something is not English but also whether it’s for example Russian specifically or Ukrainian, otherwise you can’t give a message that will be understood. There’s all the obstacles mentioned in the question and a few more. You can throw more rules on something like that until it has an acceptable level of false results but by then you’ve put more effort in than it’s worth. And this was for a single site, only detecting a limited set of languages.

(At this point someone inevitably suggests using an existing third party API for language detection but this wouldn’t solve all the implementation problems and also opens security concerns.)

Being the largest site, Stack Overflow gets the most such questions (and some answers), but even then the problem of non-English posts there isn’t so bad. All other sites in my experience are small enough that all off-topic foreign language posts there will be handled appropriately (most of them seem to be spam).

  • 1
    According to this answer, they were only looking at titles, they were only looking for a small set of trigger words and classifying the post as Spanish/Portuguese if it contained only one of the trigger words, and they were deliberately optimizing for minimal false negatives at the expense of false positives. I don't think we can draw any conclusions from this experiment about the actual accuracy of a "proper" language detection algorithm (potentially ML backed) on a full post. – Jörg W Mittag Jun 29 '20 at 6:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .