3

Basically, it's where differences in spelling, such as letters chosen or diacritics used, are treated as different queries by the Stack Exchange search engine, even if the queries are otherwise identical.

These concerns have previously been addressed through this post and this post, though it seems to have been buried amongst the junkyard of older, forgotten posts. I commented back in July asking if the issue has been addressed any further, but the fact that no one has responded since even then just serves to back up how I've stated it's gotten lost amongst the old posts. Or at least it has seemed to.

So, I'm wondering this: has this shortcoming been addressed? And if so, in what way(s)?

  • What's so dialectical about diacritics? – Deer Hunter Oct 28 '15 at 15:17
  • 1
    Certain words are given diacritics, such as accent marks, in certain dialects, whilst in others they may be omitted. For example: resume vs resumé, coordination vs coördination, cafe vs café, you get my point, I think. – SarahofGaia Oct 28 '15 at 15:59
2

As of the beginning of 2015, we have internationalized AKA "case folded" most fields that we search on. This means that diacritics (héllo vs hello) are now correctly treated as equal. Additionally, we employ some stemming to make plural and singular terms equivalent (thoughts and thought are identical).

Having said that, we have not accounted for British spelling. The reason is that we'd have to manually keep a list of all British and English terms and make them equal (a synonyms list) and it just hasn't been high on our priorities list. You'll also notice that Google does not adjust for British spelling. Making the terms equivalent reduces relevance but increases recall, which is not always a good thing (more noise in your search results).

Finally, even though diacritics are now equal and stemming is employed, there are some cases where the results sets will be quite different. An example is qué vs que. Both results sets contain the same questions and answers that match on either term, but we also search code blocks and code blocks are treated as a special case. We do not stem or case fold code blocks. So, the additional results in the que query are due to code blocks matching on that term while qué does not appear in code blocks. My personal guess is that some people spell "queue" as "que" and use that variable name in code.

  • @Jan the interesting thing there is the cost. What if you specifically wanted things that container "colour"? They are now harder to find. No search change is ever truly "free" in terms of cost. ;) – Haney Oct 10 '17 at 19:43
  • (and yes, I realize that a search for it in quotes will probably work, but my point is that the basic aka naive behaviour has changed) – Haney Oct 10 '17 at 19:44
  • Considering this is a text (= language) based search, I have a hunch that most of the time people will be interested in the concept colo(u)r rather than in the exact spelling. At least that is the case when searching for terms like this on Chemistry – Jan Oct 11 '17 at 8:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .