33

The introduction of multiple language-specific sites to the SE network has brought to light the shortcoming of the current tag system: lack of support for non-English characters. Non-English characters are either silently dropped or converted to the nearest English equivalent. I propose that the system be changed to allow language-specific sites a greater degree of self-determination in the matter of tagging. Some of the affected sites are: Russian.SE, Spanish.SE, Chinese.SE, Japanese.SE, French.SE, Judaism.SE.

This question has initially been about the Russian.SE site, but seeing how other sites are affected by it I decided to broaden it.

See also:

Improve this question
14
  • @kotekzot site specific feature requests/bug reports should go on site metas. SE staff monitors all site metas. Particularly private betas and early public betas when there are no mods.
    – wax eagle
    Commented Jun 14, 2012 at 2:19
  • @kotekzot It's a conscious choice. For your information, the existing language sites have taken different decisions about tags: French Language settled on French (with English synonyms), German Language settled on English (with German synonyms), Spanish Language initially went with English but changed to Spanish, Chinese Language and Japanese Language both use English.
    – Gilles 'SO- stop being evil'
    Commented Jun 14, 2012 at 2:22
  • And it's worth noting that those posts actually have more visibility to the people who need to see them on site metas than they do here as here they may get lost in the shuffle, but on site metas where they have much less traffic the community team is more likely to see them and address them on an individual site meta.
    – wax eagle
    Commented Jun 14, 2012 at 2:22
  • 5
    @Gilles you don't see the pattern here? Sites devoted to languages that use a latic script have tags (or synonyms) in their own language, sites that don't - don't. That seems like a technological problem to me.
    – kotekzot
    Commented Jun 14, 2012 at 2:26
  • @kotekzot The pattern is not clear. For a long time, French Language was the only site with non-English tag names that people saw. I do think it's the right way to go, but it is not yet established.
    – Gilles 'SO- stop being evil'
    Commented Jun 14, 2012 at 2:29
  • 1
    @Gilles it is pretty clear if you consider synonyms. Sites dedicated to languages that aren't lucky enough to use a latin script don't get a choice.
    – kotekzot
    Commented Jun 14, 2012 at 2:35
  • 1
    @casperOne, this could be edited as general question about allowing Unicode tags and synonyms.
    – theUg
    Commented Jun 14, 2012 at 2:37
  • @Gilles, even Latin script language sites do not seem to be using language-specific characters like eñe in Spanish.
    – theUg
    Commented Jun 14, 2012 at 2:41
  • @theUg good idea, done.
    – kotekzot
    Commented Jun 14, 2012 at 2:55
  • 3
    @Gilles The Japanese site uses English tags because there isn't really a feasible alternative. You can't auto-convert 漢字 to a supported character like you can auto-convert é to e, you have to use the English word.
    – Troyen
    Commented Jun 14, 2012 at 4:26
  • 1
    Related: meta.stackexchange.com/questions/102828/… meta.stackexchange.com/questions/58184/…
    – Troyen
    Commented Jun 14, 2012 at 4:30
  • @Troyen thanks, I've added French.SE to the list of affected sites and linked the 2 questions.
    – kotekzot
    Commented Jun 14, 2012 at 5:53
  • 2
    Judaism.SE would also certainly want it, as would many proposals still in a51 (Hebrew, Yiddish, etc.)
    – Hod - Monica's Army
    Commented Jun 14, 2012 at 12:40
  • 1
    Spanish.SE would also like this: meta.spanish.stackexchange.com/questions/237/… meta.stackexchange.com/questions/135142/… this is important as we want our tags to be in Spanish.
    – JoulSauron
    Commented Jun 15, 2012 at 14:02

1 Answer 1

Reset to default
15

Our original implementation of tags was written with stackoverflow (and similar sites) in mind, and due to technical reasons would not support much by way of non-latin characters. This has seen a lot of work lately, in particular changing the implementation so that it theoretically supports a wider character set. So: we're working on it.

We still have a little way to go, and we might need to still think about what characters we allow per-site (in particular, to avoid 7 versions of the same tag, with/without accents - there's only so much synonyms can do).

Update: this is now enabled on russian.SE, as an initial proving ground, and will be released to additional sites where it makes sense assuming no huge "eek" issues are reported.

Improve this answer
17
  • Good to know it's being worked on! Don't suppose you could give us a ballpark estimate of when it would be ready?
    – kotekzot
    Commented Jun 14, 2012 at 6:37
  • 5
    @kotekzot there's a running joke here on meta, that every time estimate is always 6-8 weeks, regardless of magnitude or existing time elapsed...
    – Marc Gravell
    Commented Jun 14, 2012 at 6:48
  • I'll make sure to come a-knockin' in 8 weeks then!
    – kotekzot
    Commented Jun 14, 2012 at 6:50
  • In French, it would be fine if you couldn't have two tags that differ only by accentuation. There are words that differ only by accentuation, but they wouldn't tend to be tag names. I can't speak for other languages.
    – Gilles 'SO- stop being evil'
    Commented Jun 14, 2012 at 7:33
  • 4
    "technical reasons would not support much by way of non-latin characters" intrigues me as to how you're currently storing them. varchar with some obscure collation?
    – AakashM
    Commented Jun 14, 2012 at 8:23
  • @Gilles - I recall that (according to French grammar) if a word is capitalized, then accentuation is not required.
    – Adam Mosheh
    Commented Jun 14, 2012 at 17:36
  • 2
    @AdamMosheh cringe No, accents are required. The lack of accents is tolerated if the typographical medium doesn't permit it, and word games such as crosswords ignore accents, but spelling rules apply to words in uppercase as much as to words in lowercase. (Besides, we were speaking about tags, which are in lowercase.)
    – Gilles 'SO- stop being evil'
    Commented Jun 14, 2012 at 17:45
  • @AdamMosheh Omitting accent's on uppercase letter's is like the grocer's apostrophe in English. Some people do it, but its not to everyones liking.
    – Gilles 'SO- stop being evil'
    Commented Jun 14, 2012 at 17:45
  • 1
    @Gilles careful - I realise the above is intentional, but you accidentally got one of them right
    – Marc Gravell
    Commented Jun 14, 2012 at 18:18
  • 1
    @AakashM you really want to know? Well... We were kind of abusing sql-server's full-text search by tricking it into working on a munged version of the combined tags. This let us do a range of "all of" / "any of" tag-based queries against a fast index. This falls to pieces when multiple cultures get involved - our hacks no longer work. Most of the code has been tweaked to compensate for this (meaning: not using this hacky hackety hack-hack): in many ways, it is now a case of removing code that is there to enforce simple tags
    – Marc Gravell
    Commented Jun 14, 2012 at 18:22
  • 1
    It seems to me that canonicalizing to the lowercase form of the C normalization form is not going to be enough, because you aren’t actually doing Unicode casefolding for matches then. So you will have a problem getting says tchüss and tshüß to match, or Σίσυφος and σίσυφος and σίσυφοσ. I suggest running everything through a level-1 comparison under the Unicode Collation Algorithm, as the primary sort keys ignore both full Unicode case as well as diacritics. You already seem to ignore case correctly on chat searches, in that looking for tschüß finds TSCHÜSS correctly.
    – tchrist
    Commented Oct 1, 2012 at 17:22
  • @tchrist happy to get the feedback, but in many ways we dodge the most complex scenarios through a: auto-complete, b: synonyms, and c: the human factor. If there's something I'm missing, I'd love to hear the scenario (in the context of tags). Tagging != text search.
    – Marc Gravell
    Commented Oct 1, 2012 at 17:56
  • This answer needs updating.
    – msh210
    Commented Apr 3, 2013 at 7:49
  • @MarcGravell: What's the procedure to officially request accented tags on Spanish.SE?
    – Flimzy
    Commented Oct 21, 2014 at 14:52
  • @Flimzy are they not enabled?
    – Marc Gravell
    Commented Oct 21, 2014 at 16:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .