14

This came about from a discussion on one of the betas - but it affects the main engine and my understanding is that such requests should be posted here.

Initially it was simply about supporting diacritics in the tagging system, and subsequently turned into a bit of a debate; a quick summary is as follows:

A: The tagging system is broken because it just silently drops characters with diacritics.

B: Tag diacritics are a usability nightmare because the majority of users can't type them.

A: The site can't possibly hope to attract an international audience by being so English-centric.

B: We obviously don't want to lose the international audience, but most users are from North America and we can't just throw usability out the window for them.

This went on for several more rounds and I don't wish to bore everyone with the details, but clearly both sides of this argument are valid:

  • When trying to attract an expert audience, if a word (such as the name of a food) is legitimately non-English and cannot be written correctly without diacritics, it sends a negative message to non-Americans.

  • On the other hand, I have no idea how to type an Ă without looking it up in the character map, and I'm certain the same can be said for 98% of the other users (especially keeping in mind not all languages share the same diacritics).

Blah blah blah, get to the point already...

Eventually it hit me that we actually have a solution for this, that being the newly-implemented tag synonyms feature. This allows us to:

  • Use the "correct" tag, diacritics and all, if and when the software supports it, and

  • Avoid all the associated usability headaches because English-speaking users would simply be able to type the anglicized tag - not just in searches but also when tagging their questions, which is extremely important for newbies.

Everybody wins, except the taggers and moderators, because maintaining all the synonyms would be a nightmare - especially once the site is out of beta and you need a bunch of upvotes to create tag synonyms (which of course you don't have, because the tag was just created!)

So my request is twofold:

  1. Update the tag system to support Unicode characters.

  2. Implement a feature whereby, whenever a new tag with diacritics is created, the system automatically adds an "English" (ASCII-encodable) tag synonym.

This gives everybody what they want, and frees users from actually having to worry about any of this nonsense. International users can tag the way they want, and North American users can live on in blissful ignorance, and worry about answering questions instead of fumbling with alt-codes.

You might say that these should be requested as two separate feature requests. But I happen to believe that they need to be implemented together, because if the former is implemented without the latter, the tags on certain sites could easily become an unholy mess in no time flat. That is why I've lumped them together this way.

Feature #2 might not be appropriate on every SE site (particularly the proposed foreign-language sites, like the Persian IT), but the more Stack Exchange branches out into non-technical areas, the more important this is going to become. Cooking is obviously full of these issues due to the vast number of "ethnic" foods, but I could easily see this having applications on sites like Mathematics, Physics, English/French Translation, Books, Japanese, and so on. It remains to be seen which, if any, of these proposals actually get off the ground, but if they do, the ability to support Unicode tags and/or automatic culture-synonyms would be a major benefit.

5
  • 1
    Excellent point WRT the foreign-language sites... I rather wonder if the lack of non-English tagging isn't one of the things preventing the Persian IT proposal from moving forward as well - would be great if this could kill two birds with one stone!
    – Shog9
    Jul 28, 2010 at 23:11
  • This should almost be supported already by using the collation features in the database. I'm not sure what's involved in getting it to work, though, as Jeff has always been against globalizing the engine (which myself and many others have thought was a big mistake).
    – Jon Seigel
    Jul 29, 2010 at 1:49
  • @Jon: Almost is definitely the key word; this issue came up as a result of the diacritic characters being silently dropped; "túró" became "tr", and if you search for "turo" you get no hits (although, interestingly, Google seems to have no trouble with it). So there's definitely some work to be done.
    – Aarobot
    Jul 29, 2010 at 3:23
  • IIRC, Jeff implemented the proper diacritic-less searching for comment replies already. Take a peek at one of those bug/feature requests.
    – Jon Seigel
    Jul 29, 2010 at 21:40
  • Almost six years later, I still can not create the tag büchi-automata on Computer Science.
    – Raphael
    Apr 18, 2016 at 14:07

1 Answer 1

4

I agree that silently dropping the accented characters was kind of unfriendly.

We now auto-map any non-ASCII characters in tags to their ASCII equivalents, pretty much as documented here:

Non US-ASCII characters dropped from full (profile) URL

7
  • Glad to see an official response to this. I, personally, have no problem with this, I think it's a great workaround in fact - however, some international users are rather passionate about this issue and it may only assuage them temporarily. We'll see.
    – Aarobot
    Jul 31, 2010 at 13:41
  • Ugh. Why isn't this being done using collation options in the queries/database? That would be a lot easier than maintaining your own character replacement mappings.
    – Jon Seigel
    Aug 1, 2010 at 3:17
  • @jon IMHO tags should be plain ascii anyway -- they are meant as general, generic shortcuts, not exact precise instruments. So there's no reason to store them in any other form. Aug 1, 2010 at 6:53
  • @Jeff: There are two main options as I see it. 1. If you're allowing a site in a RTL language (wildly different from English), does it make any sense to have tags in English/ascii-only? No. 2. Tags can be ascii-only, but searching should be accent-insensitive (this is already implemented in comment replies). -- So clearly if the site is global-enabled (and as I said previously, I think it would be a big mistake to ignore this aspect of the engine), #1 is your only option.
    – Jon Seigel
    Aug 1, 2010 at 13:18
  • (cont'd) And please base your decision on globalization on a design, NOT on technical reasons or limitations. Ask the board of directors for their opinion on globalization, and I think you'll find they're going to side with me on this, even if it's not slated to be implemented until some time down the road until a business model is proven. But it should happen eventually: Stack Exchange can't be the one-stop-shop for all questions and answers if it's only providing content using Latin characters.
    – Jon Seigel
    Aug 1, 2010 at 13:19
  • @jon the comment search matching uses the same function you see in the above link. Aug 1, 2010 at 18:25
  • 4
    I think this is wildly inadequate for the emerging language-specific sites like Russian, Chinese, Japanese, that use scripts with no English equivalents.
    – kotekzot
    Jun 14, 2012 at 3:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .