When editing a post that contains a non-BMP unicode character (codepoint U+10000 or up), the AJAX-loaded post when you submit the edit does not include that character. I suspect this is a JSON-encoder bug on the server side.
For example, in my answer to How to convert some character into five digit unicode one in Python 3.3?, I included the codepoint U+1D15D ("MUSICAL SYMBOL WHOLE NOTE"), then was confused by the fact that not even a placeholder character was being shown.
Normally, the post looks like this:
(where the font used to render the glyph is missing that codepoint so a square box placeholder is shown instead).
When I edit the post the preview works fine, but when I then submit the edit the page is refreshed using AJAX and I see:
There is no character between the quotes there; the JSON response for the edit gives me:
<pre><code>>>> '\\U0001D15D'\n''\n>>> '\\U0001D15D'.encode('unicode_escape')\nb'\\\\U0001d15d'\n</code></pre>\n\n
for that part of the post; again no U+1D15D character there.
Clearly, the SE software stack can handle non-BMP codepoints. The JSON standard can handle it too, almost all characters can be included literally, or you can use \uabcd escapes provided you encode these codepoints using a UTF-16 surrogate. See section 2.5 of the JSON RFCsection 2.5 of the JSON RFC.
So, what I suspect is that non-BMP codepoints are not being handled by the JSON encoder on the server side and are instead silently dropped.
Can this be fixed?
