What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

In this comment to a question on TeX.SE

... the number "5000" does not paste as such but instead as

5
0
U+200C
U+200B
0
0

The HTML source looks like this:

perhaps the easiest would be <code>\edef\tmp{\everypar{\widowpenalty\the\widowpenalty\relax}}\tmp\widowpenalty=-50&zwnj;&#8203;00</code> that may reset

Or (smaller snippet):

\widowpenalty=-50&zwnj;&#8203;00</code> that

So in the HTML, we have extraneous &zwnj; and &#8203; in between the two middle digits of 5000.

Upon pasting and trying out this code, this first led to a mysterious compiler error. The same issue happened within a (now deleted) comment thread below this answer on TeX.SE; there, these or similar two characters were inserted after the letter "r" within the string \ref ("\ref").

Note that not all browsers wrap the same way. The above screenshot was for Firefox 19.0.2, in which it breaks between \relax}} and \tmp, with 5000 insidiously left intact on the second line but with the two special characters remaining in the middle. So that is why this took me completely by surprise. Other browsers might indeed only break on the invisible Unicode characters, such as Chrome 25:

Or on a smaller screen with the mobile site, multiple line breaks might show, possibly still leaving the Unicode characters unused:

share|improve this question
1  
This is a test: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa‌​aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa‌​aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa –  nneonneo Mar 8 '13 at 18:55
1  
Yep, U+200C U+200B (ZWNJ ZWSP) is being inserted every 80 characters within a code snippet in a comment. –  nneonneo Mar 8 '13 at 18:55
11  
It's not erroneously inserted, but I guess it's problematic in this case. –  Tim Stone Mar 8 '13 at 18:57
1  
@TimStone: It's generally problematic for copy-pasting code from comments. I guess I've never run into this issue because posting 80+ characters in code without a space is unusual. –  nneonneo Mar 8 '13 at 19:02
1  
@nneonneo Yeah. In TeX spaces are more often significant than the casual user would expect (though in this example there are tons of places where spaces would be perfectly fine without causing any problems). I am guessing that one reason for omitting spaces in code within comments is the (necessary) character limit for comments. –  Lover of Structure Mar 8 '13 at 19:07
    
@TimStone By the way, if a fix is made, it might make sense to apply it retroactively, because the status quo might be breaking a lot of people's code :-) –  Lover of Structure Mar 8 '13 at 19:57
    
Comments are rendered on-the-fly, so any change will be retroactive by nature. –  Tim Stone Mar 8 '13 at 20:26
    
@Tim, I guess that would make a nice answer? (Maybe with a reference to Jeff's explanation for the why as well.) –  Arjan Mar 8 '13 at 21:27
    
@Arjan Well, it seems kind of bad that this breaks copy and pasting in a non-visible way, but I don't know if there's a good solution beyond "Copy relevant comment content into the answer". –  Tim Stone Mar 8 '13 at 21:34
    
True, @Tim, it's just that future visitors might still want to understand the cause -- which is now buried in the comments a bit? (Funny to see how much trouble this has given over time...) –  Arjan Mar 8 '13 at 21:38
    
@TimStone (and Arjan and others): See my edit to the question (the caveat at the bottom). –  Lover of Structure Mar 9 '13 at 1:51
1  
@Arjan (and all others): PDF has facilities (related to accessibility support) that can change text when it is copied+pasted. I am wondering if someone familiar with the latest HTML spec knows of a similar thing there. This would be an ideal solution. And the present thing where sometimes code breaks (because this bug/feature is not known to the majority of users here) I am not too happy with: presently I need to pay special attention and test+fix all long code snippets manually. –  Lover of Structure Mar 9 '13 at 1:57
    
@nneonneo, as an aside: the Unicode magic applies to all comment text, not just `code`. –  Arjan Mar 9 '13 at 16:16

1 Answer 1

Unicode magic is inserted in comments (not just in `code`) whenever no whitespace was seen for about 80 characters, to enforce line wrapping. Without that the layout can really be messed up.

Jeff once explained why this magic is needed:

The word-wrap:break-word suggestion is a good one.

Works fine on posts, which are fixed width, and that is implemented.

Near as I can tell, there is no way to get this to work on comments because they are variable width. That is, the actual comment size depends on whether or not the comment vote UI controls are present (fex, on your own comment, you don't have controls, if you're not logged in there are no controls, etc), and if the comment has say 100 upvotes that's wider still.

One could argue if the above really is a good reason to not have a fixed width for comments on the full site. (The mobile theme is much more fluid, but uses word-break: break-all, so probably has no need for the Unicode markers, though they are present there too. And all that looks just fine.)

Using JavaScript to remove the markers upon copying, is not going to be easy. What about:

  1. Instead of the proposed word-wrap: break-word, use word-break: break-all, just like on the mobile site. This works for me in Chrome 25, Firefox 19, Safari 6, Internet Explorer 9 (in Windows 7 on Parallels on a Mac; simulating IE8 and IE7 works too), stock browser and Chrome on Android 4.1, and Safari on iOS 6.1.2. But apparently it does/might not work in Opera? See this JS Bin example.

  2. Or, instead of Unicode magic, insert <span style="display: inline-block"></span>. Tested in the same browsers, and probably also supported in Opera; see the same JS Bin example above.

  3. Or, use JavaScript on hovering to get the actual width, remove the Unicode magic, and then explicitly set width and word-wrap: break-word. This might reflow the text on hover, but only for affected comments. (Alternatively, one could run the JavaScript for all comments, on page load and after each resize.) Also, this needs to restore the original text (with the Unicode magic) when resizing the window, to allow the browser to calculate the proper new widths:

    $(".comment-copy").hover(function() {
      var text = $(this).text();
      var clean = text.replace(/[\u200B,\u200C]/g, "");
      if(text !== clean) {
        $(this).css("width", $(this).width());
        $(this).css("word-wrap", "break-word");
        $(this).text(clean);
        if(!$(this).data("orig")) {
          // Store original for later resizing
          $(this).data("orig", text);
        }
      }
    });
    
    $(window).resize(function() {  
      $(".comment-copy").each(function() {
        var orig = $(this).data("orig");
        if(orig) {
          // Reset original text, to get the proper width
          $(this).text(orig);
          $(this).css("width", "");
          $(this).css("word-wrap", "");
        }
      });
    });
    

    (See the same JS Bin example; tested in the same desktop browsers.)

  4. Or, if the above still yields problems: after posting, check if the magic was applied, and warn the commenter about it with a shiny blue info message? Like: "Beware: your comment includes a sequence of more than 80 non-space characters, which might cause issues when copying. Consider adding some whitespace."

  5. And/or, bind to the copy event, check the content and warn if needed: "The comment you're copying includes invisible Unicode characters". However, this event might go unnoticed when copying more than just the comment.

  6. Or, even though lines might be wrapped on another character: simply insert a true space instead? At least that is visible.

Some day, <wbr> will be the solution, but not today: though IE7 supported it, IE8 and IE9 don't.

share|improve this answer
    
One additional idea: mark linebreaks where there was originally no whitespace with a curved downarrow symbol to the right of the first line (or the left of the second line). I think I've seen this in some version of emacs, but can't find a screenshot now. –  Lover of Structure Mar 9 '13 at 19:15
    
@Lover, the problem with visible markers is nicely shown in the difference between the rendering in Chrome and in Firefox: the latter is smarter, and happens to break the long text at an earlier place, making the actual marker no longer needed/used. Also, the markers are not placed after every 80th character, but only when no whitespace was seen for at least 80 characters. In the Chrome screen capture things would look much better if the marker was inserted each, say, 25 characters... –  Arjan Mar 9 '13 at 19:22
    
You are right, but it's not a problem with the idea of markers, it's a problem with the possible lack in a browser interface for us to control that markers should be there only if the browser linebreaks at a non-whitespace character. As I understand it. Correct me if I'm wrong. –  Lover of Structure Mar 9 '13 at 19:30
    
True, @Lover. If one could ensure/calculate where markers are actually needed/used, then it would even be fine to insert any visible marker. Unfortunately, it's very hard to calculate that, as not every letter needs the same width, and as the total available width is not fixed (especially not on mobile). –  Arjan Mar 9 '13 at 21:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .