Changes to syntax highlighting

Question

locked for the history - please post bugs and feature-requests separately.

-Shog9

We're changing the way syntax highlighting is done on the Stack Exchange engine.

As you probably know, we use Google Code Prettify for automatic syntax highlighting.

Since the beginning, we've had kind of a boolean setting per website:

1. code blocks are always automatically highlighted (Stack Overflow, programmers.se, etc)
2. code blocks are never highlighted (Super User, photo.se, bicycles.se, etc)

We are now moving to a more tag-based syntax highlighting method. There are one of three possible syntax highlighting hints associated with each tag:

1. specific language hint -- tell prettify to use that language
2. default language hint -- let prettify infer a language
3. null -- no hint, no syntax highlighting

(all specific languages must be in the set that prettify supports, obviously)

bsh, c, cc, cpp, cs, csh, cyc, cv, htm, html, java, js, m, mxml, perl, pl, pm, py, rb, sh, xhtml, xml, xsl.

extensions available are

apollo, css, go, hs, lisp, lua, ml, proto, scala, sql, vb, vhdl, wiki, yaml

Thus:

• If a question has no tags with a default or specific language hint, no syntax highlighting will be performed.

• If a question has at most one tag with a specific language hint, plus any number of default or null hints, then syntax highlighting will be performed only for that specific language.

• If a question has two tags that both define specific languages, it uses default and lets prettify infer as it always has.

• As long as a question has one or more tags with default language hints, but none with a specific language hint, it uses default and let prettify infer as it always has.

The default tag syntax highlighting hint is null. So the question is, which tags should have the default and specific language hints? I put my initial pass through the first 6 pages of tags here:

http://pastebin.com/AWMtu5rK

If you feel there should be changes, post your diffs as answers.

(and yes, there are plans to allow an explicit override syntax but for now we want to try to get the defaults mostly correct before worrying about the edge conditions)

Answer 1

Maybe this sort of thing could be attached to the tagwiki system?

Answer 2

One potential change: when an answer is posted, remember the default language at the time.

It would be really nasty to post an answer which was formatted appropriately, only to have someone remove a tag and turn the answer into badly-formatted (or even unformatted?) mush.

EDIT: Of course, edits make this even more complicated... should an edit take on the new default or not?

Answer 3

(Separate from my "general change" as they're different proposals.)

[iphone] = default
[c#-3.0] = default
[android] = default
[jquery] = default
[hibernate] = default

should be (IMO):

[iphone] = lang-c // Assume Objective-C for the most part
[c#-3.0] = lang-cs
[android] = lang-java
[jquery] = lang-js
[hibernate] = lang-java

(There may be more, but I need to go to work...)

Answer 4

This causes a potential problem with at least the WordPress Answers site, as explained in this meta question. Basically, because the whole WordPress Answers site is about WordPress, the [wordpress] tag is almost never used, and thus, prettify will rarely be invoked, unless a specific language tag is also used. However, because WordPress is written in PHP, the [php] tag is also commonly left off of questions, so it's not uncommon to have a question with code in it that does not have a code-specific tag and does not have the [wordpress] tag.

Is it possible to cause all questions on WordPress Answers to invoke the "default" behavior, where prettify infers a language?

Answer 5

c++0x should be highlighted as c++

[c++0x] = lang-c

Answer 6

Is it possible to have "related tags" feature that when certain tag is set on question, the syntax highlight will take into account additional tags that are always related?

For example, when jQuery tag is set, it's always good idea to also highlight "pure" JavaScript as well, so the highlighter should take into account both tags. Same with HTML, very often JavaScript/jQuery questions also involve HTML and such code is posted as well but not rendered nicely unless the html tag is explicitly included.

It's possible to manually edit each question and add those tags but would be nicer to have it done "behind the scenes" without the actual tags added.

Answer 7

Pascal code (still) isn't getting highlighted, but should be.

[pascal] = default

Answer 8

[cl] = lang-lisp
[el] = lang-lisp

This seems wrong. If I look at question tagged with [cl] or [el], they have nothing to do with lisp. You probably meant [common-lisp] and [emacs-lisp]/[elisp] here (btw the latter two should be made synonyms, but that's a different issue).

Additionally [scheme] and [clojure] should also be lang-lisp and [ml] should be lang-ml.

I also recommend setting [j], [k] and [apl] to null because those languages look horrible with default highlighting.

Answer 9

[node.js] = lang-js

See, for example, How to make an HTTP POST request in node.js?, which is tagged node.js but not javascript and gets no highlighting.

Answer 10

I have a question about this statement:

If a question has two tags that both define specific languages, it uses default and lets prettify infer as it always has.

So does that means that, for instance, I have two tags that are linked to the same specific language, say [tagA] -> lang-a and [tagA-v3] -> lang-a, does it fulfill the above condition, triggering default mode? Won't that be counter-productive?

If the above does not happen (ie. Default mode only gets triggered when two different languages are implied by the tags), then shouldn't we link libraries written in a specific language to that language? jQuery -> JavaScript comes to mind of course, but there certainly are many others. I'm asking this because there are quite a number of questions tagged jquery but not javascript.

---

Also, it appears as there's no such thing as lang-as for ActionScript. The actionscript tag is currently linked to [lang-js], which may or may not highlight correctly, since the two languages are based off different versions of the ECMAScript specifications.

Answer 11

• [sqlalchemy] = lang-py
• [webpy] = lang-py
• [cherrypy] = lang-py
• [py2exe] = default
• [beautifulsoup] = lang-py or default as it might have some HTML
• [numpy] = lang-py
• [wxpython] = default (it could be XML as well)
• [boo] = lang-py (it's not the same language, but it's close enough. Only thing, there are not too many questions.)
• [scriptaculous] = lang-js
• [turbogears] = lang-py

Answer 12

Jeff, I wonder if it would be possible to have an unobtrusive display of the language which has been chosen? That would help get the tags right.

I am concerned that people will choose the tags based on the language. As I have stated before, a question should not be tagged "C#" simply because that's the language being used.

Answer 13

Those tags needs default syntax highlighting, not null (also not xml nor java). They contain posts with a mix of Java code and XML-like markup. Both needs to be highlighted as nicely as it did before this change.

jsf, jsf-2.0, jsp, servlets, servlet-filters, jstl and el.

Answer 14

is there a, or do we have a need for, a [lang-pseudo] for when you're trying to explain something in pseudocode? (often for my needs [lang-c] would be fine ;] )

Also, does this say what happens to answers formatting? What if I'm discussing [asp.net] with someone and the question is tagged c# but they really need to update the aspx instead of the cs and so I put some HTML in my answer in the codeblock, then how does it filter that my code is HTML and not C#?

Am I overthinking this?

Answer 15

None of the shell scripting languages have highlighting anymore

[bash] = lang-bsh
[shell] = lang-sh
[sh] = lang-sh
[csh] = lang-csh
[awk] = default

Answer 16

Octave code should be highlighted like Matlab.

[octave] = default

Answer 17

Well it appears the language inference has been pulling its weight thus far. There is however a minor issue here.

I experimented with the question Webclient.UploadFile which is tagged tagged c#, .net, vb.net and webclient.

When I tried answering the question with VB.NET code [which is perfectly valid based on the OP's tags] the code I entered was formatted as C#! Quite unbelievable!

Well I did not really answer the question. I wanted to test the code inference system and from the answer preview I got, it failed to infer the language properly.

I believe including a way to explicitly state which language should be used to highlight code entered, especially on the answerer's side, would be very much appreciated.

Answer 18

There is an inconsistency between the preview and the final result with respect to multiple language blocks.

E.g: I specify a language <!-- language: lang-cs --> right in front of this code block:

var test = ThisIsACSharpObject;

Then I have some text, and then again a code-block:

var test2 = ThisIsAnotherCSharpObject;

In the preview (on Stack Overflow proper), both blocks are formatted like they should as C# code, but when I post this, only the first one is formatted right. I have to specifically add the language tag in front of all the different codeblocks for them to be rendered right after posting.

(Apparently this doesn't work on meta... I suppose the code formatting thing is not enabled here ==> I think it should be enabled, as how can we otherwise reproduce problems concerning this here?)

Edit: Ow hey, apparently, here on meta this has a different behavior: I never see the code formatting in the preview while editing, but after posting, it is actually formatted (wrongly in my opinion, as only the first code block is formatted right), so we can actually reproduce this problem here... as you can see.

Answer 19

Questions tagged mercurial often mention Python code, so I propose

[mercurial] = lang-py

Answer 20

I miss the old formatting, which while imperfect at least didn't suck enough for me to complain about. So here's one I'd like to see:

[tcl] = default

Answer 21

The linq and linq-to questions that doesn't have a specific language tag should use default highlighting. Maybe even all LINQ tags?

particularly:

[linq]             = default
[linq-to-sql]      = default
[linq-to-entities] = default
[linq-to-xml]      = default
[linq-to-objects]  = default

Answer 22

Most of our GWT questions/answers currently show unhighlighted code.

It probably needs default highlighting, because the questions very often include a combination of Java, HTML, CSS, XML and JavaScript code blocks, without being specific questions for these tags. _{(Especially, GWT is not Java, it just looks like Java code, and adding a Java tag often results in answers that work in Java, but not in GWT.)}

[gwt] = default

Answer 23

QTP scripts are based on VBScript

[qtp] = lang-vb