17

Moderators can set the default syntax highlighter on tags by selecting a value from a drop-down list:

Code language with drop down, lang-c selected

The community maintains a list of supported languages by inspecting the Highlight.js code file served by Stack Exchange: What is syntax highlighting and how does it work?

You would expect the dropdown list and supported languages to be in sync.

It turns out that is not the case, first observed by Cody Gray on Request to set the TOML tag to default syntax highlighting to lang-toml.

Here is the comparison between the values selectable by moderators and the supported language according to the FAQ post.

Moderator option missing, and syntax highlighting is available

Mod option Syntax highlighting
lang-atom
lang-cc
lang-cjs
lang-cl
lang-clojure
lang-coffeescript
lang-console
lang-csharp
lang-cson
lang-cv
lang-cxx
lang-cyc
lang-delphi
lang-dfm
lang-dpr
lang-el
lang-erl
lang-fs
lang-gemspec
lang-go
lang-gyp
lang-h
lang-haskell
lang-hh
lang-hpp
lang-http
lang-https
lang-hxx
lang-iced
lang-ini
lang-ipython
lang-irb
lang-javascript
lang-json
lang-jsp
lang-jsx
lang-kt
lang-kts
lang-less
lang-lsp
lang-m
lang-mak
lang-make
lang-makefile
lang-mathematica
lang-md
lang-mjs
lang-mk
lang-mkd
lang-mkdown
lang-mm
lang-mma
lang-none
lang-objc
lang-ocaml
lang-pas
lang-pl
lang-plaintext
lang-plist
lang-pm
lang-podspec
lang-protobuf
lang-python
lang-rc
lang-rkt
lang-rs
lang-rss
lang-ruby
lang-s
lang-scheme
lang-scm
lang-scss
lang-shell
lang-shellsession
lang-ss
lang-svg
lang-tex
lang-text
lang-thor
lang-toml
lang-ts
lang-tsx
lang-txt
lang-typescript
lang-vbnet
lang-vbs
lang-vhd
lang-wl
lang-wsf
lang-xhtml
lang-xjb
lang-xsd
lang-xsl
lang-yml

Moderator option available, but no available syntax highlighting

(I can't easily check if a moderator ever selected those two options, but if they did, I consider this now a data error. This might need fixing as well, preferable by adding highlighters for proto and regex, but if that is too much of a stretch then I propose to remove the language setting from those tags.)

Mod option Syntax highlighting
lang-proto
lang-regex

Moderator option matches available syntax highlighting

Mod option Syntax highlighting
lang-bsh lang-bsh
lang-c lang-c
lang-clj lang-clj
lang-coffee lang-coffee
lang-cpp lang-cpp
lang-cs lang-cs
lang-csh lang-csh
lang-css lang-css
lang-dart lang-dart
lang-erlang lang-erlang
lang-golang lang-golang
lang-hs lang-hs
lang-html lang-html
lang-java lang-java
lang-js lang-js
lang-kotlin lang-kotlin
lang-latex lang-latex
lang-lisp lang-lisp
lang-lua lang-lua
lang-markdown lang-markdown
lang-matlab lang-matlab
lang-ml lang-ml
lang-objectivec lang-objectivec
lang-pascal lang-pascal
lang-perl lang-perl
lang-php lang-php
lang-py lang-py
lang-r lang-r
lang-rb lang-rb
lang-rust lang-rust
lang-scala lang-scala
lang-sh lang-sh
lang-sql lang-sql
lang-swift lang-swift
lang-vb lang-vb
lang-vhdl lang-vhdl
lang-xml lang-xml
lang-yaml lang-yaml

I would prefer if the first two tables would have no entries and all available combinations were matched in the last table.

Can you update and align both the Highlight.js code and the option list of the select input with id lang-class-value?

Here is the code used to produce the three tables:

void Main()
{
    var modOptionSet = GetValues(new Regex("\"(lang-\\w+)\""), GetModOptions());

    var highlightSet = GetValues(new Regex("`(lang-\\w+)`"), GetHighlightJSLang());
    highlightSet.Remove("lang-code"); // lang-code is used as an example

    OutputSet("Mod option missing", highlightSet.Except(modOptionSet), "| | {0}");

    OutputSet("Highligthing is missing", modOptionSet.Except(highlightSet), "| {0} |");

    OutputSet("Okay", modOptionSet.Intersect(highlightSet), "| {0} | {0}");
}

private void OutputSet(string title, IEnumerable<string> items, string format)
{
    Console.WriteLine(title);
    Console.WriteLine("| Mod option | Syntax highlighting");
    Console.WriteLine("|-|-");

    foreach(var item in items) {
        Console.WriteLine(format, item);
    }
    Console.WriteLine();
}

private HashSet<string> GetValues(Regex regex, string value)
{
    var list = new HashSet<string>();
    foreach(Match match in regex.Matches(value))
    {
        list.Add(match.Groups[1].Value);
    }
    return list;
}

private string GetModOptions() {
  return new HttpClient().DownloadString(@"https://pastebin.com/raw/hKudvBMq");
}

private string GetHighlightJSLang() {
  return new HttpClient().DownloadString(@"https://api.stackexchange.com/2.3/posts/184109?order=desc&sort=activity&site=meta&filter=!)qCrPFKQawtVrr(YU3kc");
}

class HttpClient:WebClient
{
    static HttpClient() {
        ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
    }

    protected override WebRequest GetWebRequest(Uri uri)
    {
        var request = (HttpWebRequest) base.GetWebRequest(uri);
        request.AutomaticDecompression = DecompressionMethods.GZip;
        return request;
    }
}
1
  • 1
    How many of these are duplicates that have an option available but are just an alias of it, though? We definitely do not want to include all the aliases as selectable options because it stands a better chance of breaking highlighting due to conflicts. E.g. lang-js and lang-javascript appearing on different tags to the same question. Last I checked there were only a few rather simple languages that couldn't be used.
    – animuson StaffMod
    Mar 12 at 23:13

2 Answers 2

8

To address animuson's comment, I've restructured rene's tables to account for aliases which should make it more clear what markdown is or is not supported in the mod options.

The following supported languages are not available in moderator language code options:

Name Language Codes
http lang-http, lang-https
ini lang-ini, lang-toml
json lang-json
less lang-less
makefile lang-makefile, lang-mk, lang-mak, lang-make
mathematica lang-mathematica, lang-mma, lang-wl
php_template lang-php_template
protobuf lang-protobuf
scheme lang-scheme, lang-scm, lang-ss, lang-rkt
scss lang-scss
shell lang-shell, lang-console, lang-shellsession
typescript lang-typescript, lang-ts, lang-tsx

The following languages are available in moderator language code options, but not supported:

Language Codes
lang-proto
lang-regex

(For completeness) The following languages are available in moderator language code options and supported (the bold language codes appear in mod options):

Name Language Codes
bash lang-bash, lang-sh, lang-bsh, lang-csh
c lang-c, lang-h, lang-cyc, lang-m, lang-c-like
clojure lang-clojure, lang-clj
coffeescript lang-coffeescript, lang-coffee, lang-cson, lang-iced
cpp lang-cpp, lang-cc, lang-c++, lang-h++, lang-hpp, lang-hh, lang-hxx, lang-cxx
csharp lang-csharp, lang-cs, lang-c#
css lang-css
dart lang-dart
delphi lang-delphi, lang-dpr, lang-dfm, lang-pas, lang-pascal
erlang lang-erlang, lang-erl
go lang-go, lang-golang
haskell lang-haskell, lang-hs
java lang-java, lang-jsp
javascript lang-javascript, lang-js, lang-jsx, lang-mjs, lang-cjs
kotlin lang-kotlin, lang-kt, lang-kts
latex lang-latex, lang-tex
lisp lang-lisp, lang-cl, lang-el, lang-lsp
lua lang-lua
markdown lang-markdown, lang-md, lang-mkdown, lang-mkd
matlab lang-matlab
objectivec lang-objectivec, lang-mm, lang-objc, lang-obj-c, lang-obj-c++, lang-objective-c++
ocaml lang-ocaml, lang-ml, lang-fs
perl lang-perl, lang-pl, lang-pm
php lang-php
plaintext lang-plaintext, lang-text, lang-txt, lang-none
python lang-python, lang-py, lang-gyp, lang-ipython, lang-cv
r lang-r, lang-s
ruby lang-ruby, lang-rb, lang-gemspec, lang-podspec, lang-thor, lang-irb
rust lang-rust, lang-rs, lang-rc
scala lang-scala
sql lang-sql
swift lang-swift
vbnet lang-vbnet, lang-vb, lang-vbs
vhdl lang-vhdl, lang-vhd
xml lang-xml, lang-html, lang-xhtml, lang-rss, lang-atom, lang-xjb, lang-xsd, lang-xsl, lang-plist, lang-wsf, lang-svg
yaml lang-yaml, lang-yml

The above markdown is generated by the script below. I used EvgenKo423's workaround from this answer to pull supported languages and aliases. I (manually) copied over the <option>s into a <select> from Cody Gray's pastebin and computed the differences.

This should make it easier moving forward check what languages are available to moderators.

const languages = hljs.listLanguages();
const addLangPrefix = (s) => `lang-${s.toLowerCase()}`;

const reversedCustomAliases = Object.entries(customAliases).reduce(
  (acc, [k, v]) => {
    acc[v] = (acc[v] || []).concat(k);
    return acc;
  }, {});


const aliasesMap = new Map();
const languageFamilies = [];
for (let language of languages) {
  let info = hljs.getLanguage(language);
  let langCode = addLangPrefix(language);
  let langCodes = new Set([langCode]);
  if (info.hasOwnProperty('aliases')) {
    // Include aliases in the list of lang codes
    info.aliases.forEach(e => langCodes.add(addLangPrefix(e)));
  }
  if (reversedCustomAliases.hasOwnProperty(language)) {
    reversedCustomAliases[language].forEach(e => langCodes.add(addLangPrefix(e)))
  }
  // Track language families with "parent" lang code and corresponding aliases
  languageFamilies.push([langCode, langCodes]);
  // Inverse map of all lang codes to the "parent"
  for (let alias of langCodes) {
    aliasesMap.set(alias, langCode);
  }

}


// Pull all available options from the list of options
const modOptions = Array.from(document.getElementsByTagName('option')).map(e => e.value).filter(e => !['<none>', 'default'].includes(e));
// lang-none is <none>/default
modOptions.push('lang-none');


// make a map to easily remove language families
const notSupportedLanguageMap = new Map(languageFamilies);
// Test for availability in modOptions
for (let o of modOptions) {
  if (aliasesMap.has(o)) {
    // Find "parent" language code and remove from notSupportedLanguageMap
    notSupportedLanguageMap.delete(aliasesMap.get(o));
  }
}


const sortByLangCode = ([lc1, a1], [lc2, a2]) => lc1.localeCompare(lc2);
// Display supported languages not available in mod options
let markdownString = "The following supported languages are __not available__ in moderator language code options:\n\n|Name|Language Codes|\n|:--|:--|\n";

Array.from(notSupportedLanguageMap).sort(sortByLangCode).forEach(([parentLangCode, aliases]) => {
  markdownString += `|${parentLangCode.slice(5)}|${Array.from(aliases).join(', ')}|\n`;
})
markdownString += '\n';


// Display unsupported languages in mod options
markdownString += "The following languages are available in moderator language code options, but __not supported__:\n\n|Language Codes|\n|:--|\n";

modOptions.filter(e => !aliasesMap.has(e)).forEach(langCode => {
  markdownString += `|${langCode}|\n`;
})
markdownString += '\n';

// Display supported language families
markdownString += "(For completeness) The following languages are available in moderator language code options and supported (the bold language codes appear in mod options):\n\n|Name|Language Codes|\n|:--|:--|\n";
languageFamilies.filter(([k, v]) => !notSupportedLanguageMap.has(k)).sort(sortByLangCode).forEach(([parentLangCode, aliases]) => {
  markdownString += `|${parentLangCode.slice(5)}|${Array.from(aliases).map(a => modOptions.includes(a) ? `__${a}__` : a).join(', ')}|\n`;
})
markdownString += '\n';

console.log(markdownString);
<!-- Setup from https://meta.stackexchange.com/a/372141/578924 by EvgenKo423 (https://meta.stackexchange.com/users/578924) -->
<script>
  var StackExchange = {}; // prevents highlightjs-loader execution error

  // hack to get custom Stack Exchange aliases
  Object._keys = Object.keys;
  Object.keys = function(obj) {
    if (obj.none) {
      window.customAliases = obj;
      Object.keys = Object._keys;
    }
    return Object._keys(obj);
  };
</script>
<script id="webpack-public-path"></script>
<script src="https://cdn.sstatic.net/Js/highlightjs-loader.en.js"></script>
<script src="https://cdn.sstatic.net/Js/third-party/highlight.js/additional-langs/lang-mma.min.js"></script>
<!-- End setup -->
<span>Current Mod Menu</span>
<!-- Manually populated from https://pastebin.com/raw/hKudvBMq-->
<select>
  <option value="<none>" selected="selected">&lt;none&gt;</option>
  <option value="default">default</option>
  <option value="lang-bsh">lang-bsh</option>
  <option value="lang-c">lang-c</option>
  <option value="lang-clj">lang-clj</option>
  <option value="lang-coffee">lang-coffee</option>
  <option value="lang-cpp">lang-cpp</option>
  <option value="lang-cs">lang-cs</option>
  <option value="lang-csh">lang-csh</option>
  <option value="lang-css">lang-css</option>
  <option value="lang-dart">lang-dart</option>
  <option value="lang-erlang">lang-erlang</option>
  <option value="lang-golang">lang-golang</option>
  <option value="lang-hs">lang-hs</option>
  <option value="lang-html">lang-html</option>
  <option value="lang-kotlin">lang-kotlin</option>
  <option value="lang-java">lang-java</option>
  <option value="lang-js">lang-js</option>
  <option value="lang-latex">lang-latex</option>
  <option value="lang-lisp">lang-lisp</option>
  <option value="lang-lua">lang-lua</option>
  <option value="lang-markdown">lang-markdown</option>
  <option value="lang-matlab">lang-matlab</option>
  <option value="lang-ml">lang-ml</option>
  <option value="lang-objectivec">lang-objectivec</option>
  <option value="lang-pascal">lang-pascal</option>
  <option value="lang-perl">lang-perl</option>
  <option value="lang-php">lang-php</option>
  <option value="lang-proto">lang-proto</option>
  <option value="lang-py">lang-py</option>
  <option value="lang-r">lang-r</option>
  <option value="lang-rb">lang-rb</option>
  <option value="lang-regex">lang-regex</option>
  <option value="lang-rust">lang-rust</option>
  <option value="lang-scala">lang-scala</option>
  <option value="lang-swift">lang-swift</option>
  <option value="lang-sh">lang-sh</option>
  <option value="lang-sql">lang-sql</option>
  <option value="lang-vb">lang-vb</option>
  <option value="lang-vhdl">lang-vhdl</option>
  <option value="lang-xml">lang-xml</option>
  <option value="lang-yaml">lang-yaml</option>
</select>

5

You're right that the UI is confusing here. We've cleaned up the languages available in that dropdown so that all base languages are selectable. The intention here is that you select the base language (in bold) from the list of language codes, even though it doesn't match your tag's name.

I agree that it would be useful to have all of the base languages and synonyms available here. It'll take a bit more work to fix this, so for now I'm marking the current behavior as . We'll update the status of the post again when we plan on fixing this.

The first part we fixed is that here were two languages in the list that weren't supported: I removed lang-proto and lang-regex. These were supported with Prettify but are no longer supported in highlight.js. I expect that this has been broken since we migrated in late 2020. The only tag to use one of these options on Stack Overflow was [protocol-buffers]. It was just falling back to a default highlighter in highlight.js. I've changed that tag's code language to be default so it continues to use that highlighter.

The second part we fixed was that we added the base languages that we already supported that were not available in the dropdown:

  • lang-bash (replaces lang-bsh)
  • lang-http
  • lang-ini
  • lang-json
  • lang-less
  • lang-mathematica
  • lang-php-template
  • lang-protobuf
  • lang-scheme
  • lang-scss
  • lang-shell
  • lang-typescript

I did not look at existing post content to see if anyone had manually set lang-proto or lang-regex, but those will also continue to fallback to the default highlight.js highlighter.

6
  • 2
    I'm a bit confused by your statement: "It seems like all of the supported languages are selectable with some base language and no other base languages need to be added" The post which sparked this conversation is about fact that there is no supported base language for ini/toml. The first table in my post is a complete list of base languages that are missing entirely from the selection menu (at least that which is available on SO I know mathematica is not available there so it makes sense that it is absent from the list). May 9 at 23:05
  • I also had a different question about the bash language family. There appears to be three separate languages in the list that are selectable. (lang-sh, lang-bsh, lang-csh) Is this also a remnant of a previous system? May 9 at 23:07
  • 4
    @Henry Oh uh wow, yeah my assumption on all supported languages being there is wrong, not sure how I missed those types of tags. Let me take another look into it, I expect we can add these supported but unlisted items. I'll also look at lang-sh, lang-bsh, and lang-csh, not sure what the history on those is.
    – Kyle Pollard StaffMod
    May 10 at 0:04
  • 1
    @HenryEcker we took a second pass at the fix and I think we've resolved the problems I missed
    – Kyle Pollard StaffMod
    May 24 at 18:33
  • 1
    Much appreciated. Thank you for both the ping and adding these missing options. I'm not sure if you want to revisit the status-by-design piece. I believe Rene's initial question was primarily about the missing base language which might make this now status-completed? May 24 at 18:41
  • 1
    I think so! Personally I'd like to see all languages in the dropdown instead of just base languages (and we do some translating to base languages on our backend) but for now I think it's fair to call it status-completed. Thanks for your help, Henry!
    – Kyle Pollard StaffMod
    May 24 at 22:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .