TL;DR: We’re making a substantial repayment on our JavaScript technical debt. Apologies in advance for any bugs as we work through it!
An administrative note: Rather than posting bug reports as answers to this post, in this case we'd prefer you to post a separate question for each bug you find.
Stack Overflow's JavaScript setup hasn't substantially changed since, like, 2008. Here are a few fun facts about the way we build our JS.
The process to install third party libraries is:
- Download a copy of the library's code
- Paste that code into a folder
git commit
Seriously! We have a
third-party
folder in the repo. It contains (very old versions of) a variety of libraries. Some of those libraries have multiple versions in subfolders. Some of those libraries have had ad-hoc edits made to them. It's difficult to upgrade dependencies and it's difficult to manage transitive dependencies.If you want to split your code up into multiple files, you need to manually include each of those files in the correct order using
script
tags.- This goes for files in the aforementioned
third-party
folder too. - It’s painful enough to discourage developers from splitting code up. We have quite a lot of 5000-line monster files.
- In many cases, splitting code into files to make it more maintainable can be actively detrimental to performance.
- Code gets written in shared files and then included everywhere. Most of our pages are downloading way more JS than they actually need.
- This goes for files in the aforementioned
Files communicate with each other through global variables.
- It's difficult to track down all the usages of a given function.
- Intellisense tooling doesn't know which globals are available in which contexts. The namespace is cluttered up with irrelevant definitions and it's easy to accidentally use a function on a page where it isn't actually available.
- Consistency across the codebase in this regard is not high. Some files use TS namespaces, others write to a global
StackExchange
object, others install variables directly into the global scope…
We have an idiosyncratic home-made file-loading/module system (called
StackExchange.using
) with a variety of similar-but-different usage patterns.
We've bolted things on to this system over the years — most new code is written in TypeScript, for example — but the overall story of how we build and deliver JS hasn't really changed.
Anyway, that's enough background. You may be aware that we've recently integrated Webpack into our JS build pipeline. I'm writing today to let you know that the second phase of this migration is underway: we are rewriting our code to use ES modules. This'll help us manage dependencies (both internal dependencies and third-party ones, using NPM) and make it easier to write reliable JavaScript as time goes on.
However, this is one of those big change-every-file migrations, and of course each file has its own idiosyncrasies. I've set things up so we can mostly work one file at a time, and we will obviously test each piece of the migration before we ship it, but Stack Overflow is a complicated 13-year-old system and it's difficult to catch every possible edge case in testing. In other words,
The work on this has already begun. I've built out all of the infrastructure (I think), and we're working file by file. Every developer in the department is converting a handful of files each, and in most cases each of those conversions can and will be shipped independently.
We have quite a lot of files to convert (hundreds but not thousands) and we're working on this migration in parallel with other ongoing projects, so I'm not expecting it to be a particularly quick job. I'm about to go on a six week sabbatical and I'd be astonished (and delighted) if the work was 100% complete by the time I'm back! But hopefully it'll be mostly done by that point.
The ask for you Metazens is simple: please let us know if you notice something's broken. In other words, keep doing what you're doing! We really appreciate it when you inform us about bugs.
Thank you for your patience and help!