258

Over the years, a number of community members have developed unofficial browser userscripts to add or supplement functionality on our platform. While we do have an official API, these scripts sometimes use the global JavaScript object StackExchange to get information or perform actions on the page. That said, the StackExchange object is an undocumented way to interact with a page on our platform on the client-side and was just an artifact of the way we were structuring our code internally.

This StackExchange object has been the main namespace that we have used for making global functions available on the client-side across the site. However, we're currently migrating our JavaScript to ES modules. A direct repercussion of this is that we'll stop using the global StackExchange object and will instead import the properties and methods we need directly onto each page based on the functionality that is hosted there.

This leaves us with the question of how to handle the global StackExchange object once we stop using it internally. We know that userscripts depend on this object existing and that removing the properties and functions that it makes accessible will prevent many scripts from continuing to work. Breaking changes are frustrating and we don't want to irreparably break userscripts.

We're reaching out to the userscripting community here in order to better understand your needs. Our objective will be to provide a stable set of client-site properties and functions that can be relied on by userscript authors. To that effect, we are interested in knowing:

We are going to make an effort to avoid breaking functions and properties in the window.StackExchange namespace that are used by userscripts in the near term, however at this point we cannot commit to supporting every specific function or property. That said, we believe that having documented and stable client-side resources available will be very good for the health of the userscript community in the long run and we are exploring our options for providing a stable, official scripting API.

Improve this question
12
  • 70
    Thanks for the heads-up, and the intent not to break community code.
    – Andras Deak -- Слава Україні
    Commented Dec 14, 2021 at 18:04
  • 34
    "what improvements could we make for you?" - providing a way to be able to select semantically important elements on a page would go a long way (an id or, perhaps a stable class name - nothing fancy). For example, this #mainbar-full div > ul:nth-child(2) is a selector for the user profile stats - if the container could be selected via document.getElementById("user-stats"), it would be immensely helpful (of course, this is not to say you should not be free to change whatever on the page - just having a sane way to access such "blocks" when present would be cool)
    – 0Valt
    Commented Dec 14, 2021 at 20:48
  • 15
    @OlegValter Its entirely ok to flesh that out and post that as another answer :D
    – Journeyman Geek Mod
    Commented Dec 14, 2021 at 21:20
  • 2
    @JourneymanGeek it is sort of a placeholder for future expansion :) But it seems like Redwolf beat me to it - the point is essentially the same: having a stable way to interact with elements on the page (which as you know can be quite painful right now)
    – 0Valt
    Commented Dec 14, 2021 at 21:23
  • 1
    To address the elephant in the room - will the rework also deal away with the CHAT global object [ducks and covers under the nearest pile of dishes]?
    – 0Valt
    Commented Dec 14, 2021 at 21:34
  • 13
    @OlegValter Related: Could we revisit/reconsider the lack of chat API? :D
    – Journeyman Geek Mod
    Commented Dec 14, 2021 at 23:01
  • 1
    @JourneymanGeek seen it the day you asked :) combined with a stable set of integration points for main sites, this would be a great present to the community (somewhat bittersweet as it would invalidate at least 4 packages, but heck, I'd gladly let go if there was an official way).
    – 0Valt
    Commented Dec 15, 2021 at 8:10
  • If you're using ES Module you could still bootstrap the StackExchange Window object to the ES Modules window.StackExchange = ..., it would mean making code purposefully to support backwards capabilities but I would say don't update the bootstrap to get people to stop using it and move over to the Correct API's for any new functionality
    – Barkermn01
    Commented Jan 5, 2022 at 11:51
  • 1
    Before any major changes go live, it might be helpful if accessing any of the existing functionality would generate a message in the browser console like "this API is deprecated, see URL for details". Script maintainers may not see these meta posts and would appreciate the advance warning.
    – bta
    Commented Jan 6, 2022 at 5:10
  • 3
    A quick, out-of-context observation: I'd suggest retitling this post something like, "Designing a public JS API for userscripts" or "Seeking input on a stable JS API to replace window.StackExchange" — the current title makes it sound like the API is already finished and you're now announcing a rollout, not that the design is just starting and now would be the time for anyone who depends on the unofficial API to make sure their needs are addressed in what comes next. As such, it feels like you're burying the lead with the current title.
    – FeRD
    Commented Jan 12, 2022 at 7:26
  • 3
    In an upcoming sprint, we've planned to address the short-term needs of providing essential user script dependencies as best we can so we don't break existing userscripts. As part of that work, I'll spend time responding to all the feedback on this post and the other post. Thanks for all of the feedback so far!
    – Kyle Pollard StaffMod
    Commented Feb 1, 2022 at 16:53
  • Why was this suddenly protected by the community user? EDIT: just rechecked my notifications, and it had a spam answer.
    – AncientSwordRage
    Commented Sep 8, 2023 at 12:49

7 Answers 7

Reset to default
117

As a quick preface, I think I will not be alone in saying this: thank you. This is very good news, and a sign of goodwill towards the community - the project has the potential to streamline userscript development of the platform, which is greatly appreciated.

If we provided a more stable interface for this information, what would you like to see from it? For example, would officially published TypeScript definitions be helpful?

An ideal situation would be to have TypeScript definitions published as part of DefinitelyTyped (for those unfamiliar, this is the repository where @types/* packages live). That would be a great contribution to open source as well as provide the community with a way to contribute too.

We also created and maintain unofficial TypeScript definitions for StackExchange and CHAT (also augmenting the Stacks namespace) namespaces - this could become a joint effort if there was at least some sort of a changelog available for us.

Improve this answer
45

Reliable HTML/CSS and UI manipulation

Relying on SE's HTML/CSS to work a certain way is, from what I've seen, the most common thing that breaks userscripts.

As an example, adding items under the title of a question (next to last active, creation date, view count, etc.) is used by multiple userscripts, and as the developer of a smaller one of those, trying to make it play nice with the others is very painful.

A second example would be CGCC's gradscript, an ages-old userscript which adds custom styling and some code golf specific features to the site (including a leaderboard, the main reason it's still so widely used). I don't use it, but it's broken several times in recent memory due to changes to IDs or classes on elements.

I'm not entirely sure of what a solution to this problem might look like, but it would be, in my opinion, the most useful thing an API could provide. It would significantly reduce the number of scripts which mysteriously stop working, reduce the likelihood of two scripts being incompatible, and make it much easier to keep a consistent look with the rest of the site's UI.

Improve this answer
7
  • 3
    UI changes unfortunately have the largest impact on userscripts, not much you can do about it aside from just build a more robust script, or push updates more frequently to react to versioning. I suppose if there was a documented UI versioning then userscript authors would have somewhere to look to in case of breaking changes and could update faster.
    – Travis J
    Commented Dec 16, 2021 at 21:07
  • 12
    @TravisJ But there are steps that can be taken, like having a certain class or ID added to various parts of the UI that's guaranteed not to change, or having functions which can apply the necessary styling to various elements so you don't have to just copy SE's and hope it doesn't change.
    – rydwolf
    Commented Dec 16, 2021 at 21:10
  • 3
    From my experience, the entire UI has changed at one point or another with regards to structure; some more nuanced than others. You also don't have to hardcode copy SE elements. You can template them on the fly from the page, which will be much more likely to endure changes.
    – Travis J
    Commented Dec 16, 2021 at 21:14
  • 2
    @TravisJ Sure, the UI will undergo massive changes on occasion. But there are plenty of small ones. Changes to class names, slight updates in things like margins or font sizes, etc. And hardcopying SE elements often works, sure, when the IDs, classes, ordering of nodes, etc. doesn't change :p
    – rydwolf
    Commented Dec 16, 2021 at 21:20
  • 4
    Don't get me wrong, I've hardcoded stuff when I wasn't terribly concerned with what it would look like, but that type of approach certainly ages poorly. I definitely think a changelog for UI changes would help userscripts stay relevant.
    – Travis J
    Commented Dec 16, 2021 at 21:49
  • 8
    A way to allow this to work better would be adding an api like StackExchange.getQuestion().insertToolbarItem({ children: 'New example', title: 'Click this button for an example', onClick: (e) => {}})
    – Ferrybig
    Commented Dec 19, 2021 at 10:41
  • 1
    I have StackExchange.ready(...) in a UserScript on the traditional view of Dev Story FWIW, the purpose being to rearrange and format the page the way I want for PDF export / printing. That's pretty easy to replace, and I only know about that event from, well, working at Stack. Point being, it's not just core Q&A that warrants consideration.
    – Andrew
    Commented Jan 1, 2022 at 0:10
32

Custom events on post creation/deletion/edition, etc.

Userscripts that add buttons to the post menu, that interact with the comments section or change the review interface need, more often than not, to detect when new comments are added, a new answer is posted, a post is edited or a new review item has loaded.

To do that, they usually hook all requests in a page or use $().ajaxComplete(). However, the first extends prototypes, while the second is unstable as it relies on jQuery (i.e. if the requests weren't made with $.ajax() it wouldn't work).

Instead, how about dispatching some custom events we could listen to, like new-post, new-comment, loaded-review, possibly including the HTML element and/or the post/review/comment id? This would make userscript development much easier and simple.

Improve this answer
8
  • 2
    Watching the requests is much heavier weight than watching the DOM, a very large portion of this could be handled with mutation events.
    – Travis J
    Commented Dec 16, 2021 at 21:06
  • 2
    @TravisJ did you mean mutation observers? Mutation events were deprecated precisely because of performance issues.
    – 0Valt
    Commented Dec 16, 2021 at 21:08
  • 5
    @Oleg - Yes, observers, thank you for the clarification.
    – Travis J
    Commented Dec 16, 2021 at 21:11
  • 1
    @TravisJ NP - just noted for future readers. Speaking of observers, though, can you clarify why would watching requests by in heavy? SE's code just needs to define callbacks that would fire custom events or post messages in some strategic places - no real performance concern that I know of. The only issue with custom events is synchronous dispatch, but that can be mitigated with postMessage to self instead.
    – 0Valt
    Commented Dec 16, 2021 at 21:24
  • 1
    @Oleg - If SE were to include defined callbacks that would be very efficient, however I find that unlikely. My comment was referencing the situation where you need to watch every request and determine whether or not it was referencing an event you were interested in.
    – Travis J
    Commented Dec 16, 2021 at 21:47
  • 1
    @TravisJ well, yes, I was thinking about SE just dispatching custom events in the success/error callbacks (unlikely, but would be immensely cool to see realized). I think double-beep meant the same thing. Agreed that intercepting every request to peek if it is the correct one is not ideal, hence the improvement request (no pun intended)
    – 0Valt
    Commented Dec 16, 2021 at 21:51
  • 2
    @Oleg - Indeed. I don't think it is a bad idea to have custom events, I would welcome that, however I doubt they would modify their development in order to accommodate that. My main point was mostly minor, with regards to the current approach and in response to the note "To do that, they usually hook all requests in a page or use $().ajaxComplete()."; not necessarily in response to the idea of custom events or custom callbacks.
    – Travis J
    Commented Dec 16, 2021 at 21:55
  • 1
    That's not what extending the prototype is FYI. Your linked example is just hooking a native function that already exists and forwarding the call to it. Very useful pattern in userscripts
    – a cat
    Commented Dec 20, 2021 at 17:08
19

(Keep) eating your own dogfood

Otherwise known as dogfooding, it would mean that you made sure that whatever API you provide for the community, you make sure are used internally as well.

I am not very involved in the StackAPI/Userscript community here, and at a cursory look I think you're already doing this? If so, please continue. Otherwise, please start doing so! :D

The benefit is that the community has confidence in the API and real-world examples to work from.

Improve this answer
15

If we tried to make userscript development better, what improvements could we make for you?

As always documentation is key. Maybe a "getting started"-document with the stable(!) ids and classes you can permanently depend on as well as a short example of that new API usage would lead to more people being willing to experiment.

Improve this answer
2
  • 12
    "permanently depend on" may be too unrealistic. Softening that to "depend on for a long time" sounds better.
    – iBug says Reinstate Monica
    Commented Dec 22, 2021 at 14:07
  • If the document is updated appropriately if/when the API changes, that would make it effectively permanent.
    – Toby Speight
    Commented Jan 11, 2022 at 13:04
14
  • Share the documented internal JavaScript files that userscripts will interact with on the front end
  • Share documented endpoints that AJAX will interact with
  • Share documented sockets that userscripts will register to
  • Just break the existing hooks that have large technical debt, and provide documented ones in return

Userscript authors are experts at finding places to tie into and then use. A lot of work has gone into identifying those odd places, and I am sure some authors are upset at the suggestion to break some of those odd places, but in the end it is better provided a few things are built in.

We need hooks, and they don't need to be global or even super obvious. If you fully document the type of hooks available on a page, and you fully document what those hooks do in code, then we can use them or override them.

The key here is code documentation: for the front end, for the end points, for the sockets. In the long run, having all of that documented for use will be better for everyone.

Improve this answer
2
  • 1
    And you can't document things without breaking them?
    – Luuklag
    Commented Dec 16, 2021 at 21:48
  • 7
    @Luuklag - Hopefully documentation isn't a breaking change. However, what seemed to be the overall approach being used by the team here is to limit feature sets available to the front end by locality. In this case, certain features may no longer be available to a user script through the same path as it was before. If making those changes allows for a future where pathing is more certain, and documented, then making those breaking changes now is the right choice; even if it means modifying API pathing or feature use by userscript authors in the near term.
    – Travis J
    Commented Dec 16, 2021 at 21:52
6

If we tried to make userscript development better, what improvements could we make for you?

Expose the API for creating and modifying keyboard shortcuts. Currently, the only exposed part of the API is the one exported via the revealing module pattern on the StackExchange global object as keyboardShortcuts property with only two public methods: init and destroy.

The abovementioned methods provide only the "all or nothing" API which severely limits the options of userscript developers that would otherwise be willing to provide scripts for customizing shortcuts. In the source, there is a buildShortcuts that makes use of a private Shortcuts constructor instances to add shortcuts.

Even something as simple as exposing these instances (G, inPageNav, and P to be precise) via the function's return value (exported on the keyboardShortcuts nested object) would go a long way for dramatically improving the possibilities for manipulating the shortcuts.

All the above is not an idle concern - there are numerous feature requests that could be handled by the community for the benefit of everyone (including reducing the amount of dev time needed to triage and address those requests). The following is only the tip of the iceberg:

Year Post
2022 Add keyboard shortcut for editing the list of duplicates
2022 Keyboard shortcut for "code" different than on GitHub
2021 Add ability to open selected question in a new browser tab using keyboard shortcuts
2021 Add the ability to show comments without adding a comment using keyboard shortcuts
2020 Add a keyboard shortcut to visit the Jobs page
2020 Add a keyboard shortcut to bookmark a post
2020 Is there a keyboard shortcut to follow questions/answers?
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .