Oneboxing on links — let us make plugins?

Question

There's a handful of sites that turn into nice looking boxes when we paste links in -- Such as SE questions, youtube videos, Wikipedia.

Could we extend a scheme to room owners that enables us to create our own for cases that may be less common globally but gain momentum locally?

For instance, if I keep posting to pages on the MDC or W3C specs in room that care about it, it's nice to have that feature. It might not be appropriate for it to exist globally, or for SE to be responsible for managing potentially thousands of these links-- but if we could use our trust model and have room owners specific plugins made by the community this may be a good way to get this done.

In theory this needs to be two parts:

1. The scraper does some DOM node extracting (provide a way to identify HTML nodes in a document)
2. CSS/HTML template based on the scraper
3. Trust model where room owners can enable/disable locally

This means SE doesn't have to manage thousands of links, room-specific interests get the useful preview, and it's grown from the community.

Thoughts on this?

Answer 1

I would only possibly picture this happening via JavaScript on the client, since we wouldn't be running lots of small contributed (and since it'd be using regex recognition, potentially chat-server-killing C# code). We're perfectly capable of killing the chat servers with runaway regexes ourselves (yes, it has happened a few times), we don't need help doing it!

As for the client-side approach: there are plenty of issues with allowing people to write JavaScript that runs on other user's browsers (this would be involuntarily, unlike a user script): security, browser-locking pattern recognition (bad code), etc.

I just can't see a workable model for this that doesn't involve either a) us maintaining/reviewing all the oneboxes anyway (in C#), or b) very serious client-side concerns we can't ignore.

Answer 2

StackExchange chat Onebox plugin module

JSON manifest of XPATH data

1. Provide a URL
2. Provide a token ID/XPATH pair
3. Provide HTML template

Example: Setting up the plugin

Note: URL fragment "Section_5.1.3" implies XPATH selections starts at that ID (this could resolve the section-specific linking such as seen with WikiPedia links): /id("Section_5.1.4")/

The objective is to classify a URL into grouping based on URL components which we could in theory run an XOR or some other inclusive matching over (no server-side regex, only string matching).

The source-code for the plugin (ie, user submitted code) would look like this:

{
  "URL" : {
        "scheme"    : ["http", "https"],
        "authority" : ["ecma262-5.com", "www.ecma262-5.com"],
        "path"      : [], //No special matching
        "query"     : [],
        "fragment"  : []
    }
  "Tokens:"[
    {
      "id" : "{Title}",
      ""         //implied ID matches element
    },
    {
      "id" : "{Text}",
      "/following-sibling::text()"    //Grab the adjacent text node
    },
  ],
  "Template" : "<h1>{Title}</h1> <br/><p>{Text}</p>"
}

Examine URLs for matching plugins based on string matches of URL components

Example: Onebox request client-side:

function URLToComponents(url){
    //RFC 2396 appendix B-- URL regex 
    var urlRegex=new RegExp("^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))?");

    var data = url.split(urlRegex);
    return {
        "scheme"    : data[2],
        "authority" : data[4],
        "path"      : data[5],
        "query"     : data[7],
        "fragment"  : data[9]
    };
}

var components = URLToComponents("http://ecma262-5.com/ELS5_Section_5.htm#Section_5.1.3");

We then submit the onebox reguest with the provided URL and the template JSON attached to the room.

Matching calls plugin

The provided URL "http://ecma262-5.com/ELS5_Section_5.htm#Section_5.1.3" matches the URL components as described in the plugin source, thus we request the page and assign token matches from XPath of the DOM Document.

Request then XPath

1. Send request to the URL, get the HTML.
2. Scan HTML with XPath to find the corresponding DOM nodes
3. Return content truncated at 512 chars, or empty string (if missing).
4. Convert token data to sanitized inputs (ie, no alert("xss")), flat text elements should suffice.

Build the template

We don't really need massive customization here, simply a few basic CSS classes to keep things looking clean on the SE side. Just provide the same classes, and let semantic HTML take care of the rest. Insert the text into the tokens and drop in the HTML content.

Fin.