What testing is done on 'April Fool' pranks?

Question

So, we have just had an 'April Fools' jape that

• activated on 31st March for many people
• wan't capable of being disabled site-wide (although this has been fixed)
• isn't capable of being disabled network-wide (or even better account-wide)
• presented a design (colours, moving marquee, random sparkles) that for many people renders the site physically unusable (admittedly this is based on a small sample within my own household but for those I've asked who are on the autistic spectrum (including myself, my sister, and two nephews) it is like nails down a blackboard -- unbearable
• caused problems for people with slower PCs, as slower machines may not run the feature fast enough and the site may lock up
• guaranteed that anybody encountering Stack Exchange for the first time while it is active would never ever return

The bug has now been fixed -- hooray. However, will next year's prank be:

1. tested for accessibility issues,
2. assessed for its impact on non-insiders that we would want to retain, and
3. just plain better tested?

For clarity, I'm not trying to be a spoilsport -- April Fools, Christmas Hats and the like are fixtures around here and can range from 'ROFL' to 'shrug and move on'. I am not arguing that they should be abandoned, just that they should be tested to a reasonable level (so they don't break the site and people can opt out simply via button that works), and crucially assessed for accessibility at the same level that other theme changes are assessed.

Answer 1

We began brainstorming ideas for April 1 early in February. (The first version of the brainstorming document was written on February 20, but we had informal discussions when we started March planning.) The initial specification was written on March 5 and finalized on March 18. The first commit for the project was done on March 20. The basic design was approved on March 22 with the following comment:

All this CSS really sucks.

On March 27 we noticed the globe gif was 13.4Mb, so we replaced it with a smaller gif. The Architecture team approved the pull request on March 28:

I see a few things to tidy here but no blockers . . . nice work making the site hideous!

This is a very short turnaround for a feature that changes so much on the page. Because we start timed events like this at midnight +14 UTC, our developers were awake an an unreasonable time to handle any bugs that might occur. (I should have been awake too, but I slept trough the calendar notification.) Since the launch, I count 4 commits fixing a variety of bugs, including the one you mentioned.

We've been testing things on our development environment for the past week. Some people tested disabling the feature. I'm afraid we assumed that since it worked on the Y2K question it was working everywhere in dev. (But it might be there are relevant differences between dev and prod when it comes to cookies.)

---

That's the basic story of testing for this year. To address the bigger question of what testing should be done, it's important to recall:

1. A prank like this is intended to be disposable, but not disrupt the function of the site.
2. This particular prank is intended to look like a monstrosity from the late 90s built by someone with just enough knowledge of DHTML to be dangerous.

Our safety value was the ability to turn the feature off with a button click. Not only does that mitigate the performance and usability concerns, it also give April Fools grouches an option to avoid the whole thing. Since the cookie was broken, there was a period of time when the safety value failed. While it would be nice if that hadn't happened, it was fixed fairly quickly. (Most people saw the fix before April 1 began for them. ;-)

I think this was a reasonable amount of testing and I don't really feel too bad that it wasn't perfect out of the gate. I think this level of testing is appropriate next year. (But maybe we ought to reuse the code to toggle the setting rather than rolling a new version.)

---

Finally I want to address the concern that this prank will cause permanent harm. The model I have in mind for how people start participating on the sites is they find the site via a Google search. Most of the time, they get the content they were looking for (or not) and move on to other searches or search results. Once in a while people discover the content was built by an active community and get intrigued:

1. Maybe it's because they keep landing on the a page from the site.
2. Maybe it's because they connect the site with another one they are familiar with on the network.
3. Maybe they just start poking around at other questions.

If that model is reasonably accurate, events like April Fools can really help the process of people discovering the community behind the site. Yes, there will be some people who land on an outdated monstrosity and don't get that it's supposed to be a joke. If so, they won't likely discover the community via the third route. But that doesn't mean they will never come back. In the days to come, the most likely candidates to join the site will find it again in future search results. I've talked with many people (including a theology professor just on Friday) who vaguely know about Stack Exchange because of search results. So some people will land on a site, notice the joke and discover there is something more to the site than a collection of great answers.

I could be wrong, of course. But if so it should be possible to look at it in the data later in the week. It's a pretty great opportunity to learn something about how people find the site. And we have many years worth of data to dig through with differing prominence of the prank.