18

I left out a lot of detail in my recent Winter Bash 2014 wrapup because it was too "inside baseball" for the general public. Before I forget, I thought I should put my post-mortem to metapost.

Things that worked

  • Start early—I got roped in involved with planning WB 2014 on November 6 and Laura had started work earlier than that. Even so, there were details being finalized right up to the start of the event. I wouldn't say it's never too early (because I was pretty sick of WB even before it started), but having essentially a month (modulo Thanksgiving) was luxurious.

  • Reuse and extend—At this point we have a solid framework for this event, so we can reuse a lot of the code and copy. One of my goals was to have all new hat ideas and triggers, but that turned out to be too ambitious. So we had mostly new hat triggers, all new hat illustrations (though Eureka! and the pirate hat concepts were carried over from last year), and a new theme for the Winter Bash page. Hat scaling and rotation was the main new feature that balpha added on the code side.

  • Think about what you can learn—The Red Baron trigger is a great example of something we hoped to learn more about. There were several other hat ideas that were specifically included to find out how people would respond. I haven't done the analysis yet, so I don't know if we got useful results. But I think attempting to learn something is almost always a good idea.

  • Sites opted in by default—In the past, we gave site moderators a choice to participate with the default being "opt-out". If a mod team didn't get back to us, their site was left out. This year, the default was "opt-in". Thanks to the default effect, we only had 2 sites opt out. Even though they didn't need to, we got 87 responses from moderators to opt in. I also got a number of emails from moderators expressing their excitement about the event. Besides getting more sites involved, this also made tracking of moderator responses much easier.

  • Hat ideas from the entire company—Since I spent so much time refining and thinking about hats leading up to the event, I feel like each of them was mine in some sense. But the truth is the majority of hat ideas came from other Stack Exchange employees. It was really wonderful to see the brainstorming spreadsheet fill up with such inventive and imaginative ideas. We did need to skip dozens of fun (and funny) suggestions because of technical and social reasons, but that still left many ideas that I can't claim credit for.

Things that could have been better

  • Typos and insensitivity—It's weird: none of the controversial hats were the ones I predicted would cause problems. Not much else to say except that several co-workers talked me out of a hat (already illustrated and ready to go) that would have been way more offensive than anything we actually used.

  • Eureka!—On the one hand, I feel like we were fair and generous about who got this hat. But on the other, it probably doesn't seem like we were.

    Just before the event, I sent around an email to all the people able to give out the lightbulb with criteria and things to look for. (For instance, if you didn't have one of the hats triggered by voting, it was unlikely you'd figured out how they work.) We tracked guesses around the network and verified that discoveries were probably independent. We gave partial credit and co-credit when people worked together. But all of that scrutiny needed to remain secret in order to keep the hat triggers secret. Even if we got it 100% right (and I know of at least one person who missed the hat because we noticed the discovery too late), it would still seem arbitrary and like we played favorites.

    I'd personally be glad to ditch it next year since it just added stress for me. But I also like the idea of rewarding people for applying the scientific method to solve our puzzles. So I'm torn.

  • I wimped out on Secret hats—My initial draft included several secret hats that were downright diabolical. I got worried that they would be too tricky to guess, so I dialed down the difficulty a bit. Some hats became non-secret and others were given easier-to-guess triggers. As a result, all the secrets were (more or less) guessed in the first two days. We need trickier secret hats next year.

  • Fewer people than expected participated—Stack Exchange saw phenomenal growth in 2014. But that wasn't reflected in Winter Bash 2014 participation. Compared to last year, only about 10% more people earned hats. There are lots of possible reasons:

    • people got bored of the event
    • hats were harder on the whole to earn
    • fewer hats were available

    I suspect that the majority of people earning hats got them by doing ordinary stuff on a site rather than by setting out to earn them. If so, the reason for less-than-expected participation has more to do with the frequency of triggering actions (and fewer hats) than anything more worrying.

Thank you all, for a pleasurable and successful Winter Bash 2014. I look forward to getting excited about hats again in November.

If you have any ideas for next year please post them as answers to Winter Bash closing remarks - ideas for next year. On the other hand, if you have critiques (that aren't already raised on meta), stick 'em here.

  • 3
    Personally, I would have loved to see the diabolical secret hats! The ones this year was fun, but after everyone figured them out they got to be a bit boring. In fact, I wouldn't even be opposed to making all hats secret, although that might be going a bit overboard. – Doorknob Jan 8 '15 at 23:19
  • @Doorknob: All secret hats except some time-based ones would be fun. (Thanks for the edit. It only feels like I've been working at this for a year.) – Jon Ericson Jan 8 '15 at 23:28
  • Or actually, they don't KNOW what Winterbash is! – 54D Jan 9 '15 at 3:21
  • 2
    WHYYYYYYYY you removed the "diabolical secret hats"?? The secret hats this years where too simple. Go for the diabolical ones (btw, just out of curiosity, mind leaving just one sample in a comment?)! – SPArchaeologist-様 Jan 13 '15 at 9:15
15

I wimped out on Secret hats—My initial draft included several secret hats that were downright diabolical. I got worried that they would be too tricky to guess, so I dialed down the difficulty a bit.

This was exacerbated by Eureka; not only do people want to guess and be correct, but they want to do it in public so they get the reward. I suggest that removing Eureka (and thus the incentive to share the secrets you've learned) will help keep the hats more secret to begin with, and once you're not rewarding the guesses then who cares if you throw in a couple that are truly hard? We get a puzzle to solve, and you'll reveal it in the end anyway.

  • Concur.... I felt light downvoting every single "how do I get this secret hat question and answer". It ruins the fun and secret for everyone – psubsee2003 Jan 9 '15 at 0:13
  • 2
    People will share the secrets with or without Eureka. – RubberDuck Jan 9 '15 at 11:42
  • 1
    @RubberDuck you don't think the "cool" factor of knowing how to get a secret hat -- which is only cool if it's not widespread -- will dominate? I do, based on watching how "competitive secrets" work in other communities. Gossip is propagated; things that convey an advantage are held close, in my experience. – Monica Cellio Jan 9 '15 at 13:51
  • 1
    Personally, I like a good puzzle, but to answer your question, no. I don't think it would happen that way. People would share the secret to gloat about figuring out the secret and out of a desire to help other people who aren't quite savy enough to have found it. – RubberDuck Jan 9 '15 at 15:42
  • 2
    @RubberDuck: I agree that the secrets will be revealed inevitably on a Meta question as well as on chat. But Eureka gives you an incentive to share the secret elsewhere, as you certainly did not find out something independently if you share it on the Meta question on secret hats, for example. And this is something that the removal of Eureka will reduce. – Wrzlprmft Jan 21 '15 at 22:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .