135

Back when the idea of beta sites was young, the expectation was that beta was a fairly quick process. Even now, Area 51 sets the expectation that 90 days is a good "start thinking about graduation" checkpoint, and today I got this in email for a new private beta:

Q: How long is the beta?
A: After a few months, when it becomes abundantly clear that it's Making The Internet A Better Place, we'll slap a proper logo and design on it, and make it a full-fledged citizen of the Stack Exchange network.

This hasn't been true in a very long time; even when a site is deemed ready to graduate, after a year or two (or more), it can easily take three months or more to get from "decision" to "graduation" because of limited design resources. (Also, I'd hardly refer to the fine work our design folks do as "slapping a proper logo and design" on it.)

I'm not complaining about the timeline; sites take time to bake and we shouldn't expect things to move that quickly.

But...could we update what we say about the process to be more realistic? On just about every beta site I've been active on, I've seen "when will we graduate?" queries starting several months in, because people thought it would just be a few months. Maybe we should set better expectations.

Improve this question
12
  • 43
    Pretty much every single hard number on Area 51 is misleading, all of the values were chosen rather arbitrarily when the whole thing started, and none of them were ever changed.
    – Mad Scientist
    Sep 16, 2014 at 20:57
  • 14
    @MadScientist Maybe in 6-8 years we'll get Area 52, and all the numbers will change... ;)
    – hichris123
    Sep 16, 2014 at 21:41
  • 6
    if graduation is too hard to predict (who knows wen beta site will build 3K questions backlog), it would better be replaced with something that can be realistically scheduled. Graduation Progress Review, #1, #2, #3 etc can probably hold every 3-4 months without setting wrong expectations
    – gnat
    Sep 17, 2014 at 8:35
  • 2
    I would especially like at least a little more transparency and progress information for the time between getting eligible for graduation and waiting for the design to finish (since that is according to SE the only thing to be done once graduation is decided), as three months for this period are quite an understatement with a year being far more realistic. Maybe some monthly or at least quarterly information might already suffice, or a way to check the current status of the apparently huge graduation queue or at least updated information thereof.
    – Christian Rau
    Sep 17, 2014 at 10:50
  • 5
    In addition to people asking about when the site will graduate, there are also some people panicking about the site being shot down, when the public beta is about to reach 90 days.
    – Wrzlprmft
    Sep 17, 2014 at 21:18
  • 26
    To put the "90 days" / "a few months" in perspective: there are currently 31 sites that have been in beta for at least 1000 days.
    – Stijn
    Sep 24, 2014 at 13:12
  • 7
    Broadly speaking, this seems like a good idea. The sites that have graduated since Jan. 1, 2012 spent 17, 13, 20, 12, 6, 19, 21, 22, 42, 39, 26 and 26 months in beta. Music and Chemistry, the two sites most recently approved for graduation, spent about 40 and 30 months in beta, respectively. On average, that's almost two years per site. What specific changes are you proposing here? Copy changes to A51 pages and e-mails, and what (if anything) else?
    – Pops
    Sep 24, 2014 at 20:43
  • 1
    @Pops Are those number referring to graduation or getting approved for graduation (as is the case with Music and Chemistry)?
    – Christian Rau
    Sep 25, 2014 at 9:42
  • 2
    @ChristianRau Good question, I was hoping nobody would ask. Unfortunately, they're actual graduation dates. We don't store "approval dates" anywhere. I could potentially figure out some of the more recent ones, but it would come down to looking through team meeting transcripts by hand.
    – Pops
    Sep 25, 2014 at 16:15
  • 2
    Bounty for a week, and only 187 views... that is somewhat sad.... :(
    – rolfl
    Oct 1, 2014 at 11:51
  • 1
    @rolfl agreed. I was hoping the bounty would go to an answer that said "yes we're going to do that", instead of to my proposal for what to do. (I also realize that you had broader aspirations about expectation-setting than just the misleading info about timing.)
    – Monica Cellio
    Oct 1, 2014 at 12:43
  • @hichris123 I'd be worried about the quality of sites that come out of Area 52, given how bad the NES multicart Action 52 was.
    – Damian Yerrick
    Dec 20, 2014 at 22:00

1 Answer 1

Reset to default
76
+100

Pops wrote in a comment on this question:

Broadly speaking, this seems like a good idea. The sites that have graduated since Jan. 1, 2012 spent 17, 13, 20, 12, 6, 19, 21, 22, 42, 39, 26 and 26 months in beta. Music and Chemistry, the two sites most recently approved for graduation, spent about 40 and 30 months in beta, respectively. On average, that's almost two years per site. What specific changes are you proposing here? Copy changes to A51 pages and e-mails, and what (if anything) else?

I propose the following changes:

1. Change the Area 51 text on beta pages. Currently, once a site is in beta, its page says:

Sites remain in beta for at least 90 days to build up a critical mass of users, questions, and participation.

Forget about setting a lower bound; nobody graduates in 90 days. Instead, just say something like:

Sites remain in beta until they build up a critical mass of users, questions, and sustained participation.

(In other words, let's just be vague there.)

I don't see anything about timeframes in the Area 51 FAQ. That's good.

2. Change the text in the private-beta invitation (quoted in the question) to something more like this:

Q: How long is the beta?
A. The beta is broken into two parts, a short private beta and a longer public beta. During the private beta a core group of users like you begins to develop a body of questions and answers, starts to determine scope and tagging, and generally builds a site that can "open for business". After a few weeks, if all goes well, it moves into public beta where you'll help build a broader, sustainable community. When it becomes abundantly clear that it's Making the Internet a Better Place we'll prepare the site for graduation. Public beta could last several months to several years, but on average it's about two years. (Optional: link to, or a sentence about, why it takes a while.)

3. Stop linking to the "7 essential questions" blog post. If you can't use this instead, then at least remove or massage the one about site design. Nobody should be proposing site logos in month 1 (or even month 6) of a beta; suggesting that people ask that sets the expectation that it'll be needed soon.

I think those three changes would make a big difference in setting expectations for the length of beta.

Improve this answer
3
  • 4
    "Sites remain in beta until they build up a critical mass of users, questions, and participation" I suggest making a slight tweak -- if we want to be vague, at least we should point at what the end goal is rather than having people wonder what 'an extended period' is.
    – jmac
    Oct 10, 2014 at 2:25
  • 5
    @jmac good point; edited. I further tweaked the tweakL "...and sustained participation", because we're looking for a pattern and not just a burst of activity to drive up the question count.
    – Monica Cellio
    Oct 10, 2014 at 2:28
  • 9
    This will also cut down on people trying to game the system by getting the questions per day up. The 90 day mark motivates a lot of people to ask low quality questions and post low quality answers. This puts the focus back on sustainability.
    – jmort253
    Oct 10, 2014 at 3:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .