Let's look at the reasons why we have a private beta stage and whether or not it does what it is supposed to.

  • It is supposed to be private to prevent random visitors who don't know where they are and what they are supposed to do on an empty site.

It would be valid if a site was located at some public domain where random type-ins could occur. Currently you need to go through the multi-step procedure to access the site, so random visitors are probably non-existent.

  • Private beta testers are supposed to create initial quality content.

As most likely there are no random users, it is basically just preventing public beta users from visiting the site, which doesn't serve any purpose in terms of quality content in my opinion. Are private beta users expected to ask better questions than public beta users 7 days later? What is the reason for such assumption?

  • Limited amount of users can ask limited amount of questions, and most likely those questions would be asked in the first day, if not hours of their visit.

Let's look at the rough numbers of questions for current Gaming private beta:

1 day - 250 questions (80% of which were asked in first hours if I remember)
2 - 170
3 - 60
4 - 14
5 - 17
6 - 25

So in the first few hours there are already plenty of questions. Stretching private beta beyond 1-2 days doesn't do much in terms of content. If you now go to a gaming site it is pretty boring, no questions, no discussions. It just sits there waiting for fresh blood.

So, to sum everything up:

I think it wouldn't hurt if the private beta got dropped altogether, or at least would last a day or two. (I also think the whole commitment stage is unnecessary red tape, but it is another story :) ).

Thanks for reading.

  • 4
    "No discussions" on the parent site is a good thing.
    – Grace Note StaffMod
    Jul 13, 2010 at 17:32
  • 5
    I would remove the statement about the commitment phase. It muddies up the focus of your question.
    – jjnguy
    Jul 13, 2010 at 17:35

4 Answers 4


There's a very strong benefit for the private beta, which is in structuring the site to prepare for the public. The people who committed are confirmed (or at least, supposed to be) supporters of the site's existence, who are vested in the success that it entails. I already wrote a lot on the Gaming Meta about why we personally suffered in terms of the quantity of content on the parent site.

But this private beta helps us prepare the site for when a huge number of people might flood in. For example, proper tagging disputes can be settled by the people who are more vested in the site structure. The same goes for allowed topics - it would be absolute chaos if we started off with everyone on the net to be able to post whatever fancy they wanted. Laying down the groundwork then gives a reference for when the general public comes onto the scene.

Yes, the people in the public beta will also contribute to building the site, which is the purpose of the beta anyway. But there are essential elements that need to be gotten through immediately on creation, some of which will be site specific. It falls on the people who are both experts on the subject and are invested in the life of the site - those of us who committed. We start the foundation of a wonderful construction, which then gets shaped by the larger community as a whole. Even incomplete, just having a functional start to what guidelines we need to work off of is much better than giving a blank slate for everyone to write on.

  • 2
    +1 - Whenever I want to answer a question, you beat me to it and post exactly what I'm thinking... get out of my head!
    – juan
    Jul 13, 2010 at 17:49
  • 2
    additionally, the private beta is a good time to identify any pro-tem moderator leadership that emerges! Jul 13, 2010 at 20:45

One very important aspect of the private beta that you're overlooking is the meta site. It's crucial to have a (relatively) small group of users to discuss topics such as subject matter boundaries, look and feel, moderators (who need to already be selected for the public beta to go smoothly!), and so on.

If you just make it wide open to thousands of screaming voices then you have no hope of accomplishing any of this in a sane, civilized fashion.

As for the SE site itself, I'd just flip your logic around; if the private beta is sufficient to produce several hundred questions that are at least decent in quality, why is it so important to rush ahead to public beta? Let the people who actually committed have their hand in shaping the site first; they're probably going to comprise a significant part of your core contributors anyway.

  • I wonder how web apps is managing without moderators... at least it's the only site on public beta for now and the team can handle it
    – juan
    Jul 13, 2010 at 20:01
  • We've been... kinda slouching in moderator nomination on Gaming... uh oh...
    – Grace Note StaffMod
    Jul 13, 2010 at 20:04
  • @Juan: I guess I wasn't really trying to say that a public beta can't survive without moderators - especially when all of the rep thresholds are reduced and everybody is a community moderator. I was more getting at the fact that once you have thousands of users, it becomes much harder to get a clear consensus on decisions like moderators or even really know any of the nominees well enough to make an informed decision. These decisions will get made, one way or another - just not in a very sane, civilized fashion. ;)
    – Aarobot
    Jul 13, 2010 at 21:09
  • I was actually agreeing with you, it wasn't sarcasm :)
    – juan
    Jul 14, 2010 at 1:42

You are looking at the beta process through a rose-colored view of these first few sites, because these first sites were hand-selected for this audience.

True, if we just launched these specific sites directly to the public, everything probably would have went fine. But we are testing the process for those sites that come after; sites that wont have such a large audience of seasoned Stack Overflow-engine users.

Even if you dismiss all possible arguments about the ability of the "committed" users to start off the sites, there's one thing you can't deny: When starting off any site with an arbitrarily smaller audience, it is much easier to fix if something goes horribly wrong. Whether it's a technical problem or just a fundamental mis-start in the direction the community takes off... it is much easier to correct when there aren't 10's of thousands of users posting hundreds of messages an hour.

Everything about launching a new site is much simpler and safer with a smaller audience. What's the rush?


Well, we have had plenty of experience with private betas and more than enough data. Despite many changes in the site lifecycle recently, the previous answers still apply. In addition, we've found that some topics have real difficulty getting off to a good start and are closed in the private beta. As painful as those closures are, it's far more difficult to close sites that have been in public beta for a while.

We also close sites that don't have enough activity in private beta. There's a strong relationship between the number of questions a site gets on its first day and the number it gets in the next 90 days:

Predictive power of the first day

For the statistically minded, the r2 for the model was 0.18. The longer the period we use as input into the model, the better the fit. If we compare question rate for the first month to the following 3 months, the r2 goes up to 0.69:

Predictive power of the first month

The complete data I used can be found in a Google spreadsheet. I've included tabs that use the first month, fortnight, week and day to predict the following 90 days. As you might expect, the longer the initial time period, the more accurate the prediction of the following time period. (Note: the initial and following ranges don't overlap, so that doesn't explain the improved prediction.)

We've more or less settled on two week private betas. One week doesn't seem to be quite enough time for a site to demonstrate it's ready for its public debut and a month seems excessive for most sites.

  • Thanks for the graphs. Some more information could be gleaned by highlighting in red those data points where the site was closed (I'm not nagging, just thinking aloud, will d/l the spreadsheet to see for myself). Aug 15, 2015 at 19:21
  • 1
    @DeerHunter: Unfortunately it's difficult to extract the data from closed sites. It is vital information and should be included in the analysis, but I couldn't include it this time. Aug 15, 2015 at 20:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .