Today SE removed the "beta" label from 29 established sites, recognizing a need that has been plaguing many of our sites for years. Until today, under SE's rules there were two states: "graduated" if a site meets the requirements and "beta" otherwise. One of those requirements is to sustain an average of ten questions per day, which just doesn't work for some of our smaller sites. High-quality sites have thus been branded with the "beta" label for, in some cases, almost a decade, long after there were questions about those sites' quality, scope, and ability to maintain their communities.
All that changed today -- yay!
Well, it started to change. In the announcement, Catija says that this is a start and SE is still working out what the rules and thresholds ought to be. Basically, we don't know what the criteria are, but by any reasonable criteria these 29 sites qualify, so let's start with these ones. We've been wanting to break up with graduation for a while, moving from this binary system to one where milestones happen when they make sense, but nailing down the specifics is hard.
I've been one of the most vocal people about how the "beta" system is broken, so the least I can do is try to help fix it. In this question, let's focus on what the criteria should be for removing the "beta" label from a site. (We can discuss other milestones, like when to raise privilege levels or when a site gets community ads, separately.)
When a site is brand new it's clearly a beta. The community has to work out through actual use what the scope is, what requirements to apply to questions or answers, and so on. It has to build a community of people working together, on meta and on the main site, to classify, improve, and curate content. It has to figure out what it wants to be. As a community grows, as the body of Q&A grows, and as time passes, what the site is starts to firm up. Eventually you have very-old sites clamoring to advance. It would be good to have some heuristics to indicate both to a community and to SE that a site is approaching, or has reached, the point where the "beta" label should come off.
What are those heuristics? When is a site no longer beta?
In a post on Writing Meta a year and a half ago I outlined some things that I felt justified our advancement. My list was subjective (there's nary a number in the whole lot!), but here are some factors to consider:
- a sizable body of questions, with old ones still getting new answers, and a steady flow of new questions (how do we recognize this? what's "sizable"?)
- content quality (measured how?)
- engaged users who not only ask and answer but also guide new users, review, curate, and use their moderation privileges
- an active meta
- enough users at various reputation levels to support higher privilege levels
- enough active users to support moderator elections
- site longevity: seven years is too long, seven weeks is too short -- when has a site stood the test of time?
What factors should communities and SE be looking at? How do we know when a site is "baked"?