Today SE removed the "beta" label from 29 established sites, recognizing a need that has been plaguing many of our sites for years. Until today, under SE's rules there were two states: "graduated" if a site meets the requirements and "beta" otherwise. One of those requirements is to sustain an average of ten questions per day, which just doesn't work for some of our smaller sites. High-quality sites have thus been branded with the "beta" label for, in some cases, almost a decade, long after there were questions about those sites' quality, scope, and ability to maintain their communities.

All that changed today -- yay!

Well, it started to change. In the announcement, Catija says that this is a start and SE is still working out what the rules and thresholds ought to be. Basically, we don't know what the criteria are, but by any reasonable criteria these 29 sites qualify, so let's start with these ones. We've been wanting to break up with graduation for a while, moving from this binary system to one where milestones happen when they make sense, but nailing down the specifics is hard.

I've been one of the most vocal people about how the "beta" system is broken, so the least I can do is try to help fix it. In this question, let's focus on what the criteria should be for removing the "beta" label from a site. (We can discuss other milestones, like when to raise privilege levels or when a site gets community ads, separately.)

When a site is brand new it's clearly a beta. The community has to work out through actual use what the scope is, what requirements to apply to questions or answers, and so on. It has to build a community of people working together, on meta and on the main site, to classify, improve, and curate content. It has to figure out what it wants to be. As a community grows, as the body of Q&A grows, and as time passes, what the site is starts to firm up. Eventually you have very-old sites clamoring to advance. It would be good to have some heuristics to indicate both to a community and to SE that a site is approaching, or has reached, the point where the "beta" label should come off.

What are those heuristics? When is a site no longer beta?

In a post on Writing Meta a year and a half ago I outlined some things that I felt justified our advancement. My list was subjective (there's nary a number in the whole lot!), but here are some factors to consider:

  • a sizable body of questions, with old ones still getting new answers, and a steady flow of new questions (how do we recognize this? what's "sizable"?)
  • content quality (measured how?)
  • engaged users who not only ask and answer but also guide new users, review, curate, and use their moderation privileges
  • an active meta
  • enough users at various reputation levels to support higher privilege levels
  • enough active users to support moderator elections
  • site longevity: seven years is too long, seven weeks is too short -- when has a site stood the test of time?

What factors should communities and SE be looking at? How do we know when a site is "baked"?

  • 6
    Throwing out a vague idea that doesn't qualify as a good answer or suggestion yet, I like the idea of having each of those items you list contribute to some kind of overall score. Once the total score reaches a threshold, then it can be considered at least a candidate for leaving beta. This means not setting a static number that must be met in every category, but allowing it to meet (or exceed) numbers in some categories, while not meeting them in others. Commented Aug 2, 2019 at 0:43
  • 3
    @JasonBassford I like where you're going with that. I hope you (or someone else, if not you) develops it into an answer. Most of these should have some minimum levels; it doesn't matter how active your meta is if nobody's curating the content on main, for example. But high bars across the board are how we got to where we were until today. Commented Aug 2, 2019 at 0:47
  • 1
    A related question I asked recently, asking why SE specifically chose 10 questions per day as the (pretty much) sole criterion for graduation in 2015. Commented Aug 2, 2019 at 0:48
  • @MonicaCellio Yes. You could fail to meet one of the traditional cutoffs in one metric, but exceed them in others. Better than required performance in one area would offset the lack of performance in a different one. Just like how you can graduate from university despite performing poorly in a few courses. It's your grade point average that allows you to graduate. Commented Aug 2, 2019 at 0:49
  • 4
    @JasonBassford Like a report card? Must have a B average to leave beta?
    – Catija
    Commented Aug 2, 2019 at 1:25
  • 4
    And no failing grades? Commented Aug 2, 2019 at 1:31
  • 1
    Nah. Just actionable suggestions for how to improve supported by help center articles.
    – Catija
    Commented Aug 2, 2019 at 1:37
  • @sonic we're on the meta site for SE; what did "some people" get confused by? I don't see the point of that edit. Commented Aug 4, 2019 at 15:35
  • 1
    @MonicaCellio There was this answer posted by a new user who thought this question was asking for general criteria for when general software should leave beta. The edit was an attempt to curb similar answers without resorting to protection. (If such an answer hadn't been posted, I wouldn't have edited it.) Commented Aug 4, 2019 at 15:37

11 Answers 11


Avid Users

We need a better idea of how many avid users a site has.

Right now Bob who answered one question 5 years ago that hit the Hot Questions list and got 200 reputation from it before disappearing off the face of the earth is counted as an avid user.

Users who show up and write one awesome answer and then leave are great but they really aren't involved in the day to day moderation such as editing/voting/closing etc.

I think that a better definition of Avid Users would include,

  • Interacting with the site in some way commenting/asking/answering/voting/flagging/reviewing etc.

  • Within a recent amount of time say last 30 days for example.

Right now the number of Avid Users only goes up with time, this way it could go down as well if users stop showing up and participating. Letting it fluctuate with the number of people who are participating in a recent amount of time would give us all a much better idea of how the site is doing.

  • 1
    This sounds like the term that many web companies use in quarterly reports: "Monthly Active Users". That combined with reputation thresholds would seem to make sense for each individual SE site.
    – hazzey
    Commented Aug 2, 2019 at 14:35
  • 1
    I'm curious how the number of avid users impacts whether a site should leave beta. I think knowing this is interesting but how does it relate?
    – Catija
    Commented Aug 2, 2019 at 20:15
  • 3
    @Catija Why does the number of Avid Users currently show up on the scorecard on Area51? Commented Aug 2, 2019 at 20:25
  • 3
    Likely because, historically, graduation meant elections and higher rep levels! There is/was a fear that having too few users meant that these features would cease to function. With beta elections, we've found that they can do just fine with low user counts... and we can hold off on raising privileges until we have sufficient recent avid users who can participate in editing, closure and deletion of posts. :) One of the great features of exploding the "Graduation" process to smithereens is that we can only change things as we need to.
    – Catija
    Commented Aug 2, 2019 at 20:29
  • 2
    I've been really thinking a lot about the various elements of graduation and why they exist and they all make sense if you think through things... :D I don't really want to prevent a site from leaving beta because it'd also mean forcing rep levels on them, though... If they're doing great with closing/editing/moderating with the beta levels... that's enough (from my POV). All this to say, I absolutely agree this is what we need to look at when we talk about when to move the rep to the default levels, just not sure if it's right for leaving beta. Feel free to convince me, though!
    – Catija
    Commented Aug 2, 2019 at 20:32
  • 1
    @Catija My point being, it that information was useful to know in the old system, then it should be useful in the new especially updated to pull in more accurate numbers Commented Aug 2, 2019 at 20:44
  • 1
    ... But why? I want to question all 8+years of assumptions about the beta process, so "because we used to" isn't enough of a reason. If I understand correctly, Monica's trying to compile a list of metrics that should be used to determine when a site should leave beta. How does knowing this help decide that a site is ready to leave beta? What number of avid users is enough and why? I can definitely see why it's useful for looking at rep levels - help me understand how it matters for beta status?
    – Catija
    Commented Aug 2, 2019 at 20:50
  • I'm going to propose an alternative definition for "avid user." The question isn't, "how many people interact with the site in X days?" The question is, "how many people interact with the site on any roughly regular cadence, or have continued to interact with the site over an extended period of time?" This is a little more nebulous, but it's not accurately modeled by saying "30 days" or so. What I'm trying to take a cut at here is engagement. Does someone find the site valuable enough to keep coming back?
    – user206222
    Commented Aug 3, 2019 at 6:50
  • 2
    I think there's a sense of avidity that means "there's a community here and the site isn't a ghost town". It interacts with site activity, so maybe it doesn't need to be a separate measure, but it feels somewhat distinct too. Worth thinking about. Commented Aug 4, 2019 at 15:20
  • General avid users is an easy measure of two separate (but related) things. 1. The site has enough users who keep coming back to do things (for whatever reason). 2. The site has enough users posting new questions/answers to keep the users of #1 happy to keep coming back. Either of those things is hard to measure individually by other methods. Number of questions or answers by themselves don't mean that they are good enough to keep people coming back for more, and number of users doesn't mean that they are coming back for reasons other than habitually typing in the address.
    – hazzey
    Commented Aug 5, 2019 at 2:34
  • I get the distinct impression that no one wants to change how an avid user is defined, but 200 rep earned in two days or even a week of activity and then silence for five years, does not define an "avid" user in anyone's book, let alone mine. Commented Aug 6, 2019 at 7:27

This question is basically asking something much more abstract: what's the purpose of beta? That is, why do we need this beta state to live between Area 51 idea and a "non-beta" site?

This question matters even more now, since the "beta" designation is now decoupled from most of the things we presently recognize as being consequences of being in beta: appointed moderators, lower rep thresholds, etc.

As I understood it, the thinking was that beta was about doing the following:

  • Firmly establishing the scope and domain of the site. What is on/off topic, what do good/poor questions look like, how much subjectivity do we allow, etc. This should ultimately result in a relatively clear set of enforceable rules, typically on the site's Meta.

  • Proving that the community is functional and growing. This includes having enough questions being asked that new answerers have a place, a body of experts that has room to grow, maintenance tasks being performed accurately, etc.

The recent release of beta sites seems to be designating "beta" is being primarily or solely about the first one. Most of those sites have clearly figured out what they're about, but many of them function at a lower level in various ways than most previous sites functioned at upon graduation.

So, we could essentially decide that beta is mainly about scope, with the matter of the functioning of the site having a much lower bar.

Scope definition is easy to determine. The site needs to have had enough questions, both open and closed, to allow for a large enough sample size to know what the site is about. There must be meta discussions about scope, with sufficient participation to have some idea about what the general consensus is.

That brings us to the question of how we define "functional". There are a number of criteria which could be objectively measured:

  1. User growth is an important aspect to any site. A site which isn't growing its base of active users is set for slow decline.

    So we should determine the number of users per-month who transition into being active users of the site, relative to the number of total active users. "Active user" of course is something that would have to be nailed down, but visiting the site regularly (every few days) and making some number of posts within that monthly timespan would be a good start.

  2. One characteristic of smaller sites (or just overly narrow topics) is that there are a small cabal of answers who are involved in almost every question. This would be fine for a single tag, but on a whole site, it is indicative of an insular, isolated community of a small number of experts feeding on a drip-feed of questions. It also makes it difficult for new experts to appear, since if all the questions are handled by the existing experts, there's nothing left to do.

    To detect this, we could find take the reputation of the answerer for each answer, then compute both the average for a week and the median. Note that this counts on the basis of the answer, so one high rep users answering 3 questions would count their rep 3 times. The lower the median is to the average, the less likely it is that a small cabal of users are dominating the site.

  3. Community question closure. Elected moderators are exception handlers; in a well-functioning site, questions get closed reasonably quickly by regular users performing close votes. If moderators have to spend time patrolling the site just to make sure questions are being closed expeditiously, that's a sign that the site lacks sufficient engaged users willing to perform "curation" tasks.

    One could measure the ratio of questions closed by moderators vs. those closed by the community. The focus of the measurement should be posts that only get 1 or 2 non-moderator close votes, relative to the number of closed questions.

  4. Community consensus on closure. A well-functioning site is one where the community is pretty clear on what is an appropriate question and what is not, and they will vote accordingly. So finding a way to measure this consensus would be useful.

    One objective measurement might be looking at how many close votes age away over a particular span of time, relative to the number of closed questions. Of course, the number of votes should also be taken into account; a spurious single close vote shouldn't count as much as 3 or 4 close votes.


A while ago graduation was based on a fairly thorough manual review. Back then I once checked a bunch of graduated sites, compared them to older sites that appeared to be stuck in beta and tried to figure some patterns that could make reviewers feel that particular site is good enough to graduate.

The only thing I saw in common in these manually approved sites was that all of them seemed to have "question bank" above 3K (total questions asked excluding any that have been deleted). Upon thinking about this a bit I decided that this could make a fairly solid indication that topic covered by the site is good enough to provide an extensive, reliable knowledge base, meaning it's reasonably safe to graduate.

I would prefer that our criteria for leaving beta would somehow take this metric into account, because it appears to be sort of "verified" by past manual reviews.

In case if we would want to get sites out of beta automatically, I would probably prefer a bit stricter criteria that additionally accounts for features of the system, something like requiring above 3K questions that are older than, say, one year (better one and half, to let 365-days cleanup scripts make reasonable impact).

In this case another possible additional criteria could be some minimal activity check - if we would want to prevent automatically promoting sites that have lots of old questions but weren't active for weeks or months.

  • 2
    What do you mean by “question backlog above 3K”?
    – PolyGeo
    Commented Aug 2, 2019 at 12:22
  • @PolyGeo a number of questions asked. It is easy to assess by checking numbers in 2nd column in the list of all sites, right after site name
    – gnat
    Commented Aug 2, 2019 at 12:24
  • 4
    So just the total questions asked excluding any that have been deleted since.
    – PolyGeo
    Commented Aug 2, 2019 at 12:28
  • 1
    right @PolyGeo - total questions asked excluding any that have been deleted since. I remember being fairly surprised back then after I found that sites manually picked for graduation all follow this simple heuristics (not questions per day, not users, not traffic)
    – gnat
    Commented Aug 2, 2019 at 12:34
  • 1
    Does question status (open, duplicate, closed for another reason, locked...) factor in? Commented Aug 2, 2019 at 14:44
  • @MonicaCellio per my recollection of that past research, question status didn't count at all, there were graduated sites showing barely above 3K questions total (that is, only deleted questions didn't count). I remember being somewhat surprised by that then, but then I figured that in theory, closed and locked questions can be reopened / unlocked and it kinda clicked. Following this, I would hesitate to suggest making it stricter than worked back then.
    – gnat
    Commented Aug 2, 2019 at 14:52
  • 2
    backlog: a large number of things that you should have done before and must do now. Until I got to the comments, I thought you were proposing that the number of unanswered questions was the important metric. I see that Wiktionary suggests that some people use the word to mean an accumulation (although even there it says "especially of unfilled orders or unfinished work"), but I think that this answer would be less confusing if it instead used e.g. "question bank". Commented Aug 5, 2019 at 11:15
  • 1
    @PeterTaylor thank you, edited this in
    – gnat
    Commented Aug 5, 2019 at 11:30
  • @MonicaCellio FWIW I think I recalled when I studied these things: this was probably in times of Workplace graduation - that is, about 5-6 years ago
    – gnat
    Commented Aug 6, 2019 at 11:38


enter image description here

Looking at the objective of Stack Exchange, we are different from other forums and QA sites in the case that a community is focused on particular topic and provides quality and reliable information about that topic.

Stack Exchange is a network of 174 communities that are created and run by experts and enthusiasts like you who are passionate about a specific topic. We build libraries of high-quality questions and answers, focused on each community's area of expertise.

This indicates that the essential things for a beta site in order to be recognized as full member of Stack Exchange Network: well defined scope and have considerable quality question answer. If a beta site (no matter how many years in beta) has been suffering for these two issues, then it doesn't meet the graduation qualification. Another unique feature of Stack Exchange is "Moderation". If on-topic is a bhakti, high-quality is jñāna​, moderation is yoga​ and all three are required to attain the goal.

We have The Real Essential Questions of Every Beta which applies as to do list as soon as proposal enters in beta and the same can be applicable as check list for reviewing the health of beta. Including the points which are discussed, following is criteria according to my understanding of Stack Exchange.

1. Well defined scope

This is 1st issue listed for private beta. As public beta site grows, more user joins the site and site usually faces the issue of on/off topicness for various subjects. So, it's desired that several major issues should be discussed on child metas and most of issues should have solved with enough consensus on child meta e.g this or this. Consequently, the possible issue should be addressed and clarified with respective meta links e.g this on /help/on-topic page of help-center.

2. Well organized subjects

Tags are subjects of site. So, a beta site should have well defined tags with tag synonyms. Many a times, it is observed that users don't take the tagging issue seriously. So, the issue of improper tagging, removing unnecessary tags and that for tag synonyms should be contained by child meta. e.g. this, this Another thing is considerable subjects should have been discussed on main site e.g if a site is about religion then philosophy, scriptures, sects, history, mythology, deities all the various topics should have discussed. In other words, a site should contain all types of question answers. This will make a site complete in its subjects. 100 distinct tags with well described excerpt, all having with at least 20 questions tagged.

3. Documentation

Apart from generic help-center, a beta site should have guidelines for new users for asking and answering questions in accordance with the objective, perspective and usefulness of community. e.g. this. The child meta should contain FAQ referring scope, guidelines and established policies. e.g. this

4. Number of Users

This is the soul of community without which it can't live effectively. The effective numbers are Avid users stated on Area51. Here I suggest to have good number (say 70+) of "Established Users" (who is having 750+ reputations). Analysis of number of active users are available for moderators site analytics. 30 users active for 5 days in a week would be a mark. Another thing is how many users are actively participating on moderation activities i.e reviewing, flagging etc. Check out how many times a "reviewer badge" is awarded. I think number 20+ would be a good sign.

5. Visits and Votes

As stated by Area51, "A great site benefits people outside the community. Eventually, 90% of a site's traffic should come from search engines" I think 5000+ visits per day is a good sign. 5 questions with 50+ upvotes would be a good sign.

Note that I have linked some meta posts from the site I moderate as an example only.


Review Time

How long on average does it take for a post to go through the review queues?

  • The faster rude/spam/low quality posts get deleted the better.
  • Closing posts before they get answers helps everyone while reopening them quickly encourages fixing them.
  • Quick turnaround on suggested edits encourages the editors.
  • Helping new users by reviewing and editing their posts helps them get started.

On the other hand if posts are taking forever to get reviewed or it has to wait for mod to review them is much worse off.

Getting things reviewed in a timely matter (I would argue no more than 48 hours) is a sign of a healthy community especially if the community is able to handle the reviewing without moderator intervention.

Of course, all that requires active users with enough reputation to handle the review tasks which is also a sign of a healthy community.

  • How do you count items for which the review task expires without being completed? I don't think you can put "infinity" into the usual arithmetic mean and get a useful number out the other side.
    – Kevin
    Commented Aug 2, 2019 at 4:44
  • 3
    @Kevin On a small site that's coming out of beta the odds of having so many things to review that the queues get overwhelmed like they do on Stackoverflow is really small. Commented Aug 2, 2019 at 4:50
  • 1
    48 hours is an extremely long amount of time. If spam can't be deleted within even 4-6 hours without mod intervention, I'd argue the site isn't "passing" on any community moderation metric. Even for suggested edits that's a long time as the post is locked to new suggestions and there's weird interactions with regular edits and unapproved suggested edits. There was a table posted on meta a few years ago that showed average review times per site and 7-8 hours was an outlier.
    – Troyen
    Commented Aug 2, 2019 at 21:37
  • This is complicated to define clearly. One close vote puts the question in the close queue, but the remaining close votes don't necessarily come from the queue: they might come from people seeing the question on the front page or in a tag. On the other hand, reopen votes following an edit probably (I have not calculated any stats) come mainly from review. Should the different queues be weighted? If so, how? Commented Aug 5, 2019 at 11:30

In broad terms, I believe the question of when a site comes out of beta should be looked at from a holistic perspective. I don't think it should have to be entirely successful in every category, or meet every specific criteria as has been the case historically, but that it should, synergistically, accomplish an overall goal. Some sites may be stronger in some areas, and weaker in others, than other sites. This should be part of their overall nature—and weaknesses shouldn't be considered in isolation.

In thinking about this, I find it humorous that we have used the word graduate, because I think of graduating from university after having achieved a particular grade point average across all courses that have been taken. In university you can get a poor grade in one course, yet still graduate by making up for that poor score by doing better (overall) in other courses. In short a score below a cut-off value in one area can be overcome by scores above the same cut-off value in other areas.

Here is my answer in terms of a broad approach:

  • Each individual category that defines part of a site's health (or success) should contribute to an overall health score (or report card or transcript, if you will).
  • Once the site has reached a certain overall health score, it should be eligible to graduate.
  • Depending on how this is modelled, it could either graduate as soon as the overall score is reached or it could need to maintain that overall score for a certain period of time.

There are some other considerations:

  • The existing criteria for failure should still apply. In other words, there need to be minimums that have to be met for the site to continue even in beta form. If those aren't met, then the site has failed—just as a student would have flunked out of school.
  • It seems that there is now also at least one criterion for immediate graduation: having remained in beta for seven years. Whether this is now an established precedent isn't quite clear. It would be nice if it were clearly defined. In terms of the overall health calculation, the number of years a site has been in beta could contribute on some kind of scale—where seven years adds a value equal to what's need for graduation, but where increased longevity before that adds an increasing number to the total score.

Once the specifics of what contributes what amount to a site's total health score have been defined, then each site's current individual (categorized) scores and total score could be shown. This would be a quick way of evaluating exactly where each site is with respect to the minimum score required to stay in existence as well as the target score needed for graduation.

I offer this as a change in the general approach to the idea of graduation. If it should be accepted in theory, then specifics could start to be defined.

  • 3
    I like this idea a lot, especially because there is no exact threshold for each particular metric that separates success from failure for every site. Meeting all of the minimum requirements for a sustained period should count toward graduation. Failing at a metric for a sustained period should probably put a site on a “performance improvement plan” where it gets some help and a period of time to correct the problem before a determination is made.
    – ColleenV
    Commented Aug 5, 2019 at 11:35

It could be basic metrics, like, total questions/answers, just to see grow.

For example, for SOru, in its early stage, we can see grow of q/a in total:

SOru q/a grow

  • все сообщения => all posts.
  • вопросы => questions.
  • ответы => answers.

This metrics are not enough to say about meeting graduation criteria. But it could be used with other metrics.


My starting point would be to have a small number of clear numerical indicators. Ideally, those indicators should be stable in time so that the values do not jump all too much between adjacent days. With a sufficiently small number of indicators there is less need to calculate averages.

I would like to see something like this:

There are three numerical criteria:

  1. The number of active users (see below) must be at least 50.
  2. The number of good questions (see below) in total must be at least 3000.
  3. At least 75 percent of flags are handled within 24 hours.

If these criteria are met and both staff and moderators show green light (see below), the site graduates. (I pulled the bolded numbers out of a hat and they can be adjusted.)

These indicators are discussed below. One of the leading principles behind this thinking is that a site does not have to be fast-paced to be useful and healthy. I don't think 10 questions per day are necessary for a site to function well.

Level of activity

I think the most important measure of maturity is the number of active users. Here is one possible definition of an active user:

A user is considered active if they have completed at least 10 actions (ask, answer, vote, comment, edit, moderate) and has been seen on at least 5 days within the last 30 days.

The numbers 10, 5 and 30 can be adjusted, but I think the order of magnitude should be this. The point of this definition is that an active user is someone who keeps consistently coming back. The exact rate of contribution per user does not have to be sky high; the key is consistent contribution.

If the site is, say, at least six months old and has a good number of active users, some of them are likely to have relatively high reputation. Consistent use often leads to high reputation.

Consistent contributions require that there is something to contribute on. Therefore this requirement will also ensure a decent influx of new questions. A question that many people contribute to by voting, commenting, and answering, is worth more than a question with no attention. Thus I think no separate indicator like number of questions per day is needed.


A site should also be stable enough and should be a valuable repository of knowledge. A simple measure (hopefully) correlating with both is the number of "good pieces of knowledge", which could be defined as follows:

A question is "good" if it has an accepted answer and both the question and the answer have at least score 2. The question must be open and not deleted.

Questions that meet that criterion are likely to be helpful for people searching for information. The score threshold should not be too high. A nice answer to a narrow question on a niche topic will have a hard time attracting tens of votes, but it is just the kind of thing the network exists for — at least from my point of view.

It is possible for a site to change course later in its life, but I find a large bulk of good questions to be a decent guarantee of stability and well-defined scope.

A site doesn't have to be fast-paced to be healthy and useful. I don't think questions per day is all that useful a measure.


A necessary property of a healthy site is the lack of issues. The question is: What indicates problems?

The simplest answer seems to be unhandled flags. I propose measuring this by the median time to handle flags over some period of time. Median rather than average, so that the rare cases where letting a flag stand for days is justified do not contribute absurdly to the indicator. If half the flags are handled very quickly, things are pretty good. Perhaps the "half" could be replaced with another fraction, but not all of the flags.

Human judgement

Finally, human judgement is needed. Perhaps the indicators can be gamed. Perhaps there can be something wrong with a site that is not caught with these indicators. Perhaps the users think that a major scope decision needs to be made before going off beta. Perhaps no flags are unhandled because nobody cares to flag the bad content they see. Perhaps the community doesn't exercise their moderator privileges. Opinions from people are crucial, whether users or staff.

If, say, two staff members and local moderators agree that everything is fine, that should be enough. A meta question can be posted soliciting concerns to allow regular users to voice any issues they see.

The human opinion should be just a simple check, not requiring too much work. The moderators know where their site is going anyway, and staff should quickly spot issues that need addressing. The point is just to have a safeguard in case the algorithm goes awry for whatever reason.

This should be a simple yes/no decision, not a numerical evaluation.

  • This is a good approach -- more durable measures, less eccentric. I'd quibble with some details (e.g. I don't care if the OP accepted an answer; I care if the community validated answers through votes), but overall I like this. Commented Sep 1, 2019 at 3:16
  • @MonicaCellio Fine details like that can easily be adjusted. For example, we could require that a good question has an accepted answer of score 2 or any answer of score 5. Let me know if you want to discuss these ideas further.. Commented Sep 1, 2019 at 9:00
  • 1
    RIght, I wasn't arguing against the approach, and SE would adjust details of any suggestion anyway. I was just qualifying my upvote. I don't get to decide what the criteria should be, but I'm hoping that the community-provided input will help SE make improvements. Thanks for contributing! Commented Sep 1, 2019 at 17:28

Apart from a well defined scope I would define health as:

  • People come and ask questions and get their questions answered. This is the number of questions per day, the number of unique people asking a question, the rate of visitors that ask a question, the rate of questions that get positively scored or accepted answers.
  • People come and write answers. This is the number of answers written, the number of unique people writing answers, the rate of visitors that write an answer, the rate of answers that gets positively scored.
  • People vote on questions and answers. This may be the rate of non-zero scored questions and answers.
  • People visit and read questions and answers. This might be the total number of visits per day, but also the average number of views per question per time duration.
  • People do not see garbage or garbage is cleaned up quickly. This could be the ratio of zero or lower scored questions and answers (especially if the total number of votes per question is rather large).

The decision for a possible graduation could be based on a non-linear classifier of all these indicators. Using historical data of functioning and failed StackExchanges (maybe time binned to increase statistics a bit), one could maybe see some limits.

My impression is that problems are not the same everywhere. Some communities may have quality problems (lots of questions, which must be closed), others may not have enough experts to answer (low ratio of positively scored questions that get at least one positive answer), others may just survive but being unable to really grow to their potential (half alive).

  • 1
    There are so many StackExchanges (174 currently), it's not really clear if this fragmentation is really the best thing for the total health of them all. Maybe some would be better together with others that have overlap. But it's difficult to say really. Commented Aug 6, 2019 at 16:19

Lasting value

Most activities, including votes, happen on new posts in some days after creation. There is a roughly exponential decay, with a half-life of some days.

Beside that, there is another source of the activities, what is roughly time-independent and is typically much slower.

We could approximate the activities as the sum of an exponentially decaying, and a constant function. The second, the constant part, is the lasting value of the posts. The sum of the lasting value of the posts is the lasting value of the site.

Unfortunately, SEDE is probably not enough strong to make such model fitting tasks correctly, but anyways it is not a particularly hard problem.

My proposal is to bind the graduation to the produced lasting value of the sites: measure the lasting value of the current graduated sites as they left beta, and make a simple numerical limit, based on this information.

Side effect: publishing this information could also help a lot to motivate people to create content with higher lasting value.

  • 1
    I'm intrigued by the idea of having a metric for "lasting value" but I'm unclear on what "activities" you mean - would this just be based on votes? I'd guess on most potential betas the typical window of activity is short enough that you could simplify the calculation and just do something like "average post score after 30 days". (Or possibly "upvotes per day after the first 30 days"? which could give a hint at how useful the content is for passers-by rather than regulars.. though maybe that's not as relevant for deciding beta status)
    – Em C
    Commented Aug 4, 2019 at 2:51
  • @EmC It could be votes, views or any other. The most obvious are the votes and the views, both has its advantages and disadvantages. The exact ruleset should be probably more complex and not disclosed by the SE.
    – peterh
    Commented Aug 4, 2019 at 11:47

Moderator Elections

With the current state of Beta, over the last few years. The primary factor for closure has been community involvement sufficient to keep 3 (or so) moderators.

In the last few months we have begun having moderator elections on beta sites. I am strong supporter of recurring elections on Beta sites and further believe that the same can be used a simple measure of community involvement.

A site that has had 2 moderator elections, separated by at least 6 months, with 2 or more qualified candidates running for election is ready to move past Beta.

  • No previous election; Set previous count = 0
  • Last election was < 6 Months ago; Set previous count = 0
  • Last election did not have 2 qualified candidates: Set previous count = 0
  • 2 qualified moderators stand for election; Increase count by 1
  • We don't have mod elections on a schedule, only when mods are needed. As such, some sites have never had elections. We rarely have more than one election per year on a site and often have gaps of 2-4 years between elections.
    – Catija
    Commented Aug 5, 2019 at 17:07
  • @Catija I knew that when I posted this answer. I believe this is a simple yet effective single measure of readiness. In the old days, there was an election on graduation, now it is as needed. For the most part the busier a site, the sooner moderators replacements are needed. and the number of candidates is directly related to community involvement. An SE site can not survive without community involvement, most of the answers here are trying define how to measure that. If a site has evolved to the point I describe, it has proven it's community involvement. Commented Aug 5, 2019 at 17:25
  • But this doesn't actually do that. A site could actually game this by just having their mods step down every 6 months... and it if was something we were waiting on to happen before we kicked them out of beta, that'd mean sites could spend years in beta... which we don't want, either.
    – Catija
    Commented Aug 5, 2019 at 17:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .