What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

How is the commit percent decided for Area 51? I know it is based on rep, but how exactly does it work?

share|improve this question
28  
Please add a link to this post in the Area51 FAQ. It's very confusing why some proposals with several hundred committers are behind others with 100 or 150, and the FAQ doesn't explain what you need to accomplish to activate a site. –  Jason S Apr 10 '11 at 1:28
add comment

5 Answers 5

up vote 134 down vote accepted

My answer here:

In the first round, we only want to launch sites that will DEFINITELY succeed. So we require (1) lots of users, and (2) lots of users with high rep, because we know that those users contribute a lot and know how the system works. The formula right now is weighted heavily towards those people.

Once we have more data from some successful sites, we'll start to loosen it up a bit. Right now we really don't know if 200 committers with 1 reputation means 200 users on day 1, or 20 users on day 1. We're going to be fairly cautious with the first sites and gather as much data as we can with them.

With that caveat, the formula we use right now works like this: We give each user a "score" based on how likely we think they are to contribute to the site. It's a bit kludgey right now because we don't have a whole lot of data. The one piece of data we have that tells us a lot and is hard to game is a user's reputation on the existing sites.

  • If you have a lot of reputation, you're much more likely to actively use the site, because you've shown that you actively use similar sites

  • If you have a significant amount of reputation across multiple sites, you're even more likely to actively use the site, because you've shown that you actively use many such sites

  • On the other hand, if you're some random person off the internet with no reputation, you're very hard to quantify but there's a good chance that you won't contribute very much

Here's the formula we have right now. It's almost certainly wrong and we'll be tweaking it as we go:

    UserScore = SUM(Reputation >= 200 ? 0.233 * ln(Reputation-101) - 0.75 : 0) + 1.5

where SUM is over each site a user has an account on.

To get a feel for it:

  • A user with no reputation gets a score of 1.5
  • A user with 200 reputation on 1 site gets a score of 1.82
  • A user with 200 reputation on 3 sites gets a score of 2.46
  • A user with 10000 reputation on 1 site gets a score of 2.89
  • A user with 10000 reputation on 3 sites gets a score of 5.68

The way to read this is that we consider a user with 10000 reputation to be almost 3x as likely to contribute as a user with no reputation.

UPDATE

We've tweaked the formula a few times since this was originally posted, based on the data from the first site betas. I've corrected the formula above to be the very latest. In general, we've tweaked the value of high-rep users downwards since they don't seem to be that much more likely to contribute, and given brand new users a bit of a boost as well.

The final commitment percentage of a proposal is equal to the MINIMUM of these three numbers:

  1. Total Commitment Score (above) / 500
  2. Total # of committers / 200
  3. Total # of committers with 200+ rep on a single site / 100

Put another way, every proposal that launches must satisfy these three criteria:

  1. A total Commitment Score of 500,
  2. 200 committers
  3. 100 committers with at least 200 rep on a single site

Here's a confusing graphical way to see how much any one committer contributes towards each of those three criteria:

Confusing graph of the badp brand.

UPDATE #2

We've added a decay factor to commitment votes on Area 51. This applies only to the Commitment Score portion. Basically, the older a vote is, the more it decays. This is very gradual: something like 10% over 6 months. If a user is very committed, they can "renew" their vote by visiting the proposal while logged in.

share|improve this answer
8  
And how much of those UserScore points does a site need to reach beta? –  Tadeusz A. Kadłubowski Jun 15 '10 at 16:36
13  
This is not a fixed number. We will generally create the sites with the highest commitment first, depending on our capacity to launch sites, which we'll decide as we go along. Eventually we hope to get to a steady state where we have a good empirical idea how many points are needed to make a site successful. –  Joel Spolsky Jun 15 '10 at 17:05
5  
@Pollyanna right now we count all associated sites except Area 51 –  David Fullerton Jun 15 '10 at 19:49
8  
So, the Blender 3d modeling site, which is likely to have users with an average rep below 100 - will require roughly a gazillion users to be confirmed? :) And most sites will be like that - only the technical sites are likely to have a high percentage of pre-existing SO users. Even the Persian IT site, despite being one of the first confirmed, is only at 3%. Seems a little too under-weighted... Wups - just saw the note that Area51 doesn't count - that means the Blender average rep is closer to zero... –  John C Jun 15 '10 at 20:26
2  
@John - I expect the first few through the process are going to run a very rough gauntlet. After awhile the rules will be slackened, and additional ways to get proposals into beta status without high rep users will be explored. It's hard to figure out if a site will succeed, though, and it's tough to define success. After nearly a year of stackexchange 1.0, very, very few sites could actually be considered successful. I suspect there are external reasons for that, but all this lends credence to the idea that it should be hard to start a site. –  Adam Davis Jun 15 '10 at 21:06
11  
Well, the GIS site sure looks like a sure winner to me, and it's only at 15% with a large number. On the other hand, there's 'Apocalyptic Defense'. The investors, I imagine, will be tickled pink with the press release when that goes beta first. This policy doesn't make it difficult to launch a site without complete overlap with the existing trilogy, it makes it impossible. How to piss people off and send them to some other platform in one easy lesson. –  Rosinante Jun 15 '10 at 21:12
3  
This is great info. Thanks David! –  squillman Jun 15 '10 at 21:16
7  
@waiwai933 - the "?" is shorthand for if (Reputation >= 200) then score = SUM(Reputation-101)^0.45 else score = 0 –  ChrisF Jun 15 '10 at 21:22
3  
@Pollyanna, thanks for the info. Although I understand the concept of setting the bar high and then lowering it - I am worried that it could create a self-fulfilling prophecy that will cause most sites to fail. If a lot of (non-SO) users commit to a site and only see it at 1%, won't that discourage more people from committing? –  John C Jun 15 '10 at 22:32
4  
If you don't allow 200 users with a reputation of 1 to open up a site how will you learn if low-rep users committing to a site are effective or not? –  Mark Elliot Jun 21 '10 at 17:27
3  
I'm sorry but I'm having a hard time following this logic: you don't know what will happen when no rep users make a site, so you'll play it safe by creating sites that high rep users want first? When, then, will you ever know how low rep sites will fair? It seems to me you'd want to allow low rep user sites at first to see what happens. It should be tested in the early stages to see how many people are actually active. –  Bob Jun 25 '10 at 16:44
4  
@Bob, they're not really experimenting to see what might work - they want to make sure a site will work. I can only presume that their revenue model or investors will wreak hideous punishment upon them, should a site fail to attract huge numbers of high-quality experts. –  John C Jun 25 '10 at 17:38
2  
Decay is a good idea in principle to weed out inactives. But I don't like that you need to recommit manually. It should be enough to visit the proposal on area 51. Few people know about the decay. And visiting the proposal on Area51 shows that the user is still interested in it. –  CodesInChaos Jun 10 '11 at 10:51
2  
Am I correct in inferring from this that 10000 on three different sites is the same as 30000 on a single site? –  corsiKa Aug 21 '11 at 22:33
2  
When you say, "A user with no reputation gets a score of 1.5" - 1.5 what? It doesn't appear to be percent - at least, I've seen three low-rep users join my proposal, and it went from 68% to... 68%. –  John C Sep 19 '11 at 19:13
show 15 more comments

It's not (just) the total reps.

  • The Gaming site has total rep of 307,812 and 18% commitment percent, while
  • The Apocalyptic Defense site has total rep of 354,402 but only 11% commitment percent.

Probably the percentage is a mixture of people count (41 for Gaming, 26 for Apocalyptic Defense) and total rep.


Edit: Based on @David's answer, I've calculated the total user score for:

  • Wordpress Answers (2%) = 45.265422137288866,
  • Apocalyptic Defense (14%) = 316.8734350894245,
  • Gaming (23%) = 428.1480785888331

so the score of 100% is

  • 1810.61 < x < 3017.69
  • 2185.33 < x < 2347.21
  • 1821.91 < x < 1902.88

Hmm the ranges do not intersect. But it seems 2,000 is the answer.

(The scores per site of each user are extracted with Firebug with

$('#committer-list .sites img').map(function(x,y){
   return Math.pow(y.title.match(/[\d,]+$/g)[0].replace(/\D/g,'')-101,0.45);
})

on each page. The lists are summed, divided by 10, and the total number of committers are added to it.)

share|improve this answer
    
Referrals are not double counted. See David's statement here –  Grace Note Jun 15 '10 at 13:38
1  
@ccomet: Thanks. That subtracts 4,175 reps from that count, and strengthen my argument more. –  KennyTM Jun 15 '10 at 14:05
2  
Since then, something ... apocalyptic ... happened to Apocalyptic Defefense. –  Andrew Grimm Dec 18 '10 at 3:16
add comment

In the first round, we only want to launch sites that will DEFINITELY succeed. So we require (1) lots of users, and (2) lots of users with high rep, because we know that those users contribute a lot and know how the system works. The formula right now is weighted heavily towards those people.

Once we have more data from some successful sites, we'll start to loosen it up a bit. Right now we really don't know if 200 committers with 1 reputation means 200 users on day 1, or 20 users on day 1. We're going to be fairly cautious with the first sites and gather as much data as we can with them.

The WordPress Answers proposal is one exception. A lot of people came over from the wp-hackers list on WordPress.org when I announced it, and as such, only have 51 rep. The comments show great activity potential, but the rep shows almost none. This is, of course, the ultimate goal, to bring in outsiders.

share|improve this answer
    
Right now, it's 24% complete. 109 committed. –  Arlen Beiler Jun 25 '10 at 16:12
add comment

Same here for the GIS website. This kinda dissapoints me, but I'm sure the GIS website will be created at some point. Or not?

Almost all users who followed it, already commited, so I'm not sure if this will grow higher.

share|improve this answer
5  
After spending community cred convincing others to join us here and get excited about it, I'll be more than a little disappointed if it fails to launch. If I'd known the intentions/parameters in advance, I would have waited to commit rather than jumping into the process and risking a "once bitten" community that will never again commit. –  JasonBirch Jun 18 '10 at 6:12
    
Although many GIS folks don't use SO for more than a reference, we are an extremely technical, generalist, adaptive bunch. I'm worried that SO is over-emphasizing their specialness, just as we do on occasion. –  JasonBirch Jun 18 '10 at 6:12
3  
I agree. I am concerned that the large number of potential GIS-overflow users who signed up won't bother a second time if a "round-two" proposal is required. Despite the fact that there are, at time of writing, an equal number of committers to GIS and the currently leading "web apps" site, it looks like the rate at which people are committing is plateauing. –  fmark Jun 20 '10 at 23:08
2  
I can understand the need for people who have shown that they will be active but isn't the aim to get experts talking and support others on the back of that? By giving SO users so much bias surely you are only encouraging hobbyists in non-programming areas. It seems crazy to me that GIS have over 400 committed and yet is still so far behind in progress. –  JamesRyan Jul 1 '10 at 8:53
1  
@EK01: That's one thing that's bothering me. There are areas where Trilogy residents might want answers, but we might not get experts. Software Law is one of them. One of our target experts is likely to not have Trilogy rep, while the curious onlooker might have thousands. –  David Thornley Jul 2 '10 at 19:23
1  
I echo Jason Birch's comments. I've been lobbying a lot of communities who know nothing about SO to come and commit. Some individuals do so, and then there's nothing for them to do except perhaps check the commit/percent counter every day or so, and see that there is still nothing to do. At present we have almost 460 committers in this limbo holding pattern. I think we're going to lose a significant number due to non-action and that it will be hard work to gain trust/interest again. –  matt wilkie Jul 9 '10 at 18:16
add comment

The following Python 3.x script can be used to compute the commitment percent based on the new formula. The scores of the 3 criterion will be emitted in the end, showing what kind of users is needed to improve the percentage.

From a few proposals I have tested on, it turns out the strictest criteria is "100 committers with at least 200 rep on a single site".

The numbers of that criteria may be inflated because of Is reputation from Area 51 counted towards the commitment score?.

import lxml.html
import urllib.request
import urllib.error
import re
import math
import time

def integerize(s):
    return int(re.sub('\D', '', s))

def collect_reps_for_user(ud):
    for site in ud.find_class('sites')[0].iterfind('a'):
        site_title = site.find('img').get('title')
        if not site_title.startswith('Area 51'):
            reputation = integerize(re.search('[\d,]+$', site_title).group())
            yield reputation


def collect_user_data(html):
    total_score = 0
    significant_committers = 0

    for committer in html.find_class('committer-info'):
        ud = committer.find_class('user-details')[0]

        user_link = ud.find('a')
        name = user_link.text
        user_id = int(re.match('/users/(\d+)', user_link.get('href')).group(1))
        print('<-> User: #{0} ({1})'.format(user_id, name), end='')

        rep_list = collect_reps_for_user(ud)
        user_score = 1.5 + 0.233 * sum(math.log(rep-101) for rep in rep_list)
        total_score += user_score
        if user_score > 1.5:
            significant_committers += 1

        print(' ... score = {0}'.format(user_score))

    return (total_score, significant_committers)




print("<?> Proposal ID: ", end='')
proposal_id = input()
url = 'http://area51.stackexchange.com/proposals/' + proposal_id

print("<:> Loading [{0}], please wait...".format(url))
try:
    with urllib.request.urlopen(url) as resp:
        doc = lxml.html.parse(resp)
        html = doc.getroot()

        phase = html.find_class('phase-name')[0].text
        proposal_title = html.find_class('proposal-hyperlink')[0].text

        print("<-> Found proposal #{0} ({1}).".format(proposal_id, proposal_title))
        print("<-> Phase: ", phase)

        if phase == 'Definition':
            print("<!> The proposal is still in Definition phase.")
        elif phase != 'Commitment':
            print("<!> The Commitment phase for this proposal has already ended.")
        else:
            committers = integerize(html.find_class('vote-count-post')[0].text)
            print("<-> Number of users committed: {0}".format(committers))

            page_links = html.find_class('page-numbers')
            pages = integerize(page_links[-2].text) if len(page_links) >= 2 else 1
            print("<-> Totally {0} pages".format(pages))

            (total_score, significant_committers) = collect_user_data(html)

            base_url = url + '?page='
            for page in range(2, pages+1):
                url = base_url + str(page)
                print("<:> Loading [{0}], please wait...".format(url))
                time.sleep(1)
                with urllib.request.urlopen(url) as resp2:
                    doc2 = lxml.html.parse(resp2)
                    html2 = doc2.getroot()
                    (extra_total_score, extra_significant_committers) = collect_user_data(html2)
                    total_score += extra_total_score
                    significant_committers += extra_significant_committers

            print("------------------------------------------")
            print("Committers:   {0:3}% [{1}]".format(min(committers//2, 100), committers))
            print("Significants: {0:3}% [{1}]".format(min(significant_committers, 100), significant_committers))
            print("User scores:  {0:3}% [{1}]".format(min(int(total_score/5), 100), total_score))


except urllib.error.HTTPError as exc:
    print("<!> Encountered '{0}' while loading [{1}]".format(exc, url))
share|improve this answer
    
Which version of python is this written for? I get loads of errors using 2.5.2 –  Bobby Jack Feb 25 '11 at 10:37
2  
@Bobby: "The following Python 3.x script...." in the 1st line –  KennyTM Feb 25 '11 at 13:32
1  
Sorry Kenny - wearing the stupid hat today! ;) –  Bobby Jack Feb 25 '11 at 23:14
    
I love it! I was thinking of writing one for Ruby to help me better learn Ruby. :P –  Melanie Apr 20 '11 at 23:12
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .