Area 51 Commit Percent

Question

How is the commit percent decided for Area 51? I know it is based on rep, but how exactly does it work?

Answer 1

My answer here:

In the first round, we only want to launch sites that will DEFINITELY succeed. So we require (1) lots of users, and (2) lots of users with high rep, because we know that those users contribute a lot and know how the system works. The formula right now is weighted heavily towards those people.

Once we have more data from some successful sites, we'll start to loosen it up a bit. Right now we really don't know if 200 committers with 1 reputation means 200 users on day 1, or 20 users on day 1. We're going to be fairly cautious with the first sites and gather as much data as we can with them.

With that caveat, the formula we use right now works like this: We give each user a "score" based on how likely we think they are to contribute to the site. It's a bit kludgey right now because we don't have a whole lot of data. The one piece of data we have that tells us a lot and is hard to game is a user's reputation on the existing sites.

• If you have a lot of reputation, you're much more likely to actively use the site, because you've shown that you actively use similar sites

• If you have a significant amount of reputation across multiple sites, you're even more likely to actively use the site, because you've shown that you actively use many such sites

• On the other hand, if you're some random person off the internet with no reputation, you're very hard to quantify but there's a good chance that you won't contribute very much

Here's the formula we have right now. It's almost certainly wrong and we'll be tweaking it as we go:

PrelimScore = SUM(Reputation >= 200 ? 0.233 * ln(Reputation-101) - 0.75 : 0) + 1.5
UserScore = PrelimScore * 0.9 ^ (DAYS / 180)

where SUM is over each site a user has an account on, and DAYS is the amount of days since they first committed to the proposal, or visited the proposal after committing.

To get a feel for it:

• A user with no reputation gets a score of 1.5
• A user with 200 reputation on 1 site gets a score of 1.82
• A user with 200 reputation on 3 sites gets a score of 2.46
• A user with 10000 reputation on 1 site gets a score of 2.89
• A user with 10000 reputation on 3 sites gets a score of 5.68

The way to read this is that we consider a user with 10000 reputation to be almost twice as likely to contribute as a user with no reputation, and a user who doesn't revisit the proposal in a year is 19% less likely to contribute than if they revisited.

Note that you cannot see your own commitment score (but you can calculate it on your own), neither can you see the exact value of the total commitment score (you can only see the percentage, which is divided by 5 and rounded down).

UPDATE

We've tweaked the formula a few times since this was originally posted, based on the data from the first site betas. I've corrected the formula above to be the very latest. In general, we've tweaked the value of high-rep users downwards since they don't seem to be that much more likely to contribute, and given brand new users a bit of a boost as well.

The final commitment percentage of a proposal is equal to the MINIMUM of these three numbers:

1. Total Commitment Score (above) / 500
2. Total # of committers / 200
3. Total # of committers with 200+ rep on a single site / 100

Put another way, every proposal that launches must satisfy these three criteria:

1. A total Commitment Score of 500,
2. 200 committers
3. 100 committers with at least 200 rep on a single site

Here's a confusing graphical way to see how much any one committer contributes towards each of those three criteria:

UPDATE #2

As of April 5, 2011, we've added a decay factor to commitment votes on Area 51. This applies only to the Commitment Score portion. Basically, the older a vote is, the more it decays. This is very gradual: something like 10% over 6 months. If a user is very committed, they can "renew" their vote by visiting the proposal while logged in.

Answer 2

It's not (just) the total reps.

• The Gaming site has total rep of 307,812 and 18% commitment percent, while
• The Apocalyptic Defense site has total rep of 354,402 but only 11% commitment percent.

Probably the percentage is a mixture of people count (41 for Gaming, 26 for Apocalyptic Defense) and total rep.

---

Edit: Based on @David's answer, I've calculated the total user score for:

• Wordpress Answers (2%) = 45.265422137288866,
• Apocalyptic Defense (14%) = 316.8734350894245,
• Gaming (23%) = 428.1480785888331

so the score of 100% is

• 1810.61 < x < 3017.69
• 2185.33 < x < 2347.21
• 1821.91 < x < 1902.88

Hmm the ranges do not intersect. But it seems 2,000 is the answer.

(The scores per site of each user are extracted with Firebug with

$('#committer-list .sites img').map(function(x,y){
   return Math.pow(y.title.match(/[\d,]+$/g)[0].replace(/\D/g,'')-101,0.45);
})

on each page. The lists are summed, divided by 10, and the total number of committers are added to it.)

Answer 3

In the first round, we only want to launch sites that will DEFINITELY succeed. So we require (1) lots of users, and (2) lots of users with high rep, because we know that those users contribute a lot and know how the system works. The formula right now is weighted heavily towards those people.

Once we have more data from some successful sites, we'll start to loosen it up a bit. Right now we really don't know if 200 committers with 1 reputation means 200 users on day 1, or 20 users on day 1. We're going to be fairly cautious with the first sites and gather as much data as we can with them.

The WordPress Answers proposal is one exception. A lot of people came over from the wp-hackers list on WordPress.org when I announced it, and as such, only have 51 rep. The comments show great activity potential, but the rep shows almost none. This is, of course, the ultimate goal, to bring in outsiders.

Answer 4

Same here for the GIS website. This kinda dissapoints me, but I'm sure the GIS website will be created at some point. Or not?

Almost all users who followed it, already commited, so I'm not sure if this will grow higher.

Answer 5

The following Python 3.x script can be used to compute the commitment percent based on the new formula. The scores of the 3 criterion will be emitted in the end, showing what kind of users is needed to improve the percentage.

From a few proposals I have tested on, it turns out the strictest criteria is "100 committers with at least 200 rep on a single site".

The numbers of that criteria may be inflated because of Is reputation from Area 51 counted towards the commitment score?.

import lxml.html
import urllib.request
import urllib.error
import re
import math
import time

def integerize(s):
    return int(re.sub('\D', '', s))

def collect_reps_for_user(ud):
    for site in ud.find_class('sites')[0].iterfind('a'):
        site_title = site.find('img').get('title')
        if not site_title.startswith('Area 51'):
            reputation = integerize(re.search('[\d,]+$', site_title).group())
            yield reputation


def collect_user_data(html):
    total_score = 0
    significant_committers = 0

    for committer in html.find_class('committer-info'):
        ud = committer.find_class('user-details')[0]

        user_link = ud.find('a')
        name = user_link.text
        user_id = int(re.match('/users/(\d+)', user_link.get('href')).group(1))
        print('<-> User: #{0} ({1})'.format(user_id, name), end='')

        rep_list = collect_reps_for_user(ud)
        user_score = 1.5 + 0.233 * sum(math.log(rep-101) for rep in rep_list)
        total_score += user_score
        if user_score > 1.5:
            significant_committers += 1

        print(' ... score = {0}'.format(user_score))

    return (total_score, significant_committers)




print("<?> Proposal ID: ", end='')
proposal_id = input()
url = 'http://area51.stackexchange.com/proposals/' + proposal_id

print("<:> Loading [{0}], please wait...".format(url))
try:
    with urllib.request.urlopen(url) as resp:
        doc = lxml.html.parse(resp)
        html = doc.getroot()

        phase = html.find_class('phase-name')[0].text
        proposal_title = html.find_class('proposal-hyperlink')[0].text

        print("<-> Found proposal #{0} ({1}).".format(proposal_id, proposal_title))
        print("<-> Phase: ", phase)

        if phase == 'Definition':
            print("<!> The proposal is still in Definition phase.")
        elif phase != 'Commitment':
            print("<!> The Commitment phase for this proposal has already ended.")
        else:
            committers = integerize(html.find_class('vote-count-post')[0].text)
            print("<-> Number of users committed: {0}".format(committers))

            page_links = html.find_class('page-numbers')
            pages = integerize(page_links[-2].text) if len(page_links) >= 2 else 1
            print("<-> Totally {0} pages".format(pages))

            (total_score, significant_committers) = collect_user_data(html)

            base_url = url + '?page='
            for page in range(2, pages+1):
                url = base_url + str(page)
                print("<:> Loading [{0}], please wait...".format(url))
                time.sleep(1)
                with urllib.request.urlopen(url) as resp2:
                    doc2 = lxml.html.parse(resp2)
                    html2 = doc2.getroot()
                    (extra_total_score, extra_significant_committers) = collect_user_data(html2)
                    total_score += extra_total_score
                    significant_committers += extra_significant_committers

            print("------------------------------------------")
            print("Committers:   {0:3}% [{1}]".format(min(committers//2, 100), committers))
            print("Significants: {0:3}% [{1}]".format(min(significant_committers, 100), significant_committers))
            print("User scores:  {0:3}% [{1}]".format(min(int(total_score/5), 100), total_score))


except urllib.error.HTTPError as exc:
    print("<!> Encountered '{0}' while loading [{1}]".format(exc, url))