Mafia Wars: the Area 51 commitment phase

Question

We're getting closer to launching the commitment phase on Area 51, but before we do we thought we'd gather some feedback on how it's going to work.

Joel describes the goals of the commitment phase in this question:

So, we created a commitment phase... a phase where we gather a list of people for two purposes:

1. So that we can email them all when the site opens, bringing in a critical mass of people all at the same time.
2. So that we can gauge whether there are enough people interested in Q&A on a given topic.

Since the primary goal of the commitment phase is to get people to sign up, it makes sense to reward people for getting others to sign up.

We imagine it working roughly like Amazon affiliate links (or Mafia Wars, as Jeff likes to refer to it):

• You email anyone you think might be interested a special link which includes a token (available now in the "Share It" popup)
• If they follow the link, they get a cookie with your referrer token in it
• If they commit to the proposal, you get +5 reputation (tentative)
• If they follow through on their commitment and participate in the site, you get +25 reputation (also tentative)
• On the commitment page, you can see your referrals, and see how many people others have referred (a list of top referrers, perhaps?)
• New badges will be available for referring your first user, 10 users, 25 users, etc.

Questions? Concerns? Love it? Hate it? Already figuring out how to game it?

Edit: We should also credit this question for the initial idea of rewarding referrals

Edit: Based on feedback, we're now proposing giving a small amount of rep for the commitment, and a big chunk of rep if they actually follow through. Also keep in mind that we won't just be looking at number of committers in order to get to Beta, we'll be looking at how those users have contributed to existing sites.

Answer 1

One caution I would raise is that there needs to be sufficient friction of some kind to keep people from committing just because someone told them to.

Commitment is supposed to be a reasonable approximation of how many people will use the site when it opens. Yet, petitioners can wander the mall and gather thousands of signatures on any nonsensical issue because it is often easier to sign than to muster up the mental effort to ignore them. Zero friction = high compliance.

Answer 2

reward people for getting others to sign up.

What can you give me that would beat seeing a site I want to participate on go live with a healthy group of contributors? Are you looking to build communities, or some shady MLM...?

If you want to track referrals for fun or internal use, then maybe throw a badge or two at the referrers. But don't make too big a deal of it. This seems like exactly the sort of scenario where you really, really don't want to screw with the motivating factors.

Answer 3

I agree that we want the system to encourage getting the right experts and enthusiasts to commit, but I think too much of a rep reward for getting just any individual to commit may backfire:

Even if we think we can control for outright sock-puppeting, you're still going to get a large number of commitments from friends and family of real 'users':

Dear loved ones with unique email addresses,

Please click this link and commit to this proposal, so my friends and I can start the world's best belly-button-lint site. It takes 30 seconds and is free!

While I think it's crucial to garner support and commitment from users outside the A51/trilogy community for a successful launch, if you don't have a core group of users that have already demonstrated some commitment to sites like these, you may wind up with ghost towns.

Maybe the rep reward (of N) should be given for:

• Anyone you refer who commits and has (or subsequently earns) a rep of at least 15 on any of the sites (this allows poeple to show a little commitment to the proposal and system)
• Any five people (or whatever) you refer who commit and provide a legit(confirmed?) e-mail address, but have no history of participation.

Answer 4

(was going to put this in a comment, but you did ask about gaming it)

How well does sockpuppet recognition work if all the puppets use different emails and don't vote anywhere? Should there be some further requirement (rep?) for the referred users before you get the bonus?

Answer 5

Clarification:

My opinion on this is that rewards for referrals shouldn't be immediate - at least not tangible rewards. A badge or two is fine, but anything beyond that would just encourage referral-spam.

Instead, referrals should be thought of as investments in the site - that is, if and when the site goes live (beta), tangible rewards would be provided if the users whom you referred actually participate. Only when there's actually a site can you actually measure the value of those referrals; it seems silly to me to reward numbers when you have no idea what they mean.

You could maybe award a few badges on Area 51 for referrals alone, and transfer them to the beta site when it goes live, but I don't see the point. Joel keeps telling us that this incubation phase is all about getting the diehard fanatic users, and they already want to see the site succeed; they don't need any additional incentives in the commitment phase, because a commitment is, well, not really a commitment.

In fact, it's probably more important to discourage frivolous referrals, otherwise you risk launching a site too early - and if that happens, and the beta site dies in utero, it will become known as a flop and have almost no chance of ever succeeding, no matter how many retoolings it goes through. People should only refer those who they seriously believe will participate - and the best way to incentivize that behaviour is to pile on the rewards when those people participate.

Let's not be like the nightclub industry where promoters spam all over the place to pad their guest lists, knowing full well that 90% of the people on it won't ever show up. Instead, we should be like the open market, where people invest in an idea and their investments pay off great dividends if the idea succeeds.

---

So, here's what I'd like to see:

• A bronze Pioneer badge for either asking or answering any question during the commitment phase. New users will see the badge being awarded and go "hey, cool!" and be x% more likely to continue participating.

• A silver Committed badge for attaining X rep during the commitment period from questions and answers (not referrals). I'd peg X at maybe 50 or 100. If it's complicated to disambiguate referral rep from "real" rep then make it based on total upvotes instead.

• A token rep reward of, say, 10 or 20 points to the referrer for each new referral.

• A more significant rep reward (say 50-100 points) for each referral that becomes Committed.

• A tiered set of badges: Referral, Advocate, and Evangelist for referring 1, 5, and 25 users respectively. Badges are ultimately pretty meaningless so I don't think we need to try to "protect" this from being gamed, it's just to give people tangible goals (rep is less easily defined).

The first two are to reward the people who actually accept the invitations, which is an important element that's been neglected from the original idea. It's a trivial reward, obviously, but new users seem to like the shiny badges, and it's really important to reinforce their positive behaviour in the beginning.

I think that overall this is pretty hard to game while still being useful. If you refer 50 users who all do nothing, you get 500 rep - whoopie, who cares. But if they all actually participate then you could grab 2500 rep, and well-deserved in my opinion.

Answer 6

One thing I'm curious about is that Atwood's Angels must have thought of adding this feature to the Stack Overflow core at some point but never did. I'm wondering why such a feature wasn't added to Stack Overflow proper.

Now, it seems fairly easy to game, as mmyers points out, but the argument for having it in Area 51 also applies to Stack Overflow: SO is better the more good users it has, so it makes sense to reward people for bringing in demonstrably good users, right?

I want to know why SO doesn't have this feature, and whether that reason applies to Area 51 as well.

Answer 7

I would think nothing should happen if I'm logged in and I click my own affiliate link but I get this popup.

alt text http://img37.imageshack.us/img37/8269/affiliatepopup.jpg

If I were to unfollow after I got my affiliate link and after coming through the link refollow I don't count towards my own reward right?

Also, I'm noticing that my referrer ID isn't unique per proposal (and the links work even if I'm not following a proposal), couldn't this encourage unscrupulous individuals to blast out e-mails with links to every proposal with their single referral id tagged on the end?

Answer 8

I do not think reputation should be given for this.

You want people to follow a project because they want to see the site become a reality. Same thing with getting people to join. I have a feeling that rewarding rep for this will lead to one of two things

• Reputation becoming worthless on Area 51. Basically this means that no one cares about getting reputation, therefore no user-desire to post example questions etc.
• Reputation becoming precious. Everyone is trying to hoard rep. People follow proposals only to ask questions so they have a chance at getting rep, same with referrals.

Both of these we definitely want to prevent.

Therefore I propose only badges or maybe at the most like 1 or 2 points of rep per referral. Users should not be trying to refer people to get reputation though, they should be trying to make the proposal get to the next step.