35

The Problem

For background on this issue I will refer people to the following discussion:

The team is on record several times as having said that this is not how Area 51 is supposed to work, that the only people who commit to a proposal should be people who actually intend to participate in the site.

Needless to say that many people (in fact, the vast majority) either missed all of the memos or simply don't care. My objective isn't to single people out, but I just saw this comment in a random proposal I looked at the other day:

Well-wisher comment

Now maybe I misinterpreted this comment, but it certainly reads as though this person has no actual interest and certainly no expertise in the area, he just wants to push it one step further toward beta.

I like to call these people well-wishers. I don't think they're necessarily committing out of sympathy - they probably genuinely want the community to do well - but unlike, say, friends or family or even associates, a well-wisher takes a completely passive role, celebrating your victories but doing absolutely nothing to help achieve them, then drinking all the champagne.

These are not isolated incidents. The statistics speak for themselves:

These stats are dismal. Less than 1 out of 5 people are even trying to pay their dues, and when they do try, they're usually not passionate or knowledgeable enough to go all the way.

Now for the sake of thoroughness, not all proposals are like this; many of the proposals that you would expect to be popular with computer geeks, like Gaming and Apple, have excellent participation rates and reasonably good fulfillment rates. But for the most part, the message is clear: Most proposals that aren't either fun or closely related to programming seem to be inundated with well-wishers who don't fulfill their commitments and often don't even bother to sign up. That's a horrible reflection on the overall Area 51 community and culture.

The team could simply use a fudge factor and encode this assumption into the system, but that's not fair to the proposals that have legitimate support. And I know the team vets proposals before they go to public beta, but looking at the number of proposals in the system, I don't see how that's going to scale over the long haul.

My Proposal

Supposedly the "commitment tokens" (max commit to 3 proposals) is supposed to discourage well-wishing behaviour, but it doesn't seem to be working very effectively. Bottom line is that there are millions of users in the Stack Exchange community and at least several thousand of them with enough rep to make a difference, and most of these people just don't care. So let's make them care.

I think we should give commitments an up-front cost. Instead of being empty promises with no teeth, make them like investments. An investor has to take on some degree of risk, and if the decision was a good one, there will be an even greater reward. Also, investors who really care will actually get involved and attend shareholder meetings, not just sit passively and wait for a return.

Here is how my hypothetical system would work:

  • Investors pay reputation for their commitments, like bounties.
  • Investors receive a 100% refund for fulfilling their commitment.
  • Investors receive an additional 50% bonus if the site goes live.
  • Investors can also choose how much reputation they wish to stake, possibly up to some predefined limit (again, like bounties).
  • Committing can be done for free, but the value of a 0-rep commitment from a high-rep user is identical to the value of a commitment from a 1-rep user (i.e. virtually nothing).
  • Users with no reputation on Area 51 itself can siphon reputation from one of their other linked accounts, but this would be one-way and non-refundable.
  • If a proposal gets closed or merged during commitment phase or mid-way through the beta then all "fees" are refunded.

The (positive) consequences of this are manifold:

  • Fulfilling commitments becomes the most effective means to gain Area 51 reputation over time.
  • Area 51 reputation actually measures trust, like it is supposed to.
  • Members who undermine the system with empty promises must work to earn back that trust.
  • Members have a strong positive incentive for participating (to earn their rep back).
  • Members have a strong positive incentive for committing/investing before participating (rep gain).
  • Members who help to spawn successful sites still get something back, even if they didn't/couldn't participate directly - but they still end up "out of pocket".

I really believe that this is how reputation on Area 51 should work. Creating and helping to define proposals is still valuable and should of course confer some rep, but the only thing that's really worth a damn in the long haul is actual participation, because if people don't participate, then the sites will either fail or take forever to reach half-decent traffic levels. Anyone can come up with an idea; the real value is in its execution.

Right now all you get for doing the most important thing is a few worthless badges, while folks spamming junk proposals and questions rack up the points and members who seem to be actively working to undermine the system suffer no consequences whatsoever. So instead of basing the progress simply on how much reputation people have, base it on how much they're willing to spend, and give them something (semi) tangible for following through.

Anyway, that's the proposal. Question and comments are welcome of course.

Improve this question
32
  • 2
    Huh, you were not the person I would have expected as the author of this. I guess this is what happens when I open a question without looking at the author.
    – Grace Note StaffMod
    Jan 27, 2011 at 15:35
  • 1
    @Grace: Why, is this so different from the philosophy I apply everywhere else?
    – Aarobot
    Jan 27, 2011 at 15:35
  • 2
    No, philosophically it's not different from what I'd expect of you. I just didn't know you were paying attention to this specific issue, and was figuring it be one of a handful of Area 51 devotees that would've come to this first.
    – Grace Note StaffMod
    Jan 27, 2011 at 15:37
  • 3
    @Grace: Oh, I visit Area 51 plenty, I just don't spend a lot of time in proposals that I don't legitimately think I'd have the time or inclination to support (which is most of them). Which is why I wrote this; I hope that it will at least get people thinking about ways to focus participants a little, to get them to contribute to what matters most (making the beta Exchanges good) as opposed to reveling in the social and play aspects of Area 51 itself.
    – Aarobot
    Jan 27, 2011 at 15:42
  • 2
    By the way, I didn't want to waste dwindling space in the proposal itself with this, but I will step forward and say that I personally am guilty of this exact behaviour in the past, with the Android proposal. I didn't really stop to think about what I could contribute, and I think having a system like this in place would have made me think twice about committing - but the site itself probably still would have made it to beta, as Android has a pretty high signup/fulfillment rate compared to most.
    – Aarobot
    Jan 27, 2011 at 15:45
  • 7
    Be careful with your statistic conclusion. There are many reasons why sign-ups may not be fulfilled. It is true that sympathy may be one of them - but it's not true that sympathy is the only reason this happens.
    – rlb.usa
    Jan 27, 2011 at 16:44
  • 2
    @Shog: There's the "sunk cost" angle, too - even if it doesn't necessarily stop everyone from committing frivolously, it might motivate a few of them to get off their butts and try to earn some of that rep back. I'm a lot more likely to shlep down to the theatre if I've already got the ticket sitting in my hand, even if I don't think it's going to be that great a movie. Even in my own experience attending and organizing various get-togethers, I've found that attendance is better when there's some kind of material commitment, even if it's just a $1 donation.
    – Aarobot
    Jan 27, 2011 at 17:20
  • 1
    Ok, so what would happen for sites like DBA - where a ton of people signed up for a Databases site, and then the site went into beta as Database Administrators? I've tried to participate, and I could have fulfilled my commitment for the site I originally signed up for,(as reflected in the definition questions), but it's really not the same site I committed to. If people get burnt like that, they just won't commit again.
    – testerab
    Feb 6, 2011 at 22:52
  • 1
    @tester: My opinion is that changes to the scope during or before the beta should be treated like merges, i.e. commitments refunded. Allow people to re-commit ("re-invest") if they want. This would have applied to sites like Programmers.SE as well.
    – Aarobot
    Feb 6, 2011 at 23:04
  • 1
    I think you're missing one huge point: What the site needs is, if we take a step back, not "committers", people who promise to ask three question and then go on to ask three questions. It needs regulars. People who stay on the site, even after it leaves beta, and ask a million questions and answer even more. And at a certain level, that is fundamentally incompatible with a "reliable committer", because the latter is someone who always commits to new sites, giving him less time for the old ones.
    – jalf
    Feb 13, 2011 at 17:29
  • 1
    So what you're doing is rewarding people for asking a few questions and then moving on to the next proposal, which is just a more elaborate way to do a "sympathy commitment".
    – jalf
    Feb 13, 2011 at 17:30
  • 1
    by the way, wouldn't it make sense to edit your proposal out into a separate answer, leaving the question to contain only the problem description? Would be a lot easier to discuss (and up/downvote) the two separately then.
    – jalf
    Feb 13, 2011 at 17:31
  • 1
    @Aarobot - one query: when I first came to the IT Security Beta, I had never heard of stack exchange. If I had I would have been all over it - but your proposal for requiring rep would have left me utterly unable to contribute. As it is, I'm now a moderator of a rapidly growing se group. Your proposals would definitely have put me off!
    – Rory Alsop
    Feb 23, 2011 at 16:24
  • 1
    @Aarobot - but if I had heard about it in initial proposal phase, I wouldn't have been able to give a commitment because I had no rep. I could now, obviously, but that's too late for that particular one.
    – Rory Alsop
    Feb 23, 2011 at 18:55
  • 1
    @Aarobot - okay, that makes sense, I think. Almost totally the opposite of what I read in your post though.
    – Rory Alsop
    Feb 23, 2011 at 20:06

4 Answers 4

Reset to default
24
+150

I think you're going in the wrong direction. Sure, part of the problem might be that it's too easy to commit, so you get a lot of people doing it out of sympathy, or because "well, why not".

But I think it's more useful to turn it upside down. The problem is that too few follow up on their commitment, not that too many commit.

I posted a question about that a long time ago, here, and I still think it's true. The current commitment process is pretty much designed to turn people off. It's designed to make it as hard as possible to live up to your commitment.

It is designed to appeal to the people who live on Meta, who are hooked on chasing rep and arguing about rules. But it offers absolutely nothing to those who are actually interested in the proposed site's domain.

Here's what happens in far too many cases:

  1. You hear of a nifty new site proposal, one that would actually be useful to you.
  2. You commit to the site.
  3. You realize you can't actually use the site yet, because it's stuck in an arbitrary commit/definition stage.
  4. you search out other ways to get your question answered.
  5. a month passes, and you forget all about the site.
  6. you get an email telling you that the site is now in private beta
  7. You can't remember what questions you wanted to ask, and you no longer really see the point.

Once committed, why shouldn't a user be able to ask and answer questions immediately? That would be useful, and it would allow all the people who discover the proposal because they have an actual question they want answered, to actually ask the question.

It's really similar to how all the StackExchange sites don't require you to sign up. They assume that you arrive at the site because you have a question you want answered. So they let you ask your question, without pulling you through arbitrary registration procedures.

Why does A51 do the opposite, not just asking you to solemnly promise to "commit" to a site, but then making you wait for weeks on top of it?

And why is anyone surprised that the process turns out a lot of "empty" commitments?

Finally, a concern with your proposal is that I think it is tailored to encourage the wrong crowd. It is all about increasing rep, about gaining fame and glory on A51.

As a potential user of a new SE site, what do I care about that? Most people just don't care about A51. They might care about "their" site (rewarding good behavior with additional rep or badges on SO works great, because SO is the actual site people use. A51 is just a bus stop on the way to the site you want.

If you want to solve the commit problem, then reaching out to the Area 51/meta/rep-addicted users isn't going to help you. You need to get hold of the actual users of the proposed site. The ones who have questions about that domain.

And a good first step towards that would be to allow them to use the site when they first hear of it.

Improve this answer
14
  • Once committed, why shouldn't a user be able to ask and answer questions immediately? - Because there's a coordination problem here: few people will put effort into asking questions and answering them before it looks like the proposal has serious momentum. I think the current private beta system is a good solution to that problem.
    – Charles Stewart
    Jan 28, 2011 at 19:33
  • 1
    Sorry, I'm not feeling this one. If a couple of months is all it takes for you to completely forget about everything you could possibly have wanted to ask, then you probably don't have much interest/expertise in the first place and shouldn't be committing. Needless to say people also usually need to answer questions to fulfill their commitment and this post says nothing about that. This suggestion simply exacerbates the chicken-and-egg problem that already exists; my recommendation already addresses the positive incentives for getting people to follow through on commitments.
    – Aarobot
    Jan 28, 2011 at 22:37
  • Furthermore, I fail to see what meta has to do with any of this, I never mentioned it and it's not related to Area 51 at all. And, finally, your concept of "chasing rep for fame" is based on the concept of reputation as it exists today. My whole proposal is about changing what reputation means; consistently following through on Area 51 commitments would mean that the system trusts you more and would hence allow you to advance a proposal further, and that's exactly what's needed most in an Area 51 commitment; trustworthy people who will do what they say they will.
    – Aarobot
    Jan 28, 2011 at 22:41
  • Awarding the bounty to this post because it makes for good discussion, though I don't agree that Aarobot is wrong. Actually, I think you two are more on the same page than you realize.
    – Pops
    Feb 5, 2011 at 19:34
  • 2
    I find the commitment (and to some degree, even beta) phase strangely counterproductive as well. The first beta I joined (electronics), I went to the site without knowing of area 51 only to be suddenly cut off. Apparently participating in the site is forbidden without "commiting" at another site. Which is really the commitment? For the second proposal I'm following, I personally know several domain experts. They're interested in the proposed site, but have absolutely zero interest in Area51. Being counted less isn't going to attract them.
    – Yann Vernier
    Feb 13, 2011 at 1:39
  • 3
    @Charles: true, but that shouldn't be the prospective user's problem. When committing, I want to use the site. Make it work! ;) And @Aarobot: a couple of months is enough to make me forget that the site existed. When I am reminded of it, it is simply "the site that didn't exist". When you want to build a community, turning people away and saying "come back later" isn't going to work.
    – jalf
    Feb 13, 2011 at 17:14
  • 2
    @Aarobot: Meta and A51 have a lot in common: both are populated with people who want to be a part of StackExchange, people who want to say "I helped make this happen", people who are interested in playing the system, rather than asking questions on a specific topic. A new site doesn't need these generalists who are in it for the fame, it needs specialists, and those aren't attracted by a promise of gaining more power over other proposals.
    – jalf
    Feb 13, 2011 at 17:17
  • 1
    Exactly as @Yann says, in short, no matter how much you bling out the rewards on A51 for following through on a commitment, you're still reaching out to the wrong crowed: the group who hangs out on A51, rather than the group who intend to hang out on the new site. The ones who don't give a damn about A51 or what they can do there.
    – jalf
    Feb 13, 2011 at 17:20
  • 1
    I'm sorry but that is such utter BS. How long do you see people waiting on tenterhooks for as soon as they hear an unverified rumour about the newest iProduct? Anticipation is more often than not a positive thing. You're asking to make the system more convenient for people who only have a fleeting interest to begin with; those are precisely the same people who I'm trying to rule out with this proposal.
    – Aarobot
    Feb 13, 2011 at 17:35
  • 1
    @BS? I thought you were trying hard to be polite? ;) Here's whats BS: the process currently doesn't work, and you refuse to accept it, even when faced with concrete examples of it. Anticipation works when you have a unique proposition, something (percieved as) new for which no alternative exists. To a domain expert who's currently participating in a dozen mailing lists and forums on his subject matter, the promise that "an unknown number of months from now, a Q&A site on your area of expertise will launch" is not going to get him hooked.
    – jalf
    Feb 13, 2011 at 17:39
  • 3
    And it's not because he "has a fleeting interest" (now that btw, is BS. You seem to use that argument to dismiss everyone and anyone who don't fit into your diagnosis). These are the most hardcore users you have, the ones who already spend a huge proportion of their time answering questions on the subject. Their interest in the subject, and in answering questions about it, is far from fleeting. But their interest in "channel #263 for questions about the subject" is very fleeting, yes, because they have 262 other channels for it already. To get those people on board, you need to deliver.
    – jalf
    Feb 13, 2011 at 17:41
  • 2
    The people you're trying to rule out with your proposal are the ones who want to ask/answer questions on the subject no matter what, the ones who currently rely on mailing lists and whatever else, but who would be extremely valuable to the proposed site. And the people you're trying to encourage are the ones who proposed to 14 sites before this, and who, once their proposal has been fulfilled, will commit to another one. I think the former is a more valuable type of user.
    – jalf
    Feb 13, 2011 at 17:44
  • 1
    I noticed that this happened in the Persian IT proposal: run the example questions through a translator and you'll notice that many of them got actual answers instead of off-topic/on-topic discussions.
    – Jan Fabry
    Feb 23, 2011 at 22:10
  • @Aarobot, It doesn't take folks a couple of months to forget a site. The amount of time it takes depends directly on how much activity they have done on that site. If no activity has been done, it is surprising if they would stay on even after a few weeks. I had came across many "almost done! new site launching soon" sites which I routinely forget after 3 to 5 days or less. Truth is most people have a life outside of SE/, even when they "commit" to a SE proposal.
    – Pacerier
    Jan 20, 2016 at 21:04
5
+50

I'm convinced by the diagnosis, and in terms of avoiding beta-harming, reputation-seeking behaviour from SOFU grognards, I think it is an excellent solution. It will need tuning, I guess.

There is another issue that is a cause of the Area 51 process not working out as planned. Jeff originally articulated a vision that I'll summarise as "Get the domain experts to commit and crowds will enthusiastically participate in the beta". The trouble is, the experts are not keen to add to demands on their time, and I think they will tend to look like your well-wishers (I have a particular domain expert in mind): they want the site to succeed, but they have nothing invested in the SX model, and they are reluctant to move the centre of their attention from the half-a-dozen mailing lists they currently work with to the site unless it has obviously become a community centre.

It makes a big difference if these people make themselves visible early in the beta. I think this proposal currently will act as some sort of deterrent to experts, as it stands - another barrier to participation, making some weird kind of investment in the site whose consequences are unclear. I think we need to think how the investment idea, pitched at insiders, should work alongside the invitation model, pitched at outsiders. I made an altogether different proposal, Making experts more visible before Area 51's beta phase that maybe provides the germ of an idea for extending the the investment model to attracting outsiders in a way that provides some psychological incentive to experts to participate.

Improve this answer
1
  • As far as outsiders go, I don't think my proposal would change anything for them, since it would still be possible to commit with no Area 51 reputation. I do also agree with your assessment that subject matter experts (without SE history) could use some additional incentives. This should be a two-pronged strategy, really: first, limit the ability of the well-wishers to actually advance the proposal to beta if they don't plan to contribute, and second, provide some additional incentives for the outsiders (who are probably more likely than the SOFU crowd to be experts).
    – Aarobot
    Jan 31, 2011 at 15:51
2

You have some very interesting points made in this question. I'm going to propose something that is a bit of a combination, but that I think might make this an interesting case still.

  1. I think allowing the definition phase of a site to continue is still an okay thing in the commitment phase. If nothing else, it gives the community a bit more incentive to organize itself prior to beta, and it'll have the benefits of making it easier for users to remember what questions they were going to ask.
  2. I'm fine with not being able to answer, I think that's an okay thing to do.
  3. A better method definitely needs to be determined to enter the beta phase, see What can be done with Area 51?
Improve this answer
2

I also think it's not about sympathy commit.

The main cause could be lot of disillusion and disapointment.

I committed to a few Area 51 proposals with real interest and passion (yes including this one), and all sites that went to beta phase did not meet my expectations. The saddest example is http://audio.stackexchange.com that will probably die soon with it's 215 questions.

Maybe we should be more strict with requirements by increasing the number of committed users and example questions needed to enter in beta phase.

Improve this answer
16
  • I don't wish to point fingers here, but since you've called out audio.SE, it would seem to me that your one question there (which happens to be a gear recommendation) really fits the definition of a well-wisher rather well. I'm sure - positive, in fact - that you had 100% good intentions, but what's needed is for people to actually step up to the plate and contribute. Cooking.SE had 94 questions on the first day - a lot of them sucked, but the point was that people genuinely had questions they wanted to ask and a group of us worked (hard) to improve the quality over time.
    – Aarobot
    Jan 29, 2011 at 20:05
  • I'm also a little frustrated that everybody keeps referring to "sympathy commit". I didn't invent that term, and explicitly called it out as being misleading. It's not necessarily out of sympathy, it's just a question of how "committed" those people really are (often the answer is - not very). Anyway, I think that requiring committers to stake actual reputation would increase the commitment requirements naturally; instead of having members with 100k network rep making huge dents in the progress, they'd chip away at it gradually with the 1k (or whatever) they actually contribute.
    – Aarobot
    Jan 29, 2011 at 20:08
  • 2
    Aarobot: that's a perfect demonstration of what I was talking about. I was motivated by that proposal, but it did not meet my expectations. Audio is a very large subject, and the part I was interested in is not really represented there. So I lost my interest. Looks like that's the problem we face. We probably need more commiters to avoid that effect.
    – user150926
    Jan 29, 2011 at 20:11
  • 1
    The definition phase should have clearly specified what part would be represented. I'm not sure how you can say that it didn't meet your expectations when there's no evidence of you having tested those expectations. Maybe there's something I'm missing - deleted questions or somesuch - but with all due respect, this sounds more like rationalization after the fact than an objective explanation of what factors were at work.
    – Aarobot
    Jan 29, 2011 at 22:55
  • 1
    @Aarobot: It's my theory. Take it or leave it. I hope your appreciate the time I spent to contribute to your question here ;)
    – user150926
    Jan 29, 2011 at 23:16
  • 1
    @Aarobot: I have a general observation: rather than trying to change the user's behavior (you know it's impossible), I think it would be more productive to put all efforts on adapting the Area51 strategy. I'm not an expert, so I'm probably naive when I suggest to increase the number of required commiters to open the private beta.
    – user150926
    Jan 29, 2011 at 23:18
  • 1
    Why do you think it's impossible to change behaviour? Really the entire Stack Overflow and Stack Exchange system is built around precisely this principle - that with the right incentives, people can be coaxed into doing the right thing. People actually change their behaviour quite readily when given the right motivation. As for the rest - I already addressed your position in my original post; raising the requirements might help a few proposals (or might not), but one thing it would definitely do is hurt the ones that actually have real support.
    – Aarobot
    Jan 30, 2011 at 0:20
  • @Aarobot: IMHO, SEN reputation system is more an exploitation of existing human nature than a attempt to change behaviors ;)
    – user150926
    Jan 30, 2011 at 0:58
  • All behaviour modification is an exploitation of human nature. Or any animal for that matter. A cat really doesn't care that you want him to sit or follow a pointer, but if he thinks there's a slice of turkey in it for him, he'll do it. If the rep-based system works for Q&A, why do you think it can't work for Area 51?
    – Aarobot
    Jan 30, 2011 at 1:19
  • @Aarabot: the modification takes a lot of work of conditioning, and you need real influence & power on the subject. Working on behavior of masses is impossible for that reason. In our case Area51 is just one website over billions with not much influences on the people that matters: the contributors. Without them, the community is useless. That's the problem right? What is needed is marketing.
    – user150926
    Jan 30, 2011 at 9:39
  • Contributors don't even exist until the site is created, which happens after commitment. This system is essentially asking people if they plan to be a contributor or not, and over time, unlike the current system, the proposed system would learn whether or not individuals can actually be trusted to contribute when they say they will. Saying that behaviour modification is "impossible" when it's already worked so well on Stack Overflow and the other Q&A is more than a little silly. Marketing's good too, yes, but it's hardly a panacea, and ultimately Google does the best marketing.
    – Aarobot
    Jan 30, 2011 at 19:37
  • 2
    @Aarobot: I will leave you with your insults. Good luck with your attempt to change the people.
    – user150926
    Jan 30, 2011 at 19:50
  • Funny, I don't see anything in there resembling an insult, and I was frankly trying very hard to be polite. Apparently, responding to an illogical argument with logic is considered an "insult" now. Sorry, not going to indulge you here; passive-aggressive rhetoric is a waste of my time and yours.
    – Aarobot
    Jan 30, 2011 at 20:38
  • 1
    @Aarobot: most of your comments on his answer have been very condescending. You're basically telling him that he's wrong, and that he's lying about why he stopped using a site. And you completely dismiss the actual points he made. If people give up on using the sites because they get disillusioned and disappointed, then you're not changing anything by saying "well dont do that". You need to fix the reasons why the SE users feel that way.
    – jalf
    Feb 13, 2011 at 17:27
  • @jalf, I have seen absolutely no logical or verifiable points in this exchange. You and Pierre both ask that we take people's subjective evaluations of their own actions and unreliable memories of their decisions at face value when in fact that's exactly the type of thing psychologists know is unreliable (choice supportive bias). But it's irrelevant in any case; call it "disillusionment" if you want, but what's needed is a system that plans for this and takes it into account, not one that pretends it can solve it (which is 100% impossible).
    – Aarobot
    Feb 13, 2011 at 18:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .