What can be done with Area 51?

Question

I'm going to argue that Area 51 is not the optimal solution for Stack Exchange. Right now, as it stands, the following occurs:

1. Someone, probably an expert or at least a strong enthusiast, proposes a site.
2. People randomly follow this site. These people aren't committed to the proposal at all. These people with no commitment vote on the questions that define the community.
3. The site goes into a commitment phase, where people commit. Some of these will be followers from the definition, but most of them will be new people. These people can't really do anything other than try to get new people to join.
4. The Private Beta starts, and usually the first thing thrown out is the questions at the definition stage, as only a small portion of the committers were involved with the definition stage, and many of the questions proposed were written by those with only a passing interest, voted up with those with a passing interest.

This whole process results in a bit of chaos, especially in the private beta and early public beta, where the community has to re-define itself.

So, my question is, is the Area 51 model broken, and if so, what can be done to improve it?

Answer 1

I think that the most important part of the problem is the commitment phase and, as you mention in your own answer, that it needs to be rethought.

Currently, the process is slanted toward heavy users of Stack Exchange. While that may work for sites where the expertise is already widely present in the current userbase - programmers, web apps, etc. - it isn't as much of a good idea for most subjects. For example, I suspect that few users are researchers in economics. As a result, no matter how many researchers - real experts in their field - you persuade to commit, you'll still end up with a SmugMug: unable to proceed to beta because of a lack of users experienced with Stack Exchange.

Meanwhile, users with little knowledge of the subject but a lot of Stack Exchange reputation have a significant impact on the process which further helps diluting the original userbase.

I brought up this point a while ago and Robert Cartaino made a great suggestion:

It might better be implemented by asking the user what their role/interest is in the site rather than asking "Are you an expert?"

Something like this:

What role do you play in [Photography]?

○ Professional or Expert
◉ Avid Enthusiast or Prosumer
○ Academic or Reasearch-Level Student
○ Beginner or Learner
○ Merely Curious

Which Pollyanna then expended upon:

It would likely be better to allow anyone to join, but if they identify themselves as something less than a professional in the field then their commitment accounts for a smaller portion of the commitment required to start the site.

For instance, a site would require 40% of its commitments from those that identify themselves as professionals with 10+ years of experience in the topic, while the remaining 60% can be filled by those with less experience.

This would ensure that enough of experts join prior to starting the site, and allow the site to grow to fill everyone's needs ...

This number should be tweakable on a per site basis (perhaps have people rate it on a scale of 1-5, where 5 means only for experts, 1 means no experts required, and average the results during the following phase to give a ratio of expertise). Mathoverflow like sites would require 100% experts/graduate/post-graduate researchers, while a gaming site may be fine with 80% people who are mere enthusiasts, but not experts.

I think that these changes, along with removing the requisite of 100 users with at least 200 reputation on a site, would greatly improve the quality of sites. It would ensure that experts are on the site, right at the beginning of private beta, and that the site isn't occupied by the merely curious and the beginner.

Answer 2

That's definitely a good assessment of the problem with the process. I'm not sure that the specific problem that needs to be solved is one of commitment level; as I've described in the past, we already have a problem with throwaway commitments - most people who commit, don't follow through, for whatever reasons, so I don't think this would actually help the definition phase.

Here's how I would frame the problem:

• The definition phase defines the scope of the site - what is on or off-topic.

• Almost anybody is allowed to vote examples up or down.

• On a live site, the equivalent action is voting to close/reopen. However, on a live site, this is a privileged action available only to those who have established themselves as having some level of subject-matter expertise through reputation.

• Therefore, the Area 51 definition phase is outsourcing a critical and generally privileged action to those who have not established themselves as being qualified to act.

The whole process is actually too democratic. As another member pointed out in chat, it is a bit like allowing a Canadian or American citizen to vote on government policies in Europe. Of course we'll vote for higher taxes in order to subsidize hotels and touristy restaurants. It'll come in very handy when we go for our vacations there.

Of course we'll vote up the "How do I fight my speeding ticket?" example question in the Law proposal. No lawyer would ever want to touch that with a 10-foot pole, but it's the kind of question we'd want to be able to ask.

Part of this is definitely motivational - having no skin in the game, so to speak - but commitments are purely symbolic and I can think of no practical way to make them any more "material" - except to give them a real material cost, but I doubt that's ever going to happen (nor am I saying it should).

The larger problem is that, even if these people are all acting in good faith and genuinely want to create the best proposal possible, they aren't experts in the field. Every proposal should identify its experts at the outset - Stack Overflow was for Programmers, Cooking was for Cooks/Chefs, Physics is for Physicists, Law would be for Lawyers, and so on and so forth. You really don't have the experience necessary to vote on Area 51 questions unless you're a practicing member of that community.

Maybe you don't need to be an actual lawyer - maybe a law student is fine - but "programmers interested in law" is a lousy seed audience for a site that's supposed to eventually attract lawyers.

As I've said before - having a site defined by amateurs/dabblers only serves to guarantee that the site will be for amateurs/dabblers. It's right there in the Area 51 FAQ - "Ask Real, Expert Questions." But if you're not an expert, how can you even identify an expert question, let alone ask one?

How is somebody with no plumbing experience even going to think of a question like, "If you run 2.5 GPM through 50 feet of 1/2" galv pipe, how many psi will be lost to friction loss?" I don't even know what some of those abbreviations mean! No, that won't happen, participants are almost certainly going to ask and vote for the "How do I unclog a drain?" question. It's not that they're actively trying to undermine the proposal, they just don't know any better.

I've heard a number of suggestions that I think could work, but I honestly don't know enough to rationalize which ones would work, so I'm just going to list them in no particular order:

• Ask members what level they're at - see Robert's answer there, but instead of doing this during Commitment, do it during definition, and don't allow votes from the beginner and curiosity crowds. (N.B. Anybody can still propose questions, the restriction only applies to voting). This relies on member honesty to no small degree but, I think most people are pretty honest.

• Engage individual professionals/organizations to help define the sites. This might cost a lot of money, but it is very likely to produce good definitions.

• Engage same professionals/organizations as curators, i.e. moderators of a sort, who could weed out the really poor questions popular with beginners and dabblers. This would probably have less of a capital cost but could still be reasonably effective at preventing crap proposals from going out the door.

I'm honestly not terribly crazy about any of the above suggestions, I'm just spelling out what I've heard. I do think that we need to somehow raise the bar for definition phase, and that forcing a commitment is not going to be enough unless the meaning of "commitment" is changed substantially.

Answer 3

It seems to me that the biggest problem in the system is that the people voting on the proposal's questions have little to no commitment in the final product. In other words, the people who make the decision have no commitment over the final product, which anyone who knows anything about managing knows that that's a very bad thing. You want the people who make decisions to be held accountable for those decisions, and I feel like that isn't the case.

I think this could be solved by limiting the number of proposals that a person can vote on. If you have to think really carefully about what subjects you are going to participate, you aren't going to go crazy about voting. You're either going to be an expert in the subject, or a suitable enthusiast who knows what's going on. This commitment to a proposal should continue on to its beta. If you drop out in the early proposal stage, then your votes go away. And once it reaches the commitment, you can't drop out (or at the very least, there's some kind of a period of time that you are committed to stay in).

I'm not going to get specific at this point in time, but I will say that there are lots of things that would need to change, including:

1. The number of proposals that one can vote on might have to decrease, or perhaps limiting voting to a proposal you have followed.
2. It should be made a bit easier to gain reputation in Area 51.
3. The commitment phase should also be re-thought somewhat, possibly leaning towards allowing non-SE people to have a higher weight than they do now, so a site of experts in a field can join without having to have so much help from heavy SO users.
4. Something needs to be done to keep people going back to a project they have committed to. Without this, they have little reason to pursue it, unless they REALLY REALLY want the site to happen. And let's face it, how many people commit to a site that they really want to happen?

Anyways, I welcome all thoughts on the subject.

Answer 4

To issue a verdict of "broken" (or not), you have to look at what comes out of the other side.

You are looking at Area 51 as a vending machine where you insert proposal and get that site. It doesn't work that way. A proposal is not a site; It's an idea. A successful Stack Exchange site comes out of months of poking and prodding by hundreds of people — small incremental units of work by a BUNCH of people, all pushing in the right direction. It's not a "master plan" by some visionary, all neatly organized in committees and procedure, to implement their original vision.

What you call a "process that results in a bit of chaos", I see as a process of chaos that self-organizes ONLY when all the required pieces are in place. That's what Area 51 is about: taking someone's random idea and seeing if you have the people, the desire, and the talent to build it up onto a workable solution.

If ALL the pieces are not in place — if you do not have the the right people with the right knowledge in the right place at the right time — the process falls apart. The process should fall apart.

It is so easy to say "There should be a site about …" But that's where 99.6% of those ideas fall flat. That was the failed model of Stack Exchange 1.0.

There are plenty of folks who think we should should just go ahead and create these sites as a means to see what will work. Others feel there should be much more activity and discussion before launching a site. Area 51 meets both of these groups somewhere in the middle.

Answer 5

One possible solution is to completely rework Area51 to be a filter to create branches off a generic Q/A Site.

More specifically: Create a generic Q/A site that is a bucket of questions for non-established stack exchange sites. Then break off a branch of that site into a new site when a "critical mass" of questions and experts is identified.

For example the giant spawning pool site would collect enough questions under tags relating to firearms that at some point you have hundreds of questions with a set of tags related to firearms and many users that have rep from those tags. The Area51 site would be about defining what tags met the criteria for a new site and voting on whether 'actual' questions were on topic. Once enough questions fall into those tags, the questions are migrated and the site will begin with many expert users already having rep for the answers they have previously submitted. (Anyone going to the generic site who tried to use the tags that got moved would be directed to the new site). This would also allow brand new users to participate immediately instead of committing to a site and forgetting about months later when it actually starts.

Edit:

To clarify some things based on the comments. I am not talking about a generic sandbox where ideas can be testing and experts weeded out from people who are just interested on real questions as opposed to people committing and debating about "example questions". As Neil put nicely: "We'd still need to attract the non-SE users, but it'd let us find out what topics would work with the SE model from the start."

Answer 6

The problem is that the sites are too narrow, for example Webmasters is basically covered by Serverfault and stackoverflow, and if it's not fully covered then Serverfault and/or stackoverflow should be tweaked so it is covered.

Also another problem is splitting a subject into 2 sites, one for high-level (eg Physics Research proposal) and one for low-level (eg Physics.stackexchange.com). This means the people with the answers hang out on the high-level site, and the questions on the low-level site basically get ignored.

An another example of a narrowness is science stackexchange sites, biology, physics and chemistry are huge areas but there isn't a big enough audience to warrant three separate sites. Maybe one day when there are enough users on a general science site they could be split up.

People see the way stackoverflow works, and thinks it's great (because it is, and questions get answered quite quickly and "goodly") then they want to ask a question not programming related and so propose a site, when all they want is to ask a question.

There needs to be a cut off point where a very general (not completely general though) stackexchange site is made.

In sum: the sites are too specialized, this can be good if there's a large potential audience, but in most cases there isn't