There is a lot of effort put into determining what is on topic for a site heading towards commitment. Sites can't enter the commitment phase without having a certain number of good quality questions.

The Chinese Language & Usage site which is almost ready to go into beta and is picking up steam has a set of questions that I think aren't the best example of what should be on topic for the site:


Why are “Peking” and “Beijing” two different names for the same place?

This is a wikipedia type question and can be found with very little research


How can you type Chinese characters on a computer?

I think this is borderline off-topic. This more of a personal preference type question or I'm having trouble installing the software after doing a Google search and downloading.


What are the common parts and history of the Chinese and Japanese written languages? What are the differences?

There could be a book written on this. It's not an answerable question.


How do you convey sarcasm in Chinese?

I would argue this can't be reasonably answered. It's like asking how can I say something impressive in Chinese. It all depends on the context and your skill level to be able to pull it off, not saying the right thing.


So what to do now? Do low quality proposal questions really matter? If they don't matter do they serve a real purpose for defining a site or do they just hold a proposal up until there is enough interest?

share|improve this question
1  
The first one could be replaced with why China has two names for them same thing (with a similar meaning as well), which might be something cultural and help answer similar questions ;-) Anyway, doesn't this belong on discuss.area51? –  Ivo Flipse Nov 29 '11 at 10:02
    
@IvoFlipse - Sounds fair enough, however this is a really basic question and is already answered on the wikipedia page for Beijing. The FAQ of all sites states don't ask things that are too simple. –  xiaohouzi79 Nov 29 '11 at 10:05
add comment

4 Answers

Remember, proposals are not sites; they are ideas. So the example questions are important for fleshing out that idea to see if there is (ultimately) interest in the site. But once you've demonstrated that interest (during Definition), the example questions are not necessarily binding on the final product. Most of the work of actually developing the site and its governance is one in the final phases — in beta.

The issue you raised is important

Some proposal ideas are somewhat predisposed to becoming sites on sole basis of being of being an interest to our current audience… even if that interest level is being merely curious. Unfortunately, that predisposition pushes many proposals through despite egregious Definition shortcomings.

We're trying to change that.

Poorly executed proposals have put more-recently-launched sites on a very precarious footing. As proposals continue to expand outward from our core group, the execution of the proposal process is getting weaker and weaker. Weakly developed proposals are limping across the finish line, while great site ideas are launch by groups ill-equipped to build them.

Area 51 is a crowd-sourced activity and it takes an engaged group of users to build a great proposal. Our first step towards improving engagement in Area 51 is re-assserting those behaviors which have been proven to work in Stack Exchange. Live sites actually fight for the quality of their systems, and we need to instill that same sense of purpose in Area 51. See…

Announcement - Changes to Area 51 Voting

This is one step — only a first step — towards building an improved Area 51 experience. By emulating a more-familiar Stack Exchange experience, I'm hoping it will engage users into asking actual questions they will ask on the live site… and prompting users to jealously protect their proposal (i.e. the future site) from potentially poor content.

What You Can Do Now

Long gone are the days where Area 51 "accidentally" creates great sites. If you want to see a growing network of subjects that interest you, it's time to take another look at Area 51. Browse through your favorite proposals. You might be shocked what the "founders" of those sites are passing off as "good example questions."

Look through the definition of your favorite proposals. If a question is good (for the actual site), vote it up. If the question is lame — exact duplicate, not constructive, not a real question, too localized — close it. We need more of that type of engagement.

share|improve this answer
add comment

Joel mentioned this during Episode #21 of the Stack Exchange Podcast, while they were discussing improvements to the voting system used for the example questions. Here's a relevant excerpt, starting at 9:41:

Joel: [...and the problem was] the sites weren't getting formulated correctly, and we would see these sites in the private beta, and we'd look at the questions that were getting asked and we said "this is an awful site". Because they're asking like "What is the capital of France?", you know? There's something deeply wrong with the site. And then we went back and looked at the Area 51 [Proposals] for those broken sites and we saw that, lo and behold, they had all, in the Area 51 process, come up with a bunch of sample questions that were "What is the capital of France?". And everybody was like "Yeah, that's totally on-topic for Travel!".

And yeah it is, but that's not the point, I mean that's not– the question's too easy or too general-referency or, you know, there are other reasons why, there are specific reasons why that is not a real question. Actually, "Not A Real Question" would be a perfect close reason for that. And so that led us to the idea that we had to—in the Area 51 process—catch the sites that aren't going to have... what was the one that was– I think Artificial Intelligence was actually like that, where we discovered right away that nobody in the creation process of the site was actually an artificial intelligence partitioner. They didn't have real questions or real answers; they were all just kind-of interested in it. And during the private beta they were asking questions like "What are some good blogs about AI?", "What is the first book I should read about AI?", "How can I go about learning AI?".

And this is not a real AI site, right? This is like a fake standalone site where people pretend to be having a site. And we went back to the Area 51 record, so to speak, to see what they thought were good AI questions and lo and behold, that's what they were! And so we thought "Hey, this is a good way we can catch that, nip it in the bud, basically. If there's going to be a site where it doesn't actually have experts or that has people asking things that aren't real questions. Hopefully the Area 51 process will catch that a little bit earlier, where you have a site that's not actually viable.

share|improve this answer
add comment

There is a contractual element in the mechanics of Area51. If the site gets enough followers and on-topic questions, it moves to commitment. The quality of the questions is actually irrelevant in this process, the only thing that matters is that people agree on which questions to spend their votes on. A lot of the comments announcements to the effect of:

New Followers: Please vote! We need one more question marked as Off-Topic to get into commitment phase! A question is considered off-topic with at least 20 votes and 4 times as many off-topic votes than other votes. So, three more off-topic votes for the closure question would get us going.

have been deleted, (and the above is based on obsolete criteria anyways) but

  1. They're still being generated
  2. They're still having an effect on commitment-phase, beta, and launched proposals
  3. The mentality that generated them in the first place is unchanged.

The Area51 FAQ states:

When at least ten questions have a score of at least ten net votes (up minus down), then the proposal is considered "defined."

Why would someone vote down if upvotes are necessary to advance the site?

The answer: They wouldn't. There's barely any incentive to the followers to generate quality questions in this phase. The followers just want to see the site launched; in their minds any disagreements on scope can be resolved later. There is a threat that the proposal might get shut down from above, but that's remote and unmeasurable. Given the option to downvote a mediocre question and wait for better ones to bubble up, versus upvoting that question and moving to the next phase, human nature will generally choose the latter.

A different metric is necessary to measure whether the site is well-defined. I'm not sure what it is yet, but I think it will require a change to the current Area51 model of 2-sentence questions with comments, and vote minimums. Additionally, a hypothetical close somewhere down the line won't cut it; more and faster feedback from experienced SE users that are less invested in the launch of the site and willing to be critical is absolutely necessary.

share|improve this answer
add comment

The bad news is that these are indeed bad questions.

The good news is that the Area 51 example questions aren't very important.

The example questions on Area 51 have two uses: determine the audience of the site, and determine the topic of the site. Here, the topic is clear. Regarding the audience, the essential question will be how much you're targeting natives and how much you're targeting second language speakers.

A key point in this is the language of the posts. Among the existing language beta sites, there are several approaches: German has a majority of English posts, but a healthy German minority; Spanish is heading for English-speaking audience only, and Japanese is explicitly primarily for learners; French has a majority of French posts, but a healthy English minority. Only French has a significant amount of meta participation in French, and has French tags. Since the example questions on Area 51 are all in English, it is likely that the committers will push towards a mostly-English site; but this is up to the people who are active in the beta. But this is not a given; if there are enough people in the beta who want to ask and answer in Chinese, then the site will accept posts in Chinese.

The private beta is an important time to establish ground rules and set the tone for the quality of the site. I recommend to be especially firm about rejecting bad questions (not constructive, general reference, etc.) during the private beta, then progressively relax the rules (but not too much). It's easier to first build a high-quality site then gradually build up an audience than to first build a low-quality site then gradually up the level. The first approach brings you experts, the second approach doesn't.

share|improve this answer
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .