How can Area 51 value high real-life reputation?

Question

It's very important for new sites aimed at some field to attract genuine experts in that field who have a high real-life reputation in that field. At a healthy mature SE site such a genuine expert will typically also have a high on-site reputation. However, for a newly launched site such a person will typically have very low prior reputation because, by definition, their main area of expertise was not already covered by existing SE sites. Thus, during the Area 51 process attracting such a person would be essentially useless to the process even though it's vital to the health of the site.

However, it might be possible to fix this problem relatively easily. Even if the existing SE communities have very few genuine experts in a new field there are still lots of people whose knowledge of that field is more than adequate to assess whether someone is a genuine expert! Thus rather than requiring a new expert to amass their own reputation, there might be a way to let the community notice an expert and thereby increase that person's clout in Area 51.

Is there a good way to make the suggestion in the last paragraph practical? Are there any obvious problems with it? Is this something that Area 51 would/should consider doing? Is there another better way for Area 51 to value real-life reputation?

For a concrete example, Terry Tao was an early adopter of Math Overflow and is clearly of more value to the site (both in terms of activity and in terms of making the site credible in the math community) than a dozen high "reputation" prior SE users. Furthermore, you don't need to be an expert in math research to identify that Tao is a likely huge asset to the site (just read his wikipedia page and note that he's already a big supporter of math on the internet both via his blog and his position on the arXiv advisory committee). Although this is an extreme case, it's also pretty easy to check that people with not quite as stellar credentials are nonetheless credible experts in the field (for example, you can see if the person is a professor at a research university).

Answer 1

Interesting idea. To scale such an ability, perhaps SE could look to an external service to try to extrapolate a user's reputation that way. Unfortunately, I don't know of any service that can automatically give a reliable reputation value to a given person, but here are some ideas:

• Twitter
  • Pros: A number of new startups are trying to determine a Twitter user's reputation nowadays. Perhaps one of them could provide a reputation value to SE.
  • Cons: There are lots of easy ways to game one's reputation on Twitter.
• Wikipedia
  • Pros: Having a relatively quick-reacting human-moderated resource like this means any Joe Schmoe can't create an entry for him/herself. Those who do have an entry probably have a fairly high reputation within their respective fields.
  • Cons: How the heck would you get a reputation value from Wikipedia??
• Google Search
  • Pros: If a person's name is referenced in a number of respected publications and blogs, there's a fair chance they are well-respected.
  • Cons: Or, the person could be referenced because they are an idiot. Also, how would you get a reputation value from this?
• Human moderated
  • Pros: The owner of the SE site manually gives a known expert a high reputation score.
  • Cons: This would have to be a manual process. Also, what if the site owner disagrees with that person's level of expertise, as the community sees it?

Okay, so I didn't have any great ideas. Maybe someone can take this as a jumping point for an even better suggestion. This is an interesting idea though.

Answer 2

See Joel's answer here.

How should "critical mass" be calculated?

Personally, I believe that reputation is leaky... it leaks from place to place. That's because the true experts are leaders in their field, which means they have followers. And they can mobilize their followers, and that is far more important than just sitting around answering questions themselves.

For example, imagine someone who is the world expert on ponies, but has not participated in Stack Overflow before. She doesn't have a lot of reputation on our system yet, so we don't know that she is an expert.

But by virtue of being an expert, she has a following. Somewhere. On her blog, twitter, email newsletter, or when she speaks at pony industry conferences. And through this influence, she can mobilize her tribe to support the new Stack Exchange site. This mass of people will be far more valuable as she is... just like the people who I brought to Stack Overflow from Joel on Software were far more valuable than I was... and indeed, our algorithm recognizes this, giving little credit for "expertese" but a lot of credit if she can mobilize a lot of people.

Answer 3

Here's a concrete proposal. During the "define" stage people decide on some objective measures of what constitutes an expert (e.g. for MO, grad student = 1 expert point, PhD = 2 expert points, Tenured prof = 3 expert points). Then during the commit phase people with enough on-site reputation can identify experts who have committed. To exit commit phase you would need some function of the number of experts recruited and the number of high reputation SE users.