What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 133 Stack Exchange communities.

Which questions need Community Evaluation? What's the criteria to be added to the review queue for Community Evaluation?

What is done if a review gets the result Needs Improvement or Satisfactory?

share|improve this question

2 Answers 2

up vote 6 down vote accepted

If memory serves me (which it might not, but forgive me - it's New Years and I'll have to look up the specifics to make sure)... the questions right now are picked like this: up to 10 questions that are open, with at least one answer, and asked between 30 and 40 days ago.

We are currently working on some improvements to the guidance in the review queue itself as well on meta, but the gist of the point of these site evaluations is that once in a while it helps for a community to look at its site and see how its doing. Think of it as a periodic sanity check and a way to make sure that the site's still doing well. Are the questions of high quality? Are the answers better than what someone can easily find through Google?

What we see a lot on various sites is that someone will bring up a specific question for review, but it's incredibly rare for someone to initiate a broader discussion. That is what these evaluations are supposed to facilitate.

This review queue appears for one week on a schedule (60 days after the start of the public beta; every 90 days after that), and it's an opportunity for the users to take a look at a sample of the questions and see if the site really is helping "make the Internet better" or if it falls short in some ways. Naturally, if it's the latter, the next step would be to get together and figure out how to correct the course.

For beta sites, they will, along with other metrics, also help us on the community team to gauge whether the site is ready to be self-sustaining and helpful for a long time. For graduated sites, it's a check for us to see whether we need to offer some advice or if things are going along smoothly. In either case, these evaluations are intended to help the site's community see how they're doing.

Right now this queue isn't deployed everywhere (see above: improvements are needed), but we are planning to roll them out on all sites soon. There's also not much of a link between the /review queue and a discussion... and that's the next thing on my list to fix, but in the meantime, if you see something your site does really well or really poorly, do feel free to open a discussion on that site's meta.

See also: Strategies for handling beta community quality evaluations for some background.

share|improve this answer
    
Out of curiosity, do the questions look the same to each reviewer? In other words, if a new answer appears or a post is edited, do new reviewers see the new content? (Ideally, the answer would be that every reviewer sees the same content. ;-) –  Jon Ericson Feb 4 '13 at 17:28
    
@JonEricson The questions in the queue will update over time if they're changed. This isn't necessarily bad - for example, if the question is edited and improved or gets a fantastic answer, future reviews during the evaluation will reflect that and the results should still come out fairly accurate. What we've seen in the past is that most questions under evaluation aren't modified, but we'll keep an eye on this with this queue and see if that's still the case. –  Anna Lear Feb 4 '13 at 18:06
    
Another question (more urgent this time ;-): why was I given only 8 questions, but I see others have evaluated 9 or 10? I assumed only 8 questions fit the criteria. Did more get added? How can I evaluate the other two questions? (My queue is empty.) –  Jon Ericson Feb 6 '13 at 17:49
1  
@JonEricson Looks like you asked two of the questions that were chosen for review. We don't ask folks to review their own. –  Anna Lear Feb 6 '13 at 18:00

These are pretty much picked randomly (IIRC, certain types of questions are excluded, obviously including closed ones)

The purpose of this is not to review questions, it is to review the site overall. The results of the eval (shown to mods) describe the overall post quality of the site. It is a streamlined version of this.

That being said, mods may later improve/close posts which have too many "needs improvement" votes. I did so when chem had their eval (the new eval, not the old one), though I waited till the eval was over to prevent inconsistencies in the results.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .