19

I noticed the new community eval queue on Chem. I like the concept, but I feel that is should allow for separate voting for the Q and As:

enter image description here

Currently, the categories talk of the combined quality of the Q & A (Well, all except Satisfactory which seems to talk about answers only). Most of the time, I've come across OK-ish questions with good answers (or a good collective answer pool), and sometimes vice versa. It's pretty hard to vote for these. Giving another set of voting buttons for the collective answer pool may make this more intuitive (as well as helping pinpoint which part of the site needs improvement)

Also, can we have eval-specific comments? In our evaluation, I found the comments much more helpful than the vote counts. Yes, I know the data is tabulated in the end, but we could add a link to a list of anonymized eval-comments per post. Even better, make the list public -- the old evals didn't have any privacy regarding comments, and that was a good thing.

Just an afterthought: In the mod interface, an extra column "Net score" would be nice (Each "Excellent" is +1, "Satisfactory" is 0, "Needs Improvement" is -1).. The current "total" column just shows how many people voted.

Improve this question
8
+500

Just an afterthought: In the mod interface, an extra column "Net score" would be nice (Each "Excellent" is +1, "Satisfactory" is 0, "Needs Improvement" is -1).. The current "total" column just shows how many people voted.

This has been implemented for a while now basically exactly as you describe. The "Total" column is still there, but there's a net score beside it as well.

Also, can we have eval-specific comments?

Unfortunately we can't do comments within the review interface short of posting them all directly on the questions, which would be ... suboptimal. At best, it would add noise for anyone visiting later. At worst, some feelings could get majorly hurt.

We are now creating an automatic meta post (example) when an evaluation starts that can be used as a discussion area. Feel free to post your thoughts as answers there, either addressing specific questions or broader patterns, good or bad.

Improve this answer
  • And it'll automatically have a separate "answer" post for each question under review? That way it's not crammed in all under the "question". – Troyen Feb 7 '13 at 3:58
  • @Troyen I was planning on keeping it more free-form. Anyone can post an answer with whatever they'd like to discuss. Prepopulating answers tends to have the unfortunate side-effect of discouraging additional answers. – Adam Lear♦ Feb 7 '13 at 4:09
  • I suppose that makes sense. – Troyen Feb 7 '13 at 6:00
  • 2
    @AnnaLear: To me it makes more sense if each eval post has a meta answer, directly linked to from the queue. Explicitly encourage extra answers in the meta question. Not prepopulating answers has the unfortunate effect of people going "too much effort to write an answer, forget it" (that would be similar to my reaction -- if I have to write ten comments, it's fine. A meta answer -- I get lazy, though I'll end up posting anyway) – Manishearth Feb 7 '13 at 7:25
  • @Manishearth Unfortunately, the other concern with prepopulating posts is that it adds confusion to the process - if I vote on the meta answer, am I participating in the evaluation? Finding additional non-automated answers when there are already 10 on the page is also difficult. We've seen this with the manual evals we've posted on meta before. It's likely not a perfect solution, but I want to keep it as simple as possible at least to start. – Adam Lear♦ Feb 7 '13 at 16:35
  • @AnnaLear: Alrighty then. As long as the meta post tries hard to ask folks to post their opinions, and is highly visible in the queue :) – Manishearth Feb 7 '13 at 17:27
16

I also found the comments on the meta-evals more interesting than the voting. Particularly when folks linked to external resources for comparison - and in my own evals, I often thought an answer pretty decent until I compared it to those found elsewhere. Since one of the primary goals here is to see how your site stacks up against The Greater Internet, that seems like rather a critical part.

I don't really see the point in breaking out question vs answer votes. I mean... That already exists - just visit the question and vote on the answers! The goal of this review is to evaluate the combined Q&A as a whole - if the information is available (or missing, or poorly presented) anywhere on the page, that counts. Also this creates more work, more choices, and more ambiguity in what is already a slightly-too-complicated process, IMHO.

Net Score sounds useful, if only because it doesn't count the "Meh" "satisfactory" votes.

Improve this answer
  • You're right about the Q&A -- a commentary box is much more informative and useful :) – Manishearth Nov 20 '12 at 15:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .