(Prologue: I was told this question was supposed to be here vs. meta.stackexchange.com so I've copied it from over there.)

While taking a look at the current "Answer Ratio" for WordPress Answers it occurs to me to ask if this is really an appropriate metric on which to judge a site?

As you can see I do answer a lot of questions over there but many times I don't answer a question if someone else has already given a really good answer, especially if the asker has selected one of those answers as the correct one. To add another answer would make it look like I was just "piling on" and fishing for reputation points.

Metrics for WordPress Answers on Oct 21, 2010

I'd wonder if it it wouldn't be better to have a more complex algorithm here, one that ignored questions that have had a "best answer" marked, or even that factors in positively all questions with a best answer.

And if a question has a single answer with lots of up-votes shouldn't that count more than a question with lots of answers but few if any up-votes?

I'd also wonder if down-votes shouldn't also be considered, but negatively?

Not sure what the best formula to propose would be but in summary I think this metric could stand revisions. Thoughts?


Note that all of these metrics are guidelines, not absolute rules. "Worrying" does not mean that a site will fail the beta, it just means that that aspect needs some extra scrutiny. "Okay" means what it says; the site is doing fine, but there might be room for improvement. If the answer ratio is in the "Okay" range, then you probably don't need to worry about it.

That said, a healthy question and answer site will have a good number of questions with multiple answers. Not all questions will, some will just have one answer and that's sufficient; but if all of your questions are so easy to answer that a single, quick answer is all they need, then your question pool probably isn't very good.

Remember, even for questions that have definitive "right" or "wrong" answers, and already have a "right" answer posted, that answer might not explain the concept very well, or might be a simple link with no explanation of its own. It is frequently useful to post another answer, with more information, a better explanation, or something of the sort. Not everyone will get the answer right, or will nail the answer with a great explanation the first time; if people are avoiding posting new answers just because someone's already posted one, then you might not be getting the highest quality answers.

So, if all questions are getting answered with exactly one answer, something might be off. The questions might be too easy, too Googlable already; a few of these questions are OK, but the majority being these is not. Or maybe people posting answers are being too shy of posting a better answer than the one that's already there, even if at its core the answer is the same. Or it might indicate that you have too few experts; there's a handful that are answering all of the questions, but not enough that you get two people writing answers at once.

It looks to me like the latter might be the case for you; there are a few experts, but very few users with even 200 rep. With that few experts, answering that many questions, the answers will be spread a bit thin, and you won't get the benefit of multiple experts looking at, thinking about, and discussing answers to someone's problem.

So, I'd say try to get more people to answer questions. Try to vote answers up more often, to encourage those who are answering questions well by giving them upvotes and rep; and when answers aren't good enough, leave more comments asking for clarification, or edit questions to clarify and hopefully elicit better answers.

  • @Brian Campbell - Thanks for your comments. I tend to write long and complete answers (like bit.ly/flickr-for-daily-photo-blog) but if I do there's less likelihood someone will backfill my answer with more info. Frankly I love it when I need a question answered for people to be so thorough so I assume people like my answers too, but I don't want to damage the site by having my long answers result in a "Worrying" answer ratio. So it worries me that a metric might not accurately reflect the value of a site which is why I asked about it. – MikeSchinkel Oct 23 '10 at 21:41
  • @MikeSchinkel I am definitely a fan of the big, complete answers. But sometimes, someone just needs a short, quick answer. Big complete answers take time to write; so people should probably also be doing short, quick answers while people like you and I are writing our big complete ones. And some people should write medium ones, too. In the question you link to, there are two answers, one of which came a day earlier. I don't think that the big complete answers are what is hurting the site; I think it's the lack of many other people also trying to answer questions, with differing answer styles. – Brian Campbell Oct 24 '10 at 3:26
  • That said, I've taken a look at your community again, and seen several of the one-answer questions, in which that one answer is timely, and appears to solve the question at hand. So, I think that there are certainly cases in which this metric is not perfect. As I said at the beginning, these are guidelines, not hard and fast rules, and your answer ratio is in the "Okay" range, not the "Worrying" range, so you probably don't need to worry about it. – Brian Campbell Oct 24 '10 at 3:35
  • Thanks for the detailed follow up! – MikeSchinkel Nov 1 '10 at 10:22

I don't know which way to jump on your questions, but I'd like to observe that topics for which there is exactly one, really definitive, all-right-all-the-time answer to most questions may not be the best candidates for Stack Exchange. Wouldn't they be better served by an encyclopedic FAQ?

The other way that metric might be telling is about how fast questions are getting answered. If you're getting multiple response when there is just one right answer it suggests people typing in parallel, which suggests fast responses.

I could still be talked out of this, but I think it is a reasonable metric, though perhaps one with a peak in the preference landscape: less than one answer is bad, two or three is good, dozens are also bad.

  • I thought it was encouraged behavior to delete your answer if you see that someone else posted a better one. By measuring the number of answers, you're discouraging people from deleting redundant answer (or from not posting them if they notice someone else answered the same thing while they were typing). – sepp2k Oct 23 '10 at 12:16
  • @sepp2k: Better is one thing, or even wholly equivalent. I delete mine if it is late or once another has commanding lead in votes. But "easy" questions on Stack Overflow often attract a collection of mostly equivalent answers that come at the problem from different angles. I don't delete mine in that case. – dmckee --- ex-moderator kitten Oct 23 '10 at 15:32
  • 1
    @dmckee - Thanks for taking the effort. I don't buy the first argument especially since StackExchange wants to encourage objective questions; if they are objective then often there is really only one answer. OTOH I see your point about "typing in parallel" indicating fast answers but I wonder if the other metrics don't cover it too. And @sepp2k has a point about answer deletion. So, hmm... – MikeSchinkel Oct 23 '10 at 21:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .