What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 122 Stack Exchange communities.

Question: How can difficult answers be justly rewarded, as typically, they do not earn nearly as many points as easy answers.

In order to cast a (merited) up vote, I think the following are necessary:

  • The question must be interesting enough that I view it
  • I must be able to understand the question
  • I must read and understand the answer – and determine if it correctly answers the question
  • I must decide that the answer conveys the solution well

The first and third points are notable I think: - Easy questions are heavily viewed, do not require much effort to understand, and answers are often short, minimally technical, and presumably easy to judge for correctness. - Difficult questions tend to focus on the details and require much more scrutiny in reading. Both the question and any answer tend to be longer and often at least moderately technical.

As a result of the above, answers to difficult questions – which require more effort to compose – usually earn votes that are disproportionately few in number. Since the intent of this site is presumably more toward solving the difficult questions, this discrepancy does not seem to be in line with the objectives of the site. (I will grant that difficult questions are much more interesting, and are quite likely to get answered regardless of votes.)

Factoring ‘difficulty’ into the weight of votes provides a bit of an additional reward for valid answers to difficult questions. One possible self-regulating measure of difficulty is the relation between the time that a question goes without up voted answers and the number of views – a question that accumulates many views, but goes unanswered (without being closed, of course) suggests that the question is not easily answered (or it is too vague to be answered – but a valid answer in this case may be more notable anyway).

I grant that while interesting, the above is not an ideal solution. Hopefully though, a viable approach to the problem can be found.

share|improve this question
add comment

migrated from meta.serverfault.com Jan 8 '12 at 20:30

This question came from our discussion, support, and feature requests site for system administrators and IT professionals.

1 Answer

up vote 10 down vote accepted

You're missing a fifth voting point:

  • Must be clearly better than at least one other answer

Three equal answers will sit there at zero votes for quite some time. Or 1 vote to all. When a 4th answer comes in that is wrong is some way, those other three good answers will start rocketing up in rep.

This is possibly a SF-ism, I'm not sure if that dynamic holds on other SE sites.

Since the intent of this site is presumably more toward solving the difficult questions, this discrepancy does not seem to be in line with the objectives of the site.

Answering the difficult questions requires building enough of a community of experts in that field to handle that kind of question. But simple population dynamics demonstrates that there are vastly more apprentice or journeyman level people out there than masters, and as a result we'll get a lot more apprentice/journeyman level questions than master-level.

Master-level questions are interesting, and tend to be full of nuance. The few such questions we get tend to be well upvoted simply because our electorate knows a good question when they see it, and want to encourage this. Unfortunately, many of these questions linger unanswered for some time. In part this is because there is insufficient master-level community in that topic, but it's also because it's hard to ask a master-level question in the format of StackExchange without a lot of back-and-forth commentary to refine the question down to something answerable.

Where ServerFault shows its strength is in the middle. Askers know enough about what they're doing to know how to ask the questions, but not so much that the only questions they have left are obscure edge-cases. That's a population we have critical mass in, and we handle these well.


Back to your question, about how to reward difficult answers that don't get high votes.

I'll answer with a few observations of my own:

  • People who answer hard questions begin to figure out who the other users who know WTF they're doing in that field, simply by seeing who the other answerers are on those questions.
  • Answerers of such questions earn unscored-reputation among other SF users as "someone who knows WTF they're talking about".
  • People who answer the hard things are usually pretty good at also answering the middling questions that get a lot more attention.
  • People with a reputation for knowing WTF they're talking about earn upvotes faster than User123456 who would post the same content.

So when you get someone who can answer the hard questions weighing in on a middling question in that field, they tend to earn more rep on those answers because they've proven their worth. This is all about how the system of actual reputation works within our community, not the codified one the SE engineers baked into it.

So no, the hard questions tend to not get much scored reputation. But the real reputation such answers generate is rather significant, especially on a site that values deep knowledge. That's it's own reward right there.

share|improve this answer
    
This is possibly a SF-ism - it is not. There actually have been psychological studies about how the presence of an additional, clearly inferior choice improves the perceived value of the other choices available that can easily be related and compared to the inferior one. –  syneticon-dj Jan 9 '12 at 0:03
    
Frankly, I'd favour 3 upvotes to no votes (although, I expect that multiple answers are uncommon for difficult questions) - I wonder which is more common. Adding a poor choice to the mix is a common marketing technique to steer consumers towards a more expensive 'better value' choice. On the SE sites it is amplified because the down vote costs rep, while the up votes don't. I fully agree with your analysis of question types, and that it is the middle ground that SF excels at. I do agree about your point regarding 'implicit reputation earned' - however, I am not certain it is a good thing. –  cyberx86 Jan 9 '12 at 5:51
    
Arguably, it is a bias in the voting toward the higher rep users (more higher rep users vote - and they usually know (of) each other; the leading answer scores even more votes; and the user with the highest rep doesn't always have the best answer). As you point out, it doesn't treat the new user equally - which means that the new users may be somewhat deterred from the difficult questions, even if it within their capabilities to answer. That said, every system has its checks and balances, and since one isn't built in for this purpose, I imagine this version will have to do. Thanks. –  cyberx86 Jan 9 '12 at 5:51
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .