15

I was wondering whether there's any evidence to be had as to whether it's harder to write good questions or good answers.

For example, from the data available in SEDE or previous surveys.

I suppose "good" should be measured by score/age, but reasonable alternatives are acceptable.

  • 5
    It's a well-known fact that people do not vote as often on questions, so I'm not sure it's a reasonable criteria to go for for a fair comparison, but I don't have any ideas if any set of criteria would fit. – M.A.R. Nov 6 at 14:02
  • @M.A.R. the goodness of questions and answers could be scaled appropriately – Stop Harming Monica Nov 6 at 14:03
  • 3
    Define what is a "good question" and a "good answer", and you might have your answer. – Mari-Lou A Nov 6 at 14:10
  • @Mari-LouA I gave a definition... – Stop Harming Monica Nov 6 at 14:11
  • 8
    The score? I have seen very simple questions earning massive upvotes because of a great answer posted. I have seen great questions that failed to attract a single answer because the expertise needed to post an answer was way above their pay grade. When a question hits HNQ it can attract 50 or more upvotes. Does it mean it's "good" or it's popular? – Mari-Lou A Nov 6 at 14:12
  • @Mari-LouA score over time, normalised – Stop Harming Monica Nov 6 at 14:12
  • Or just use a binary count of non-closed, non-deleted, postitive-score. – Stop Harming Monica Nov 6 at 14:17
  • 3
    Does it matter which is “harder” to do or which we need to encourage more of (good questions or answers)? – ColleenV Nov 6 at 14:23
  • 1
    I’m aware of the context, which is what prompted my comment. I think that focusing on the effort isn’t as helpful as focusing on what we need more of and what we might be able to do to make that happen. – ColleenV Nov 6 at 14:36
  • The question is too subjective, does harder mean "better"? A very interesting question could be easy for its author to write. A difficult question on Mathematica that shows effort, research, and careful planning could be very hard to set up but the right answer could be posted in minutes. – Mari-Lou A Nov 6 at 14:59
  • @Mari-LouA no "harder" means "requires greater effort to produce". Why would you think I meant "better"? – Stop Harming Monica Nov 6 at 15:05
  • 1
    @Mari-LouA I'm not asking about whether being hard to write makes something good – Stop Harming Monica Nov 6 at 15:19
  • 3
    It depends upon site, question, and answer being compared - and how to compare so many things so different, and by which standard. – Rob Nov 6 at 17:25
  • 2
    Many of these comments would be good answers (and should be, instead of answering in comments). – V2Blast Nov 7 at 7:07
18

Well, for starters, Good Answer is awarded 7285 times but Good Question only 5710 times (on Meta Stack Exchange). On Stack Overflow, there are about twice as many Good Answer as Good Question badges. This SEDE query shows that that is the general case.

Does that mean it's easier to write good answers than good questions? I doubt it; users are (on average) less inclined to vote on questions. This alone makes it very hard to compare question and answer quality, at least with quantitative data.

  • 12
    I would exclude Metas, as they voting patterns are completely different – Stop Harming Monica Nov 6 at 14:04
  • 5
    but there are also more answers than questions, so the numbers should be normalised by the total number of questions and answers respectively – jknappen - Reinstate Monica Nov 6 at 14:24
  • @jknappen-ReinstateMonica although, maybe that's because it's easier to write good answers... – Stop Harming Monica Nov 6 at 14:25
  • 2
    @Glorfindel that'll learn ya – Stop Harming Monica Nov 6 at 14:26
  • 2
    What's the most pressing problem on SO, the number of correct answers or the lack of stimulating questions? – Mari-Lou A Nov 6 at 14:43
  • 6
    It's much easier to write a good answer than it is a good question. A good answer just requires knowledge of the subject matter; a good question always starts from a position of relative ignorance. – Robert Harvey Nov 6 at 14:59
  • 2
    One could argue that the reason people are less likely to vote on questions is because there are fewer of them that are worth up-voting, therefore it’s more difficult to write a question that will get up-voted than it is to write an answer that will get up-voted. There’s no way to get a reasonable measure of a subjective thing like “quality” from a pool of uncorrelated voters. If I had to guess, answers get votes for simply being correct, not necessarily good, and questions only get votes for being “good”. – ColleenV Nov 6 at 20:36
  • @RobertHarvey I don't see how that makes a good question better than a good answer? It seems like you're implying that being relatively ignorant is better than having knowledge of a subject matter... – JMac Nov 7 at 14:41
  • @RobertHarvey some of my most upvoted questions were when I know the answer, but it was not explained well at the time anywhere on the Web and many people were asking themselves the same question, for example stackoverflow.com/questions/24011575/… – Ian Ringrose Nov 7 at 15:22
  • @IanRingrose: So you're making my point, right? Your best questions were the ones you already knew the answer to. – Robert Harvey Nov 7 at 15:52
  • I think also that we’re looking at posts as if they aren’t constantly being improved over time. I wonder if answers have more edits and therefore more visibility than questions. Do many question authors disengage once they get their question answered? – ColleenV Nov 7 at 16:23
12

Aside from there potentially being more answers than questions, as well as people more likely to vote on answers, answers seem to be easier.

Case in point, extremely well-read questions like this have caused a single Great Question Badge, but a multitude (22) of Great Answer Badges. This is of course an extreme example, but I think the point stands.

But I think it's about what you're posting about - if you're writing a question, you need to establish all your context aside from the subject of the site, while when writing an answer it's in the already established context of the question.

In my opinion, that makes writing a good answer easy, because:

  • For an answer, you can often just address the question point by point.
  • For a question, you need to come up with all important things instead, as well as related side-concepts.

Of course, the best answers often also reference related side concepts. But I think it's easier.

  • Could you produce some numbers for that? – Stop Harming Monica Nov 6 at 15:54
  • @StopHarmingMonica this number? – Gloweye Nov 6 at 15:58
  • so far you've provided opinions and a datum, not data. – Stop Harming Monica Nov 6 at 15:59
  • Glofindel already gave the available data, but I think it's not really quantifiable how "hard" something is to do, when it is inherently subjective. I agree with the on-hold that it IS primarily an opinion-based question, but I just don't think that's reason for closure.. – Gloweye Nov 6 at 16:02
  • It's easy to miss the link, darkish blue on white is similar to black. – Mari-Lou A Nov 8 at 10:28
8

I would argue that it is ‘harder’ to write consistently well-received questions than answers. This is probably why the SE development team created the Socratic badge. The following are the stats for the number of socratic badges earned on 20 different sites.

MetaSE is excluded because many of the "questions" are in fact bug reports, feature requests, or requests for support but for the curious here's the link.

Socratic
Ask a well-received question on 100 separate days, and maintain a positive question record. This badge can be awarded multiple times.

  1. Academia
    Awarded 1 times.

  2. Arqade
    Awarded 55 times.

  3. Ask Different
    Awarded 7 times.

  4. Biology
    Awarded 2 times.

  5. Christianity
    Awarded 4 times.

  6. Cross Validated
    Awarded 9 times.

  7. EL&U
    Awarded 18 times.

  8. French Language
    Awarded 8 times.

  9. GameDevelopment
    Awarded 0 times.

  10. History
    Awarded 5 times.

  11. Japanese Language
    Awarded 12 times.

  12. Mathematics
    Awarded 428 times.

  13. mathoverflow
    Awarded 77 times.

  14. Puzzling
    Awarded 9 times.

  15. stackoverflow
    Awarded 4049 times. (Yes, it's 4K)

  16. SuperUser
    Awarded 20 times.

  17. Worldbuilding
    Awarded 6 times.

  18. The Workplace
    Awarded 0 times

  19. Travel
    Awarded 17 times.

  20. User Experience
    Awarded 2 times.

enter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description hereenter image description here enter image description here enter image description here

  • 1
    "Ask a well-received question... Academia: Awarded 1 time.": irony :-) – Raedwald Nov 8 at 10:50
  • @Raedwald Well-known discrepancy, it's been mentioned on Meta several times already :) – Mari-Lou A Nov 8 at 10:54
  • ELL has 42 awarded ell.stackexchange.com/help/badges/111/socratic with 31 questions per day and ELU has 18 awarded with 34 a day. There are significant variances across sites, and I don’t think there is a single answer for the entire network. And there are a lot of mediocre questions in those well-received ones that were upvoted because they were a question a lot of people had. – ColleenV Nov 8 at 12:17
  • @ColleenV did you get the latest stats off SE? Because I remember EL&U was closer to 45 questions per day. Yes, there are all sorts of divergences, for example on ELL there is one user who has posted a total of 900 questions (excluding deleted one) but "only" earned 1 Socratic badge. I wouldn't be happy seeing them earning a huge chunk of rep. – Mari-Lou A Nov 8 at 12:59
  • @ColleenV In the answer, I said "consistently well-received questions" Not a word about them being "good" or not. – Mari-Lou A Nov 8 at 13:03
  • I'm not arguing with you - I got the numbers from stackexchange.com/sites#traffic. I'm just pointing out that asking in general "which is harder, asking a well-received question or answer" is ignoring a lot of variables. I think looking at the Socratic badges was a good idea. I think the badges earned also illustrate that it depends a lot on your audience and not so much just the traffic to the site. It is quite strange that answer badges are for straight score and out-scoring other answers but question badges require "consistently well-received". – ColleenV Nov 8 at 13:20
  • Not arguing, just talking. – Mari-Lou A Nov 8 at 15:50
  • 3
    I think the illuminator badge is also interesting. Edit and answer 500 questions (both actions within 12 hours, answer score > 0). I think a lot of questions need help from the experts in the field to highlight why they are interesting and worthwhile questions. If they get that help, they may end up being well-received when they might not have been without that help. – ColleenV Nov 8 at 17:29
  • What is the story behind these statistics for different sites? How must they be interpreted? – Sextus Empiricus Nov 8 at 18:28
8

We can look at the distribution of scores for questions and answers.

The image below shows the results for the websites StackOverflow and CrossValidated (statistics).

Note both the axis

  • The y-axis is log scale
  • The x-axis groups scores into categories of growing size further away from zero 0, 1, 2-3, 4-7, 8-15, etc. (this is done because constant bin size may end up with zero values)

distribution

We can see effects mostly in four regions:

  • Negative score

    Especially on StackOverflow questions receive negative scores relatively more often than answers. On CrossValidated the difference is not so large (this relates to a larger amount of deleting negative score posts, see the image below which is based on statistics that include deleted posts).

  • zero score

    Questions have more often a score 0 than answers.

    percentage posts with zero score
    
                    SO      CV
     answers        39%     18%
     questions      46%     27%
    
  • Moderately positive scores >1

    Answers receive more often a positive score >1 than questions.

    But the distribution among positive scores is very similar for questions and answers. (this can be seen by the curve on the logarithmic scale plot being higher for answers, but just with a more or less constant factor)

  • Extremely high positive scores

    The distribution for positive scores are very much the same. We can compare them a bit more easily when we take the ratio of the two values. This is done in the image below. Now we can see that for the very high scores >100 for cross validated or >500 for Stack Overflow, the ratio changes a bit in favor to relatively more questions (in the case of Cross Validated the ratio turns even >1 which means the density of extreme scores is higher among questions than among answers, even when you do not correct for the many deleted and negative scoring questions).


Conclusion:

It is more easy to write an answer with positive score than a question with positive score.

But among questions and answers with positive score there is not much difference between questions and answers in how many will be good (very high positive score).

Only for the extremely high scores it may be easier to obtain them with a question than with an answer.


The image below is the same script but including the deleted posts. This increases the number of negative score posts.

withDeleted


The image below shows the ratio of the distribution for questions and answers. This makes it possible to better compare the curves in the other images which are very close to each other.

likelihood ratio

  • @user644520 I have added images for both cases (with and without deleted posts). – Sextus Empiricus Nov 8 at 18:15
7

It's comparing apples to oranges. They're both fruit (posts), but they differ in taste, texture, etc (requirements, structure, etc).

Writing a good question is different from writing a good answer. As anecdotal evidence, Jon Skeet wrote a separate blog post about writing answers and about writing questions. If you compare these posts, you'll see that there really are differences in what's required of both.

Also, is is pointed out by user Rob in their comment, it can vary per Stack Exchange site.

I think this question was inspired by the rumor, that Stack Overflow Inc wants to increase the amount of reputation points for question upvotes. Ultimately, our purpose is to be a repository of knowledge. The gamification is merely a means to that end. The amount of points you get for any action, should be proportional to how well that action contributes to the goal.

  • 2
    "The amount of points you get for any action, should be proportional to how well that action contributes to the goal." - Then the question becomes: Will increasing the vote score for questions help us towards that goal? – House- 'Reinstate Monica' -man Nov 7 at 14:49
5

I find it more difficult to write a good question.

When I don't know the answer, I often don't know how to ask for it. I lack the proper terminology, and I often even don't know what field my question belongs in. I find it much more difficult to make my ignorance understood than to answer a question that I know the answer to.

I often notice that for many questions it takes some clarifying comments and first answers for the asker to realize that they have been seeking a solution to their problem in the wrong direction or for the answerers to understand what problem the asker is attempting to address.


Note.

All of you who are doing statistics on votes or badges need to take all the questions into account that

  • are duplicates,
  • get closed and disappear from the site (and the statistics),
  • are asked on the wrong site and need to be migrated,
  • or don't get posted at all, because the asker finds the answer while working on phrasing their question or because they get discouraged with the requirements of asking.

Those are questions, too, and they bear witness to how difficult it is to ask a good question.

4

With a good answer you're building on

  • a specific, hopefully well scoped problem
  • years of experience, sometimes irrelevant
  • The entirety of the internet

With a good question you're basically

  • laying out your ignorance as best as you can
  • in the hopes that with a roadmap of your failures
  • and maybe you can find and leave enough bread crumbs for the people in your path to find what you need.
  • You may have specific requirements, and an end goal, but everything in between is hazy.

Good answers, IMO are certainly easier.

  • 1
    This appears to be an SO and SuperUser centric type of answer. It suggests that askers are basically ignorant fuzzy-minded helpless beings while answerers are omniscient contributers. I find this answer to be incredibly offensive to users such as myself who have earned the Socratic badge. Do you know how often I come across incorrect answers on language sites? Aplenty! P.S How come you changed questions to answers in the last line? – Mari-Lou A Nov 8 at 7:52
  • with a good question you're basically laying out your ignorance as best as you can nice, really encouraging to hear that. – Mari-Lou A Nov 8 at 8:33
  • 2
    Cause I made a mistake when writing it - I meant answers, not questions. And I'm mostly speaking out of my own experience, and the occasional really fun rabbitholes. And I still think a good question is built around a vacuum. – Journeyman Geek Nov 8 at 8:43
  • Edited for clarity: If a user is basically dumb, which seems to be implied here, rest assured that user does not ask interesting or good questions. Successful discoverers in medicine, inventors, developers, scientists, astrophysicists, mathematicians etc. searched the answers to their own burning questions. They are/were not ignorant (in the sense of knowing next to nothing as seems to be implied in the answer) nor are/were they building around a "vacuum" whatever that means. – Mari-Lou A Nov 8 at 11:21
  • 1
    @Mari-LouA I have given a very similar answer from my experience on both SO and non-programming sites such as Psychology and Writers. I don't see this as an answer specific to SO. – I am not the way you speak Nov 8 at 19:25
3

Another aspect that hasn't been mentioned so far is that a good answer almost always requires a good question, in order to get recognition.

The dominant factor in determining how many votes an answer will get is the number of views of the question. For my experience there are two main avenues for your answer to get recognition:

  • It is best and/or one of the first answers to a question that manages to become a highly ranked result in a popular search engine.
  • It is best and/or one of the first answers that becomes popular in the "Hot Network Questions" list.

Being able to write a very good answer quickly is important, but unless you are able to quickly identify the questions that will get popular due to search engines or the HNQ, your answer is unlikely to earn more than a few votes.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .