22

For years, people have been able to spend their hard-earned imaginary Internet points to add "bounties" to questions in order to attract more attention to them. (Because, really, "attention" is the only thing that you can be reasonably sure a bounty will give you.)

But does this extra attention translate into better results?

Granted, any individual question may not be really answerable or is still boring even with the extra carrot. And a question might be so interesting that it would have gotten a ton of (good) answers even without the bounty.

Do questions that have bounties get more/better answers—in aggregate—than questions that never had a bounty? Does this vary by site? If so, do bounties seem more effective at a certain site "size"?

Or is it really just a wash, and that all anyone ever gets for their bounty is extra attention?

  • 5
    I thought bounties were about getting attention. That certainly works. Getting good answers is a nice result of that attention, provided the question is actually answerable and of reasonable quality. – Martijn Pieters Aug 28 '14 at 19:47
  • @Martijn: Yes, granted, but one hopes that increased attention will, generally, result in more and/or better answers. – Fish Below the Ice Aug 28 '14 at 19:48
  • Scott Pilgrim vs the World had a lot of hype for it. But it didn't smash the box office. Bounties are marketing campaigns. On turds, or ambergris. – random Aug 28 '14 at 19:50
  • 1
    Trying to estimate the effectiveness of bounties from the available observational data is hopeless. Questions that get bounties are not representative of questions in general. One would need a controlled experiment: set bounties on randomly chosen unanswered questions, and select a control group as well. – user259867 Aug 28 '14 at 20:04
  • 2
  • @Thursday: Well, I wasn't suggesting double-blind trials or anything. I guess I was just hoping that someone with a more statistical bent (than me) might look at the Data Explorer with some well-crafted queries. – Fish Below the Ice Aug 28 '14 at 20:23
  • @Thursday Try doing that on the small sites. I'm not sure you could even do that on mid-sized sites. – fredsbend Aug 28 '14 at 20:49
  • Yes, it does. – Adam Davis Aug 28 '14 at 21:57
  • @AdamDavis Well, when you drop 2000 rep like it's free ... just saying. – fredsbend Aug 28 '14 at 22:03
20
+100

The bounty system does bring more views to bountied questions. On average, Stack Overflow questions get 1455 views. Bountied questions average 2631. It stands to reason that the more people who see a question, the better the odds it will get a quality answer.

To test that, I divided all answers into three groups:

  1. All answers whether their question has been bountied or not (the control),
  2. Answers that arrived before the bounty (before), and
  3. Answers that arrived after the bounty (after).

Since questions can have multiple bounties, an answer might belong to all three groups. A slightly cleaner test would only look at questions with just one bounty.

Score is our usual method of determining quality. The query is based on public data and I've included deleted posts.

group    N       avg_score bountied_rate accepted_rate 
-------- ------- --------- ------------- ------------- 
before    112270      18.8         0.122         0.123
after     388660       2.9         0.259         0.243
control 26548476       2.3         0.004         0.306

Despite the extra attention, answers posted after the bounty are barely better scored than answers in general. Existing answers get most of the voting benefit of being featured. It seems entirely possible that this is a result of the reward existing answer bounty reason. But new answers are more than twice as likely (26%) to receive a bounty as existing answers (12%).

The odds an answer will be accepted has more to do with the total number of other answers to the question than it's actual quality. So let's look at questions instead:

group  N         avg_score avg_max_score avg_bounties accept_rate avg_views 
-----  --------- --------- ------------- ------------ ----------- --------- 
bounty    161510       2.8          9.45        1.022       0.658      4536
control 14181661       1.5          3.26          N/A       0.572      2242

Questions that have been bountied get higher scoring answers on average. And the highest scoring answer is also markedly higher on average. But as I mentioned above, that could be a function of the greater number of views. There is an increase in accept rate among questions that have been bountied. The surprising thing is how many questions get multiple bounties:

bounties N        answers avg_score avg_max_score accept_rate avg_views 
-------- -------- ------- --------- ------------- ----------- --------- 
 0       14020151     2         1.5           3.2       0.571      2214
 1         158289     3         2.7           8.3       0.659      4269
 2           2979     4.5       5.8          46.1       0.574     14550     
 3            182     7        12.9         189         0.555     48050     
 4             35    10.5      13.5         166.5       0.714     72940     
 5             15     6.1       7.6          25.6       0.467      5346      
 6              4    12        30.8         417.5       0.25     127626    
 7              3    14.7      36.8         392.3       1         96343     
10              1     5        63.8         170         1         46912     
13              1    40        28.4         430         1         67442     
14              1    97       444.2       26940         1       1135600

I venture to guess that a question really isn't getting better answers after it's first bounty. Every measure of quality except accept rate increases with multiple bounties.

Summary

Bounties certainly increase attention (as designed). There is some indication that a single bounty also increases answer quality, but multiple bounties probably don't increase anything but attention. I'm only looking at Stack Overflow, but spot checks other sites show similar results.

  • 2
    I should raise a bounty on your answer ! – Thomas Ayoub Oct 7 '14 at 9:15
  • When the author goes back to look at a question, the view count goes up. Questions with bounties are questions people care about. I'd bet a significant part of those extra views are people checking on their own questions (yes you do get notified, but people check anyway). – SteveWash Sep 25 '15 at 13:55
  • 1
    @SteveWash: Our viewcounter is a bit more sophisticated than that. It tries to avoid counting the same person more than once. In any case, it's unlikely a single person would push up a question by 1000 or more views. – Jon Ericson Sep 25 '15 at 15:58
  • Is it? I have a bounty on a question right now that's had a total of 32 views. If I leave the page and go back, it tells me I have 33 views. I've done that four times now, and the total has increased by 1 each time. Not likely its getting new views from other users since its been up for days with little traffic. And with only 36(now) views, it doesn't seem the increased "exposure" is working, either. – SteveWash Sep 25 '15 at 21:36
  • "It stands to reason that the more people who see a question, the better the odds it will get a quality answer." That's not a great argument at all -- a thousand people who don't answer, is no better than 10 that don't. A better judge is to see how many Bountied questions have accepted answers, which is something explored here: meta.stackoverflow.com/a/320145/199700 – Django Reinhardt Sep 5 '18 at 9:34
4

Bounties gain attention in their own way on each SE site. Each SE site is a culture of it's own, sometimes with overlap into other SE sites, and sometimes with hardly any overlap at all.

I have offered bounties on a few sites with mixed results:

  • On Christianity.SE I have offered a bounty to reward an already existing answer that was quite superb. I earned this one. I have seen, many times, a bounty bring in equally superb answers as many as 100 days after the question was asked.
  • On MSE, when it was MSO, actually, I offered a bounty on a feature request that was not my question to get the SE staff to notice it. It received no further attention from anyone except a few more upvotes and a worthless answer that basically said "yep, me too," which three people upvoted for some reason.
  • I just offered a bounty on SU for a question that went three weeks with no activity at all. We will see.

On the bigger sites, the quality of interaction seems to decline the bigger it gets. They are a victim of their own success. Hence, I don't think the bounties are too effective for those sites.

On the mid-sized sites, the bounties seem to work very well. There is a high possibility that a regular user can answer very well, but for the few upvotes they might get, they are not really willing to put in the time. The promise of a bounty gives them that extra push.

I have no experience with the small sites, but I suspect the bounties are hit and miss, mostly because there is actually a good chance that none of the regular users can provide a good answer.

See that your only experience is on SO and MSE, I can understand you frustrations.

Here's some examples:

  • You did not offer a bounty on the SU question, or it would be recorded in the post history. This is the history of C.SE question. I don't see a bounty on your MSE question either... – user259867 Aug 28 '14 at 21:04
  • The MSE question was not the one I bountied. I put the bounty on a question that was not mine, specifically to get an answer from SE staff. I cannot find that question, and I'm not sure if it is on MSE or MSO. The result is that it did not get any useful attention. I'm not even 100% sure if the question I bountied was on the same subject. The point is that I resorted to reposting instead of a bounty because the bounty did nothing. – fredsbend Aug 28 '14 at 21:12
  • @Thursday I've updated. One of my SU posts has a current bounty on it right now. – fredsbend Aug 28 '14 at 21:20
  • The question you bountied on MSE is this one. – user259867 Aug 28 '14 at 21:26
  • @Thursday Thanks. I've updated again. My search voodoo is off today, I guess. Just curious, are you always "Thursday" or does your name change daily along with the weekly calendars? – fredsbend Aug 28 '14 at 21:31
  • Sadly, display name can be changed only once in 30 days. So, my name is accurate 1/7 of the time. Maybe I should switch from days of the week to months? For now, I have reasons to like the present name. – user259867 Aug 28 '14 at 21:41
  • @Thursday Cool. There's a lot of Chesterton fans on C.SE. – fredsbend Aug 28 '14 at 22:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .