Asking days badges

Question

A couple of months ago, I suggested a set of badges for asking questions. The responses were... mixed. The top answer can be summarized:

In practice, all this would do is encourage people to ask yet more stupid, worthless questions and we have quite enough of that as it is.

We believe that one of the main things holding back many of our beta sites is that when people have questions they aren't in the habit of asking on the site. Even our flagship site, Stack Overflow, doesn't get many questions from its most knowledgeable users. Perhaps they don't have questions, but far more likely they find answers without needing to ask. Our subtle bias against asking (questions => clueless) means we lose out on what might be our best source of excellent questions.

So we went back to the drawing board and designed a badge set that should not encourage worthless questions:

Asked a well-received question on X separate days, and maintained a positive question record.

In order to award badges, we need to define our terms:

• positive question record

(total questions - negative questions - closed - deleted)/total questions >= 0.5

It triple counts a question that is downvoted, closed, and deleted. If you ask one that hits that trifecta, you'll need to ask three better questions to dig yourself out. My own record of asking does not meet the criteria:

(44 - 0 - 12 - 12)/44 = 0.4545...

My public record looks a lot better; I had a bunch of questions from the beta period that have been subsequently (and rightfully) deleted. With care, I could bring my record into positive territory with 4 questions. The takeaway is that people can't sweep past indiscretions under the rug by deleting them. And they can recover from a bad start as long as they don't ask more bad questions.

• well-received question on X separate days

  A well-received question is one that's open, not deleted, and has a score > 0. If you ask at least one well-received question in a UTC day and none of your questions that day are deleted, downvoted, or closed, you get one more day of credit toward the badge. Whenever the badge criteria is checked, previous asking days are re-evaluated so that there's an incentive to fix downvoted and/or closed questions. The no-deleted-questions criteria is to prevent people from asking several questions at once and deleting any that aren't upvoted or answered to ask another day. (Though this is a losing strategy in the long run.)

My proposed badge names and levels:

Asked a well-received question on 5 separate days, and maintained a positive question record.

Asked a well-received question on 30 separate days, and maintained a positive question record.

Asked a well-received question on 100 separate days, and maintained a positive question record.

In addition, since some outstanding individuals ask on more than 100 days, the gold badge may be awarded multiple times. Over on English, Oishi-sama (now a community moderator) would have a remarkable 5 Socratic badges. Bronze and silver badges are only awarded once.

Neither the names nor the levels nor even the descriptions are set in stone. Please play around with this query to get an idea of how the badge works using our public data. Since the query does not account for deleted questions, it's a bit more generous than the actual badges would be.

This version of the badge set should encourage a habit of well-received questions over time and encourage people to fix poorly received questions.

---

I'd like to express my gratitude to the Parenting community for participating in our experiments.

Answer 1

I ran the proposed badge-award criteria against the top 124 users on English Language & Usage by reputation, and discovered that this would produce 2 golds, 20 silvers, and 75 bronzes amongst those 124 users, from a total of 38.5k questions asked (by anyone) and not deleted.

Here’s the table sorted first on Days and then on Reputation, where Days is “the number of good question-asking days” as described in the original posting. Some further notes on reading this table, with followup observations following it:

• The Rank is just the line number of the listing.

• ELU moderators are marked with the traditional ♦ flair, U+2666 ʙʟᴀᴄᴋ ᴅɪᴀᴍᴏɴᴅ sᴜɪᴛ, while moderator candidates in the current ELU election ending in under five hours as of this writing are marked with the ad-hoc ♢ flair, U+2666 ᴡʜɪᴛᴇ ᴅɪᴀᴍᴏɴᴅ sᴜɪᴛ.

• Because this is taken from SEDE, the values of Days shown here may be somewhat lower once deleted questions are taken into account, but I do not have ready access to that data. It would be interesting to compare graphs of running it both ways.

 Rank      ID Name                     Reputation Days Bronze   Silver     Gold
 ===================================================================================
   1.    3119 Yoichi Oishi ♢                21468 567 Curious Inquisitive Socratic
   2.     305 Brian Hooper                  19811 109 Curious Inquisitive Socratic
   3.    4763 Uticensis                     10183  76 Curious Inquisitive 
   4.   17953 Noah                           7867  71 Curious Inquisitive 
   5.   10041 Daniel                        34832  68 Curious Inquisitive 
   6.     127 Mehper C. Palavuzlar          16184  66 Curious Inquisitive 
   7.    2637 FumbleFingers                 78103  63 Curious Inquisitive 
   8.   23608 coleopterist                  21824  59 Curious Inquisitive 
   9.    4801 Jez                            6472  57 Curious Inquisitive 
  10.     252 kiamlaluno                    31745  52 Curious Inquisitive 
  11.    9505 SF.                            6569  47 Curious Inquisitive 
  12.   11268 Armen Ծիրունյան                6560  46 Curious Inquisitive 
  13.   53966 WS2                           11814  45 Curious Inquisitive 
  14.   13711 Terry Li                       5040  45 Curious Inquisitive 
  15.    6006 MrHen ♢                       22264  44 Curious Inquisitive 
  16.   44619 Mari-Lou A                    16415  35 Curious Inquisitive 
  17.    3479 F'x                           27021  34 Curious Inquisitive 
  18.    8360 KitFox ♦                      18359  33 Curious Inquisitive 
  19.      61 Chris Dwyer                    5230  32 Curious Inquisitive 
  20.   17795 Bravo                          9551  31 Curious Inquisitive 
  21.    1606 Claudiu                        5675  29 Curious  
  22.    3782 Manoochehr                     4946  29 Curious  
  23.    2303 Robusto                       89611  28 Curious  
  24.    6509 RiMMER                        15429  28 Curious  
  25.    5964 z7sg Ѫ                         8980  26 Curious  
  26.   11433 jwpat7                        48950  25 Curious  
  27.     212 b.roth                        12409  25 Curious  
  28.     300 RegDwigнt ♦                   58211  23 Curious  
  29.    4972 Mitch                         26139  23 Curious  
  30.    1798 JoseK                          5833  22 Curious  
  31.    3559 Matt Эллен ♢                  19454  21 Curious  
  32.    1547 Marthaª                       19588  20 Curious  
  33.   59527 David M                       11432  20 Curious  
  34.   66974 Josh61                        13476  19 Curious  
  35.   19894 JAM                            5742  19 Curious  
  36.   65205 Elian                         10357  18 Curious  
  37.    4040 Malvolio                       9213  18 Curious  
  38.   25030 terdon                         8863  18 Curious  
  39.   11762 Jim                            5888  18 Curious  
  40.    2085 tchrist                       50161  17 Curious  
  41.    1506 Jon Purdy                     19378  17 Curious  
  42.   64985 ermanen                       10922  17 Curious  
  43.   11550 Mahnax ♢                       7536  17 Curious  
  44.    7451 Urbycoz                        6978  17 Curious  
  45.      30 Edward Tanguay                 6514  17 Curious  
  46.    9001 Hugo                          41045  15 Curious  
  47.   20739 Ste                            5690  15 Curious  
  48.      48 JSBձոգչ                       36731  14 Curious  
  49.    5822 Callithumpian                 19106  14 Curious  
  50.   51137 Ryereḁd                       13358  14 Curious  
  51.   36232 Sven Yargs                    10866  14 Curious  
  52.   42788 Bradd Szonye                  10776  11 Curious  
  53.   23491 bib                           33090  10 Curious  
  54.   14666 Kris                          20734  10 Curious  
  55.     100 waiwai933 ♦                    9307   9 Curious  
  56.   10287 Tom Au                         6120   9 Curious  
  57.    6534 snumpy                         4929   9 Curious  
  58.   11396 Jeremy                         5322   8 Curious  
  59.    5432 Alain Pannetier Φ             13956   7 Curious  
  60.   21883 Cool Elf                       7463   7 Curious  
  61.    8732 aedia λ                        6606   7 Curious  
  62.      39 nohat ♦                       40985   6 Curious  
  63.    2022 mplungjan                     18941   6 Curious  
  64.      77 ShreevatsaR                   18501   6 Curious  
  65.   21655 Edwin Ashworth                17057   6 Curious  
  66.    4526 chaos                         15002   6 Curious  
  67.    6683 JeffSahol                     14891   6 Curious  
  68.    7662 Fraser Orr                     8089   6 Curious  
  69.   31341 jlovegren                      5269   6 Curious  
  70.    8019 TimLymington                  21052   5 Curious  
  71.   14073 MετάEd                        17271   5 Curious  
  72.     380 Mr. Shiny and New 安宇        15741   5 Curious  
  73.    2587 PLL                           14593   5 Curious  
  74.   13799 Irene                         10165   5 Curious  
  75.   16205 Pitarou                        9876   5 Curious  
  76.   18220 J.R.                          40901   4   
  77.   24489 StoneyB                       31743   4   
  78.   13140 Jay                           21559   4   
  79.    2007 Dusty                         12831   4   
  80.    9186 T.E.D.                        11995   4   
  81.   11146 Shoe                          11897   4   
  82.   11035 Lynn                          10718   4   
  83.    3534 phenry ♢                       9420   4   
  84.     184 Noldorin                       7254   4   
  85.   11928 David Schwartz                 7067   4   
  86.    3946 Kate Gregory                   6884   4   
  87.   42683 p.s.w.g                        5975   4   
  88.   32623 rhetorician                    5926   4   
  89.   12952 Barrie England               104265   3   
  90.   18696 Andrew Leach ♢                42757   3   
  91.    3036 Cerberus                      35363   3   
  92.    5754 Peter Shor                    30250   3   
  93.   10893 simchona ♦                    24113   3   
  94.    6336 mgb                           17052   3   
  95.   13141 onomatomaniak                 13565   3   
  96.    8377 Unreason                       9743   3   
  97.     110 VonC                           9686   3   
  98.     588 Ed Guiness                     7225   3   
  99.   15581 Brett Reynolds                 6400   3   
 100.     140 Vincent McNabb                 5188   3 
 101.   58761 medica ♢                      24673   2   
 102.   17956 Jim                           16982   2   
 103.    4941 Neil Coffey                   14309   2   
 104.    8816 Monica Cellio                 12203   2   
 105.   16264 Jim                           10403   2   
 106.   49727 user49727                      7529   2   
 107.   38065 John M. Landsberg              5878   2   
 108.    5894 HaL                            5865   2   
 109.    4951 mgkrebbs                       5455   2   
 110.    3820 Rory Alsop                     5197   2   
 111.   15299 John Lawler                   48084   1   
 112.     449 Kosmonaut                     34448   1   
 113.     547 Colin Fine                    30249   1   
 114.    2490 Hellion                       27887   1   
 115.   18655 JLG                           16858   1   
 116.   16191 Mark Beadles                  13770   1   
 117.    8656 MT Head                       10828   1   
 118.    5262 The Raven                      9488   1   
 119.   54363 oerkelens ♢                    8429   1   
 120.    9887 KeithS                         7478   1   
 121.    3626 slim                           7044   1   
 122.    7396 FrustratedWithFormsDesigner    6176   1   
 123.    2386 Jonathan Leffler               5290   1   
 124.    5450 James McLeod                   5041   1    

---

Conclusions

As you see this makes the new badge trio fairly rare — but quite possibly not unreasonably so. By these criteria, this makes the bronze badge one of ELU’s rarest bronzes. If you flip to the silver and gold pages, you’ll see that the relative rarity of the proposed new badge compared with other badges in that respective class remains. That is, just as it would be a rare bronze among bronzes, it would also be a rare silver among silvers and a rare gold among golds.

However, that does not necessarily mean that the criteria for badge awards are “too tough”. An alternate and completely viable hypothesis is the comparative paucity of question-asking badges (and Days in general) speaks more to the issue which these badges are attempting to address than it does to a problem with their selection criteria.

My overall take on these numbers is this this is probably pretty reasonable. The number of non-deleted questions on ELU was 38,507 total at the time this was run, although the SEDE data will have been slightly older. Please see the longer ELU.meta posting titled “How curious are you? — or — Tallied ELU ‘Asking-Day Badges’–Proposal Results” for more discussion, especially if it is specific to ELU instead of to SE in general.

Answer 2

I decided to dig into the positive record of questions formula a bit more. I looked at the distribution of (total questions - negative score - closed - deleted)/total for people who have asked 5 or more questions. If the distributions are similar on most of our sites, I think we can use a single threshold for this badge. Starting with my baselines for designing the badge, on the left is Parenting and on the right is Stack Overflow:

The x-axis is the positive record formula from 0 to 1 and the y-axis is the number of users with 5 or more questions (including deleted ones). In order to smooth the curve, user are binned in increments of 0.1 on the x-axis. (To get an idea of what the query looks like, see the public version which does not include deleted questions.)

Now for some sites in between: Gaming and Server Fault:

Mathematics and Skeptics:

The curves are similar, but strangely the smallest site (Parenting) has the least noisy-looking data. Deletion plays a huge part:

Site           Percent deleted
----           ---------------
Skeptics                    38
Server Fault                22
Stack Overflow              17
Gaming                      16
Mathematics                 12
Parenting                    9

When deleted questions are backed out, Stack Overflow and Skeptics look a lot like Parenting:

(All of these charts may be clicked on to see the bigger version. I point this out now because the Skeptics distribution is rather sparse when you throw out deleted questions.)

I haven't looked at every site's distribution, but it seems as if ignoring deletions results in most people having a perfect record. Very few people manage a sub-0.5 record without having their questions deleted. Or perhaps more accurately, people with a poor record of asking have their questions removed. Deleted questions also set people back a day toward earning the badges.

Long story short: including deletions makes the criteria for the badge self-correcting. It should be harder to get on sites like Stack Overflow, which needs better questions, not more questions. It should be easier on sites like Parenting, which struggles to find a consistent source of new content. Thanks to our Roomba, the balancing doesn't even necessarily need humans to delete question. Just ignoring them does the trick.

Answer 3

The real problem I have with these badges is the "good question" phrase in all of them. It gives no indication of what the badge is about and is confusing, seeing as we already have a badge called Good Question which is completely unrelated (this sounded like "Get x Good Question badges"). The badge description should say the exact criteria:

Ask a question with at least one upvote and no flags on x separate days

Answer 4

The only remaining worry that I have left, then, is people begging for upvotes on questions, and complaining more bitterly about downvotes. Not a problem now, but if badges were on the line...

Answer 5

Self-deleted questions should not count towards the deleted count in the positive question record formula.

The user is doing the site a favor by cleaning up their unnecessary questions, and knowing that it gives them -1 towards a badge would only incentivise them to leave it undeleted instead.

I know of at least a few times I've asked a question on SO and then deleted it later (sometimes after it got upvotes) simply because I realized my mistake and no longer needed an answer, so figured I'd remove it to help keep the site clean by removing unnecessary/invalid questions.

Users already get -1 for having their question downvoted or closed (-2 for both actions), and it makes sense to give them another -1 if their question wasted other users time to evaluate and delete the post, however if they deleted it themselves it should not give them a -1.

Answer 6

I think the criteria for a good question "has a score > 0" is set too low.

A person could post 4 or more (OKish) questions a day, five days in a row, and legitimately qualify for a bronze badge = Has a positive record of questions and asked good questions on 5 separate days People might very well post a higher number of questions in the hope that at least one, gets an upvote, which on EL&U is always possible. If it doesn't take five days due to closed/deleted questions, the length of time to earn a bronze badge might take two weeks, which I think is still too easy.

I suggest that the bar needs to be raised: "has a score > 3" (greater than three)

This makes it a bit more challenging. A "good" question which attracts quality answers will always be upvoted. People will naturally forget about badges and just be impressed by the quality of answers a decent/good question attracts.

I like the idea, it would (should) encourage new users to at least do a minimum of research, check that there are no duplicates, and discourage general reference type questions.

Answer 7

I think self deletion of questions should not count against a user in terms of this badge. Self deletion can be a sign that a person knows they asked a bad question, and if these count against the badge, it might encourage people to not police their own work.

Answer 8

Two quick suggestions:

1. Make the gold badge re-earnable for every additional 100 good posts?
2. Tweak the wording so that the primary criteria comes first. Something like: "Asked a good question on 5 separate days, and has a positive question record."

Answer 9

I like the idea, some sites like Arqade are very much question-limited and could easily answer much more questions than they get at the moment. Encouraging good users to ask more would be helpful there.

I'm a bit wondering though if the positive record criteria are strict enough. The case I'm thinking about are users that ask a large amount of mediocre low effort questions, which get occasionally downvoted but generally not closed.

There is another aspect of the formula connected to the last paragraph, and that is the rather large difference in voting between sites. My intuition would be that the criteria are rather strict on SO as getting a score of zero for a reasonably good question is not uncommon there. While on many smaller sites votes are given much more freely, and it is unusual for any question to stay at a score of 0.

This tendency might not be a problem, as encouraging questions is more useful on the smaller sites anyway. But if the difference is too large the formula might need some adjustment between different sites.

Answer 10

The new badge is very nice idea, but I have doubts about that criterium:

A well-received question is one that's open, not deleted, and has a score > 0.

Well, it's unrealistic to expect a popular question will have no downvotes, because there are always malcontents, but questions that has, for example, 20 upvotes and 19 downvotes is not well-received, it's controversial.

I'd suggest the change to

has a score > 0 and the number of downvotes is less than the number of upvotes / 2

Answer 11

First of all, I think the description can be somewhat more descriptive. I came here looking for the meaning of the badge I earned. Especially the phrase 'maintains a positive question record' was hard for me to grasp.

Apart from that, I oppose to these badges.

I see so many questions of which I'm really surprised they are asked at all, that still get upvoted. Sometimes a question is fairly well written, but still shows no research at all. Nevertheless people will upvote it. If a good question is a question that has a positive score, than I think too many questions match this criterium.

And I think these badges will encourage people to ask more questions they otherwise wouldn't have asked. Maybe people asking bad questions will try harder to get this badge, although I doubt so. I think there is a better chance that people asking good questions are tempted to ask more of them.

Now, more good questions doesn't sound bad, but I believe it is a bad thing if people will ask these questions just because of the reward they get out of it. It means that even though the question is essentally good, it shouldn't have to have been asked at all, because the person asking it could have tried just a little harder to find the answer themselves.

I'm happy with the bronze badge I earned, but after a membership of over 3,5 years, I'm stuck at 10 questions on SO. Three One of them doesn't even have upvotes, which might rather be because it is too specific than because of the poor quality. ;-)

I wonder whether I should start asking more questions about more 'obvious' subjects that appeal to a broader audiance, just to get the silver and gold badges as well.

If most people agree that the answer to that is "Yes, you should", then these badges make sense. Otherwise, these badges attract undesirable behaviour.