41

"Generalist" is a badge awarded to community members for broad knowledge of a site's most commonly-asked-about subjects. It's earned by providing answers that accumulate 15 total score per tag in 20 of the top 40 tags. But, there's a caveat - in order for the Generalist badge to be awarded, there must be 40 "top tags" - defined, in this case, as a tag used on at least 200 questions.

It turns out, this is a major inhibitor for many sites (not just small ones!) to even start awarding this badge, as I found out when investigating a question on Retrocomputing Meta: Why has nobody ever been awarded the Generalist badge? If you do the same back-of-napkin math I did, you discover that for the badge to even be awarded, there must be at least 1600 questions (and that assumes that those same 40 tags are used equally across those questions, five tags each i.e. 40/5 x 200). Most of the time, these tags are used only one or two per question, making the necessary number of questions much higher.

For sites at the scale of Stack Overflow or even some of the large to mid-sized Stack Exchange sites, that's completely doable, but for smaller sites that are sitting between 2000-8000 questions, it's nearly impossible. One of the reasons that we set these limits is that we wanted to ensure that the "top tags" on a site had stabilized somewhat before starting to hand out the badge - and that's a reasonable intention - but we wondered whether 200 questions each in 40 tags was the right number to determine that.

As such, I explained all of this to Slate and asked her to use her magic SQL skills to dig into the data on SEDE. Essentially, my question was this: what would be the impact of adjusting the number of questions/tag needed (currently 200) and/or the number of total tags with that many questions (currently 40)? I dropped this all in her lap before going on vacation for three weeks - thanks, Slate!

When I got back, she'd prepared a very nice suite of data for me. She'd looked at several cases, each one step away from the other:

  1. The status quo - 40 tags with 200 questions each.
  2. Require fewer "top tags" - 20 tags with 200 questions each. (Note: we can't go below 20, because we're not re-evaluating the "score of 15 in each of 20 top tags" criterion.)
  3. Require fewer questions per "top tag" - 20 tags with 100 questions each
  4. Require even fewer questions per "top tag" - 20 tags with 50 questions each

In each case, we didn't change the requirement for the badge itself - it was still necessary for a user to get a score of 15 in 20 of the top tags ("top tags is defined as the 40 most-used tags with at least n questions in that tag).

She re-ran the data using these new triggers to see the impact of how many users per site would have earned the Generalist badge and the data was pretty awesome. For the larger sites, nothing changed - if site already met the qualifications for the higher requirements, no new badges are awarded, but for sites that don't meet the requirements, there was a sizable increase.

With the current badge requirements, only about 75 sites had even one user getting this badge (brown line below). With the others, as many as 150 sites had at least one Generalist - leaving only a couple dozen sites on the network unable to award the badge.

Graph titled "Generalist badge awardees under the proposed scenarios" The vertical axis is "Number of Generalist badge awardees (logarithmic)" and ranges from 1 to 500 and the horizontal axis is "Site Number (sorted by volume of awardees) and ranges from 1 to 175. There are four lines that all start at (1, 500) and slope down to around (50, 25) where they diverge quite a bit with the lines ending at around 75, 100, 125, and 150 respectively.

An important note about the graph above - while it looks like there's a change in the number of badges awarded for the first 25 sites, this is deceptive - none of the sites with any Generalist badge winners saw any additional qualifying users. When we order the list by volume of awardees on a per-case basis. In some cases, sites that had no badges awarded end up with more badge recipients than sites that currently have them, so the "Site Number" in the graph above doesn't always refer to the same Stack Exchange site, just the count of badges on the nth ranked site.

Because we knew that there wouldn't be a change in awards on the bigger sites and to better show the value of these changes on smaller sites, the graph below excludes all sites with zero change in awardees - so all of the sites that currently have any as well as the sites that have zero, both before and after the change. This graph shows, of sites that see a change in awardees, how many awards occur on those sites.

In the most limited change, about thirty new sites would see at least one person awarded the badge. Reducing the number of questions to 100 increases that to just over fifty sites and further reducing the questions needed per tag to 50 increases the count to nearly eighty sites.

Graph showing the change in Generalist badge awardees for the sites that see a change due to relaxing the requirements for this badge. The vertical axis is "Number of NEW Generalist badge awardees" and goes from 0 to 50 and the horizontal axis is "Sites sorted by volume of NEW Generalist badge awardees" and ranges from 0 to 80. The far right is above 50 with three lines sloping down to 30, 50 and 80 sites each.

To make it easier to see, the graph above was cropped at 50. One site, Code Golf & Coding Challenges, will end up with nearly 150 members earning this badge if these changes are made.

The proposal:

Based on the data, we're proposing to change the requirements for awarding the Generalist badge to the following:

  • Provide answers that accumulate 15 total score per tag in 20 of the top 40 tags (unchanged)
  • We lower the number of questions required for a tag to count as a "top tag" to 50.
  • We remove the requirement for there to be 40 "top tags" with at least 50 questions each. Because the badge requires there to be at least 20 tags with 50 questions, it's similar to lowering the requirement to 20 but a simpler query.
    • In other words, for example, if the site only has 30 tags being used on at least 50 questions, and a user earns a score of 15 on 20 out of those 30 tags, they would now qualify for the badge. If the site only has 20 tags being used on at least 50 questions, a user would qualify if they have a score of 15 on all 20 of those tags.

As a note, only tags with at least 50 questions will be considered "top tags" and only the top 40 tags will qualify to earn the badge, even if there are more tags with at least 50 questions each.

Given the data above, do you have any concerns about this change? Are there other variations of this that we should consider before changing this?


The Code

For those of you who speak SQL, I've shown the query below for the before and after for the proposed change. While I can't guarantee this is the specific way the devs will change the badge query, the end result should be the same and we'll update y'all with what the devs end up doing if we do make this change live.

On the left - the SQL (roughly) as it is now. On the right - what we're proposing it'll become. The parts that differ are highlighted in bold.

Before After
SELECT u.Id AS UserId,
u.DisplayName
FROM Users u
INNER JOIN UserTagTotals utt
ON u.Id = utt.UserId
WHERE utt.TotalAnswerScore >= 15
AND utt.TagId IN (SELECT TOP 40 Id
FROM Tags
WHERE [Count] > 200
ORDER BY [Count] DESC)
AND (SELECT COUNT(*)
FROM Tags
WHERE [Count] > 200) >= 40

GROUP BY u.Id,
u.DisplayName
HAVING COUNT(u.Id) >= 20
SELECT u.Id AS UserId,
u.DisplayName
FROM Users u
INNER JOIN UserTagTotals utt
ON u.Id = utt.UserId
WHERE utt.TotalAnswerScore >= 15
AND utt.TagId IN (SELECT TOP 40 Id
FROM Tags
WHERE [Count] > 50
ORDER BY [Count] DESC)



GROUP BY u.Id,
u.DisplayName
HAVING COUNT(u.Id) >= 20
9
  • I'm not sure whether this is official announcement aka "We are going to do that", or just a non-official feature request that have same chance to be done as any other feature request on Meta. Can you please clarify? (Example for the later, with the feature pending to this days, for over 7 years.) Sep 20, 2022 at 13:53
  • 5
    It's neither, I think? That example is significantly more complex to build. This is changing a couple lines of a query. This is a dummy check - is there anything we missed that should prevent us from asking the devs to do this? I can't promise when it'd be done but I can't imagine it'd be difficult to do.
    – Catija
    Sep 20, 2022 at 13:56
  • 1
    It might be troublesome, but I wonder if adjustments would be difficult for other similar badges for smaller sites Sep 20, 2022 at 13:57
  • 3
    @JourneymanGeek Splitting the badges' requirements per-site would most likely be much more complex than changing it universally. That's why this is such a simple change. It doesn't impact the bigger sites at all - no change in numbers of badge holders.
    – Catija
    Sep 20, 2022 at 13:59
  • 1
    Also, very much related, this suggestion might even fit as answer there. Sep 20, 2022 at 14:01
  • 4
    See, that request is 10 years old! We're just working through the backlog, @ShadowTheKidWizard :P
    – Catija
    Sep 20, 2022 at 14:03
  • 1
    I find it very hard to believe that SO will not gain extra awardees when the criteria are relaxed. Are you guys 100% sure of the SQL used here?
    – Luuklag
    Sep 20, 2022 at 14:20
  • 11
    @Luuklag the top 40 tags on SO don't change, and the necessary tag score doesn't either. Basically, the condition for tags is relaxed, not the one for users.
    – Glorfindel Mod
    Sep 20, 2022 at 14:21
  • 4
    Yes, precisely as Glorfindel says - if we'd increased the list of qualifying tags to the top 80 instead of the top 40, we likely would have seen an increase on SO, @Luuklag but because we're not touching the list of tags that qualify, so having the qualifications be 50 questions instead of 200 doesn't impact it.
    – Catija
    Sep 20, 2022 at 14:25

2 Answers 2

15

It's great that you're looking into some badges again. Now that you are, could you reconsider making it a gold badge? SE's current response there promises that you'd "come back to this when we're ready to look at badges more closely".

5
  • 4
    I'm not sure what the level of effort is to change the badge from silver to gold - I'm pretty sure changing the query is relatively simple. I'll certainly take this discussion internally to see whether we want to make this change and, if so, how simple it would be. My concern is that making this too complex a change might end up with it being less likely to get built.
    – Catija
    Sep 20, 2022 at 19:19
  • 1
    By lowering the bar, it will be easier to get the badge than before, which somewhat invalidates the claims on the post.
    – Bubbler
    Sep 20, 2022 at 22:42
  • 10
    @Bubbler that's true but it's still extremely rare. Even on Stack Overflow it's only about 1000 people. That's fewer than all but 3-5 gold badges.
    – Catija
    Sep 21, 2022 at 3:27
  • 4
    @Bubbler not really true. It will be made possible to earn, not really easier. Sep 21, 2022 at 7:08
  • 1
    @Catija, Stack Overflow is unusual in just how general a generalist you need to be. For a site like Information Security, "Generalist" would be a middling gold badge, while on Electrical Engineering, it would be the fifth-most-common. (On both, there are about a dozen less-common silver badges, and at least one less-common bronze badge.)
    – Mark
    Sep 28, 2022 at 2:53
10

Have you checked the effect on Meta sites? Meta Stack Exchange and Meta Stack Overflow are the only two sites to have them right now, but if my SEDE query is right, we'll see some new ones on the meta sites of Mathematics, Super User, the Russian and Portuguese Stack Overflow and maybe a few others.

(This post also doubles as a way for users to check what will happen on their favorite main site; the Stack Exchange Data Explorer has a switcher to run the query for another site. It's updated once a week, on Sunday morning, so the results may be a bit stale.)

1
  • 6
    We didn't check meta sites as part of this - honestly our focus had been on the main sites, so we didn't even think about it. It's worth calling out and, I think, only further indicates the value of this change.
    – Catija
    Sep 20, 2022 at 14:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .