41
  1. Take the set of 500 of most popular tags on the site in question. Find and store the correlation of pairs of these tags, e.g. the proportion of the time that a question tagged c# is also tagged .net.
  2. For each user, assemble a score by tag; for each non-CW question/answer, for each tag on the question, they receive a score in that tag equal to their number of upvotes, divided by the number of tags on the question, times 1 minus the correlation between this tag and the other tags on the question, per each tag (i.e. if c# is correlated 0.8 with .net and 0.3 with linq, then the user's score for c# in a question that is also tagged .net and linq is multiplied by 0.2 * 0.7 = 0.14). Scores for tags not found in the correlation table are not adjusted for correlation, i.e. they're simply votes / num_tags.
  3. For each tag score, apply this formula: score = (score * 50) / (score + 50). (This caps the output at 50 and imposes an extremely tough curve of diminishing returns over 25.)
  4. Sum the resulting tag scores. If the result is >= 500, the user qualifies for the Generalist badge.

The various constants in there are wild-assed guesses; the core idea is the methodology. It at least attempts to address the tag overlap issues that have plagued discussion of implementing Generalist. I expect it to be torn apart, but perhaps it's a start. :)

Please see also my suggestion for Polymath, which would be a gold badge with the same logic but 5x the awarding threshold (so 2500 if the 500-point threshold above were adopted).

7
  • 1
    Do you have results as to which users would have earned that badge?
    – jjnguy
    Jul 13, 2009 at 21:00
  • I do not. Maybe one of our friendly neighborhood data miners can help...
    – chaos
    Jul 13, 2009 at 21:05
  • Note: edited to change the diminishing returns mechanism to a capped version. The previous formula was open-ended, which isn't really appropriate to this application, since it allows the possibility of getting the Generalist badge for sufficient upvotes in a single tag.
    – chaos
    Jul 13, 2009 at 23:04
  • 2
    I really hope this gets implemented.
    – jjnguy
    Aug 11, 2009 at 16:30
  • According to your second point, if I answer to a question with two loosely related tags (correlation 0.1), I get no score for that. Why is it worse than question with 1 tag (where I get 1 point for each upvote)? Aug 16, 2010 at 22:22
  • It does seem that big correlation should lead to reduced value of answer, not the other way around. Aug 16, 2010 at 22:50
  • Well, it's moot at this point because another implementation is in place, but I don't think you're reading my math right.
    – chaos
    Sep 1, 2010 at 14:10

5 Answers 5

12
+100

I will look at this, but I can't give a specific time frame.

The general philosophy of the badge, is to award people who dabble in a lot of different topics but don't necessarily go deep on any particular one.

That's why awarding it is tricky, because we don't want to pile more badges on people who already have a slew of the things anyway.

7
  • 1
    In that case, where can I sign up for the beta of that badge? I'm more than willing give it a test run.
    – Eric
    Aug 11, 2009 at 11:49
  • 24
    Why discourage going deep? If somebody is so active that they go deep in one tag and are active in many tags, I think they should get two badges. That said, here are two more options to consider: (1) you can only earn generalist if you don't have any specialist badges (I don't care for this one), or (2) you earn generalist if you have at least N rep and there are k different tags so that at least X percent of your rep comes from each tag. If (N,k,X)=(1000,10,10) and you have 1400 rep, you must have at least 140 upvotes in 10 different tags. The badge should never be taken away. Aug 11, 2009 at 16:08
  • 8
    Obviously the time frame is six to eight weeks. :)
    – chaos
    Aug 11, 2009 at 16:17
  • What would happen when someone is awarded the Generalist badge, however then goes on to specialise in one particular tag after?
    – Justin
    Dec 21, 2009 at 20:33
  • 5
    I don't think any badge should be unable to be earned with an existing account.
    – deleted
    Jan 12, 2010 at 4:21
  • 1
    ...to award people who dabble in a lot of different topics but don't necessarily go deep on any particular one. Or more specifically, to award people who dabble in a lot of different very popular topics. No incentive here for anyone not riding the top of the bell curve, which is kind of disappointing. What's the reasoning for limiting it to popular tags, anyway? If someone provides valuable answers in lesser-traveled tags, isn't that useful to SO? Doesn't it build value in those tags so that those communities eventually bring their Q&A here?
    – T.Rob
    Nov 26, 2011 at 3:52
  • The only people with this badge on SO are the elite anyhow. No one with less than 10k has this badge. There's only 200 or so with it now. Dec 11, 2011 at 0:13
16

I have a simpler suggestion for the Generalist badge:

  • Generalist (silver): 5-10 (tweak this number) silver tag badges;
  • Jack of All Trades (gold): 15-25 silver tag badges or 5-10 gold tag badges or something like that.
9
  • 13
    I'm miles off my first tag badge, let alone 5!
    – ChrisF Mod
    Jul 23, 2009 at 13:33
  • 8
    This heavily favors users in the most popular tags. Generalist is an chance to give some love to people occupying more obscure corners. Aug 4, 2009 at 15:51
  • 5
    @dmckee: Why? "Generalist" means active in many areas, not active in obscure areas. If you want to reward such activity I think you need a different badge for that.
    – cletus
    Aug 4, 2009 at 23:27
  • 1
    @cletus: Yeah. That comment was ill-thought out. I meant something more like sticking with number of specialist badges emphasizes the popular tags even more than the badges, because most of the popular tags are in coorelation with at least one other popular tag: C# with .net, c with c++, java with jquery, javascript with html... chaos's proposal reduces this effect. Aug 5, 2009 at 2:19
  • One issue is whether we want to reward those with knowledge in obscure (at least for SO) areas. It's a lot easier to get a tag badge in a popular language than an unpopular one. Do we want to reward the ones who are extremely helpful to anybody who's still interested in Snobol or Mac OS 9? (Other than the rep, of course?) Another issue is, as pointed out, correlated badges. Somebody with a C# badge is likely to have a .net, while nothing's quite that correlated with C++ (for example), although mfc and c are going to be there. Aug 5, 2009 at 13:57
  • 1
    I'd call the gold one "Leonardo da Vinci". Or maybe "Jon Skeet", since he already has five gold tag badges.
    – mmyers
    Aug 5, 2009 at 20:35
  • 10
    IMO this is really awful. For one thing, I think you dramatically underestimate how hard it is to get a tag badge outside of the "golden" family of topics centered on C#. I have 25K rep and only just recently got one topic badge, a silver in PHP, and might not have that if I hadn't worked on it specifically. For another, the tag badges reward specialization in popular areas, which seems to me to make deriving a concept of being a generalist from them nonsensical.
    – chaos
    Aug 11, 2009 at 16:16
  • 1
    There are only FOUR people on SF who have won ANY 'tag' badges (and only 2 tags at that). At this rate nobody will ever get any version of this badge on SF. Jan 11, 2010 at 21:55
  • Hahaha, is this still the case chaos and @Mark? :D
    – user1136431
    Apr 1 at 1:06
10

I think that it's better to go for a simpler implementation at the expense of statistical perfection. The point of a badge is to reward a behavior; in striving to get a badge, people do something positive. If the criteria for the badge are convoluted, people are less likely to strive to fulfill them and therefore less likely to exhibit the good behavior you wanted (any more than they would "naturally").

I propose the same answer that cletus did, but with tweaked numbers. Rather than having to earn 5-10 silver tag badges (400 upvotes on a given tag) to get [Generalist], you have to get 200 upvotes on 10 different tags (for which there exist tag badges).

7
  • I realize that this has the downside that you don't know whether you've earned 200 upvotes on a given tag, which again makes the criteria less transparent, but I like the idea of a lower threshold than 400 upvotes. Aug 5, 2009 at 15:02
  • 5
    Why should my Generalist-ness depend on the existence of someone else having at least 400 votes in one of my tags? Aug 8, 2009 at 3:07
  • And you can see how you're doing in a tag now. Select the tag to bring up the tag specific page, then choose the "stats" tab. Aug 8, 2009 at 3:07
  • @dmckee: I meant that the only tags that should count towards your Generalist badge should be the ones for which it is possible to get a tag badge; there doesn't actually have to be anybody that has the tag badge. For example, getting lots of points in [subjective] shouldn't contribute towards your Generalist-ness. Aug 8, 2009 at 4:44
  • 1
    Specialist badges are generated automatically as soon as any user has the requisite number of upvotes. Aug 8, 2009 at 17:07
  • If you look you'll note that there are 8 people with specialist badges in "subjective" and one of them has the gold. Care to guess who? Aug 8, 2009 at 17:09
  • @dmckee: Wow, you're absolutely right. I really misunderstood how tag badges are implemented. Now that you've set me straight, I have to disagree with the current implementation. There should be a restricted set of tags for which specialist badges exist, and [subjective] shouldn't be one of them. But I guess that's getting far afield of the question of how [generalist] should be implemented. Aug 10, 2009 at 3:42
5

If my math is right -- which it rarely is -- on SO, the 501st tag still has 194 uses. I'm not sure I'm comfortable with that cut-off, especially considering that the cut-off will increase over times. I'd include any tag over a certain threshold. How about 50 to coincide with the Taxonomist badge (which is often earned through gaming the system)?

14
  • The cutoff is totally arbitrary, and is mostly just to keep from having to assemble a ridiculously huge correlation table. If +50 uses is feasible and not going to blow out anybody's disks, I'm down.
    – chaos
    Jul 13, 2009 at 21:10
  • I feel like I should get the Taxonomist badge now. Is it really that easy to retag 50 posts to something new?
    – jjnguy
    Jul 13, 2009 at 21:10
  • Pesto is just saying that to taunt Jeff Atwood.
    – chaos
    Jul 13, 2009 at 21:11
  • 4
    I just want Jeff to hold me close, even if he's just doing so in order to slam me to the ground. Jul 13, 2009 at 21:19
  • @chaos: Unfortunately, as the system grows older and older, the correlation table is going to have to get huge, or else be faced with replacing Generalist with Most-Popular-Tags-ist. I'd have gone with "Populist", but that badge already exists. Jul 13, 2009 at 21:23
  • The effect of the correlation table is actually in the direction of devaluing more popular tags. You'd get Generalist faster under the methodology I'm proposing by cruising under the correlation table's radar and getting votes in obscure tags.
    – chaos
    Jul 13, 2009 at 21:26
  • @chaos: But only those obscure tags that still crack the top 500. I suppose a large number of them qualify as obscure, to be honest. Fair enough, rock on. Jul 13, 2009 at 21:28
  • 3
    I don't know: this sounds expensive enough to calculate as it is. Jul 13, 2009 at 21:31
  • @Pesto: Note the bit about scores for tags not found in the correlation table -- "left unadjusted". Not sure if it's clear, but that means that they're treated as if they weren't correlated with anything, i.e. they get full value instead of being dinged for correlation.
    – chaos
    Jul 13, 2009 at 21:37
  • @chaos: Oh, I stand corrected, then. I misread that. I withdraw my complaints. Jul 13, 2009 at 21:42
  • @Pesto: Understandably. Edited to clarify.
    – chaos
    Jul 13, 2009 at 22:04
  • @Joel Coehoorn: Yeah, it's pretty heavy. My thought would be to run it maybe once a week. Since we've gone this long without Generalist being awarded at all, I don't think an infrequent schedule will hurt anything.
    – chaos
    Jul 13, 2009 at 22:06
  • I suppose you could make it fast enough by adding tables to support current correlation values/user scores that update whenever posts are added/edited, but I doubt they'll want to alter the db schema for this. Jul 13, 2009 at 22:14
  • 2
    "The cutoff is totally arbitrary, and is mostly just to keep from having to assemble a ridiculously huge correlation table. If +50 uses is feasible and not going to blow out anybody's disks, I'm down. – chaos 16 hours ago" I COMPLETELY READ THIS WRONG THE FIRST TIME AROUND
    – TheTXI
    Jul 14, 2009 at 13:14
3

Having thought about this for a while in the shower this morning -- it's where I do all my important thinking -- I'm wondering if this still isn't too complex. I haven't followed any earlier discussion, so forgive me if this has been shot down before, but has any thought been given to a system like the following:

foreach answer in user.non_cw_answers
  foreach tag in answer.tags
    tags[tag] += answer.upvotes / len(answer.tags)
tags_meeting_criteria = 0
foreach score in tags
  if score >= S
    ++tags_meeting_criteria
if tags_meeting_criteria >= T
  award_generalist_badge()

Where S is the minimum number of upvotes needed in a given tag to qualify, and T is the number of such tags required for generalist. We could probably improve it further by having user.non_cw_answers only return the highest scoring answer for a given answer in the case of multiple answers.

Has such a system been discussed before? If so, what were the flaws seen? If not, what comments do you have?

2
  • 1
    The objection raised to that has been that there's a lot of overlap between tags, like c# and .net, and so people get assigned credit for being a generalist across those tags when they really aren't. The correlation table mechanism is meant to address that.
    – chaos
    Jul 14, 2009 at 14:32
  • That's why I'm proposing that the upvotes on a given question be split amongst those tags. It's a simplified approach intended to lower the overhead. Jul 14, 2009 at 14:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .