What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

I just looked on SO, and found that there are about 12,880 tags with only a single use. There are 26,250 used less than 10 times.

That's just too many for the community to fix. I think we need some automatic cleanup here.

I suggest you add a cleanup process to remove all tags used less than a certain number of times more than a month after their creation. That would give a legitimate tag a while to become used more, while removing those that will never be used more than a few times.


Edit: Here are the tags from page 512. I've highlighted the ones I think are most likely bad tags. I even fixed a few:

  • installer-class
  • inspector
  • installation-tools
  • insert-worksheet
  • insets
  • insomnia-mysteries
  • inorder
  • inprivate
  • inprocfactory
  • inprocserver
  • in-progress
  • input-button
  • input-buffer
  • input-mask
  • inputformsection
  • insert-nested-select
  • insert-parameters
  • insert-img-to-option
  • insert-data
  • insertdate
  • insert-delayed
  • insertafter
  • insertchildbefore
  • input-mode
  • input-simulation
  • inputpanel
  • inputrc
  • input-redirection
  • inquiry-management
  • inputstreamreader
  • input-string
  • input-testing
  • infocards
  • infochannel
  • infogrid
  • inflector
  • influence
  • info-plist
  • inforeader
  • inform
  • inform7
  • information-base
  • information-card
  • information-dispersal
  • informed-search
  • information-sources
  • information-technology
  • information-ownership
  • information-science
  • info-vis
  • informx (fixed now)
  • info-window
  • infralution
  • inheritence (typo - fixed)
  • initialcapacity
  • initial-context
  • initialization-block
  • initfileurlwithpath
  • inline-comments
  • inline-cpp
  • inline-edit
  • inlinec
  • initialize-request-handl
  • injectable
  • injectors
  • init-script
  • inittab
  • initwithcontentsoffile
  • initwithcontentsofurl
  • inlinks
share|improve this question
1  
However, those tags may be needed again...I suggest only cleaning up the really uncommon tags...but then again, you never know. –  studiohack Apr 30 '10 at 17:58
1  
John - How many tags are there in total? –  user27414 Apr 30 '10 at 19:32
1  
@studiohack: I meant the single-use tags, if they're still single-use a month after their creation. If they're needed again, they can be created again. –  John Saunders May 1 '10 at 1:10
6  
@Jon: as of this moment, there aer 35,840 tags on SO. That means 36% of all tags are only used once. 73% of all tags are used fewer than 10 times. Now, we should also take age into account, but these are numbers that concern me. –  John Saunders May 1 '10 at 1:13
    
@John S: oh ok. makes a lot more sense. +1 –  studiohack May 1 '10 at 2:21
    
@John - that's a lot more serious than I had thought. Ouch! –  user27414 May 1 '10 at 3:45
16  
We can't get rid of [insomnia-mysteries]!!! –  Earlz May 17 '10 at 18:31
9  
What happens if a tag is deleted, and the question only has that one tag? Can there be questions with 0 tags? –  Corey May 18 '10 at 8:20

15 Answers 15

up vote 39 down vote accepted

I don't know -- can anyone make a case for these single use tags existing for more than a month?

edit: I am convinced this is a useful cleanup task, to remove these orphaned tags from the system automatically. I've implemented a routine that removes tags created more than 6 months ago which have been used only a single time. It will run monthly.

Right now I am running it by hand over the tiers and I got these stats so far:

(note: where you see multiple posts being updated, it's because there are "extra" deleted posts with that tag.)

I spot checked a whole bunch of them, and removal seemed correct to me in every instance. It's almost always a case of over-tagging, where the question asker sprayed 1 or 2 "extra" tags on the question of extremely low value, when the more general tags they also included were perfectly adequate.

There is some code at the end to deal with questions which end up with ZERO tags after removal of the one-use orphan tags -- it tags these . But that is exceedingly rare.

Any tags not currently in use, or that happen to get orphaned through unusual means (rollback, merge, etc), are removed every 24 hours by a background process.

share|improve this answer
18  
Surely there are technologies that are either too new or too rare to be used more than once in the first month. –  user27414 Apr 30 '10 at 17:07
6  
@Jon B: If they are too rare, the tag isn't really that helpful, and people probably aren't seeking it out much, anyhow. If they are too new, the tag will be recreated later with more success once the technology becomes more popular (which means a small number of questions might improperly lose their tags, but this is OK imo). Killing a tag does not imply it will never exist again. That said, this number could be upped to 3 months (which is a LONG time), if you're paranoid about the time it takes. –  Brian Apr 30 '10 at 19:19
2  
@Brian - 3 months might be a little safer. Once the initial cleanup is done, I doubt the problem will get out of hand in a 3 month period. –  user27414 Apr 30 '10 at 19:34
2  
@Jon: I don't have access to the data dump right now, or I'd do some more analysis of these tags. In particular, I think the time since their creation might be interesting as well. For instance, what percentage of the single-use tags are over six months old? –  John Saunders May 1 '10 at 1:15
2  
@Brian: Why would it be okay to have a presumably reasonable tag removed from a question that it is appropriate for? If I add a Blorgle tag to a question, and it fits that question, and it times out after three months, then another question is added that has the Blorgle tag...but now the first question doesn't? Why not? The tag is now considered "okay" again for the time being. –  beska May 18 '10 at 15:25
3  
@Jeff: I'll grant a lot of these single use tags are useless and silly...but to globally delete low use (but useful) tags because of a non-critical-mass userbase is to pretty much ensure that you'll never reach that critical mass userbase...you have to have these questions around with reasonable tags so they'll show up on search engines, so that those people know they can come to the SO site for their questions. –  beska May 18 '10 at 15:28
3  
@beska: Because perfect is the enemy of good. The benefits of zapping extraneous tags outweighs this. –  Brian May 24 '10 at 21:45
    
Loving the (temporarily?) new rubik tag ;-) –  Arjan May 30 '10 at 22:14
    
@arjan yeah that was the 1 in 20 where a rare tag was actually justified. –  Jeff Atwood May 30 '10 at 22:47
6  
What if the [untagged] tag has only 1 question in 6 months? Will it also get deleted? –  Maxim Zaslavsky Jun 4 '10 at 19:50
1  
Wow, this is truly sad, now I have to go back and add in tags that haven't got a population here yet. We need people to be able to google these tags, so that they come here and we can build the population. –  Lance Roberts Jun 30 '10 at 18:28
1  
I've just had a look at Unix's single use tags. Some are useless and need culling (e.g. myths). Many others are legitimate (many are names of applications that get little traffic, e.g. mutt, nano), enough that I fear the automatic removal will do more harm than good. –  Gilles Feb 18 '11 at 0:17
1  
@Jeff: There are two untagged questions from the early days of the site a little over 6 months ago, so I guess the script has run once already. I wouldn't be surprised to see quite a few legitimate tags see no activity in 6 months. The biggest problem is the questions that have multiple tags, one of which is something like a rare unix variant or rare application: we'll have no warning that the question has become hard to find. –  Gilles Feb 18 '11 at 0:46
5  
@gilles if it's that rare it should be in the title or body and findable via search.. not everything HAS to be a tag –  Jeff Atwood Feb 18 '11 at 0:53
3  
Just my 2 cents worth, but it seems as if a low use tag with a "follower" would be "more valid" as someone is interested and has taken the time to denote that they are, even if the tag has low use. –  Mark Schultheiss Aug 18 '11 at 12:59

What is the point of keeping tags that are used by one or few users? When tags is quite popular, I can hope that someone will view questions with this tag and, coming across the problem he knows the solution, he will answer it. When tag has 10 questions, who will visit its page regularly? Except for search engines?

Such tags, in my opinion, poisons the tag system, because many users use such tags, when other, more popular, would be appropiate and would increase their chances to get answered. When there's too much options, making good choise become more challenging.

The exception are, of course, new technologies, which began to be popular and tags for them should be created, so that other users will see it and use when needed.

share|improve this answer

How many of Stack Overflow's current tags have at least a 6 month gap between their first and second use? Probably not very many (if any - would make an interesting odata query). Certainly not enough to justify keeping the single-use tags around.

share|improve this answer

Coming to the party late, I have a minor concern.

Some people don't include the name of the technology they are using in the questions, assuming that putting it in the tag is sufficient.

Some of the untagged questions are going to degenerate into meaninglessness if you remove their tags.

Would it be possible to automatically modify the question text to append say "[Tag removed: xxyzy]"?

share|improve this answer

Okay...I am the "perpetrator" of one of these tags. Currently, my tag, "inform7" has only two questions, and I've asked both of them.

However, I've gotten good, quick answers for both of my questions, and neither question was voted down, so presumably both questions are at least semi-reasonable. Given that, my question is: if my tag is removed, what should I have tagged them with? Nothing else is very appropriate, and I think my tag actually is pretty spot on. Assuming we want more people to start using the site, and get a critical mass of this (admittedly niche) userbase, shouldn't the tag remain?

I'm all for tag consolidation, but auto-removing low use tags seems to be a bad solution (and, to be honest, if we're not talking about duplicate or joke tags, I'm not clear on what the "problem" is.)

share|improve this answer
    
No one is talking about banning these tags or other new tags. We're talking about a one-time cleanup that has the unfortunate side-effect of also catching some good existing tags, and pairing this with a new tag workflow that puts more emphasis on re-using existing tags to help avoid the need to repeat this in the future. –  Joel Coehoorn May 18 '10 at 15:49
4  
So we are talking about doing a brainless simple operation that isn't actually correct because we are too lazy to do the right thing? –  Christopher Barber May 18 '10 at 16:27
1  
@Joel: I understand the idea...but I'm wondering how much will be lost. Your point about adding a new workflow is critical...without that, I don't think there's a point...but with it, I can see the utility. Sending messages out to originators of affected questions would also be a good thing...I, for one, would be happy to follow a more rigorous workflow to recreate the (hopefully useful) tag so that the question wouldn't essentially be permenantly "tag-orphaned". –  beska May 18 '10 at 16:53
1  
@Joel: Also, while you say "we're talking about a one-time cleanup", I don't think that's what everyone is talking about. It seems as though people are suggesting things like an ongoing process which auto removes tags that haven't been used for a month, which is the idea that I'm more fervently against. –  beska May 18 '10 at 17:00

I added a tag racket with only one question because the PLT-Scheme project are changing their name to Racket (currently offline). This was at the request of a user who wanted to add the tag themselves but could not because they didn't have enough reputation at the time. If the tag is removed then the same problem will happen again the next time someone tries to ask a Racket question. Removing rarely used tags every time it hits 1 usage will make it unlikely that it will ever be possible to use them. In this case an alternative would be to make plt-scheme a synonym for racket.

Another tag I created with only one question is android-send-me-logs for the project android-send-me-logs. Again the user that asked the question would not have had sufficient reputation to create this tag themselves.

Your proposal will result in the removal of these two tags, plus I'm sure many other tags in a similar position. Will this benefit the site? And will leaving these tags cause any harm?

share|improve this answer
    
The proposal will only get rid of these tags if they remain with only a small number of uses. –  John Saunders May 30 '10 at 18:04
    
yes, they'd have to stay that way for 6 months under the current ruleset. –  Jeff Atwood May 30 '10 at 20:37

New technologies come out all the time, and it will take a while for their population to be big enough to get multiple questions on StackOverflow, but I'm sure we'd like the questions to be there one day. If someone asks a question for a specific technology early on, then he might get very little response, like here, but it would be a shame for the tag that categorizes that question to be removed. It will also cause duplicate questions to be asked eventually, since the logical tag for that type of question isn't attached for it.

While base-db is just getting started as a concept, I've also created a tag pi-db for OSIsoft's PI database that has a huge userbase all over the world, but the community here just hasn't been built yet.

I don't think we should aim StackOverflow to just the hot topics, we should be seeking to attract all programming communities, to make this THE preeminent place to ask programming questions. (Note this is biased from my own laziness, since I want to only have to go to one site to find all my answers).

share|improve this answer
    
No one is talking about banning these tags or other new tags. We're talking about a one-time cleanup that has the unfortunate side-effect of also catching some good existing tags, and pairing this with a new tag workflow that puts more emphasis on re-using existing tags to help avoid the need to repeat this in the future. Also, I think questions where the single-use tag is the only tag would of necessity be left alone. –  Joel Coehoorn May 18 '10 at 16:45

I think any tag that is only used a few times over a few months should be hidden and not shown on any questions or in the list of tags.

If it gets type into the tag box, it should then be unhidden for a month.

That way if it takes a few month before a tag is used more then 1 or 2 times, we don't loose the tagging info from the first questions it was used on.

share|improve this answer
    
Why? I don't understand the win. –  beska May 18 '10 at 16:53

Here's a datadump query that will give single-use tags older than 3 months:

With SingleTags As
(
  SELECT Tag FROM PostsTags GROUP BY Tag HAVING Count(*) = 1
)
SELECT st.tag, p.ID As PostID, p.Title, p.CreationDate 
FROM SingleTags st
INNER JOIN PostsTags pt ON pt.tag = st.tag 
INNER JOIN Posts p ON p.id = pt.postid 
WHERE p.CreationDate < DateAdd(m, -3, getdate())
ORDER BY p.CreationDate

And I know it's not stable right now, but here's the StackQL results:
http://jcoehoorn.dyndns.org/stackql/default.aspx?qid=10012
And here's a similar query on the StackExchange Data Explorer:
http://odata.stackexchange.com/stackoverflow/q/427

Quite a few of these tags are already cleaned up.

share|improve this answer

This engine is all about promoting democratic processes, right? Why not allow voting on the tags themselves?

You get bonus rep for accepted tags, you lose rep for rejected tags. If tags are considered meaningful data, then they should be treated like all other meaningful data.

share|improve this answer
    
I like this idea conceptually...but I wonder if it would be just too much information/work for people to worry about. Implementation might be difficult. Still, the general concept seems good. –  beska May 18 '10 at 16:56

Before you go arbitrarily removing tags, ask yourself "why?" What are you gaining?

It's not as if you are trimming down the tags list to a manageable size. People don't browse tags like a book. Tags are a valuable data-mining tool. Not a popularity contest. You're removing valid tags; possibly on rare, niche topics... arbitrarily, with little-to-no benefit.

The next time I am looking for a question about Infralution, I'd like to think the original authors question would be found. Not orphaned because the tag didn't meet the minimum-popularity quota.

share|improve this answer
    
you have to look at the data, though. This is like posts of 30k length. Are they valid? Perhaps. But the data shows us that the VAST VAST majority of 30k length posts are absolute garbage. –  Jeff Atwood May 18 '10 at 1:20
5  
@Jeff Atwood - The OP's list of most likely invalid tags shows more than a third are valid. So the VAST VAST majority criteria may not apply. Either or -- my point was philosophical. Nothing I think will make a lick of difference to the system either way. –  Robert Cartaino May 18 '10 at 1:39
4  
instances of 1 "valid" tags can also mean people are tagging at too low/granular a level. If I tag this discussion as "robert-jeff-may17", it's a theoertically valid tag, but not a useful one. –  Jeff Atwood May 18 '10 at 3:05
1  
Robert, you could help by saying which of the tags in my list are valid, and why. –  John Saunders May 18 '10 at 5:10
    
@John Saunders - I went by your statement "I've highlighted the ones I think are most likely bad tags." –  Robert Cartaino May 18 '10 at 14:33
    
@Jeff - "...can also mean" true. So let me ask you a different way. From hereon in, if someone see a post (older than a month) and one of the tags is only used once, delete it? Automatically? –  Robert Cartaino May 18 '10 at 14:46
    
@Robert: when I said "are most likely bad tags", I meant, "appear to have the highest probability, based on my areas of knowledge, of being bad tags". My areas of knowledge do not include a lot of things that are discussed on SO (PHP, Python, etc.), so I left unbolded the set of tags that I thought might possibly be valid. Note also that the tag list is an edit - the original question did not include a tag list. –  John Saunders May 19 '10 at 2:03
    
I think a neat solution would be that any 'tagless' questions resulting from this clean-up should be tagged with the 'untagged' tag (if that makes sense). This would make a clean-up after the clean-up much easier. –  Benjol May 19 '10 at 8:18

I started going through the single-use tags manually periodically, but the tags list on SO is over 500 pages long and growing fast, so it takes a while to go backwards in the tags to get to the 2-usage ones.

Certainly, there are a lot of single-use tags, but what I've found is that the vast majority of them are used properly. It would definitely help if I had 10k tools on SO to audit new tags...

In any event, with regards to an automated system, I don't think any such thing could work very well, as it's really a human problem. The context of the tag needs to be observed to determine whether or not it should exist. Enhancement of the 10k tools might be a better option (although I can't really suggest anything because I haven't used them before).

Once tag synonyms are implemented, it's tough to say if that will reduce or exacerbate the problem, but certainly there will be fewer tags overall in the system, which should help.

share|improve this answer

I really dislike the idea of automatically deleting perfectly legitimate tags just because they are not used frequently. If someone tags a question about the "xyzzy" programming language with the tag "xyzzy", then it seems rude to delete it even if there are only a couple of instances. You would basically be telling people with questions about rare or unusual technologies not to bother to come here.

share|improve this answer
3  
Agreed. As I said in my answer, most of the low-occurrence tags I've seen have been used properly. I think the issue is more of removing the "garbage" tags -- misspellings, aliases of other tags which already exist, etc. –  Jon Seigel May 17 '10 at 18:17
1  
Well, in reality, obscure technologies are obscure and SO wouldn't be much help I don't think. I use some rare/obscure technology, and there are 0 tagged questions in that area. SO is very much a non-niche community. –  Paul Nathan May 17 '10 at 18:22
    
I think we're to the point where a one time fix is justified, if it's combined with Stephen Steel's answer where something is put in place to make it easier to use existing tags vs creating new ones in the future. –  Joel Coehoorn May 17 '10 at 18:25
1  
@Joel: I really wish it was easier to get data on when tags were created. I would be very curious to see how many tags with 1 occurrence are >= 1 year old. –  Jon Seigel May 17 '10 at 18:29
3  
Obviously popular technologies are going to see the most traffic here but I do not get the feeling that the intent of the community was to exclude any technology from consideration. The proposal to delete tags just because they are uncommon seems at odds with the tagging concept. –  Christopher Barber May 17 '10 at 18:29
    
@Chris - No one is talking about deleting any questions. The questions would still remain. And if this is the only tag associated with a question, than either we have to also keep the tag or the script will have to use a special tag - something like "obscure". That could even be a benefit for these questions, as I can imagine a lot of people would want to browse the obscure tag. –  Joel Coehoorn May 18 '10 at 13:26
    
@Jon - you can't know exactly when a tag was first created, but you can use the datadump to find the oldest question that still has a given tag. –  Joel Coehoorn May 18 '10 at 13:28
    
@Christopher - You must remember that infrequenly used tags carry considerable cost: they dilute the useful tags. The reason it is constantly necessary to merge tags for synonyms is that nobody bothers to look throgh 500 pages of existing tags to select appropriate ones - they just enter whatever occurs to them. –  Stephen C. Steel May 18 '10 at 14:19
5  
How do infrequent tags "dilute" useful ones? I can see how infrequent tags that refer to the same concept as a popular tag are bad, but no one has yet explained how an infrequent but unique tag presents any problem that could not be solved by better tag navigating UIs. –  Christopher Barber May 18 '10 at 14:42
    
Xyzzy...how appropriate for me! My tags on this list are "inform7"... –  beska May 18 '10 at 15:07
    
@Christopher - Some of the low use tags are good: if my question concerns the little used xyz programming language, then tagging it with "xyz" is appropriate. Then the small number of SO users familar with xyz might notice it. But many of the low use tags are not so useful. Many of these questions are not so unique that they really need to introduce new tags. While a better navigation UI might make it easier to wade through 500 pages of tags, but it will still take far more effort than most questioners are willing to expend. –  Stephen C. Steel May 18 '10 at 20:52
1  
This is in essence a question of scalability. In the long run for SO to remain successful it is going to have to solve the problem of having large number of tags. Crude automated tag deletion rules aren't going to solve that problem. It is actually pretty clear that the existing tagging scheme is overly simplistic and does not scale well. –  Christopher Barber May 19 '10 at 13:22

I think the problem is likely to continue to get worse, as it arises from a simple fact: it is MUCH easier to introduce a new tag than it is to find existing ones that are suitable. Worse, the asymmetry in effort becomes more and more pronounced as the growing number of low use tags dilute the good ones. The frequent need to merge tags that are straightforward aliases of each other is proof of this (vs2010/visual-studio-2010, subversion/svn, mercurial/hg, etc.)

Ideally, it would be best if it required more effort on the part of the user to introduce a new tag than it takes to select ones that already exists.

share|improve this answer

I think automatically zapping tags is too heavy handed. There will be legit tags that squashed as a result, and some posts may end up with no tags at all (which could be a serious problem, depending on how the queries work).

I agree, though, that we have a problem here. Perhaps there is a way for tags to be queued up for mods or 10K users to review. In this manner they could be cleaned up slowly over time.

share|improve this answer
    
Obviously, if zapping the tag would result in questions without any tags, then the tag should not be zapped. Other than that, I say zap away! –  nb69307 Apr 30 '10 at 17:06
1  
Any automated tool would have to look to see if a post has only one tag, as the Tag Merge/Remove tool does now. It won't let us remove a tag if any post would be left tagless as a result. –  Bill the Lizard Apr 30 '10 at 17:07
    
A merge is different, since the post will go from one tag to a different tag. –  user27414 Apr 30 '10 at 17:08
1  
The same tool can be used to remove tags. –  Bill the Lizard Apr 30 '10 at 17:10
    
Still, I agree, I think slowly over time we can clean this up. Surely we're not creating tags faster than we can remove them, are we? –  Bill the Lizard Apr 30 '10 at 17:13
1  
@Bill At the moment, I suspect we are! –  nb69307 Apr 30 '10 at 17:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .