Should We Zap The Low-Occurrence Tags? [closed]

Question

I just looked on SO, and found that there are about 12,880 tags with only a single use. There are 26,250 used less than 10 times.

That's just too many for the community to fix. I think we need some automatic cleanup here.

I suggest you add a cleanup process to remove all tags used less than a certain number of times more than a month after their creation. That would give a legitimate tag a while to become used more, while removing those that will never be used more than a few times.

---

Edit: Here are the tags from page 512. I've highlighted the ones I think are most likely bad tags. I even fixed a few:

• installer-class
• inspector
• installation-tools
• insert-worksheet
• insets
• insomnia-mysteries
• inorder
• inprivate
• inprocfactory
• inprocserver
• in-progress
• input-button
• input-buffer
• input-mask
• inputformsection
• insert-nested-select
• insert-parameters
• insert-img-to-option
• insert-data
• insertdate
• insert-delayed
• insertafter
• insertchildbefore
• input-mode
• input-simulation
• inputpanel
• inputrc
• input-redirection
• inquiry-management
• inputstreamreader
• input-string
• input-testing
• infocards
• infochannel
• infogrid
• inflector
• influence
• info-plist
• inforeader
• inform
• inform7
• information-base
• information-card
• information-dispersal
• informed-search
• information-sources
• information-technology
• information-ownership
• information-science
• info-vis
• informx (fixed now)
• info-window
• infralution
• inheritence (typo - fixed)
• initialcapacity
• initial-context
• initialization-block
• initfileurlwithpath
• inline-comments
• inline-cpp
• inline-edit
• inlinec
• initialize-request-handl
• injectable
• injectors
• init-script
• inittab
• initwithcontentsoffile
• initwithcontentsofurl
• inlinks

Answer 1

I don't know -- can anyone make a case for these single use tags existing for more than a month?

edit: I am convinced this is a useful cleanup task, to remove these orphaned tags from the system automatically. I've implemented a routine that removes tags created more than 6 months ago (12 months on Meta sites) which have been used only a single time. It will run monthly.

Right now I am running it by hand over the tiers and I got these stats so far:

• Stack Overflow, ~7,400 tags removed (/tags went from 522 pages to 415 pages)
• Super User, 866 tags removed http://pastebin.com/HBRaFsfm
• Server Fault, 1224 tags removed http://pastebin.com/jauyQv5k
• Meta (prod), 474 tags removed http://pastebin.com/nT8h6bTj

(note: where you see multiple posts being updated, it's because there are "extra" deleted posts with that tag.)

I spot checked a whole bunch of them, and removal seemed correct to me in every instance. It's almost always a case of over-tagging, where the question asker sprayed 1 or 2 "extra" tags on the question of extremely low value, when the more general tags they also included were perfectly adequate.

There is some code at the end to deal with questions which end up with ZERO tags after removal of the one-use orphan tags -- it tags these untagged. But that is exceedingly rare.

Any tags not currently in use, or that happen to get orphaned through unusual means (rollback, merge, etc), are removed every 24 hours by a background process.

Answer 2

I really dislike the idea of automatically deleting perfectly legitimate tags just because they are not used frequently. If someone tags a question about the "xyzzy" programming language with the tag "xyzzy", then it seems rude to delete it even if there are only a couple of instances. You would basically be telling people with questions about rare or unusual technologies not to bother to come here.

Answer 3

I think the problem is likely to continue to get worse, as it arises from a simple fact: it is MUCH easier to introduce a new tag than it is to find existing ones that are suitable. Worse, the asymmetry in effort becomes more and more pronounced as the growing number of low use tags dilute the good ones. The frequent need to merge tags that are straightforward aliases of each other is proof of this (vs2010/visual-studio-2010, subversion/svn, mercurial/hg, etc.)

Ideally, it would be best if it required more effort on the part of the user to introduce a new tag than it takes to select ones that already exists.

Answer 4

Before you go arbitrarily removing tags, ask yourself "why?" What are you gaining?

It's not as if you are trimming down the tags list to a manageable size. People don't browse tags like a book. Tags are a valuable data-mining tool. Not a popularity contest. You're removing valid tags; possibly on rare, niche topics... arbitrarily, with little-to-no benefit.

The next time I am looking for a question about Infralution, I'd like to think the original authors question would be found. Not orphaned because the tag didn't meet the minimum-popularity quota.

Answer 5

Okay...I am the "perpetrator" of one of these tags. Currently, my tag, "inform7" has only two questions, and I've asked both of them.

However, I've gotten good, quick answers for both of my questions, and neither question was voted down, so presumably both questions are at least semi-reasonable. Given that, my question is: if my tag is removed, what should I have tagged them with? Nothing else is very appropriate, and I think my tag actually is pretty spot on. Assuming we want more people to start using the site, and get a critical mass of this (admittedly niche) userbase, shouldn't the tag remain?

I'm all for tag consolidation, but auto-removing low use tags seems to be a bad solution (and, to be honest, if we're not talking about duplicate or joke tags, I'm not clear on what the "problem" is.)

Answer 6

I think automatically zapping tags is too heavy handed. There will be legit tags that squashed as a result, and some posts may end up with no tags at all (which could be a serious problem, depending on how the queries work).

I agree, though, that we have a problem here. Perhaps there is a way for tags to be queued up for mods or 10K users to review. In this manner they could be cleaned up slowly over time.

Answer 7

I started going through the single-use tags manually periodically, but the tags list on SO is over 500 pages long and growing fast, so it takes a while to go backwards in the tags to get to the 2-usage ones.

Certainly, there are a lot of single-use tags, but what I've found is that the vast majority of them are used properly. It would definitely help if I had 10k tools on SO to audit new tags...

In any event, with regards to an automated system, I don't think any such thing could work very well, as it's really a human problem. The context of the tag needs to be observed to determine whether or not it should exist. Enhancement of the 10k tools might be a better option (although I can't really suggest anything because I haven't used them before).

Once tag synonyms are implemented, it's tough to say if that will reduce or exacerbate the problem, but certainly there will be fewer tags overall in the system, which should help.

Answer 8

This engine is all about promoting democratic processes, right? Why not allow voting on the tags themselves?

You get bonus rep for accepted tags, you lose rep for rejected tags. If tags are considered meaningful data, then they should be treated like all other meaningful data.

Answer 9

New technologies come out all the time, and it will take a while for their population to be big enough to get multiple questions on StackOverflow, but I'm sure we'd like the questions to be there one day. If someone asks a question for a specific technology early on, then he might get very little response, like here, but it would be a shame for the tag that categorizes that question to be removed. It will also cause duplicate questions to be asked eventually, since the logical tag for that type of question isn't attached for it.

While base-db is just getting started as a concept, I've also created a tag pi-db for OSIsoft's PI database that has a huge userbase all over the world, but the community here just hasn't been built yet.

I don't think we should aim StackOverflow to just the hot topics, we should be seeking to attract all programming communities, to make this THE preeminent place to ask programming questions. (Note this is biased from my own laziness, since I want to only have to go to one site to find all my answers).

Answer 10

I think any tag that is only used a few times over a few months should be hidden and not shown on any questions or in the list of tags.

If it gets type into the tag box, it should then be unhidden for a month.

That way if it takes a few month before a tag is used more then 1 or 2 times, we don't loose the tagging info from the first questions it was used on.

Answer 11

I added a tag racket with only one question because the PLT-Scheme project are changing their name to Racket (currently offline). This was at the request of a user who wanted to add the tag themselves but could not because they didn't have enough reputation at the time. If the tag is removed then the same problem will happen again the next time someone tries to ask a Racket question. Removing rarely used tags every time it hits 1 usage will make it unlikely that it will ever be possible to use them. In this case an alternative would be to make plt-scheme a synonym for racket.

Another tag I created with only one question is android-send-me-logs for the project android-send-me-logs. Again the user that asked the question would not have had sufficient reputation to create this tag themselves.

Your proposal will result in the removal of these two tags, plus I'm sure many other tags in a similar position. Will this benefit the site? And will leaving these tags cause any harm?

Answer 12

Coming to the party late, I have a minor concern.

Some people don't include the name of the technology they are using in the questions, assuming that putting it in the tag is sufficient.

Some of the untagged questions are going to degenerate into meaninglessness if you remove their tags.

Would it be possible to automatically modify the question text to append say "[Tag removed: xxyzy]"?

Answer 13

Here is an argument against zapping low-occurrence tags - they might be needed to meet site standards, or to convey important context for the question.

As an example, on Arqade, there is a de facto standard that each question should carry a tag identifying the game being asked about. In some cases, posters omit to identify the game in the title and/or body of the question, assuming that the tag is sufficient. (Sometimes this is intentional for entertainment value, if the question sounds particularly bizarre out of context, as in the currently top-voted question How can I tell if a corpse is safe to eat?)

But if there are very few questions about a particular game, the corresponding tag might have very low occurrence. For instance, there seems to be only one question about the game Pocket Harvest: How do I improve the rating of my crops?, and so the tag pocket-harvest has only one occurrence. If this tag were zapped, the question would be out of compliance with the "tag identifying the game" rule; worse, since the game's name is not mentioned in either the title or body, the question itself would become incomprehensible.

There are other sites with similar practices:

• On Law, tags are used to identify the jurisdiction for the question. Currently there is only one question tagged russia: What do you do if your consulate general violates the law?

• On Motor Vehicle Maintenance and Repair, tags are used to identify the make and model of the vehicle in question. Currently there is only one question tagged winnebago: Installed new alternator, now no power

• On Movies & TV, tags are used to identify the title of the movie or show in question. Currently there is only one question tagged tinker-tailor-soldier-spy, asked by me: How did Smiley know he could trust a certain person? (Caution, it contains spoilers.)

In these cases, the fact that a tag has low occurrence does not necessarily suggest that it is likely to be incorrect, nor that it can be safely removed - merely that it asks about something relatively obscure. (It should also be kept in mind that many SE sites have dramatically lower traffic than SO, so something that is "relatively obscure" for that site may actually be quite well known in absolute terms, like russia.)

In conclusion, at least for such sites, no, we should not zap the low-occurrence tags.

Answer 14

Here's a datadump query that will give single-use tags older than 3 months:

With SingleTags As
(
  SELECT Tag FROM PostsTags GROUP BY Tag HAVING Count(*) = 1
)
SELECT st.tag, p.ID As PostID, p.Title, p.CreationDate 
FROM SingleTags st
INNER JOIN PostsTags pt ON pt.tag = st.tag 
INNER JOIN Posts p ON p.id = pt.postid 
WHERE p.CreationDate < DateAdd(m, -3, getdate())
ORDER BY p.CreationDate

And I know it's not stable right now, but here's the StackQL results:
http://jcoehoorn.dyndns.org/stackql/default.aspx?qid=10012
And here's a similar query on the StackExchange Data Explorer:
http://odata.stackexchange.com/stackoverflow/q/427

Quite a few of these tags are already cleaned up.

Answer 15

How many of Stack Overflow's current tags have at least a 6 month gap between their first and second use? Probably not very many (if any - would make an interesting odata query). Certainly not enough to justify keeping the single-use tags around.

Answer 16

What is the point of keeping tags that are used by one or few users? When tags is quite popular, I can hope that someone will view questions with this tag and, coming across the problem he knows the solution, he will answer it. When tag has 10 questions, who will visit its page regularly? Except for search engines?

Such tags, in my opinion, poisons the tag system, because many users use such tags, when other, more popular, would be appropiate and would increase their chances to get answered. When there's too much options, making good choise become more challenging.

The exception are, of course, new technologies, which began to be popular and tags for them should be created, so that other users will see it and use when needed.