However, I noticed that it still exists on many other SE sites, notably Math, Physics and Chemistry. Should the tag be stopped before it explodes exponentially, or is the problem with it only programming-related, or is it simply the decision of each community?
Each community should decide for itself in its own Meta.
It could be a crucial site-building tag for one of the other sites, and it would be sad to see it disappear because it wasn't allowed a priori to have that tag.
This answer applies both the Physics.SE and Chemistry.SE, where homework is widely used.
Why we use it
I've explained this here, but I'll explain it again, in more detail:
Both sites have an HW policy (the Chem one is pretty much plagiarized from Physics). Now, identifying an HW question isn't always that easy. For example, the policy does not discriminate between questions asked because they were HW and questions asked for self-learning. (In fact, there are some questions which are HW but we don't consider them to be) It is the type of question that matters, and it isn't always easy to identify such questions.
This is unlike StackOverflow and other sites, where a question asked from HW is most of the time no different from a normal coding question. That's because programming homework is generally just "do X in language Y", which is similar to the bulk of questions on SO. On the other hand, HW questions on our sites are of the "Solve this." "Prove that." form (and the bulk of "good questions" are of the "why does..", "Explain why..", etc form -- completely different)
Why do we need to identify such questions in the first place? Simply, our policy doesn't allow full answers to HW questions. Hints or partial steps is all we provide for such answers.
Now, new users don't know this. So, when they post such an answer, we temporarily delete it with a comment like "Please provide only hints for HW questions, blah blah, link link, edit it and flag, we'll undelete. In the future, try to only give hints for questions tagged homework"
Basically, the tag is used as a sign to users "Please post hints only". It is much less obtrusive than hat-noting all HW problems or any other solution we can think of.
Chronological links to meta posts that show the evolution of the policies on both sites:
On removing it
I've already mentioned that it has a very important use (unlike most meta tags, which have no use). Till date, I still don't see anything "bad" with the tag as it is used on Physics and Chemistry. (I do see what is wrong with a homework tag used on SO, or a subjective/beginner tag used anywhere). I'd really prefer if that was clarified.
That being said, I personally don't mind it being removed, it's not that big a deal to me.