The issue of questions bumping after retagging has come up before several times, e.g. here.
I have a concrete suggestion: disable bumping for users with high enough reputation (10k, 15k or 20k), or at least for moderators.
The main reason which is given for this bumping is that retagging might be erroneous, but I think that high-rep users can be trusted.
Edit: Here's a concreter suggestion. Users with high-rep can optionally check a box saying "don't bump", as long as all they do is retagging. Their retagging is then pending for approval by a moderator (or a trusted user, if you want to spare the mods).
What we gain: we can retag to reduce clutter without disrupting ongoing questions.
What we lose: a moderator can mess up tags, either on purpose or because of negligence.
In my view, in this case
gain > loss.
Originally asked on the math meta.