I was going to suggest something similar today, until I found this question, already asked eight years ago.
My suggestion would be to have such a notification feature not only for tags which are clearly debunking a question as off-topic, but where usage indicates there is a certain risk for the question to be be off-topic. Candidates for this tag feature could be, for example, the licensing tag or the open-source tag on Stackoverflow or Softwareengineering.
Though both of this tags have well-written recommendations to ask licensing questions not on the former sites, but either on Law.SE or OpenSource.SE, new askers seem to overlook those hints regularly.
On the other hand, there are still certain on-topic questions on Stackoverflow or Softwareengineering where the former tags can be applied, which makes @ChrisF's idea for trying to remove the tags completely not really suitable for them.
For implementing such a feature, the question to be answerered here is: for which tags should this notification be activated, and who can decide about it? I can think of some alternatives here:
Only users of a certain reputation should be able to activate or deactivate it (ar least 10K), or maybe only site mods
Tags where the closage (including migration) percentage from the past was higher than, lets say, 75% should get this kind of notification.
One has to take into account that once this feature is available, it could influence the closage percentage, so maybe the feature should be activated automatically when the closage percentage goes beyond a threshold, and deactivated when only manually by a diamond mod.