Currently, the favorite and ignored tags section in the sidebar directly allows one to add new tags and remove existing ones. At least for me, adding and removing tags is something I do only occasionally, by far the most common use is passive, using the tags to navigate.
This is how my tag area in the sidebar looks on SO:
And this is how I would propose to change it:
The default view (if you have already at least one tag set as favorite or ignored) would be the passive view, that would not allow for editing of the tags. If you wanted to edit, you would click on "manage tags" which would expand the tag UI to the current state.
There are several reasons why I would change this:
The current approach wastes a lot of space on the rarely used form field to add tags. The forms are also rather eye-catching due to the high contrast on the buttons, this emphasizes a rather less important part of the page.
It is currently very easy to accidentally remove a tag. As there is no warning or undo, you have to add the tag again manually, which is rather annoying. Making the default view read-only would remove this problem.