The safe clickable area for selecting a favorite tag (especially for one letter tags, illustrated above) is quite small compared to the size of the remove button right next to it. Considering that the user is much more likely to want to filter by a tag rather than delete it, how about we move that option somewhere else.
Jeff even wrote about this situation on Coding Horror.
zat all, or adding some new UI for suppressing accidental tag removing
xin my tags. I was so pleased I came to meta, searched to find an announcement about it, found this request and left my comment.
r, where the "delete" icon takes up most of the clickable area.