I agree wholeheartedly with hammar's comment:
It serves a purpose in teaching new users how the system works, and how many votes are required to close a question, so I don't think it should be removed completely.
Thus, the best solution is to stop showing this pop-up after a user gets to a certain reputation level, when we can reasonably assume that they already know and understand how the system works.
There are already lots of other pop-ups and bells that stop appearing after a certain reputation level, like the prompt to leave a comment when downvoting or the suggestion to share your answer to a question on a social networking service. There's no reason that this can't be implemented similarly. After a user achieves, say, 10k reputation, we would stop showing the pop-up explaining how close voting works.
Then again, I could easily be persuaded that this isn't necessary at all. I cast a lot of close votes and thus see this pop-up a lot, but since I understand how the system works, I just ignore it. I don't bother to close it, since it isn't modal and doesn't interfere with my workflow. In fact, it fades away automatically after a certain period of time, which means I don't have to interact with it at all. slhck echoes these sentiments in a comment:
Is it that annoying? I'll just close the tab and move on.