“Serious problem” is relative to the post, not to the whole site. If an answer should not be on the site at all, then it's a pretty serious problem with the answer. Flag away.
There is admittedly something amiss with the wording, so you should read the text that you see after selecting the “not an answer” option and before clicking “Submit”. “Not an answer” flags are for posts that:
- contain no information at all;
- or are evidently (even to a non-expert) wholly unrelated with the question (not just relevant material with a long digression, or a subtle misunderstanding of the question);
- or should be an edit to the question or to another answer;
- or should be a comment on the question or another answer (if the comment should be on another answer and it's not immediately apparent which one, use a custom flag to indicate that).
On the other hand, if the post is relevant but wrong, this may be a serious problem, but it is not a cause for moderator action. Downvote, but don't flag.
Also, if the post is spam, there's a separate flag for that.
For guidelines as to what should be an answer, see What is an acceptable answer?