How can irrelevant answers (that don't answer the question) be removed?

Question

For concision, I abbreviate 'an answer that doesn't answer the OP' to NS, to mean 'non sequitur'.

I once asked on a Meta website somewhere (I forgot where, but please edit this OP if you find it) whether the existence of an answer to your question, correlates with the probability of receiving another answer. The answer was definitively positive: If your question received one answer already, then the likelihood of receiving another answer decreases with each subsequent answer
(ie: A second answer is less likely, a third even less likely, ...).

So how can NSs be removed, to ensure due attention and answers to questions that truly haven't been answered?

I already thought of, to no avail:

1. flagging for moderators' attention, either for the NS to be deleted or to be converted into a comment. However, I'm unsure whether moderators can do so.

2. commenting underneath the NS to raise this concern to the answerer, but the answerer may never edit the answer or comment.

3. downvoting, but this fails to remove the NS.
3.1. I personally eschew downvotes and never downvote on a Main site, because I prefer joyful, constructive criticism and positivity.
3.2. Even if I downvote, 1 lone downvote seems futile.
Moreover, an unjustified upvote for the NS would negate this downvote.

Answer 1

Generally, you should downvote (because that's exactly what downvotes are for: answers that are not useful). Avoiding use of a function that's designed to be crucial to site functionality because it doesn't feel "nice" is, I really think, misplaced sympathy. How else are such answerers — or other potential answerers reading those — to be effectively taught that such answers are unhelpful?

In any case, though, you should comment and flag as Very Low Quality — the comment is more so those who review (not generally ♦ moderators) know what to look for. On most sites this should remove the answer within a few hours, and not infrequently within a few minutes.

---

Note that a positively-scored answer cannot be flagged VLQ, so you may need to downvote in such cases whether or not you normally would prefer to.

To address your edit, single votes (up, down, close, delete, reopen...) are of course always prone to getting canceled out for no good reason. That's no justification for simply never casting them in the first place: most of the time they won't be canceled out, and in any case, where do you expect large accumulations of uncanceled votes to come from if not from individual votes that each have some risk of being canceled? A single vote has little effect, but "little" is not the same as "none" when the system is designed to encourage small efforts from many people.

Vote.

Answer 2

You should flag it as Not An Answer if it is not an answer. If it appears to be an answer (but perhaps to a different question), first comment, something like this:

The question asks how to tell when a roast chicken is done. A recipe for stock made with the carcass from a roast chicken is not an answer to that question.

You leave the comment partly to educate the answerer and partly in case a moderator looks at only the answer. Without the question a mod might conclude "this looks like an answer to me" so the comment helps to point out the issue.

You don't need the comment if the "answer" is a clarifying question, "I have this too", "did you ever solve this?" and the like. These are exactly what Not An Answer is for. The more you use it the better the site gets.