I deal with the question itself first. If it's a possibly salvageable question that the asker can fix with more effort, I comment along those lines to try and help. If it's phrasing, formatting, grammar, or something else that can be edited to fix, I do so.
If the question is beyond salvation, or after a period of time the asker hasn't improved it, I vote to close and (sometimes depending on the question) downvite it.
I frequently leave comments on answers posted to questions that are off-topic, asking the person who answered not to do so because it encourages other off-topic questions.
I also often comment on answers posted to poor or no-effort questions asking the poster not to answer them because it removes any incentive for them to edit and improve it (because they've already gotten it answered) and because, again, it encourages other poor or low-effort questions.
In either case, I usually get a comment back. They typically either are
- From the answerer, who defends their reason for answering anyway (often rather rudely), in which case I downvote to indicate disagreement with their answering the question.
- From the answerer, who says something along the lines of "Sorry. I didn't realize/know that." and removes the answer, in which case there's no need (and nothing to) downvote.
So my position is that I try to teach the person who answered the reasons to avoid answering these types of questions, give them an opportunity first to do something constructive themselves, and then if they choose not to do so I downvote the answer to discourage them from doing so in the future (and almost always leave a comment stating why I'm doing so).