It sounds like you did the right thing. I compliment you on explaining what the problem with the answer is.
Downvoting and commenting is slightly "riskier" than just downvoting: there's a higher likelihood of drama, negative reactions, or revenge downvoting. In an ideal world, everyone would respond professionally and maturely to criticism, but we're not in an ideal world, so you have to be prepared that some small fraction of users might not react well to criticism.
For that reason, I know that some people try to avoid leaving a comment when they downvote. What you want to do is up to you. If you find it extremely bothersome to occasionally encounter hostile reactions when you comment and downvote, you might consider just downvoting without leaving a comment (or if you prefer, leaving a comment without downvoting).
There is absolutely no requirement to leave a comment when you downvote. Don't let anyone pressure you into commenting whenever you downvote.
Ideally, everyone would respond maturely to criticism, and everyone would leave constructive feedback on answers that are wrong -- but in practice that won't always be the case, so be prepared, and act accordingly. Personally, I try to act according to how I'd want to see everyone act in an ideal world, and ignore revenge downvoting and other negative reactions, in the spirit of "be the change you want to see in the world"... but you can choose your own path.