Answers are, in some ways, harder to clean up than comments as they have no utility outside of moderator intervention to remove quickly. Comments can be flagged because one of the flag reasons is noise; there is no equal option for answers. There's also that whole thing where every time a new user posts a comment-as-answer... there's often a very unwelcoming surge of downvotes that follow it.
However, answers are a lot more visible to receive that clean up. Posting an answer bumps it to the front page. There is also the "new posts by new users" list which helps identify these very comment-as-answer posts. Comparatively, there is no real method to monitor for comments apart from manually opening every post or every user's activity logs. If a comment is placed on a really old post, then it can remain unseen for a very long time. Multiply that by a lot to see how problematic it can become when that's granted to everyone.
That's speculation, of course; we don't have exactly proof that this is opening some floodgate and I don't even think it'll turn that bad. But I think it would still be wiser that we have some method conducive to policing and monitoring comments better before we reduce the threshold.