Comments usually provide a relatively temporary conversation. If a series of comments provides a lot of value in itself, then it's a good idea to promote those comments as answers - either separately or as part of the original answer. At that point, they can be cleaned up.
Obviously the comments themselves aren't temporary, but I think their purpose is usually transient - things like suggesting changes to the answer. As such, so long as they get the point across, I don't think there's much need for cleanup. I certainly wouldn't start flagging comments which were simply ungrammatical. I'd view that as an abuse of the flagging system, personally.
While it's a good idea for all of us to do our best when it comes to communicating clearly, it's obvious that some people have natural disadvantages - dyslexia being one example, and not having English as a native language being another. Such users aren't being abusive, they just don't find it as easy to write in correct English.