Are you a moderator? If not, then you cannot moderate, of course. Or at least, not in something at this level.
I am not sure about what you mean by "remove votes", but if you're talking about "removing votes from upvoted comments" that would be an abuse, and even moderators cannot do it (I don't see why they should).
By the way, two points about comments:
Clarification comments and normal comments are welcome since they help the community shape the question (fix some wording, clarify a point, etc). When this is done and the changes have been applied to the question (when needed), comments should be removed. It's not like you need to do it right away, but they are not needed anymore (and here you see why comments are seen as ephemeral or second-class citizens). If you want to make sure someone is acknowledged for this you can write "Edited per username's suggestions" in the edit summary.
Comments that are only chatty and that do not add anything to the discussion should be removed (on Meta we are more tolerant), and you can flag these for moderator attention. Don't just flag everything or a comment just because they disagree with you or because they down-voted you. Flag comments that objectively are chatty and not constructive (or offensive where applicable).
Hope that helps.