I agree, here's my addition to your feature request:
You should be able undo comment upvotes, with a similar restriction as other upvotes: only within X time of casting the vote. It should have a similar UI too: click the upvote button (which is now highlighted) again.
Instead of storing a "cancel vote" in the DB (as is done for posts), the entire record should be deleted (this should alleviate the technical concerns frequently brought up when changes to regular voting are discussed). With a suitably short time limit (15-60 seconds) to change the vote, there is no potential for abuse. If comment editing is a concern for you (e.g. vote -> edit to say completely different), then simply don't vote until the comment is 5 minutes old (the current editing limit).