Upvoting comments counts as an 'action' in the First Posts and Late Answers review queue, and therefore enables the "I'm Done" button. It makes sense that when you immediately unupvote the comment, the "No Action" button is the enabled one again. However, when you upvote two comments, and then unupvote one of them, this happens as well (if you didn't do any other action like voting on the post or editing it):
If you refresh the page, the system will enable the I'm Done button again, as expected. But refresh shouldn't be necessary.