I'd like to expand on this idea so that it applies to both the Close and Reopen queues in both directions. Editing, while useful, is not always an indication that the question should be left open or that it should be reopened. As you've stated, there are sometimes cases where editing is just for the sake of editing, but you still want to close the question or you don't want to reopen the question, and it's inconvenient to have to go to the question itself to edit.
I propose that the edit screen in both these queues act similar to improving a suggested edit. Add a checkbox down below the preview so the user can indicate what primary action they want to complete along with their edit.
In the Close Votes queue:
[ ] Vote to close this question also (you will see the dialog when you save your edit)
The box should be checked by default and if the box is checked at the time of attempting to submit the edit, bring up the close dialog so they can also select the reason for closure, then submit both of them at once.
In the Reopen Votes queue:
[ ] Vote to reopen this question also
Similarly, the box should be checked by default, and upon submission of the edit (if checked), bring up the normal "Are you sure you want to reopen?" confirmation dialog for reopening questions.