This is rather difficult to describe with fancy pictures with red circles but it's easily reproducible.
Whenever I click Leave Closed in the re-open queue on MSO the "nominate this question for re-opening" pop-up appears. I normally press escape out of shock and a leave closed vote is recorded.
This is the latest review it's happened to me on: https://meta.stackoverflow.com/review/reopen/15029
You can also test it on https://meta.stackoverflow.com/review/reopen/15043, I clicked reopen this time and a leave closed vote was recorded but I also got a red popup stating that an error had occurred.
This is also occurring when the Edit & Reopen button is clicked, the edit box opens for a second before being overridden by the popup.