This question already has an answer here:
When I unaccepted an existing answer and accepted my own instead, the summary report in the drop-down to the left of my username showed -2 rep change on that question. Hovering over the -2 showed the text
--2 reputation was lost in the last 0 days 0 hours 10 minutes 24 seconds
Double negative makes positive, and a positive amount lost is a negative gain, so the math works out :) .