So there's someone today who decided to pump me up to ten thousands (said with Vegeta voice) hunting my questions and +1 them. Now I went back to 9664... fact is that when I started today I was beyond 9800 (otherwise I would have hit the cap limit for the day, and never reached 10k). So my assumption is that he removed the +1 to my questions, and also others he voted in the past (and he unlocked, probably by editing them).
The fact (bug? lacking feature? planned feature?) is that I don't see in the daily report any -10 for the removed points. This is kind of expected (if you +1 someone and then remove the +1, he does not see +10 followed by -10. The +10 just disappears) but because in general the +1 is removed and added within the same day. If this happens much later, the points disappear from the grand total, but you have no clue where they were removed from.
I mark it as bug, but it's probably by design, and in any case a minor annoyance, like the "where the badge comes from" case.