Recently I downgraded the incorrect answer on one of the questions. After that I lost around 30 points. Looking at my history I can tell that many of my old answers were recently (few hours ago) downgraded by 2 points. I suspect this was the user which question I downgraded. He later removed his answer.
How do you deal with such things?