I have not downvoted anything. I can not see in my account that someone else downvoted one of my answers. Though I drop from about 852 to 813 reputation. Why? This is not the first time.
The most likely answer is that a question or answer which gained you reputation was deleted or migrated to another site, and then your reputation was recalculated.