The "reputation" tab in your user profile already shows events that cause your total reputation score to decrease. For example, it shows downvotes that you have cast on other users' answers (-1 each) and it shows downvotes that other users have cast against your posts (-2 each).
(However, not all of this information is available to other users browsing your profile, so there's no cause for concern that people might retaliate against your downvotes. See here for details.)
The only two things I can think of that can cause you to lose reputation but are not currently shown are:
- having one of your previously-accepted answers unaccepted (effectively a -15 point penalty)
- having one of your answers flagged as spam and automatically deleted by the system (-100 points)
The former has been feature requested here, and there seems to be little to no reason to provide a history of the latter. The developers really shouldn't waste any time improving the site for spammers.