No, the detailed voting information displayed under the "reputation" tab in your profile is not available to other users, including moderators. Downvotes really are intended to be anonymous.
We can see where you lost 2 points for a particular post (either a question or answer) of yours being downvoted by others, but we can't see detailed information about the posts by other people that you have downvoted. For example, here's what I see when I snoop around on your profile:

More detailed information about when exactly you downvoted was publically available for a while after the feature was rolled out, but that was determined to be a bug and fixed.
It's also worth suggesting that in the future, you can generally verify this sort of thing either by logging out of the site completely, or by enabling your browser's "private browsing"/"incognito"/whatever feature, and viewing the page in question.