In addition to the excellent points described in Glorfindel's answer, one other thing to check is the question's timeline.
This is a chronological timeline of everything that happened on a question, including
- initial question ask
- answers
- revisions
- comments
- closure, deletion, reopen, undelete, etc
- votes
all based on when they happened in chronological order.
(source)
Note: the "post ID" is the number shown after the "questions/" part of the question's URL.
Among the more than 500 answers I've given so far on Math SE, the top 2, in terms of score, were due to at least some extent from special events listed in their timelines. As mentioned in Rob's comment to Glorfindel's answer, the post may have become a HNQ (i.e., Hot Network Question). Depending on how potentially interesting the question is, how long it's on the HNQ list, etc., it may get a lot of views, with a generally corresponding increase in votes for the question and answers. Here is the timeline of a question where my answer got many more up votes (32 in total) than I would have normally expected it to (I circled the pertinent part in red):
Although this happened just about 3 months ago, so it's definitely not on the HNQ now, there's no indication of when the question stopped being on that list. Also, there's at least one other math question I've seen on the HNQ list which I've answered & got a few more votes than I would have otherwise expected. One thing to note is that, I believe since many sites don't support MathJax, any question using MathJax in its title (e.g., like the other question I discuss below) will generally never be on the HNQ list.
Next, here is the timeline for the other question I mentioned:
In this case, the post was tweeted on StackMath's twitter account. I don't know how many other SE sites also have their own their own twitter accounts, or how popular they are. However, in this case, my answer fairly quickly got a lot of up votes, ending up at 15 in total.
Note most of my answers on math SE get at most 4 or 5 up votes, so these 2 answers are definitely not normal, or even what I consider to be among my best answers. Also, these 2 events, i.e., being on the HNQ and being tweeted, would normally happen quite soon after a question is posted, so it won't likely apply if you see something happening after a year or so. Regardless, if there are no other fairly obvious reasons, it's still worthwhile to check the timeline for the possibility of the question being on the HNQ, being tweeted, or something else occurring & being listed which may have caused the post to suddenly get at lot more views and votes.