Due to the reputation cap, you can only get 200 reputation per day so it's not like those upvotes are severely affecting your reputation. If you really feel so bad about getting it all from one answer, go answer some other questions and get upvotes that don't count. That way you can pretend that you got reputation from those answers and the other one just filled in the rest. Kind of like what I'm doing right now. I've already hit the reputation cap here on meta for the day, but I'm still answering anyways. :)
Really there's nothing to be worried about. Some questions just attract a lot of views, especially when someone like Jon Skeet answers them. Lots of views sometimes equates to lots of upvotes on unexpecting answers.