Yes please! Most questions get migrated with only a vote or two, so it will be no great loss to the question.
The ones that do come with votes are usually wildly skewed one way or the other. Bikeshed questions are hard enough to keep in their place when they don't come in with more vote weight than the target community could have put. Also answer votes are often out of balance because the source community didn't understand the question or have the right knowledge to solve it or judge answers.
Addendum: The primary concern with this historically seems to be the issue of migrating old questions. This is a legitimate concern, but I believe it should be solved another way. Migrating old questions has its own set of problems. It has been suggested that not resetting votes is a good deterrent for migrating old questions that have already collected lots of activity. I suggest that A) having votes reset on highly voted content would actually be more of a deterrent and that B) this is often the most problematic on very new questions.
By that last point, I mean that when a question comes along on a high traffic site, it can pick up answers and a whole pile of votes in just a few minutes. On a lower traffic more specific site that it gets migrated to, these votes are specifically the ones that most need resetting because they were based on the very different standards of the high traffic site.