It's been pointed out that "score" is a bit confusing for new users, especially given that it already has a perfectly ordinary meaning that's not obviously out of scope. But the OP is correct that "votes" is not merely a bit confusing, but wrong and misleading in important ways. And "vote score" or similar, while neither misleading nor confusing, is too long to fit.
Instead of any of those, then, I suggest using "voting". This has the advantage of clearly tying back to concept of votes up or down, but without implying that it actually counts the total number of votes. Its meaning is deliberately skewed from the normal in an obvious way, so users unfamiliar with it, rather than being misled, will know that the site is using it in an idiosyncratic way, and will know to find out what the actual meaning is. A tooltip can then inform them efficiently that it's the sum of up and down votes.
In other words, treat it as a short caption for a figure ("for question quality, see fig 1-21, voting"), rather than an inline phrase ("24 votes", "Score 2").