It's been pointed out that "score" is a bit confusing for new users, especially given that it already has a perfectly ordinary meaning that's not obviously out of scope. But the OP is correct that "votes" is not merely a bit confusing, but wrong and misleading in important ways. And "vote score" or similar, while neither misleading nor confusing, is too long to fit.
Instead of any of those, then, I suggest using "voting". This has the advantage of clearly tying back to concept of votes up or down, but without implying that it actually counts the total number of votes. Its meaning is deliberately skewed from the normal in an obvious way, so users unfamiliar with it, rather than being misled, will know that the site is using it in an idiosyncratic way, and will know to find out what the actual meaning is. A tooltip can then inform them efficiently that it's the sum of up and down votes.
In other words, treat it as a short caption for a figure ("for question quality, see fig 1-21, voting"), rather than an inline phrase ("24 votes", "Score 2").
scorealone. Not sure that would be the best word, but I can't think of an alternative that has voting semantics attached (perhaps someone on English can help...)
score. Score, in general sense, means how much point or runs or goals i have did.... e.g. In cricket, if i say i scored 200 runs, it means i made 200 runs without taking into account of others. No plus or minus. The term should be change from votes but
scoreisn't appropriate in my opinion.
Net Voteswould work better? it's shorter than vote score (2 only characters longer than answers for sizing concerns), more accurate than just votes and avoids the gamification issue that people have with the word score.
ratingbut it isn't a unit either.