What's broken 1: not everything is StackOverflowStack Overflow / Mathematics. Privileges and voting rights are much harder to gain on some networks. I know many on DSP.SE who are qualified to close vote, but can't.
What's broken 2: 1
-rep can't vote. Some users, especially experts, have nothing to ask or don't wish to participate in Q&A's - yet their votes remain valuable. "Non-participants shouldn't get to vote" - it takes 3 upvotes on a question to change this; good as nothing.
Two in one post because they're related, and emphasis on "not all is SO".
Solution 1: allow sites to decide their own privilege thresholds.
Solution 2: a verification system that addresses the abuse concerns of letting any new account vote. Phone + e-mail, mod review, are examples.