Communities self-moderate based on the perceived accuracy and quality of questions and information provided by users. Our system works by letting people with experience (from enthusiast to expert) select for the best information and cause it to rise to the top through voting.
Thus, we need to have a fairly good sense that you can be an objective judge of accuracy and quality in any given domain, and require a few of your contributions within that topic space to be vetted prior to unlocking privileges that let you help the community curate and sort content.
An association bonus is provided to people that have previously demonstrated knowledge of how the software works to unlock basic privileges immediately, but you still need to demonstrate a modicum of knowledge in the topic space to do things like down-vote.
This design (in theory) allows everyone from enthusiasts to experts to upvote the best stuff, while allowing only those with more proven experience to downvote. How that works in practice varies on the objectivity of the topic, and in more subjective cases, who you ask :)
We probably need to do more to help folks new to the site to get enough rep to take on more curation and review roles, especially on sites where it seems like everything you could possibly ask or answer has already been asked or answered. But that's something we plan to work on this year.
So in short, Glorfindel's answer is spot on; I just wanted to put a little more color around it.