Kinda had my hands full last week when I answered this; re-reading it today, I think my original answer was a bit glib. Lemme see if I can convey my thoughts in a way that's a bit more respectful of your concerns...
Let's back up a step and review the charter for electing moderators:
The community is the source of everything useful that happens to exist on our websites. We gladly reciprocate by trusting you to lead and govern your own community. Democratically elected community moderators are the ultimate goal of, and foundation for, every site in our network.
-- Stack Exchange Moderator Elections Begin
Who is this "community"?
Community is one of those words that gets tossed around so often that it quickly becomes trite. And yet, that is the explicit goal of these sites: to be more than just dead repositories of information, but rather a living collaboration among individuals dedicated to the subject. So who does "community" refer to here, in the context of elections?
Well, what is the outcome of an election? A team of elected moderators:
We designed the Stack Exchange network engine to be mostly self-regulating, in that we amortize the overall moderation cost of the system across thousands of teeny-tiny slices of effort contributed by regular, everyday users
...
Even with active community self-regulation, moderators occasionally need to intervene. Moderators are human exception handlers, there to deal with those (hopefully rare) exceptional conditions that should not normally happen, but when they do, they can bring your entire community to a screaming halt -- if you don't have human exception handling in place.
...
As a moderator, your actions now represent the community, so you will be held to a higher standard of behavior. You are an ambassador of trust, with the same sorts of rights that the official development team and community coordinators have.
A moderator is a representative, trusted to act on the behalf of the regular users in situations where they cannot self-moderate. This, then, is the "community" who moderators are elected to represent: anyone who has ever participated constructively, anyone trusted to perform one of the countless small actions that - combined - account for the bulk of moderation performed on the site.
This is probably a lot more people than you would tend to think of when thinking of a site, especially if you're not actively involved on the site day-to-day. Heck... On any site running elections, there's a good chance this is more people than you would tend to think of even if you are actively involved on the site day-to-day. The true power of these sites is always found in the long tail, and that applies to moderation as well: we don't rely on small social groups to possess the breadth of knowledge for a given site's topics, nor can we afford to do so when it comes to moderating them.
Why elect moderators?
Given the (relative!) insignificance of moderators when it comes to the vast bulk of moderation tasks, why do we bother with all this rigmarole when choosing them? It comes down to trust: although relatively infrequent, these elected moderators will often be asked to intervene in situations that are extraordinarily complicated, sensitive, visible, or all of the above. In order to resolve these potentially catastrophic problems, they will have to make hard decisions, take controversial actions, and express strong opinions. And they will do all of this in the name of the community that they represent.
...This is awful hard to do when the community doesn't accept that you represent them.
When sites are very new, moderators are still needed; until elections can be organized, that role is filled by employees (like myself) and then pro tempore moderators (which are appointed by employees, usually based on recommendations from the community). Finally, when the community has reached a size sufficient to allow an election to be completed successfully, moderators are elected directly.
This process is based on 6+ years of... Well, mostly trial, error and observation:
We observed that when employees moderate, the level of trust accorded them by most communities is low: on most sites, we don't actually qualify as members of the community, so it is hard to see us as representatives.
We observed that when appointed moderators moderate, the level of trust accorded them is higher but still insufficient for the most controversial of actions: even when they've earned the support of the initial core membership (as indicated by voting on meta), the vast bulk of the site's community - as defined above - see them as no different from themselves and can take umbrage at the idea that they should be given more trust.
We observed that elected moderators - when chosen by a public and well-publicized election - are frequently able to make the same decisions as pro tempore mods or employees without the associated controversy. Indeed, it has become commonplace to see a site divided by indecision over one issue or another reunite after their first election. This harmony is so valuable as to outweigh all the disadvantages inherent in running elections the way we do...
Why solicit broad participation?
So now hopefully we have enough information to answer the question that underlies your feature-request: why would we trust someone to vote in an election if they aren't involved enough in the site to already know that an election is occurring?
And the answer is simply this: we need them to trust that the moderator who is eventually elected will represent them.
In other words, the voter themselves is more important than the moderator they vote for. The election is conducted for their benefit, as a ritual to establish trust between those who care at least enough to vote and those who will eventually be elected. When, in the past, we've failed to include people in this ritual, the outcry has been immediate and insistent.
We can and should do as much as possible to ensure that voters have enough data to make informed decisions. But just as important is simply ensuring that they are welcomed and empowered to participate in the first place, so that sufficient trust exists for the moderators who are eventually elected to perform the tasks they were elected for.
As Jeff once wrote,
Democracy is a highly imperfect process. But it is a participatory imperfect process. Please participate [...] -- even if only as an observer -- and give us feedback on how we can improve the moderator election process to better serve your community.