ServerFault was interesting for a period of time until two users emailed and suggested that I stop answering questions in 'their tags'. I've since removed my email address, filtered their email addresses, and subsequently removed my account.
One user takes it upon himself to answer the question first with a brief, often incorrect answer and instantly gets two upvotes either through friends or additional accounts. After the correct answer is posted by someone else, that correct answer is downvoted and the contents copied to his higher ranked placeholder. At this point, the first, highly ranked answer gets more attention as the sheep agree, placing this person's reputation high enough that he can easily downvote, flag, and consequently edit answers. Twice, my answers were 'edited for clarity' removing enough substance from the correct answer that I was later downvoted. Once an answer has been flagged as an answer, reputation gains should stop and the answer should become immutable.
Second, the reputation mafia appears to communicate and upvote their friends answers regardless of the accuracy of the answer. Simple votelog analysis would probably make this more difficult to accomplish.
As I'm sure an upvote/downvote/reported log is kept, it should be fairly easy to determine if users are getting a number of votes from particular users that is above the norm. A modifier of the time between the answer and subsequent upvote and the frequency of those users would require additional work to game the system. Modifying an answer for clarity and adding the correct answer to an initial answer would have much different token signatures. An answer that was not similar could not be considered as simple editing. A revision history showing edits would also allow one to judge the answers. As some responders have political agendas, it is in their best interest to answer first and be upvoted then put the correct answer in.
If a high-reputation user owns a tag, don't display any questions in those tags until the question has aged or the tag owner has posted an answer/released the question. I do appreciate the questions that have arisen from December 2009 that show recent activity and two answers as 'unanswered'. Were these released by high-reputation users that gave up ownership of those tags? Documenting tag ownership would also go a long way towards building the community so that we don't improperly answer questions correctly and step on the toes of the high-reputation user that owns the tag.
Documenting that the ServerFault/StackOverflow/SuperUser community is a closed community for the elite and mentioning that newcomers will be downvoted when providing the correct answers might dissuade the newcomers that you don't want from wasting time. A double blind email system would prevent high-reputation users from harassing others, allowing those emails to be reported and blocked at the source. As I stupidly used my primary email address when I signed up, I now receive about six times more spam than ever, presumably from one of the high-reputation users that started receiving bounced emails when I blocked their email bombs.
