How is consensus determined on Meta sites?

Question

On our main sites "the best answer" is somewhat fluidly defined as the superposition of the highest voted answer and the accepted answer. Truth be told, the system is rather insistent on allowing multiple "right answers" and that's one of its strengths: you can be the judge of who you believe.

But that doesn't always translate well on the per-site metas, which serve as a sort of policy-creation body. People who are active on meta help decide how policies are enforced on the main site. Moderators who have ♦-powers tend to take mandates on meta very seriously. Practically speaking, accepted answers to meta-questions are no different than any other answer. Does that leave just votes to determine consensus?

Closely related: Not everyone participates on meta-sites. It can be hard to tell if enough people have looked and voted on questions for real consensus to be declared. Size of the site determines how many people could be voting on questions. What counts as a quorum?

Finally, sometimes people change their minds about a meta-question, but, since votes are locked on meta, it's not really possible to change one's mind without a pointless edit to someone else's answer. When should meta-questions be revisited?

---

On graduated sites, ♦ moderators are elected, which makes them representatives of the active participants, but on beta sites, we are appointed. It's doubly pressing for beta sites to determine "the will of the people" since part of the point of a beta is to get a working policy framework in place before electing permanent moderators.

I'm interested in figuring out how sites already answer these questions and what might be done network-wide to make discovering consensus a little bit easier.

---

N.B.: Obviously this assumes that consensus matters. It does to me when I'm making my decisions and I suspect it matters to others. But that isn't to say that it always makes a difference. I'm really not interested in answers that hope to use this question as a grindstone for all the axes stored up against certain people who have made unpopular decisions and have the power to make those decisions stick. Feel free to ask your own question(s). (Thank you.)

Answer 1

As Sam I am notes, it's a judgement call...

Meta sites are a perversion of the Stack Exchange model, discussion sites implemented using a Q&A engine. This works surprisingly well, if you keep the limitations of both in mind at all times:

• Voting isn't limited - you can post an opinion and down-vote a conflicting option. Or upvote it. Or find two opposing view-points and vote the same way on both. Or leave disemboweling comments without voting at all.

• Sometimes good ideas are expressed very, very poorly and are down-ranked. Sometimes bad ideas are expressed very well, and garner some popularity for that.

• Sometimes folks propose strategies that would break the entire site if followed... And they end up ranked highly because a lot of folks don't particularly like the site they're on, and wish it was the sort of site that gave away more free ice cream and less disemboweling criticism.

• Sometimes folks post impassioned pleas for help and are down-ranked because you're all sadists. Sometimes folks just post rants, and are up-ranked because you're all masochists.

• Sometimes folks post well-reasoned, insightful analysis followed by a brilliant solution... And are ignored completely because tl;dr man, ain't nobody got time for that.

In other words, you need to interpret the outcome of each discussion or proposal individually. There's no hard number or ratio that will guarantee that you're doing the right thing; let folks discuss, when they've had a chance, make a decision and implement it.

Folks are terrible at letting you know what they really want, worse at letting you know what they need... But really, really adept at informing you when you've made a bad call.

So keep your ears open, and do what you believe is right for the site until you're convinced otherwise. Meta is a tool toward this end, nothing more.

Answer 2

Judging by the history of certain policy decisions, consensus is reached when 80-90% of the community agrees on an issue, and one staff member unilaterally decides that all those people are all wrong.

Or in more simple terms: Consensus doesn't matter around here.

Answer 3

Construct meta-questions for the purpose of gaining consensus.

Sometime a per-site meta "looks like a place to go with problems." Many questions are useful for allowing folks to vent their frustration. But meta (especially on beta sites) is supposed to be where participants shape the future of their sites. When you would like to find consensus, build your questions for that purpose:

• Some issues benefit from a simple up-down vote. In that case, encourage voters to vote on the question itself and make clear what a vote means. (Usually, a simple majority means consensus. One tricky bit is that the asker can't vote on their own question.)

• Other times, there are several viable options which are clearly delineated. In those cases, pre-filling the choices and encouraging people to vote up or down each one can produce a clear winner. (Again, the person filling in the answers can't themselves vote.)

• When you don't have a clear set of predetermined options, simply ask people to write in their suggestions. It helps to answer with an example or two to get the ball rolling and to give people an idea of how to format their answers.

• For open-ended policy questions, you might get lucky and have a clear consensus emerge. But more likely, you'll get a handful of answers that overlap and contradict. When that happens, the consensus is (more or less) the intersect of the top answers. Edits or summary answers can clear up the issues, but you might need to ask a more focused question (such as those above). Or (and this is what all the other answers are getting at) you might need to live with a little bit of ambiguity.

Note that the more carefully the asker constructs their question to tease out consensus, the more useful the voting is. In no case does the accepted answer mean much more than an indication of which answer the asker themselves prefers.

Quorum has more to do with time than population.

Some people don't care about meta and that's fine. After a while (especially on a small site) you get a feel for who will participate on meta-questions. So the key is giving those folks enough time to weigh in. Some questions require more thought than others as well. If there have been a lot of changes recently, it might be time to let the dust settle and see how things are going on the main site. If questions aren't getting a lot of votes, it might be a sign that people don't care or it might be a sign that they haven't decided yet. In either case, patience is a virtue.

Try to build on past agreement.

Usually, once a consensus is found, something significant needs to come up in order to revisit it. The onus is on the person who is arguing for change, not on the status quo. If you really want to argue for a change, it doesn't hurt to open a new meta-question. It's even fine to make a trivial edit in order to change your vote, in my opinion. But the person revisiting a prior consensus really needs to show that something material to the question has changed. Otherwise, a site can end up on a treadmill of meta-questions.

Sometimes consensus isn't the right way to make decisions.

The question assumes that consensus is something decision makers should look at. But that's not always the case. Other answers seem to address this point well enough.

Answer 4

It depends on how much it doesn't keep you up at night and doesn't drag you through long comment chains.

All happy sites are alike; each unhappy site is unhappy in its own way.

Knowing when that balance is reached is a measure of how well you know that site's community and what numbers actually turn out to kick up a fuss, if any.

Some sites have piddling meta activity, while others rage like crotch burn. Here, hard numbers or percentages will only exist to make you feel bad.

Understand the users, read the arguments, and then go with what makes sense or the least amount of stupid.

Answer 5

This is a bit of a meta-answer, but whatever.

SEinc calls upon new sites to build communities. This process has just about nothing in common with the management of the network as a whole, or of the big three. Yes, things like the definition of the close-worthy questions do evolve organically out of the giant compost heap, but rarely through a discussion on meta that reaches a consensus.

Since the team asks beta sites to build communities, it's not unreasonable for Jon to ask for the experiences of prior sites in pulling this off. Obviously, some have succeeded, and others not so much. Even if there's considerable variation in their experiences, all of them are potentially illuminating. Perhaps the cynical, scarred veterans of 'meta as the meta of stackoverflow' should stand aside and let some 'meta as the meta of the entire network' light shine through.

Answer 6

The answer by @Shog9 matches my feeling of how your question is best answered but I want to add why I think a pure count of votes provides an unwise measure to use.

It is because of how numbers in the reputation leagues are heavily biased towards inexperienced Stack Exchange users. For example, on this site today we have:

Total Rep* |   Users
100,000+   |      17
50,000+    |      44
25,000+    |     110
10,000+    |     272
5,000+     |     556
3,000+     |     869
2,000+     |   1,227
1,000+     |   2,255
500+       |   4,038
200+       |  10,102
1+         | 524,589

* users with less than 200 reputation are not tracked in the leagues

The vote up privilege comes at a reputation of 15 and the vote down privilege comes at 100.

From the reputation league we can estimate that there are many more than 10,000 users who can vote down, and many more than that who can vote up.

In contrast only 869 users have the cast close and reopen votes privilege, and it is only these users who have gained sufficient trust to understand when questions should be closed and re-opened.

As a hypothetical, if a feature request is made to abolish closing of votes then I would expect a large number of inexperienced users (who may have recently seen their first question closed) might upvote it, and there might not be enough experienced users (who have come to understand how important close voting is) available to outvote it (using a combination of downvotes, and upvotes on answers suggesting the opposite).