What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

At the moment, (AKAIK), when one casts a close vote, the only way to know whether the community agrees with you is to check back on that question manually to monitor it's status. And that's fine.

But with the recent changes to /Review, and the associated up tick in editing/close voting, etc., I've been thinking about ways to provide feedback to 3k users who are new to close votes, similar to (but obviously different from) flag weight.

Just as flag weight is meant to be a single number that (sorta, kinda) encapsulates how appropriately one flags content, I'm interested in a single number that (sorta, kinda) encapsulates how appropriately one uses close votes.

For instance, personally, I would be interested in seeing the following measure: If

p = proportion of questions where I cast the first 
     close vote that were eventually closed

q = proportion of all questions with at least one 
     close vote that are eventually closed

What is p/q or possibly log(p/q)?

Edit Since two people already badly misread this suggestion, I'm going to elaborate. Note that both p and q involve only the first close vote cast. So I'm specifically picking out those questions where I voted to close with no other close votes present, and asking whether those questions were closed more or less often than "normal". Casting votes 2-5 to close would have absolutely nothing to do with any of this.

While my main interest is curiosity (and I just kind of like numbers and stuff), I also thought that this (or something like it) might be helpful for users who are learning how to use close votes to see how in line they are with the overall community's sense of what should be closed or not.

Is measuring something like this possible, and if so, could it be useful?

share|improve this question
My understanding is that close votes do expire after a time. How would that affect your metrics. For example. If your the first to vote to close, but it expires and later someone also votes to close is their vote a first to close as well? –  Some Helpful Commenter Nov 23 '11 at 5:55
@ConradFrix I don't know. That's why part of my question was whether this was even possible to measure. My hunch is that both would qualify as "first" close votes for each voter, but I'm open to other suggestions. –  joran Nov 23 '11 at 6:07

1 Answer 1

I would be very reluctant to try this out. Feedback loops are an important component in control system theory, but improperly used they tend to oscillate and produce less than useful results.

The flag weight is ok because there's a "final authority" (the mods) who judge whether the input is valid or not.

There wouldn't be anything in the loop you're proposing, though, to balance the cycle of user --> community --> user feedback or prevent/dampen oscillations. This loop would largely encourage people to try to vote with the crowd, rather than judging a question on its own merit. Want to increase your close metric? Look for questions that already have one close vote, never start voting to close on your own unless you're certain it'll be closed. If you want to increase your number more, add comments to encourage others to close the question (rather than adding comments asking the OP to improve the question - which would negatively impact your score). Bump the question with the sole intention of getting more people to close it faster.

In other words, while following the community in general is good, telling everyone that their close votes are judged bad if they aren't in line with the community isn't. It will change their behavior, but the control input will no longer be whether the question is good or bad, but will be the user's best guess at what the community will judge the question to be. And once a question gets one close vote, it'll get jumped on as others attempt to increase their stat - again, regardless of the true value of the question. It turns closing into a meta-guessing game.

Another aspect is that knowing whether your close votes in general are "good" or "bad" relative to the community isn't that useful if you can't narrow the reason down. Perhaps you're great at closing off-topic questions, but you are terrible at all the other close types, and so you change your off-topic behavior assuming it's one the reasons your stat isn't very high - or vice versa. This means we need to break the number down for users so they can see which areas they follow the community in.

While I love stats and metrics, I don't see this as having an overall positive effect on Stack Exchange.

share|improve this answer
There are pieces of this I agree with, but you misunderstood one important point of my suggestion. The metric I propose only applies to the questions where a person casts the first close vote. Hence, casting votes 2-5 would have no effect on this metric. In order to "do well" you'd have to "vote well" in the absence of any other close votes. –  joran Nov 23 '11 at 4:13
Also, you'll note that I left open the question of whether this information would be tied to any privileges/badges. Indeed, it might even only be visible to you, rather than public. Following the herd too much is indeed a problem, but note that nearly every other SO behavior is subject to strong feedback (questions/answers/flags) except close votes. –  joran Nov 23 '11 at 4:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .