84

In a recent answer on Meta Stack Overflow, Shog9 made the following note:

Accuracy is all over the map too. Some of our most prolific close voters routinely toss votes at questions that don't get closed in a timely manner - or at all.

I realized something—I don't actually have any idea what percentage of the questions I vote to close actually get closed. My ability to cast close votes is relatively worthless if I just splatter close votes everywhere like buckshot.

We don't want people throwing close votes willy-nilly—we want them to be snipers, shooting down the terrible questions like a well-oiled machine. Unfortunately, currently, it's extremely hard for a user to determine if their perception of what should be closed matches up with the rest of the userbase's.

Clearly, this data is available. It's just not accessible to the average user. I don't think this data should be public to anyone who wants to look at it, but I do think it would be extremely helpful for someone to view their own close vote history.

Ideally, I'd like to have something like this available to me:

Mockup with close votes cast, close accuracy %, questions reopened %, a graph of successful votes and votes declined, a table of questions voted on and the results

Sure, this wouldn't help people who weren't interested in taking the time to look at the stats. But I think some people would be, and it would make them much more effective at closing wisely.

Improve this question
2
  • 13
    I don't think "Closed but reopened" is very informative if it doesn't take into account whether the question was edited before reopening. In principle the reopen question might be very different from the closed one. So if that is to be an indicator of questions that got closed although they shouldn't have been, it should probably only count questions that haven't been edited before reopening.
    – Martin Ender
    Feb 8, 2015 at 14:05
  • 9
    I think this would be interesting to know, if only to see how well you align with the rest of the community, but I personally am not too concerned with trying to push that number toward 100%. I feel that sometimes reviewers jump on the CV bandwagon when they have a lot of respect for the other reviewers in the community and don't always consider as carefully as they might if they were the ones initiating the CV. I think it's important to not have reviewers operating in lockstep.
    – ColleenV
    Feb 8, 2015 at 16:09

3 Answers 3

Reset to default
17

It's possible to find an estimate1 of your close accuracy yourself thanks to @rene.

  1. Run this query with your user id which will return all the undeleted, closed questions that you voted to close. The number of rows returned is the number we're looking for here.
  2. Go to your user page, click on votes, click on closure to get your total close votes cast.
  3. Divide the number of rows returned from the query by the number of total closure votes you have done, which gives you your "close accuracy".

For example, I ran the query for myself, and it returned 4,546 results. I have a total of 7,022 close votes cast on SO. So I have a 64.7% "close accuracy".

1 This isn't 100% accurate though since the SEDE query doesn't count now deleted questions that you voted to close. So in reality, most users actual close accuracy would be higher than the one produced by this method. Although2

2 On re-examining this, I think the /votes/closure tab doesn't show the questions that were deleted. At least I wasn't able to find any that said "Page Not Found" when I opened them. So maybe this is a 100% accurate method to find your "close accuracy" or close to it. Note: I have less than 10k on SO, so maybe that's why it doesn't show/count the deleted questions in the /votes/closure tab for me, but does for users with greater than 10k. So the query/method would be a lot more accurate for users with less than 10k, and possibly really inaccurate for >10k'ers.

With that said, I strongly agree with your feature request. It could have the possibility of getting users who cast close votes on questions that aren't likely to be closed to better use their limited amount of close votes for a bigger impact overall. It could also hopefully entice users that don't close vote as much to do so more. But at the same time, it's possible some users may be discouraged from participating more in close voting if they have a low close accuracy.

Not sure how much use the reopen stat would be, it's good trivial information to see, but it doesn't really tell us much of anything. It's good to have questions you voted to close reopened, which means the user took the advice of the close reason and then edited their question well enough to be reopened. That is much more likely for most close voters I'd say than voting to close incorrectly.

Improve this answer
15
  • The query also includes the questions that were later reopened. One could exclude them, or count separately, by looking up the Reopen event in posthistory. (Don't have time for it now.)
    – user259867
    Feb 8, 2015 at 6:35
  • This is definitely helpful, though it isn't enough to really satisfy me. I'd like a somewhat more complete interface to make it easier to learn from my mistakes, not just come up with a rough estimate of what my close accuracy is.
    – Alexis King
    Feb 8, 2015 at 6:42
  • 1
    I don't think this is very accurate at all. On SF for example SEDE tells me I have 1710 closed whilst my user page says 6313 which gives me an alleged accuracy of just 27%. When I check I see there are pages and pages of deleted questions that are clearly skewing the results. It seems that, the better you are the less accurate this method is.
    – user147520
    Feb 8, 2015 at 6:47
  • 1
    @bluet FWIW, I can see lots of deleted questions on my /votes/closure tab. I have 10k reputation, though, so maybe it hides them if you have less than that? I don't know. (I didn't downvote you by the way—I can't check if someone did because I don't have the rep here, but I noticed you had +1 earlier.)
    – Alexis King
    Feb 8, 2015 at 6:49
  • @bluet Some of them are, yes. If a question is actually deleted, it shows up with a red background on that page (just like it does on the actual question page if you can view it).
    – Alexis King
    Feb 8, 2015 at 6:55
  • 10
    Your guess is right: deleted questions are shown in /close votes tab only if you have 10K. (You would not miss them: they appear with "deleted" color background). So, your method results in the % of closed questions among non-deleted questions only, which of course is not a representative group of the questions you VTCed.
    – user259867
    Feb 8, 2015 at 7:31
  • 10
    This method is wildly inaccurate as such this answer has no value.
    – user147520
    Feb 8, 2015 at 7:38
  • 1
    @Iain I disagree that the answer has no value. I found the link to the query and pointing out the different places you could get some statistics useful, as well as the discussion about deleted questions and reputation. Granted I don't think the method is accurate, but there is useful and relevant information in the answer.
    – ColleenV
    Feb 8, 2015 at 16:03
  • @ColleenV It may be interesting but it's not relevant to the question as it's accuracy is very poor.
    – user147520
    Feb 8, 2015 at 16:46
  • 5
    @Iain I would concede your point if this was an answer anywhere other than meta. The question is a discussion and not something that has a correct answer. I think it adds to the discussion, particularly by highlighting issues with calculating such a percentage.
    – ColleenV
    Feb 8, 2015 at 17:20
  • 4
    It seems like it would be much more accurate, and probably more useful, to find questions on which I've cast a vote that aren't (or if possible were never) closed. Only in exceptional circumstances will unclosed questions be deleted, so the results should be much closer to matching the actual votes cast.
    – jscs
    Feb 8, 2015 at 20:03
  • @ColleenV I agree with Iain to the extent this is a +0 not a -1. It has enough value to hang around, but not enough to be seen as answering the question.
    – Mark Hurd
    Feb 12, 2015 at 23:48
  • @MarkHurd What is the question here though? Meta is unique in that these are discussions that are outside the normal question and answer format of the main sites. The original post is a request for a feature not a question.
    – ColleenV
    Feb 13, 2015 at 0:05
  • 2
    @ColleenV This post has a title of "Help me make myself a better moderator—give me statistics about my own close vote accuracy!", which is close enough to the question "Are there statistics regarding my own close vote accuracy that can be provided to help me be a better moderator?", and this answer is an attempt at that.
    – Mark Hurd
    Feb 13, 2015 at 0:11
  • Apparently I've cast 6,810 close votes, and that query gives my 1,746 rows. So my accuracy is something like 390%. Awesome!
    – Lightness Races in Orbit
    Jan 24, 2016 at 23:40
11
+100

The proposed measures really measure agreement with other users rather than the inherent close-worthiness of posts you've voted to close, since they're all based on the consistency of your close vote with close and re-open votes by [mostly] peer users. Because it's based on consistency with other users, it says at least as much about the context of the users around you as it does about your own accuracy.

Therefore, this sort of analysis seems to me to be less useful for evaluating one's own accuracy as a close-voter than, perhaps, for highlighting points of communal disagreement. Perhaps the Close tab in Moderator Tools should include the following lists of recent questions that may be instances of communal disagreement on question fitness:

Review of these lists by people with Moderator Tools access could give them some insight into cases the community disagrees about and could also suggest posts that deserve further review. In addition, if you find anything you've voted on in one of these lists, it can be a cue to you to give the post and the reasons for your vote more thought, or to defend your point of view to the disagreeing community members.

Improve this answer
2
  • 2
    Regarding your "unclosed" category, you'd get a lot of false positives from less popular tags. I tend to be active in the less popular tags and I've got quite a few close votes that never resulted in closing, not because anybody looked at the question and thought "I disagree with this close vote" but because there are not enough people who can form an opinion about closing the question.
    – Louis
    Feb 9, 2015 at 18:15
  • @Louis Good point; thanks. I've changed that category to "Left open," focusing only on the posts about which there is explicit disagreement, expressed though votes.
    – Isaac Moses
    Feb 9, 2015 at 18:28
7

Hmm...

Part of the goal of voting to close is to educate the OP and help them to improve their question. When you cast a close vote the OP can see it, if they have sufficient rep, ideally when that happens they will make an appropriate edit and salvage the question. Or someone else may see a poor question and edit in necessary information, improve clarity, etc. and bring it back from the brink of closure.

In either situation at the time when you voted to close the question it deserved to be closed and your vote was accurate. Later on someone improved the question, making your vote invalid.

It would be really hard if not impossible for the system to account for that.

Improve this answer
1
  • 2
    Well, the system could account for that: by not taking into account questions edited after the close vote at all. Of course this would render the statistics strongly biased for fire-and-forget questions and similar and thus not very useful.
    – Wrzlprmft
    Feb 9, 2015 at 15:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .