Does it make sense for the required number of close votes to be a static number?

Question

Right now engineering.SE is early in its beta phase. We have received many questions which are unquestionably bad and for which the best response would be swift closure (examples: here, here, and here). All of the questions have now been closed but in some cases the process took more than 2 days due to the low number of high rep users.

Contrast this to a more developed site like physics.SE where 5 high rep users represent ~1% of the users with more than 1,000 reputation and ~0.01% of the total number of users. This group of 5 people gets to speak for the entire community in terms of what questions are appropriate on the site.

Would it make more sense to have the required number of close votes scale by the total number of users? The scaling doesn't have to be linear, perhaps 2-3 more people for every order of magnitude of growth in the active userbase.

Answer 1

I guess my one question is, why?

I see the reasoning in theory, but in practice I have no complaints about how things work.

Let's go over the numbers, right quick.

#Problem Statement

I'd like to know how the time it takes for questions to get closed varies based on the size of a site in the Stack Exchange network.

#Setup

I ran a query to test the amount of time it takes for questions to be closed across five network sites. As is always the case when people try to run queries like this, there are some significant biases in the data that need to be accounted for:

• Deleted questions (the result of "truly successful" closures) do not count
• Questions that are closed then reopened count once
• Questions that are closed, reopened, then closed again count twice
• Data is cached and only updated weekly (not really a big deal for this)

That said, I don't believe any of those to be a major hindrance in comparing the sizes of sites to the time it takes to close them.

The five sites a chose were, in no particular order:

• Stack Overflow
• Startups
• The Workplace
• Code Review
• Super User

#Approach

In order to find an answer to my problem statement, I ran a query to find, for each close (given the aforementioned limitations), how many hours it took between posting the question and having that close be successful. I grouped on those results, and created a table of each number of hours, and the associated number of closures on each site.

Finally, I then found for each hour x, the percentage of all closed questions which were closed in x or fewer hours.

Data

I reviewed all data since Startups (I'm a mod there, information bias) went to public beta, on August 18th of 2014. These were the total counts of recognized closed questions.

      Startups   Stack Overflow   The Workplace   Code Review   Super User
            45           52,752             825           435        2,190

I don't know why the last three are the same, and would appreciate clarification. Maybe all of this is nonsense, if my query was.

For the first day (24 hours), these were the data. I've marked each site with two asterisks when they crossed over 50%.

Hours   Startups   Stack Overflow   The Workplace   Code Review   Super User
    0        12%              30%              1%           36%          10%
    1        21%              46%              6%        ** 59%          18%
    2        33%           ** 53%             10%           68%          21%
    3        37%              59%             14%           73%          24%
    4        37%              63%             16%           77%          26%
    5        37%              67%             20%           80%          27%
    6        40%              70%             23%           82%          29%
    7        40%              72%             25%           83%          30%
    8        44%              74%             28%           84%          31%
    9        49%              76%             30%           85%          32%
   10     ** 53%              78%             32%           86%          34%
   11        56%              79%             34%           87%          35%
   12        56%              81%             36%           88%          36%
   13        58%              82%             37%           89%          37%
   14        58%              83%             40%           89%          38%
   15        58%              83%             41%           89%          40%
   16        60%              84%             43%           90%          41%
   17        60%              85%             45%           91%          42%
   18        63%              85%             48%           92%          43%
   19        65%              86%             49%           92%          44%
   20        65%              86%          ** 51%           92%          46%
   21        70%              87%             53%           92%          47%
   22        70%              87%             54%           93%          48%
   23        72%              87%             56%           93%          49%
   24        72%              87%             57%           93%       ** 50%

Also, of course, very important is the number of users who can vote to close. I also retrieved this data from SEDE. Asterisks indicate betas, with lowered reputation thresholds.

       *Startups   Stack Overflow   The Workplace  *Code Review   Super User
              24           26,256              98           483          515

#Discussion

I think the number at the 24-hour mark is the most relevant, because that's the time by which all users will have had some opportunity to see the question. I would expect slower closes for questions in the middle of the night (timezones permitting).

Given these data, I don't see any viable correlation between the number of users on a site and questions getting closed too slowly or quickly. That seems to be a factor dependent more on the culture of the site than its sheer mass of users.

I hypothesize that this might be the result of the natural increase in post quantity. In other words, where there are more users, there are more posts to moderate, and vice-versa.

Because of that, I think scaling the number of required votes would fix a problem that is already intrinsically fixed by definition.

As Famous Blue Raincoat mentioned, the assistance of moderators to pick up the slack on smaller sites tends to make up for the lack of users with privileges. By the time there are enough posts to overwhelm the moderators, in theory, there are enough users with moderation privileges.

Answer 2

The number of questions coming in also grows with the number of users. If this is multiplied by increasing number of close votes per question, the growth would become superlinear. On some large sites with many questions coming in (Stack Overflow and Ask Ubuntu come to mind) there are already not enough closevoters to handle the questions. To complicate things further, the number of users who actively close questions is not nearly as large as the number of users (even those with the privilege to close).

A major factor in close backlog at Engineering was that you did not have moderators back then. But now you do.

This group of 5 people gets to speak for the entire community in terms of what questions are appropriate on the site.

No, they speak for themselves. Their names are right there in the closure banner. If others disagree, they will reopen.

Answer 3

There are times when slow closure might be adventageous. Say a user gets a couple close votes and a downvote, so they edit their question in accordance with the helpful comments being left along with those votes. At this point, maybe the question is now okay, to the point where someone would rather just answer the question, as opposed to waiting for enough reopen votes to accumulate.

I guess I'm saying there's not too much harm in letting a question stay open for awhile while the community makes up its collective mind.

If a question is truly in need of prompt closure (something like spam, e.g.), flag it, and that should speed up the process, as it would get immediate attention the next time a moderator logged in.