Request for hot question vote statistics

Question

Elsequestion it was said:

Guess what: if your site is full of crappy questions, your site sucks - even if they're not highly-ranked by your own users, folks are finding them via Google, and that's where the vast majority of your readers are coming from. You can work to fix that - as painful as that process is - or you can bury your head in the sand and blame it on all of those stupid people from elsewhere. If you think "hot" questions are a serious problem for your site's quality, then you're already ignoring a much bigger problem. Blaming someone else is easy and fun for the whole family - but it doesn't fix anything.

This struck me as a bit odd and I'd really like to see some numbers to back it up, that the active community on the site is the one pushing the questions that are popular and being raised as issues as to why hot questions are problematic.

It's one thing if it's the core group on the site that is pushing these questions into troublesome popularity, it's quite another if it's people who are less involved in the community and the core group lacks the ability to keep it from growing too popular.

It would be really nice to get a "how many of the up votes came from people who hit the question via a hot question rather than from google". I am not sure the problem is folks finding them from Google... and without the ability to find the questions that are outliers from anonymous and low rep feedback (typically from Google?) it becomes difficult to identify the question people are finding from the Google that should be protected.

I would like to see some stats that are broken down by:

• Reputation
• Answers given (i.e. out of "20 votes, 15 of them were from people who have answered 0 questions")
• Days active out of the last 100 days

For the following questions that are ones that are currently or have been outlierly popular:

• SuperUser: How are pseudorandom and truly random numbers different and why does it matter?
• Workplace: What is a 'friendly' way to let managers know that having good developers is a privilege?
• Programmers: How has an increase in the complexity of systems affected successive generations of programmers?
• SciFi: Would the One Ring even work for anyone but Sauron?
• English: Is there a non-sexual phrase for sleeping with someone?
• StackOverflow: Why is “except: pass” a bad programming practice?

(I'm trying to get a sampling of sites where I recall the question being 'hot' and relatively recent so the data is 'fresh')

---

To an extent what needs to be distinguished and identified is "does the core group within the site have the tools / ability to be able to clean its site up or protect itself from the added popularity these questions bring?". I hope that the numbers from the above could give some added data and insight into this. Do we need to protect things faster? Would The association bonus should not enable users to vote on every site help reduce the extreme scores?

_{For context on the core group, please read A Group Is Its Own Worst Enemy by Clay Shirky. In particular pay attention to "Three Things to Accept" #2 and #3, and "Four Things to Design For" #3.}

^{Members are different than users. A pattern will arise in which there is some group of users that cares more than average about the integrity and success of the group as a whole. And that becomes your core group...}

^{The core group has rights that trump individual rights in some situations... you can see examples of how bad an idea voting is when citizenship is the same as ability to log in.}

^{...you need barriers to participation. This is one of the things that killed Usenet. You have to have some cost to either join or participate... Now, this pulls against the cardinal virtue of ease of use. But ease of use is wrong. Ease of use is the wrong way to look at the situation, because you've got the Necker cube flipped in the wrong direction. The user of social software is the group, not the individual.}

Answer 1

First off, this made me laugh:

This struck me as a bit odd and I'd really like to see some numbers to back it up

You quoted what I intended to be a tautology, an assertion that a site full of crappy, popular questions is a crappy site. But I guess we don't talk about search engine traffic that much; for reference, it makes up 80-90%+ of all traffic to all of the sites you referenced: chances are, if someone's viewing your question it's someone who found it via search. Hot questions are dramatic because they get a disproportionate amount of on-site and in-network traffic, visitors who can visibly affect the question and answers they're viewing - but this is a drop in the bucket of folks silently reading.

That out of the way, the association bonus is a common point of concern: the notion that hoards of folks unfamiliar with your site could descend upon it and skew the voting without ever having earned the privilege is worrying, and rightly so - indeed, it was one of the big concerns for the folks over on MathOverflow after they joined the network. But in practice, it's rarely an issue; for popular questions, new folks tend to vote the same way that folks already on the site vote. Indeed, the real skew shows up with obscure questions and answers that only get one or two votes total.

But talk is cheap. Below is the data for the questions you listed. Note that all voter classifications are relative to the time when the vote was cast - in other words, a voter whose first action on the site was to cast the vote but who later posted several answers, earned 15+ reputation points, etc. isn't counted as an established user in those categories.

How are pseudorandom and truly random numbers different and why does it matter?

• Total Upvotes: 340
• Total Downvotes: 2
• Upvotes from users with vote privilege earned on site: 176
• Upvotes from users with assoc bonus only: 115
• Upvotes from users with at least one prior answer on site: 182
• Upvotes from users with at least one day visited in the week prior to voting: 188
• Anonymous upvotes: 38

What is a 'friendly' way to let managers know that having good developers is a privilege?

• Total Upvotes: 198
• Total Downvotes: 5
• Upvotes from users with vote privilege earned on site: 58
• Upvotes from users with assoc bonus only: 130
• Upvotes from users with at least one prior answer on site: 45
• Upvotes from users with at least one day visited in the week prior to voting: 59
• Anonymous upvotes: 119

How has an increase in the complexity of systems affected successive generations of programmers?

• Total Upvotes: 109
• Total Downvotes: 1
• Upvotes from users with vote privilege earned on site: 66
• Upvotes from users with assoc bonus only: 29
• Upvotes from users with at least one prior answer on site: 68
• Upvotes from users with at least one day visited in the week prior to voting: 74
• Anonymous upvotes: 33

Would the One Ring even work for anyone but Sauron?

• Total Upvotes: 170
• Total Downvotes: 0
• Upvotes from users with vote privilege earned on site: 69
• Upvotes from users with assoc bonus only: 94
• Upvotes from users with at least one prior answer on site: 61
• Upvotes from users with at least one day visited in the week prior to voting: 97
• Anonymous upvotes: 105

Is there a non-sexual phrase for sleeping with someone?

• Total Upvotes: 176
• Total Downvotes: 3
• Upvotes from users with vote privilege earned on site: 79
• Upvotes from users with assoc bonus only: 76
• Upvotes from users with at least one prior answer on site: 56
• Upvotes from users with at least one day visited in the week prior to voting: 67
• Anonymous upvotes: 59

Why is "except: pass" a bad programming practice?

• Total Upvotes: 116
• Total Downvotes: 3
• Upvotes from users with vote privilege earned on site: 112
• Upvotes from users with assoc bonus only: 1
• Upvotes from users with at least one prior answer on site: 111
• Upvotes from users with at least one day visited in the week prior to voting: 113
• Anonymous upvotes: 0

Answer 2

There is more detailed data for particular site (Workplace) provided by Shog when asked on hot-question traffic here. Data is for 180 days up to 2014-03-27:

Defining "outsider votes" as votes cast by someone who hasn't yet earned even 15 reputation on the site at the time the vote was cast

And looking at all votes cast during the past 180 days

We'll define "outsider agreement" as a case where outsider votes on a post are > 0 and insider (folks who've earned >= 15 rep on the site) upvotes - insider downvotes are also > 0

We'll define "outsider disagreement" as a case where outsider votes on a post are > 0 and insider votes are < 0

Cases of outsider agreement: 2,680

Cases of outsider disagreement: 277

Cases where the insiders' votes would've brought a post's score < 0, but outsider votes were present in sufficient quantity to make the post's score positive: 147

Total posts voted on by outsiders in the past 180 days: 3870

Total posts voted on by insiders in the past 180 days: 7186

Total posts voted on by both groups in the past 180 days: 3053

Q: What is the difference between outsider disagreement (277) and insider negative but for outside votes (147)?

A: So... Imagine you & I see an answer & we both downvote it. It's at -2. Then 3 people come in with nothing but the association bonus and up-vote it. It's at +1. There are 147 cases like that

Now imagine @enderland and @RhysW both downvote the same answer, so it's at -1 again. There are 277 cases like that.

Total insider votes during the past 180 days: 29,479

Total outsider votes during the past 180 days: 12,837

Q: am I reading this right -- there were ~700 posts that only outsiders voted on (insiders didn't)?

A: something like that

I'm not breaking this down any further right now, but a really common case for that involves answers to questions being upvoted by the asker

Q: Ok, so if I'm reading all this right, we have significant "outside" participation, but only in 5-10% of cases does that participation disagree with the community.

A: Right. And if you consider that even most of those 5-10% are "controversial" in the sense that they're getting both up-votes and down-votes from insiders, you can see there's a lot to be gained from community education.

---

Two notes regarding above data. First, it is complete in the sense that it takes deleted posts into account ¹.

Second, it doesn't seem to distinguish between questions that were in hot list and those that weren't. As a result, one can only (maybe erroneously) assume that most "outsider" votes in above relate to hot questions. Probably taking data only for questions having over 500-1000 views could make a closer approximation.