49

I just wrote this totally nasty query on data.se (same query including all community wiki questions).

It is ... complicated but this is what it looks at:

  • It splits up data in to 1 month chunks (2009 - 2011)
  • It counts questions, answers and votes during the month for those questions and answers.

Voting drives the economy of Stack Overflow, this trend of decreased voting can not be helping.

Why do you think we are seeing this trend? What can we do to help increase voting?

||improve this question
  • 8
    How does that data correlate with views per post? I think that would be an important metric. Maybe the real problem is that less eyes are on the posts. (I suspect that may be it) – jjnguy Feb 23 '11 at 4:15
  • @jjnguy I would need help from the stats guy to answer that, we only store total views, not views per-month – waffles Feb 23 '11 at 4:19
  • 3
    A very similar (or dupe) topic was discussed back in Jan: Why aren't people voting as much as they use to? – slugster Feb 23 '11 at 4:33
  • 1
    @waffles could you please add the total number of users to the stats? – systempuntoout Feb 23 '11 at 8:25
  • That would have to be active users @systempuntoout, because I you can't vote if you don't log in – Ivo Flipse Feb 23 '11 at 8:38
  • Could there be a correlation between users 'flocking' to SE sites in favor of SO and taking their votes with them? Programmers for instance certainly doesn't lack votes... – Ivo Flipse Feb 23 '11 at 8:40
  • @waffles on Programmers the query returns Divide by zero error encountered. – systempuntoout Feb 23 '11 at 9:21
  • 1
    +1 Well that's a good trick! master..fnGetDateRanges How do you get to know the SE master db? – RichardTheKiwi Feb 23 '11 at 9:50
  • 4
    stackoverflow.com/questions/4877157/launch-dll-using-c-program for an appalling example (please don't pollute it with up/down votes, it's a frozen masterpiece) – systempuntoout Feb 23 '11 at 10:33
  • 4
    One factor could be noise questions: duplicates, migrated, quickly closed. They'd drive the per-question metrics down. Do we get those in enough volume to be significant? – Rup Feb 23 '11 at 13:55
  • 1
    Could we have a (freehand) plot with that? – Henk Holterman Feb 23 '11 at 14:23
  • 1
    "Invalid object name 'master.dbo.fnGetDateRanges'." -- Hmm. – Shoe Jan 16 '14 at 4:53

12 Answers 12

Active Oldest Votes
20

Show the User voting behavior to the community as an indicator of the User civic duty-ness

I'd like to see a voting rate indicator like the accept rate on the user profile.

enter image description here

This should reflect the voting behavior of the user based on the following factors:

  1. Time spent on the site (visits, questions views, questions asked)
  2. Number of upvotes/downvotes
  3. Reps? (more reps, less relaxed checks or viceversa?)

This could be indicated with some fancy arrows (green, yellow, red) or a more aggressive percentage.

The rationale is something like this:
We are aware that you spend a lot of time on our network but your upvotes/downvotes rate kinda sucks, do you mind to vote more?

This indicator should be backed with a proper section allowing the user to check his/her voting stats and avoiding MSO questions like I'm keeping voting up but my arrow is still red, why?.

I'm curious to see this metric, as a great upvoter, I'd like to see some data about my clicks rate during these months here on SE.

I haven't reflected at all of any possible drawback of this proposal, I'm just throwing an Idea that could be horrible or hard to implement.

Update:
there's a cool topvoters new feature here

Update2:
and a more fine grained voting stats discussed here!

enter image description here

||improve this answer
  • 1
    +1; I agree with the general idea, but given the number of metrics that make up civic-dutyness I've stuck up a proposal to expand it to include all those other metrics that make a good citizen. – user142852 Feb 25 '11 at 11:20
  • Replace accept rate with citizenship level – gnat Mar 14 '14 at 16:18
15

The raw number of votes is increasing, so I think what you're seeing is either:

  1. Even though there are more users, they are hitting the 30 vote/day cap.
  2. People are not seeing as many % of the posts as they used to be and what they are seeing is of declining quality (partially supported by the fact that answers/question is on a slow downward trend).

To counteract these:

  1. Raise the number of votes/day to 45 (or some other higher value).
  2. Increase the time between new questions for new and low-rep users. (If we cut down on the low-quality questions, people will see more of the high quality questions, which they are more inclined to vote on because downvoting charges a penalty).
||improve this answer
  • 3
    I'd agree on increasing the votes/day. I hit the daily vote limit rather early – Sathyajith Bhat Feb 23 '11 at 11:23
  • 5
    @Sathya ... Note ... on Stack Overflow only 30-60 people hit the voting limit ... as a comparison approx 2000 people only vote once. – waffles Feb 23 '11 at 22:44
  • I definitely agree with increasing question waiting time for new users. The Flash section is plagued with very poor questions in short intervals from new users. – null Feb 24 '11 at 0:25
15

One very simple small idea would be to make the Civic Duty badge able to be earned multiple times.

||improve this answer
14

Warning: graph- and speculation-heavy post follows.

I don't know what happened, but that trend has been reversed:

Votes per post on Stack Overflow

After a low of ~1.9 votes/post in March of 2011, the rate has settled into a range between 2.6 and 2.8.

That got me thinking about the network as a whole. For instance, I looked at the perceived drop in quality on Digital Signal Processing and concluded that a steady drop in voting was the culprit. Either good contributors are less motivated to ask, and answer questions or posts with few votes come off as lower quality. (Or both!)

What's a healthy ratio of votes to post? Should a large drop, such as SO saw from 2009 to 2011, concern us? To answer those questions, I queried our database for votes/post, total views, and total posts. I've included upvotes, downvotes, and accept votes because these have the greatest impact on participation. I'm using views as a loose proxy for success. (That's not really fair to smaller topics, such as Poker, but we take that into account when evaluating sites.)

This graph shows votes/post on the vertical axis and the horizontal axis is log scale views. The bubbles represent total number of posts:

Views/Votes per Post

The left side of the graph is populated by sites we closed and the right side is dominated by Stack Overflow. Taking out those exceptional sites we get this:

Views/Votes per Post except SO and closed sites

So there's a big cluster of beta sites that get between 2 1/2 and 6 votes per post. On the right side we have some graduated sites with low voting and a handful (Skeptics, Science Fiction & Fantasy, RPG, Programmers, CS Theory, TeX, MathOverflow, Travel, and Mathematica) with v/p north of 5 1/2. There are also a number of graduated sites that aren't extreme in either direction.

Some of the sites on the high side of this metric (Skeptics, MathOverflow, and CS Theory, in particular) suggest that votes per post is a measure of "degree of difficulty" in writing questions and answers on a particular site. On Skeptics, if you don't do a good job of establishing your claim as notable, your question will be closed. If you don't follow the rules on using sources, your answer will be deleted. It's probably our most demanding site. Therefore, many people vote on the site without contributing much in the way of posts.

On Stack Overflow, we see the problem with success. There are far more posts submitted than anyone could hope to keep up with. At best, you might be able to keep up with the questions on a popular tag, such as ...maybe. So we can easily see why so many posts get no votes at all. Anecdotally, that's a problem for people who started contributing later in the game; only exceptional posts gather many votes.

Conclusion

Tentatively, I suggest that different sites will have different "steady state" levels of voting. Drops in voting, such as SO saw between the start of 2009 and 2011, should be concerning for those of us watching particular sites. When this sort of decline is observed, steps should be taken to reverse the trend.

Now, the question remains: what reversed the trend on Stack Overflow? Some candidate changes:

Changes that came a bit late to explain the turnaround:

||improve this answer
  • Jon, is there a higher res version of the site graphs available? – wax eagle Jan 15 '14 at 21:41
  • 2
    Did you correlate these observations with "Recent feature changes..." history? Eg for March of 2011, there's an entry: "2011-03-10: Auto-deletion of low score, zero answers questions has been extended a bit." – gnat Jan 15 '14 at 21:45
  • It's the first graph I find most interesting here. Did you use post date or vote date? If you look at median votes using post date, it looks a lot worse. Median votes by vote time show a pretty stable trend over all time. I don't think things have got better; it just looks like there are more good questions that people are voting on. – ben is uǝq backwards Jan 15 '14 at 21:54
  • To speculate heavily: in the beginning all the "easy" questions got answered and so have lot's of upvotes. Then there was a growing period, while SO got big and a bit of pain as the community cracked down on what it disliked (more deletion etc). Now, the community keeps growing, which keeps the median number of votes per post down but the posts by people who've been here a while are better than they were in 2011, making the average better. – ben is uǝq backwards Jan 15 '14 at 21:56
  • @benisuǝqbackwards: The first graph is from this query. (The graph has the link to the query surrounding it, but I guess I didn't end up liking it in the text somewhere.) I used the time the vote was cast to determine which month it belongs to. – Jon Ericson Jan 15 '14 at 21:57
  • 1
    Some important factors I suspect to lead to the unusal voting on Skeptics are the accessibility and popular appeal of the typical Skeptics question combined with the very low question volume. Our answers are designed to be evaluated by someone without any particular domain knowledge as we rely on citations. And slow sites always seem to have much higher votes than the fast sites. – Mad Scientist Jan 15 '14 at 22:03
  • @waxeagle: The graphs aren't working properly, but here is the spreadsheet data. I used the "Bubble chart" to graph the data. I could try zooming in on the cluster of sites in the middle to make them easier to separate, but I'm not sure I'll do that today. – Jon Ericson Jan 15 '14 at 22:11
  • 1
    @MadScientist: Hmmm... could you expand a bit on what you mean by "slow" and "fast" sites? Do you mean the rate at which the front page changes with activity on posts? That's something we can measure. – Jon Ericson Jan 15 '14 at 22:33
  • 1
    March 2011: "Increase in the daily voting limit for questions" + "Free downvotes for questions" => hunters for Civic Duty and Electorate badges push more questions down to and under zero => autodeletion kicks in => more low voted questions go away => average votes go up – gnat Jan 15 '14 at 22:36
  • 2
    @JonEricson I mean the volume of posts, my very subjective impression is that the number of votes per posts decreases significantly if the question volume increases, and each post gets less attention. – Mad Scientist Jan 15 '14 at 22:50
  • ...(continued from above) in this "chain", badge hunter's role is to educate other users by demonstrating that voting is widespread, normal and expected. Right score is great, wrong score at least shows a clue / prompt to correct it (vote opposite). No score is a dead end, worst case. By the way, hot questions play similar role, especially since these aren't hidden in collider anymore – gnat Jan 16 '14 at 4:43
  • @MadScientist less questions also mean individual voting makes substantial impact. Take an example of the site having about 4K questions. Take four top voters who cast about 12K votes. Divide 12K by 1,5-2 to account for deleted posts, get their 6-8K votes spread over 4K questions => this adds solid 1,5-2 votes to average. You see, 4 voters matter there – gnat Jan 16 '14 at 6:43
  • 1
    Is it possible to get something like "average votes/view" by site, for both question/answer? We have been discussing this in Workplace and wonder if this affects the "hot questions" problems we have. – enderland Mar 14 '14 at 16:16
10

Maybe it's because fun/subjective questions are becoming less and less allowed over time, such as this one.

Those kind of questions have probably thousand of times the votes of a normal question (probably more than enough to screw the average votes per post on all questions). Since they are usually CW, maybe you can try to only count non-CW votes in your query and see if it makes a difference?

||improve this answer
  • +1 for re-running the query with CW posts excluded. I'd be interested to see if there's a large difference (though personally, while I do expect it to be measurable, I disagree and and think it will be pretty small) – Joel Coehoorn Feb 23 '11 at 17:24
  • @Joel @Kop I forgot to mention that this query totally excludes CW voting and CW questions – waffles Feb 23 '11 at 21:20
  • of course the same trend is there if CW is included ... see edited question – waffles Feb 23 '11 at 21:22
9

Badge

Stealing slightly from jjnguy, we have a badge for hitting the daily vote cap (Suffrage) once, but what about on several days, or lots of days? As such, I propose a silver and a gold badge for voting a lot. Here's an example of what one might look like:

A really badly drawn badge

For hitting the daily vote cap on 150 days.

Since badges are designed to encourage positive behaviour on the site, it makes sense to have a badge for voting a lot. I've hit the daily vote cap fairly often.

Potential downsides:

  1. This doesn't guarantee quality voting. That's where you have sportsmanship etc.
  2. This will have the same effect as the electorate badge, where people will vote like crazy on everything in sight.

Computer Index Score -> Stackoverflow citizenship score

This is an extension of what systempuntoout is proposing, sort of, but changed.

Yes, I'm talking about this. Let's face it, the stackoverflow profile is accumulating a lot of statistics and information and there's lots we've all wanted to see in the moderator elections, in that little badge etc.

So, as an idea, I suggest we do a Microsoft: take all these metrics, give them a score out of 10, then take an average (or something) giving a total community rating score. Then display that on the profile. This would take into account all the rewarding badges, like sportsmanship, electorate etc.

A picture of my profile with some modifications to show ratings for various attributes that make a good citizen

In short, it's kinda like a stackoverflow citizenship test. It takes the need for all this extra information in the profile signature box away. You can factor things like accept rate into it too.

This would be in addition to the existing rep/badges. This is because rep/badges are for you, and this is how well you play in the team.

Upsides:

  1. Reduces clutter.
  2. Provides a simplified "niceness" indicator.
  3. You'd still be able to see all the badges, votes, questions etc. Indeed, these stats could go under one of the profile tabs.
  4. This information is being computed anyway, for the badges.
  5. You could use it in moderator elections.
  6. When you have another niceness statistic as demanded by meta, you just factor it in to this one.

Downsides:

  1. Like all such statistics, its a face-value approximation, or a rough guide to how nicely you've played. Just like rep, basically.
  2. Everyone will have an opinion on what should form part of said statistic and how much weight it should hold and nobody will be happy with the result.
  3. I haven't made any proposal as to how these calculations should happen. I've just proposed an idea. So that would need to be worked out.
  4. Probably some more.
  5. The image is badly done and only slightly uses freehand circles. I'm sorry. I couldn't fit it all in in freehand without using a massive image. I hope the badge makes up for it, and my overuse of bold text and obnoxious formatting.

Thoughts? Opinions? Abuse? Questions? Waffles?

||improve this answer
  • But then you get stuff like this... chat.meta.stackoverflow.com/transcript/message/532247#532247 – Benjol Feb 25 '11 at 12:35
  • Well, yes... but you already have that. It's harder when you build the total metric from things like #edits compared to #closes, flag weight etc. There's only so much you can automate. And I believe there's an abuse detection system in operation anyway. But you're right, any new system and someone will try to game it. – user142852 Feb 25 '11 at 13:59
7

I would look to query figures like views as compared to votes, as well as vote spread - are we seeing a higher percentage of posts not get any votes at all (or more negative votes), skewing the results?

I might suggest some alternate theories to test for evidence:

  • The number of vote worthy ('good') posts is decreasing as the membership ratio better comes to represent the industry (Alpha vs Vocational programmers).

  • The number of duplicate and off topic posts is increasing as the number of truly new questions goes down (The Simpsons did it). I don't know enough about the workings of SO to know if your community ownership clause is catching those [closed] posts.

  • The number of Google drive by questions (account with just one question) is rising, contributing to the previous two points?

||improve this answer
  • 3
    +1, especially your first bullet sounds plausible to me. If you only consider posts that have a positive score, the ratio is pretty even. – balpha Feb 23 '11 at 21:56
  • (btw, I only added "and Score >0" to the query, so the description is off a bit now) – balpha Feb 23 '11 at 22:06
4

I was trying to point out the same thing a month ago in this question and in this answer.

And at that time this is the comment I got from Waffle's:

"So you have a graph that proves that a question that was asked a second ago was viewed less than a question asked 3 years ago. What exactly does that prove?"

and

"I have no problem and any data helps out. However as it stands this proves nothing, if you could come up with an expected trend graph and show where we are dropping from the expected trend, it would be helpful. But as it stands all that this proves is that there is a long tail."

and from Jeff:

"I agree with waffles, this is simply a graph that says "over time, older things are viewed more". It's not particularly useful."

Does the same not apply to votes?

Isn't this just saying that over time posts get more votes? Where is the trend?

||improve this answer
  • sorry, I did not mean to be abrupt. this particular query does not look at views, only votes. I partition it so I only look at votes on a particular post that happened in the month we are analyzing. so this is safe against older stuff being treated unfairly. to do the view analysis properly we would need help from the stats guys. – waffles Feb 23 '11 at 22:43
  • @waffles - I had a relook at the data as @Kyle Cronin suggested and the long term trend (across three months of data dumps) is only slightly down for views/question, not as drastic as the initial data suggests. I would suggest you do the same with the votes, it gives a much better picture. – going Feb 23 '11 at 23:04
  • I do that with vote analysis already, it is excluding votes out of the time frame analyzed. so for example when looking a June ... I count the questions in June, Answers in June and votes on those questions and answers that happened in June. – waffles Feb 23 '11 at 23:07
  • @waffles - What I mean is pick a 3 month period (not the previous month, but the three months before that) and see how that period of 3 months compares across multiple data dumps. So currently for Sep, Oct, Nov you have (1.22, 1.26, 1.29). If you look at the same 3 month periods in previous data dumps do they also trend at the same numbers. – going Feb 23 '11 at 23:31
  • 1
    yeah for views we would need to use that approach, but for votes we have a timestamp on each vote – waffles Feb 23 '11 at 23:34
3

Just a hunch... but maybe people are just getting tired of voting?

Imagine you read dozens of questions and answers every day. When you're fresh and eager you read each in detail, upvoting those you find helpful. But... as time goes by, days pass, you take less time to read each answer and you don't upvote because you didn't consider it deeply enough.

I'm sure there are people here who are as fresh and upvoters as they used to be years ago when they first joined, but I'm also suspecting there are many others with the above "symptom".

Newcomers are not relevant, they might upvote the answer that solved their problems but it's really rare they check any other questions.

So the problem IMO is with the "inner circle" of members and the solution? Good question. Probably some of those members can answer this better. :)

||improve this answer
2

I have no idea if this applies to the community at large, but as a newer user on the site I'm hesitant to down-vote other users who offer suggestions that may slightly miss the mark but aren't out-and-out wrong (especially if they have been on the site for a while or have high rep). I've also found myself less likely to vote-up an answer that is clearly correct but already has the highest score of the answers listed for a question.

On a related note, when is it appropriate to downvote? If somebody leaves an incorrect answer - not necessarily a "wrong" answer, but just "didn't answer my specific problem", should they be voted down?

I asked a question on SO earlier today. It's only my second question. The first response didn't apply - but I wasn't sure if it was my fault for not explaining well enough, or his fault for not reading thoroughly. Within moments another user had offered a suggest that fixed my problem. I'd upvoted his answer immediately (and later accepted it) but I hadn't considered downvoting the initial wrong answer from the other user.

Mostly I felt like it's sort of a dick move to down-vote a guy who tried to help me, but just missed the mark. He didn't leave bad advice, it just didn't apply to my question as I'd scoped it.

||improve this answer
1

This is similar to this question to which this was my answer:

Another thing to consider is the change to the homepage algorithm so that it now show you new interesting but unanswered questions.

Once an answer (or two) gets two or three up-votes then the question will tend to disappear from the home page, reducing the number of people that see it thus reducing the voting chances.

||improve this answer
-7

I'm not saying that this is a good idea or that I agree with it...but how about reputation bonuses for voting up?

1 rep per upvote? 1 rep per 5 upvotes?

I think that would definitely increase voting on posts. (But I also think it would encourage voting on posts of lesser quality too much.)

||improve this answer
  • 6
    I'm glad to see downvotes. This would most likely be a terrible idea. – jjnguy Feb 23 '11 at 4:14
  • Someone once said on Meta, and I can't remember who, and I'm going to paraphrase, "Reputation is for what others think of you; badges are for what you do". Or something like that. And I agreed, so that's why I personally dislike this. – waiwai933 Feb 23 '11 at 4:16
  • @waiwai, I also agree it's a bad idea. I just though it should be brought to the discussion. – jjnguy Feb 23 '11 at 4:17
  • 1
    Someone else (Jeff) once said, reputation is a measure of community trust, earned via participation and personal investment in the site. – Lawrence Dol Feb 23 '11 at 7:27
  • @waiwai When there was a feature request for removing the +2 bonus for acceptance (then the only method contradictory to that philosophy), Robert cited pretty much exactly the first part of that. – Grace Note♦ Feb 23 '11 at 16:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .