98

There have been a number of discussions about "pity" or "sympathy" upvotes.

https://meta.stackexchange.com/search?q=pity+sympathy+upvote

That is, the idea that once a post is voted down to -1, some kind-hearted user will come by and upvote the post, no matter how terrible or wrong it may be, to return it back to zero.

Does the actual voting data actually support this theory -- or is it all in peoples' heads?

Improve this question
22
  • 11
    Even if there is, it is being more than offset by crowd mentality. The majority of people aren't too kind-hearted.
    – Kobi
    Commented Jun 1, 2011 at 4:09
  • 55
    +1, even though this question sucks
    – Mike Pennington
    Commented Jun 1, 2011 at 4:21
  • 13
    I've seen users admit that they regularly pity-vote, but I don't have concrete data beyond that.
    – Adam Lear StaffMod
    Commented Jun 1, 2011 at 4:27
  • 5
    also see: stats.stackexchange.com/questions/11414/…
    – waffles
    Commented Jun 1, 2011 at 4:27
  • 8
    I see it on the PHP tags. Terrible questions/answers are upvoted to become neutral. But weirdly, it's only if it has -1. Anything below that doesn't garner much in the way of pity votes.
    – JohnP
    Commented Jun 1, 2011 at 4:43
  • 5
    Subjective title is subjective. The voting pattern exists. But it's hardly a problem. -- And yes, I've recently picked up that habit. While I was very reluctant to offset someone elses downvote in the past, I feel obliged to give pity upvotes with some of the recently caused downvote frenzies. (separate topic..)
    – mario
    Commented Jun 1, 2011 at 4:47
  • 28
    @mario: WTF? Pity upvotes should never be given no matter what the reasons are. I'd say it actually is a big problem if people start feeling obliged to do it...
    – fretje
    Commented Jun 1, 2011 at 8:59
  • 9
    I've upvoted some answers with negative votes motivated out of pity but only in situations when the answer was reasonable and the down votes didn't describe the reason for the downvote.
    – jpierson
    Commented Jun 1, 2011 at 9:59
  • 8
    Pity, sympathy? What are these things?
    – Grant Thomas
    Commented Jun 1, 2011 at 12:15
  • 2
    @Jon: probably nothing - there are plenty of good reasons to up-vote something that's already been down-voted, so without some magical ability to judge the intent of the voter... But really, this applies to all votes - past discussions have shown that folks hold radically different opinions of when and why up- and down-votes should be cast. IMHO, the only thing you can reasonably do - should a problem exist - is try to mitigate any unintended consequences of it, as we've slowly been doing (consider a certain user who at one time had ~10K from asking hundreds of awful questions...)
    – Shog9 Mod
    Commented Jun 1, 2011 at 17:49
  • 2
    Pity votes are far less of a problem than the sheer number of votes given to questions that don't actually help anyone. Every vote on fascinoma questions like this one are a "pity" vote.
    – user7116
    Commented Jun 1, 2011 at 21:35
  • 20
    @mario: You're not offsetting anyone's vote. You're rewarding someone. An upvote is worth +10, whereas a downvote is worth a mere -2. Each time you "pity upvote," you're not undoing something; you're increasing that user's reputation by 8. Ultimately, that's the real problem: pity upvotes increase reputation, rather than nullifying the loss.
    – Borror0
    Commented Jun 3, 2011 at 17:02
  • 14
    @mario: Let me get this straight: you purposefully increase the reputation of bad users? What makes you think your voting pattern is somehow defensible?
    – Borror0
    Commented Jun 3, 2011 at 17:43
  • 2
    Is there a problem? I don't know if I'd call it a "problem" but so far, every single question that I've downvoted for being poor "questions" and was the first vote on was promptly followed up by an upvote within a minute. And most of them that I've noticed never received any other votes afterwards, up or down. I'd say it is an annoyance however as it's the type of question that you would ever upvote otherwise, and the lack of additional upvotes show that. So much for doing my part in trying to bring those questions down.
    – Jeff Mercado
    Commented Jun 5, 2011 at 6:09
  • 2
    @JeffA: Perhaps and you have a point. However considering the actual volume of questions and comparing the poor questions that have been pity voted versus legitimate questions, the data will always look insignificant. I've watched this reposted question go all the way down to -6 within about 30 minutes. Yet somehow, two people (as of now) felt it necessary to upvote it. Those are pity upvotes if I ever saw them and this particular situation is not counted in your statistics.
    – Jeff Mercado
    Commented Jun 6, 2011 at 21:20

9 Answers 9

Reset to default
37

Some baseline statistics.

Stack Overflow

  • Questions: 1,681,746
  • Questions with zero score: 778,511 (46%)
  • Questions with at least one downvote: 88,874 (5.3%)
  • Questions with zero score and at least one downvote: 26,441 (3.4%)
  • Questions with zero score having exactly two votes of +1, -1: 16,730 (1%)
  • Questions with zero score having exactly two votes in order of -1 then +1: 9,873 (0.6%)
  • Answers: 3,859,602
  • Answers with zero score: 1,459,773 (37%)
  • Answers with at least one downvote: 250,477 (6.5%)
  • Answers with zero score and at least one downvote: 61,654 (1.6%)
  • Answers with zero score having exactly two votes of +1, -1: 44,238 (1.1%)
  • Answers with zero score having exactly two votes in order of -1 then +1: 17,005 (0.4%)

Server Fault

  • Questions: 78,768
  • Questions with zero score: 37,522 (47%)
  • Questions with at least one downvote: 3,090 (3.9%)
  • Questions with zero score and at least one downvote: 1,015 (2.7%)
  • Questions with zero score having exactly two votes of +1, -1: 566 (0.7%)
  • Questions with zero score having exactly two votes in order of -1 then +1: 313 (0.4%)

Super User

  • Questions: 82,315
  • Questions with zero score: 33,121 (40%)
  • Questions with at least one downvote: 4,260 (5.1%)
  • Questions with zero score and at least one downvote: 1,267 (3.8%)
  • Questions with zero score having exactly two votes of +1, -1: 564 (0.7%)
  • Questions with zero score having exactly two votes in order of -1 then +1: 245 (0.4%)

I think the most telling statistic is the questions with exactly two votes that are at score zero. How many of these are in the order downvote, then upvote?

  • SO: 59% Q / 38% A
  • SU: 43%
  • SF: 55%

Where "flip a coin" is 50%.

Based on this data, "sympathy" or "pity" upvoting is not an actual problem -- if it was, I would expect to see much higher rates of an upvote following a downvote.

Improve this answer
20
  • 6
    Does this statistic include closed questions?
    – Paul McMillan
    Commented Jun 1, 2011 at 4:09
  • 1
    @paul this is all non-deleted questions.
    – Jeff Atwood
    Commented Jun 1, 2011 at 4:48
  • 29
    I'd say the 59% down-then-up is a telling statistic. You might be able to throw out SU and SF, as the sample sizes are considerably smaller, but 59% on SO seems to indicate that there is, in fact, some pity voting happening. It's almost a 20-point spread. If it was not happening in great numbers, you'd expect the percentage to gravitate towards 50% with a larger sample, would you not? Problem or not, it appears to lend credence towards the anecdotal observations.
    – Anthony Pegram
    Commented Jun 1, 2011 at 5:01
  • 36
    But how many of those are -1, edit, +1? Surely upvoting due to an improved question doesn't count as pity, does it?
    – Gabe
    Commented Jun 1, 2011 at 5:12
  • Could you also post stats on closed questions and break them out by percentages of upvotes / downvotes?
    – Mike Pennington
    Commented Jun 1, 2011 at 5:12
  • 1
    @Gabe good point, but since the data suggests it's not a problem even with counting the edited posts, then it certainly won't be a problem when they are removed from the stats.
    – Pollyanna
    Commented Jun 1, 2011 at 5:34
  • 11
    Well, this won't include the sort of question where everyone agrees the question is bad and it gets downvoted to -5, yet when you split the vote you see someone upvoting it, for some reason. Obviously it could be someone working towards the electorate badge, but I think the criteria can be expanded to include these too.
    – Yi Jiang
    Commented Jun 1, 2011 at 9:14
  • 12
    @Adam: Anthony is quite right that 59% is a lot more than 50%, in particular since the sample size is more than 16,000. To make it more obvious: 59% is 44 percent more than 41%! Gabe's point was that this might be explained by the sequence "downvote - edit - upvote". So it would be interesting to see only those 1-1=0 questions that have no edits (or no edits between the votes).
    – Hendrik Vogt
    Commented Jun 1, 2011 at 12:40
  • @YiJiang The Electorate badge doesn't require upvotes. Although it is a common misconception that it does so you may be on to something there.
    – Adam Lear StaffMod
    Commented Jun 1, 2011 at 13:16
  • @Jeff What did you use to gather these stats? I'd love to see the spread for Programmers.
    – Adam Lear StaffMod
    Commented Jun 1, 2011 at 13:16
  • 2
    Notes on the significance of these figures (as compared to a null hypothesis of randomness, and assuming counting statistics) added as an answer.
    – dmckee --- ex-moderator kitten
    Commented Jun 1, 2011 at 14:02
  • 2
    @Jeff - something else to consider is the opposite effect of "corrective downvote". Sometimes I do this as well - I see an answer that isn't right but isn't wrong enough to DV. If it gets upvoted, though, I will DV to cancel it out.
    – JNK
    Commented Jun 1, 2011 at 14:12
  • 10
    This is good stuff, but... I'm not sure I see the need for a "0 score" criteria. In these stats, if a post gets 2 down-votes and then one up-vote, it's not counted; it probably should be.
    – Shog9 Mod
    Commented Jun 1, 2011 at 17:58
  • 7
    Yeah, I'm failing to see the reason for finding posts with a 0 score only. Surely it would make more sense for the criteria to be no votes followed by one or more down votes followed by an up vote? This is one such case; -5 followed by +2. I'm not sure pity voting is a real problem, but it's certainly annoying when you see it happen.
    – Andy E
    Commented Jun 1, 2011 at 22:39
  • 3
    Jeff, comparing these statistics may not be as good indicator for the non-existence of counter-voting as they seem. In contrast one can interpret these numbers in the reverse direction: I don't have the distribution of the votes, but it seems that the number of counter-votes is considerable higher than the probability of them (I think the probability of a question receiving a -1,+1 should be way smaller than the probability of receiving no votes). In addition these numbers seem to imply that there is significant number of anti-sympathy counter-votes: -1,+1.
    – Kaveh
    Commented Jan 22, 2013 at 2:20
23

From the Stack Overflow list:

Answers with zero score having exactly two votes of +1, -1: 44,238 (1.1%)

Answers with zero score having exactly two votes in order of -1 then +1: 17,005 (0.4%)

is statistically significant if we simply assume counting statistics

sqrt(44237) = 210
sqrt(17005) = 130

(44237 \pm 210)/2 - (17005 \pm 130) = 5113 \pm 167

or about 31 sigma.

Of course it is still 5000/3.8e6 so it is a small correction, and it go against the direction of pity voting.

Following that model for all the above data

                             Sign   Significance
------------------------------------------------
Stack Overflow   Questions    -      13
                 Answers      +      31
Server Fault     Questions    -       0.7
Super User       Questions    +       0.5

The interpretation of the table is as follows:

  • a pattern of pity voting would generate a negative value for sign
  • significance less than ~3 isn't evidence for anything
  • significance more than ~5 is evidence for something, but doesn't prove what.

Both stack overflow results show large significance, but of different signs. The Server Fault and Super User data show no significance.

Not sure what to make of that.

I have developed a half-assed theory (pure speculation, really) that could account for the difference between questions and answer. It may be the question-posters are (a) likely to vote for any answer they received and (b) likely to do so soon after it appears.

If there is anything to that it would mean that some fraction of the (+1,-1) answers are the result of people trying to de-emphasize so-so answers that the OP voted for.

Improve this answer
8
  • This is all significance against 50/50 anyway, which assumes that randomness actually has something to with voting decisions. I'm not so sure that's a good assumption.
    – Nicole
    Commented Jun 1, 2011 at 14:13
  • @Renesis: As I am only looking at those events that are (+1,-1) the only thing at issue is the order those votes arrive in. Why wouldn't random be the correct null hypothesis?
    – dmckee --- ex-moderator kitten
    Commented Jun 1, 2011 at 14:56
  • 6
    Because the set is limited to questions with exactly 2 votes but all questions are not fixed at 2 votes, it assumes that +1 and -1 questions are equally likely to get a second vote, and equally likely to not get a third vote.
    – Nicole
    Commented Jun 1, 2011 at 17:10
  • 1
    The analysis set is not limited to questions with two votes: it is limited to question with one up and one down. There is no a priori reason to expect those to be dominated by up first or down first, so the null hypothesis is that the situation is random.
    – dmckee --- ex-moderator kitten
    Commented Jun 1, 2011 at 19:41
  • Well... I usually only downvote if the answer already has a vote (- or +) because of the known problem of sympathy upvotes! Anyway, I agree with the 50/50 null hypothesis, but I think it is impossible to interpret the reasons that it fails from this data alone. I would argue, however, that this shows that previous votes influence behaviour. Is this desirable? I don't think so but I cannot think of a good way around it.
    – z7sg Ѫ
    Commented Jun 8, 2011 at 17:15
  • @z7sg: Correlation data certainly can't tell you what the cause was, but a lack of correlation is pretty strong evidence of a lack of causation. The really interesting thing about this data is the different highly significant signs for questions and answers on Stack Overflow. What is up with that?
    – dmckee --- ex-moderator kitten
    Commented Jun 8, 2011 at 18:00
  • @dmckee Hard to say of course! One difference is that it doesn't cost any rep to vote down questions. But for me at least, that doesn't influence my behaviour, it is more related to what other people are voting. It would be interesting to see how people voted if they only knew the order and not the number of votes for each answer.
    – z7sg Ѫ
    Commented Jun 8, 2011 at 18:11
  • 1
    From personal experience, I know that I give very quick up votes on answers to my questions that are at all helpful. Since the asker is almost always the first person to see an answer and understand the context, I think your theory passes the smell test. Later answers can transform mildly helpful answers into distracting answers.
    – Jon Ericson
    Commented Jun 5, 2012 at 16:10
16

The wide spread belief that “pity upvoting” is common changes how some people votes, therefore the belief that “pity upvoting” is common is a problem regardless of it’s truth.

Improve this answer
3
  • 14
    I think people remember a pity upvote like they remember a slap in the face. It stings and it will always be remembered as an offense, an affront, a slight against human dignity when a crap post at -1 suddenly goes to 0. Perhaps that's why they perceive it as common, but the data does not support this.
    – Jeff Atwood
    Commented Jun 1, 2011 at 8:32
  • What is "wide spread"? Wide spread amongst voters on Stack Overflow? Or wide spread amongst vocal users on Meta Stack Overflow?
    – Jan Fabry
    Commented Jun 1, 2011 at 9:22
  • @Jan, we don't know that!
    – Ian Ringrose
    Commented Jun 1, 2011 at 10:52
14

Here is my non-answer based on the ideas I got from the stats answer.

We could conduct some very basic A/B testing.

Divide all our users into 2 groups:

  1. Group A (odd user ids) see all questions they did not vote on which are scored -1 as 0. So they will not see -1 anywhere on the site for a bit instead it will be 0.
  2. Group B (even user ids) see the "truth".

Run this for a day, then look.

If we notice the first group is less likely to upvote, we know it may be a problem. This will also gauge the size of it.

Of course it is a very elaborate experiment that would mean we need lots of changes just to perform it, personally I do not think it is worth the effort.

Improve this answer
12

Regarding the -1 then +1 pattern: I will sometimes upvote a mediocre but not harmful/wrong answer if it is at -1, where I wouldn't have upvoted it if it were at zero.

However, I do not consider this "pity" voting - I'm not concerned about the user's rep. I'm just trying to sort the posts into order of usefulness. For me, a negative score means the content should be disregarded or taken with copious salt. An "ok" post at -1 is therefore just incorrectly classified, and I want to fix it.

So, my point is that you cannot ascribe "poor user, have some rep" motivation to all these votes.

Improve this answer
5
  • 16
    "I will sometimes upvote a mediocre but not harmful/wrong answer if it is at -1, where I wouldn't have upvoted it if it were at zero." ==> if this isn't pity voting, then what is it exactly? IMO, this is almost the definition of pity voting...
    – fretje
    Commented Jun 1, 2011 at 14:11
  • 3
    @fretje I'd think pity upvoting is more, like Jeff's italic point, voting up "no matter how terrible or wrong it may be".
    – Grace Note StaffMod
    Commented Jun 1, 2011 at 14:14
  • 2
    @fretje it's not pity upvoting because it's not motivated by a feeling of pity.
    – Blorgbeard
    Commented Jun 1, 2011 at 14:17
  • 7
    @Grace: I guess we'll first have to settle on the definition then... IMO it's already pity voting when the person wouldn't do it if the score wasn't negative.
    – fretje
    Commented Jun 1, 2011 at 14:17
  • 4
    @fretje My personal thought pattern is, I don't care what motivates the person to vote, as long as their spirit in voting is upvoting what they find useful, and downvoting what they believe is not useful. The nature of how score works makes this happen in both directions - users refusing to upvote highly voted answers because they feel "It's high enough", or users refusing to downvote negative elements (but not upvoting) because "It's low enough". Voting to "balance" works much the same, though I hope that the majority who do it are still believing that the question deserves the upvote.
    – Grace Note StaffMod
    Commented Jun 1, 2011 at 14:21
11

Just found a nice example. After posting a series of duplicate (and barely comprehensible) questions, which got down-votes and close-votes, the user edits his last one to include "guess everyone hates me". Presto - two up-votes within 30 seconds.

And another: some lovely chap posts a question saying "I know my posts suck, but who cares, you shouldn't ban me from asking" and refuses to listen to anything anyone is telling him. Goes to -8 and then 3 upvotes materialize despite his continuing childish behavior. (oh, and this doesn't appear to be his first account that's been question banned either.)

Contestant number three posted an angry, conspiracy-filled rant accusing someone of threatening to delete his account, but wouldn't even disclose which site it occurred on, much less provide any evidence. Naturally he was shocked that we wouldn't all just take his word for it and do something to someone. Then he wasted hours of everyone's time arguing in circles without actually explaining anything. Four people thought this was worthy of an upvote, meaning he gained reputation for his vague yet belligerent meta-help vampire diatribe.

Improve this answer
10

I have sympathy voted in the past.

If it is indeed a problem, perhaps single downvote questions should see a red zero, instead of a black one. That should offset the psychological aspect of things.

Improve this answer
2
  • 9
    So that would be a negative zero, as opposed to a positive zero. *8')
    – Mark Booth
    Commented Jun 1, 2011 at 13:52
  • @markBooth precisely.
    – Moshe
    Commented Jun 1, 2011 at 14:32
7

I will upvote a marginal answer from someone with a low reputation to encourage them.

I expect good answers from someone with a high reputation.

I don't think I'm the only person that feels this way.

The voting on Meta Stack Overflow is more opinion based than the voting on Stack Overflow and Programmers.

The problem I see more often is good answers in obscure topics with zero or one upvote, while fair answers in popular topics get 10+ upvotes. I don't think you can remove opinion and popularity from voting.

Improve this answer
9
  • 2
    The whole essence of voting is opinion and popularity, to be frank - it's what it is meant to represent. A highly voted answer only means something "good" because it represents something that is in popular opinion. We cannot enforce what leads a person to decide whether a post qualifies for an upvote or a downvote - but we still encourage that you focus on the merits of the post, and vote on the content. Whether some fraction of individuals express bias on account of "sympathy", it appears to be fine. Statistics show that "pity voting" is not really a problem.
    – Grace Note StaffMod
    Commented Jun 1, 2011 at 12:32
  • I agree with your statements, but I don't see what they have to do with the question at hand...
    – fretje
    Commented Jun 1, 2011 at 12:33
  • @fretje: The question in the title is "Is there an actual “pity” or “sympathy” upvote problem?" My answer attempts to illuminate one aspect of sympathy voting, and remind everyone of other vote biases. It's up to you and everyone else to determine if my answer has any merit.
    – Gilbert Le Blanc
    Commented Jun 1, 2011 at 12:39
  • 1
    @Grace: I'm not yet convinced that statistics shows there's no problem. See Anthony's comment to Jeff's answer.
    – Hendrik Vogt
    Commented Jun 1, 2011 at 12:44
  • @Hendrik It's 59% of only those with exactly two votes. Which is only 1% of the content. Certainly, a large number, quite large over 50%. Not a significant number in the grand scheme, though. I'm unconvinced that this problem is actually severe and not, in part, only 59% on account of so many people being so terribly afraid of "pity" voting that they just don't bother to downvote.
    – Grace Note StaffMod
    Commented Jun 1, 2011 at 12:49
  • 1
    @Gilbert: I'm not saying it doesn't have any merit. But you're talking about voting based on the reputation of the poster. While this question is about voting based on the presence of a negative score on a post (being pity or sympathy voting). 2 totally different things in my opinion. Although you shouldn't do either (you should always vote based on the content of a post), I don't have problems with the former... the latter on the other hand...
    – fretje
    Commented Jun 1, 2011 at 12:52
  • @Grace: OK, I should have been more explicit. My main point is that Jeff found the 59% "most telling", and Anthony's point was that 59% is a lot more than 50%. And I'm with Anthony here. (You're right, one could say this is marginal since we're talking only about a small portion of all the questions here.)
    – Hendrik Vogt
    Commented Jun 1, 2011 at 12:55
  • @Hendrik What I find most "telling" is that with questions costing 0 to downvote, and answers costing -1 to downvote, that the ratio of potential "pity" voting on answers is less than the inverse of that on questions. If pity voting was a real problem, I'm with Jeff - I'd definitely expect to see a far greater ratio, especially on answers. That'd be where people would be most reluctant to add the second downvote, and where there's so much more content to allow such voting practices.
    – Grace Note StaffMod
    Commented Jun 1, 2011 at 13:00
  • 1
    @Grace: It's really interesting that the numbers for answers are so different. But don't forget that the free downvotes on questions are rather new, so they can't play a big role in those numbers.
    – Hendrik Vogt
    Commented Jun 1, 2011 at 13:06
-2

Damnit people! That was too many downvotes for this question and not enough upvotes. Can't I just have a score of zero?

Improve this answer
4
  • Harder to equalize now, is it?
    – Lorem Ipsum
    Commented Jun 1, 2011 at 21:43
  • Dunno. I still don't have enough rep to see up vs. down votes. If I change the plea, will it help you think?
    – Paul McMillan
    Commented Jun 1, 2011 at 22:23
  • 5
    You're at -3/0 right now. You can use this timeline to get the spread easier even when you're at sub-1k.
    – Grace Note StaffMod
    Commented Jun 1, 2011 at 22:28
  • +1 because I hadn't come across the timeline feature (mentioned in a comment to this answer) before. Not a pity upvote!
    – Andrew Grimm
    Commented Oct 28, 2011 at 7:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .