-16

I have a sneaking suspicion that the decision whether to upvote / downvote content relies heavily on the reputation of the author of the question.

This phenomenon is known as the fallacy of origins.

I am looking for volunteers with high reputation to help me conduct this experiment.

The experiment design would be a classic A/B testing, in which we would ask the same question from 2 different users.

User A: Would have a low reputation <100

User B: Would have a high reputation >5k

We would measure the amount of upvotes and downvotes per view, and apply a statistical significance test to evaluate the impact of the effect.

Edit 11/11/2018: Ramhound Raised an interesting point in the comments.

Users who see the 2 versions of the question might behave differently.

Can we access the logs and query which user seen which question and when ?

closed as off-topic by Sonic the Introverted Hedgehog, Servy, fbueckert, Won't, ale Nov 9 '18 at 20:23

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to seek input and discussion from the community. If you have encountered a problem on one of our sites, please describe it in detail. See also: What is "meta"? How does it work?" – Sonic the Introverted Hedgehog, Servy, fbueckert, Won't, ale
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    The policy is to vote about question's content quality, not the OP's reputation. The outcome will be that many low rep users aren't able to provide quality answers or questions most of the time. – πάντα ῥεῖ Nov 9 '18 at 19:01
  • 3
    Policy is one thing, and user behaviour is another thing. Would you like to help me test this hypothesis ? – Uri Goren Nov 9 '18 at 19:03
  • I believe such has been already tested earlier already. – πάντα ῥεῖ Nov 9 '18 at 19:04
  • 1
    Cross-site duplicate: meta.stackoverflow.com/questions/367719/… – Sonic the Introverted Hedgehog Nov 9 '18 at 19:05
  • I'm tempted to vote to close, because I don't believe this is relevant to the SE network at large; it's a survey request, which traditionally isn't a good fit on any SE. – fbueckert Nov 9 '18 at 19:08
  • 2
    @πάνταῥεῖ , If you happen to find that this experiment had been done before, I'd love to have a look at the results. – Uri Goren Nov 9 '18 at 19:12
  • Since the ending question is a call for volunteers, I'm voting to close. (cc @fbueckert) – Sonic the Introverted Hedgehog Nov 9 '18 at 19:14
  • 1
    Considering the impetus for this question...I think a greater understanding of what SE is, and what our goal is, is needed. We are not a forum, nor help desk. Those expectations are what lead to these types of requests, and learning more about the network brings the perspective more in line with what we're trying to build here. – fbueckert Nov 9 '18 at 19:17
  • Perhaps the reason why my request for an experiment over on MSO was positively received is that I kept open the possibility of it not being the case, and I provided a clear argument. @UriGoren I think you should have a look at it to see what kind of requests get positively received here. – Sonic the Introverted Hedgehog Nov 9 '18 at 19:20
  • Well, this just got weirder. How, exactly, is this supposed to be controlled? Not much of an experiment if there's no process, no control, and no oversight. So I don't think anything constructive is going to come of this. – fbueckert Nov 9 '18 at 19:41
  • @fbueckert, this is a controlled experiment in which the only feature we are testing is the author's reputation. Classic A/B testing scheme, what do you find lacking in this experiment design ? – Uri Goren Nov 9 '18 at 19:51
  • 1
    Like I said: No process; there's no way to account for outliers at all. No control; you haven't organized this in any sense. It's an experiment born of sour grapes, not methodical thought. And finally, no oversight; you can skew the results however you want, because there won't be any. As controlled experiments go...this one isn't. – fbueckert Nov 9 '18 at 19:53
  • @fbueckert, It's pretty amusing that you said "It's an experiment born of sour grapes", because this argument relies on the fallacy of origins itself. But nevertheless, how do you suggest to conduct an experiment to evaluate the effect of reputation on up/down voting tendency ? – Uri Goren Nov 9 '18 at 20:03
  • Well, considering we have users that quite often create new accounts, to test various aspects, and have no issues gaining upvotes for them, I believe the experiment is unnecessary. Experience with the system shines through. The hypothesis is entirely flawed. – fbueckert Nov 9 '18 at 20:07
  • 1
    @UriGoren - The problem with your test is that most users in a community, will recognize the fact the same question, has been asked more than once. If I saw the same exact question twice, I would simply assume, they were the same person. – Ramhound Nov 10 '18 at 22:11
9

I have a sneaking suspicion that the decision whether to upvote / downvote content relies heavily on the reputation of the author of the question.

Nobody doubts that the fallacy of origins holds on Stack Exchange to a certain degree, even though we are instructed to vote for the content, not the user. If you've just been on the receiving end of a few downvotes as a new users, the effect feels much stronger than it actually is. However, Stack Exchange is not a place for conducting experiments, not even about the site itself.

We would measure the amount of upvotes and downvotes per view, and apply a statistical significance test to evaluate the impact of the effect.

With N = 1 question, I'm quite sure that the statistical significance will be almost zero. Also, one of the questions would almost surely be closed as a duplicate of the other, influencing the votes as well.

  • 2
    "Nobody doubts that the fallacy of origins holds on Stack Exchange" What evidence I've seen (which is a decent amount) has shown that the effect is, at best, very small. Certainly way smaller than people like the OP are claiming it is. As such, I'd say I would say it doesn't hold (to a meaningful degree). – Servy Nov 9 '18 at 19:06
  • Thank you for your answer, obviously we will need to post several questions (that could be deleted later) in order to gain a significant result. – Uri Goren Nov 9 '18 at 19:07
  • @Servy certainly, the effect looks to be bigger if you're a relative newcomer, and on the receiving end of a few downvotes. I know that my own thought process works partially like 'hey, I know that user, (s)he knows what (s)he is talking about.' (I can often resist the urge to immediately upvote, though.) – Glorfindel Nov 9 '18 at 19:07
  • 1
    @Glorfindel In such cases the entirety of the difference (to a close approximation anyway) in those situations is that newer people don't know how they are expected to act, and those that have experience on the site do, and so their actual actions are different. Whenever you see experienced users using new accounts you see behaviors similar to when using their older accounts, demonstrating that this effect is, at best, very small. That some people think it's a notable effect doesn't mean it is a notable effect. – Servy Nov 9 '18 at 19:10
  • @Servy, have you looked at the distribution of reputation ? users with less than 200 rep are more than 90% of the users. Obviously this kind of bias could have a disastrous effect on newcomers ability to gain reputation. – Uri Goren Nov 9 '18 at 19:19
  • 3
    @UriGoren Sure, most users haven't posted at all. They've literally created an account and done actual nothing (public) with it. What's your point? What is disastrous about having lots of low privileged (mostly inactive) users? – Servy Nov 9 '18 at 19:20
  • 4
    @UriGoren So how do you think people that have reputation earned it if it's impossible for low rep users to earn any reputation? How do you explain all of the people that have started new accounts recently and had no problem earning reputation? Given that all of the evidence suggests that new users posting good content have no problem earning reputation, what's the basis for your assertion that new users can't earn reputation? And, as mentioned, almost all of those low privileged accounts are entirely inactive. They're not "not getting upvotes", they're just not posting. – Servy Nov 9 '18 at 19:28
  • 3
    @UriGoren Yes, there is a survival bias. Users that post quality content get upvotes and continue posting, people that post low quality content get downvotes and don't keep posting (well, hopefully, sadly in practice they often don't stop). That said, most of the low privileged accounts aren't users that "quit" because they got bad feedback, they're users that never posted in the first place or people that created an account to ask one question and then didn't bother keeping the account (simply making a new one if they have another question at some point in the future). – Servy Nov 9 '18 at 19:40
  • 3
    @UriGoren But what the evidence doesn't show is that people posting useful content don't get upvotes just because of their existing reputation. – Servy Nov 9 '18 at 19:40
  • 2
    @UriGoren Given that you don't understand such fundamental concepts of the site I think it's pretty safe to say that you haven't done enough research to be at the point of trying to do large scale experiments. You should take the time to learn what the community considers useful content, look at the existing data already available to you on upvote and downvote rates. You appear to be lacking an understanding of too many fundamentals to either successfully implement a large scale experiment, or to meaningfully interpret the results, given that lack of understanding of core concepts. – Servy Nov 9 '18 at 19:48
  • 3
    @UriGoren You have been told before that you're expected to be doing some research before asking questions, and that was given to you as a major problem with your earlier contributions. You've claimed that being told the problem would help you fix it, and yet you've continued to demonstrate a lack of research before posting, and an unwillingness to do it. That's not a good sign. – Servy Nov 9 '18 at 19:50
  • 2
    "However, Stack Exchange is not a place for conducting experiments, not even about the site itself." Why not? This is a bad mentality to have. – user Nov 9 '18 at 19:55
  • 6
    @UriGoren As has been told to you earlier, providing evidence, in the form of your own actions, that contradicts the points that you're making, is not an ad hominem attack. It's just providing evidence. An ad hominem attack would be bringing up irrelevant personal information in an attempt to generate bias. You having experience with statistics doesn't mean you're familiar enough about a particular topic to do an experiment on it. I'm claiming you lack the domain knowledge, not the statistical knowledge. As for the link, what is that intended to show? What did you learn from the question? – Servy Nov 9 '18 at 20:02
  • 1
    @UriGoren - Using past behavior as an example, and to specifically explain how to avoid downvotes, is not an ad hominem attack. Nobody is attacking you. However, your past behavior cannot be ignored, given the fact your downvotes were issued to the quality of your questions not due to the amount of magical internet points you might not have. – Ramhound Nov 10 '18 at 22:27
  • 1
    @UriGoren Using Wikipedia's definition, "Ad hominem [...], is a fallacious argumentative strategy whereby genuine discussion of the topic at hand is avoided by instead attacking the character, motive, or other attribute of the person making the argument, or persons associated with the argument, rather than attacking the substance of the argument itself." I did not do that. I stated facts about your actions that specifically contradict the arguments that you were making, rather than stating facts about you that are irrelevant to the discussion at hand in an attempt to discredit you. – Servy Nov 12 '18 at 14:27

Not the answer you're looking for? Browse other questions tagged .