What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

The number of upvotes for a question is a flawed measure of question quality, one particular issue is that the amount of exposure a question gets matters a lot. Questions that were linked in the hot questions list tend to get an enormous amount of votes, much more than any questions gets organically on the site itself.

The kind of question that tend to get to the top spot of the hot questions list tend to be simpler, less technical questions with an appeal to a broader audience. The hot questions list distorts the voting even beyond the mere effect the increased views have. It exposes the question to users that might be interested in reading about that topic, but are not active users on the site and wouldn't have seen and voted on the post otherwise.

I think this distortion of voting is harmful, as the result might not be representative of the community of that specific site. For example, while I might enjoy reading some answers on Security.SE, I'm by far not qualified to actually evaluate them. If I voted there, my votes would not provide any actual value as I can only judge the answers in a very superficial way. Another example would be Skeptics where the community requires answers to cite references supporting their claim. This works very well and the community also votes that way, but the moment a question hits the hot questions list, voting becomes unreliable and answer not meeting our standard are upvoted anyway.

I think that this interaction between the hot questions feature and the low barrier to voting on sites where one didn't earn any reputation at all leads to a distortion of voting and dilutes the usefulness of post scores.

My proposal would be to not count the association bonus for the voting privilege. This means that in order to vote a user would have to get at least one upvote on an answer, two upvotes on a question or five accepted suggested edits. I think that this barrier is low enough to barely inconvenience anyone with a serious interest in the site, and significant enough to prevent drive-by voting by non-experts from the greater SE network.

share|improve this question
2  
+1 pending data showing this actually happens. –  AAA Jun 5 '13 at 12:13
25  
"The kind of question that tend to get to the top spot of the hot questions list tend to be simpler, less technical questions with an appeal to a broader audience." - Welcome to Stack Exchange. It's by no means a function of those that can vote by association bonus, I assure you. –  casperOne Jun 5 '13 at 12:23
5  
@casperOne Certainly not, the hot questions list and the ability for everyone to vote exaggerate and amplify this effect, but they are not the main cause of it. I still think this might be worth it as it limits the distortion somwhat. –  Mad Scientist Jun 5 '13 at 12:27
5  
The major part of the traffic to any site is made up of non-experts and beginners to the field. Hence, such questions actually help drive a lot of new people to the different SE sites. The voting is just a reflection of that. It is about popularity rather than quality. Sometimes, the activity is localized, sometimes it may go beyond that single post. If a simple post is able to get even one new user to a site who may turn out to be very active, then the post itself has actually served a much higher purpose simply because of being simple. –  AsheeshR Jun 5 '13 at 12:53
16  
+1 The majority of highest-voted questions on Mathematics.SE are the questions that SE (probably, mostly SO) users upvoted out of proportion, for their amusement value. I don't have a problem with people getting a laugh out of a 3rd grade teacher's mistake. The problem arises when such questions become highest voted on the site, and consequently get shown to new visitors as a welcome to the site. –  ˈjuː.zɚ79365 Jun 5 '13 at 14:26
1  
their privileges would then be create chat rooms, edit community wiki, set bounties, talk in chat, flag posts, create wiki posts, remove new user restrictions, participate in meta, comment everywhere, create posts, correct? –  gnat Jun 5 '13 at 14:49
1  
@user79365 that is part of a greater issue, of how the hot questions algorithm works. There is a meta question somewhere that shows that because Math.se users tend to organically vote more, a disproportional number of Math.se questions end up on the hot list. –  ryan Jun 5 '13 at 14:54
    
the reasoning feels similar to one that was proven to work for protected questions but the consequences are rather difficult to estimate. It would be very interesting to give this "feature" a trial run for a few months at one of smaller SE sites (Skeptics? Biology? :) to learn the effects it might cause –  gnat Jun 5 '13 at 15:10
5  
-1 Sometimes I visit a new site and I see a good question I would like to upvote. I make an account and thanks to the association bonus I can. What we should do is raise the minimum reputation needed to earn the association bonus. –  Cole Johnson Jul 12 '13 at 7:12
6  
Since they'd been a request for data, here's the list of MathOverflow users who are voting without having any questions or answers there. data.stackexchange.com/mathoverflow/query/137109/… –  Scott Morrison Sep 13 '13 at 1:24
1  
1  
@ScottMorrison Summary: 1333 users giving 9159 upvotes and 2 downvotes. This is of a total of 35678 users, 854611 upvotes and 53939 downvotes. So about 1.1% of upvotes are from non-posting users, and those users (non-posting but voting) make up 3.7% of the total userbase. –  Qubei Aug 8 at 7:32
1  
@Qubei, thanks for the summary. That strikes me as rather a lot, and worth continuing to try to weaken the effect of the division bonus on MathOverflow. –  Scott Morrison Aug 8 at 21:10
1  
related: Impose a 24 hour voting freeze on questions being discussed on Meta "The argument for locking voting sounds remarkably like the complaints smaller sites have had about hordes of SO users coming in and voting on hot questions because of the extra attention that they have received..." –  gnat Aug 20 at 7:33
1  
If the main concern is hot questions, what about a new "lock" or "protect" mode requiring 125 rep (same as downvote privilege) in order to vote on such questions and their answers? –  Josh Caswell Aug 20 at 22:07

6 Answers 6

I don't think this is useful.

I support forcing reputation to be earned on a site before votes are granted on meta. But I don't want to have to re-earn privileges on every site I participate in before I can vote.

I realize that you're trying to solve a problem with regard to the hot questions list. But just in the normal course of searching and browsing I often find an answer to a question before I post on a site where I have an interest. If I cannot vote on that site because I have not participated there, I cannot reward the poster of the question and answer for solving my question before I had one.

If this privilege is removed we might as well remove the entire association bonus system and force all users to re-earn their privileges on any new site.

The minimum bar reputation privileges are there in order to get users to learn their way around SE's mechanics, once they've done that once it's really unnecessary to force them to go back to being completely new users on every new site.

share|improve this answer
33  
Exactly this. It would be incredibly frustrating if I had to actively participate in each site I have some background knowledge of just to give credit where credit is due. If the hot questions list is really that much of a problem, the problem is still with the hot questions list, not with the association bonus enabling voting. –  Tim Stone Jun 5 '13 at 14:58
2  
@TimStone if it was only about questions. Drive-by / sympathy upvotes to crappy answers are also a problem, making wrong impression on what kind posts are welcome. As an example: crappy off-topic (10K-only) question / chat discussion with details here - it took efforts of 6-7 site regulars to offset score for equally crappy answers (question even wasn't hot, there were just careless passers by upvoting "entertaining" crap)... –  gnat Jun 5 '13 at 15:40
5  
...as for hot questions, it is almost guaranteed that any amount of DVs cast by regulars to crappy answers will be rolled back to zero by drive-by sympathy upvotes, again, making wrong impression on what kind posts are welcome –  gnat Jun 5 '13 at 15:41
5  
MathOverflow has an endemic problem that 'easy' questions (which often aren't even appropriate on the site) get relatively lots of votes, merely because the audience who can understand any given 'hard' question is very small. I don't want to see this problem get worse because of users coming from other sites, before they've established themselves as an active MO user. –  Scott Morrison Sep 13 '13 at 1:26
5  
@ScottMorrison I get that. But really, voting is a measure of question popularity. I know we all want it to be about question quality, but at this point we know that's not true. But voting on questions is first and foremost about how accessible the question is, how well written it is, and how broad of an appeal the subject matter has. You're going to get tons of votes on easy questions no matter if you put barriers to voting from other sites. That's just how it tends to work. –  wax eagle Sep 13 '13 at 2:17
1  
This doesn't seem like a very persuasive argument to me. You're essentially saying, "I'm against this because it would inconvenience me." Well, yes, it would inconvenience a lot of folks. That's kind of the point. Nothing in your answer provides evidence that you or anyone else is actually qualified to vote on content on a site they've spent all of 12 seconds on. Just because you would like to "reward the poster" does not mean that the reward should be yours to give. –  Aarobot Jul 28 at 16:06
2  
@Aarobot If you're going to make this argument, you should probably also make the argument that the bar for voting in the first place is too low. 25 rep is trivial on pretty much any site. My argument is that knowing what to vote on is more a function of understanding SE than the subject matter. –  wax eagle Jul 28 at 18:36
1  
I can clearly see that's your argument; what you haven't provided is anything to substantiate that argument. Does your country allow non-residents to vote in elections, just because they've voted in their own countries? –  Aarobot Jul 28 at 18:42
2  
@Aarobot no, but treating voting with the solemnity of national elections would be a huge mistake. Generally the bar for upvotes/downvotes is "is this useful?" "is this well written" if the average drive by user can't figure that out, they probably aren't going to be getting much out of SE anyways. I'm not sure why there is any burden of proof for the status quo here. –  wax eagle Jul 28 at 18:45
1  
Huh? They aren't getting much out of SE, or at least not that site in particular; that's the point, they're drive-bys! There wasn't a "burden of proof" on the status quo when the status quo was simply voting on the trilogy; the stakes have changed now that there are hundreds of Stack Exchange sites, and even more importantly now that we have the Hot Questions list that is specifically intended to direct people to sites that they wouldn't normally look at. This has nothing to do with the individual user at all, it's about the aggregate effect of the system. –  Aarobot Jul 28 at 18:49
    
@waxeagle, 25 reputation is nontrivial for non-mathematicians on mathoverflow! –  Scott Morrison Aug 8 at 21:11

Without data (requested here) it's rather difficult to tell how much of an issue this is, or what sites ate affected.

  • Say, one may ponder that issues related to bonus don't have a noticeable impact at Stack Overflow, due to it being so much larger than all the rest of the SE network combined. But then again, without data this is merely a guess.

Anyway, in order to tame the damage of drive-by votes in hot questions, I would consider more targeted approach.

I think the most straightforward way to prevent over-voting from newcomers who just arrived at the hot question would be to simply delay granting them upvote privilege for a day or two.

This approach follows one recommended in a seminal article by Clay Shirky (bold font in quote is mine):

my favorite pattern is from MetaFilter, which is: When we start seeing effects of scale, we shut off the new user page. "Someone mentions us in the press and how great we are? Bye!" That's a way of raising the bar, that's creating a threshold of participation. And anyone who bookmarks that page and says "You know, I really want to be in there; maybe I'll go back later," that's the kind of user MeFi wants to have.

Particular advantage of "delayed approach" is that it allows to address concerns of the newcomers in a non-confrontational way: If you really want to vote on that post, simply wait a day or two and get back to it to cast the vote.

Another option, also compatible with Shirky's guidance, would be to require particular citizenship level (if this feature is implemented) in addition to association bonus to grant upvote privilege.


Don't get me wrong, local reputation requirement feels fair to me and I would probably even prefer it to above, but it looks quite difficult to justify. For example, I really can not imagine how to address concern like this:

Hey I've got over 1K... 3K... 10K rep at 2... 3... 5... Stack Exchange sites, how am I less qualified to vote than a random guy who just happened to get two occasional upvotes on their single answer?


Another consideration to take into account is that setting "hard" requirement to gain on-site reputation prior to voting privilege may have a side effect of incentivizing abuse. I've seen something like this once at Programmers, in a hot question that was protected - which essentially models a requirement to gain on-site reputation prior to doing something.

User who arrived from another site just quickly posted 3 low quality answers to different questions, two of which picked quick sympathy upvotes and, before their answers were flagged and removed, they were able to break through and post into protected question.

Imagining tricks like this becoming a regular practice instead of isolated incident sends shivers down my spine.


Side note current way looks particularly ugly in the context of protected questions. Fundamental approach at Stack Exchange is that it's easier to contribute content than to rate (vote) it, and this makes good sense: one obtains a privilege to rate posts after proving their will and ability to contribute positively rated content. The way how association bonus works in protected questions totally trashes this reasoning: system doesn't yet trust user enough to allow them contribute content but somehow grants them the privilege to rate it. Weird, isn't it.

share|improve this answer
    
To address your second part, what could be done is raise the threshold at which the association bonus is awarded, instead of it being 200 maybe it should be 500 or 1000. That might make more sense than removing it's effect. –  wax eagle Jun 11 '13 at 15:58
    
@waxeagle yeah that's an interesting option, although it may be a bit difficult to implement and justify. You see, to authoritatively answer questions like "why 500? why 1000?" might take quite some effort - as opposed to that, telling a user "we expect you to spend a little time to familiarize with site specific Q&A prior to voting on these" appeals to common sense –  gnat Jun 11 '13 at 16:38
1  
I agree. I think 200 rep is a good threshold, as it is I oppose changing how the system works because I don't think it's broken. However if others do then good/better potential solutions may be out there. (as far as justification 1000 rep is the established user threshold, which might be a good thought). –  wax eagle Jun 11 '13 at 16:45
1  
@waxeagle well you might be interested in taking a look at the hotness formula damage case study - "+25, +14, -1, -1, -1" for an example of why I think the system would better be tuned in this regard –  gnat Jun 11 '13 at 17:07
1  
Regarding your last paragraph, without data it is not possible to check which of two (a user with a single answer and two occasional up-votes vs. a user with no answers and association bounce) has a larger effect. I would guess the second one happens more often on many smaller sites. But putting that aside, the existence of another problem doesn't mean we shouldn't fix this problem. We can't fix all problems at once, so if we don't fix problems because there are other problems no problem would get fixed. –  Kaveh Jul 12 '13 at 9:12
    
One thing that is (or would be) different about SE vs. MeFi is that registration itself isn't being delayed, only voting. The most likely outcome of this is to only prevent vote skewing on the first instance of any individual outside network user being exposed to a "hot question". The second time they click on a "hot question" link to the same site, their privileges will be in place. I think it's better to divorce the rule from association entirely, and simply require a certain number of actual question or answer upvotes (rather than just reputation) in order to vote. –  Aarobot Jul 28 at 16:04
    
@Aarobot I also would prefer to have it the way you suggest (see "Don't get me wrong..." section), the only reason I suggested "softer" approach is that I just can't figure a compelling justification to address potential complaints like I wrote about: "Hey I've got over 1K... 3K... 10K rep at 2... 3... 5... Stack Exchange sites, how am I less qualified to vote than a random guy who just happened to get two occasional upvotes on their single answer?" –  gnat Jul 28 at 16:19
    
And the simple answer is: You have no experience with the subject matter (as far as we know). That's the problem - we treat voting competency the same as competency using chat rooms or posting links - i.e. coming from experience with the system. It's actually experience with the subject or community that makes voting useful. Doesn't matter how great you are in other communities. –  Aarobot Jul 28 at 16:36
    
@Aarobot well 57 upvotes given to the answer that seems to counter your simple (and I believe fair) reasoning made me try softer approach. I want at least some progress, if it compromises fairness to avoid resistance, so be it. You see, this request is stuck for over a year now and I really hate that, seeing how smaller sites suffer... –  gnat Jul 28 at 17:01
1  
The scary thing is that Programmers is only a "smaller site" by the standards of Stack Overflow and Server Fault... it's bigger than almost every other site on the network, which is hardly surprising considering its audience. Anyway, I did downvote that other answer, but I think this is one of the cases where the answer votes don't/shouldn't matter much; the question votes are a measure of relative importance, the top answer is rather predictable ("but I want it!") and the dev team will have to decide if they believe it's an issue or not. I do wish they weren't just ignoring it. –  Aarobot Jul 28 at 17:59
    
No voting on first day? Goes to create accounts on all sites now in case they're needed later. –  Monica Cellio Aug 15 at 16:57
    
@MonicaCellio voting of user having a bonus and additionally smart enough to do it that way somehow doesn't worry me much –  gnat Aug 15 at 16:59
    
@gnat It seems that delay would only help with the first hot question a user sees from the particular site. They will create an account, find they can't vote yet, and go away disappointed. Next week, another question from the same site will hit the charts, and we are back to square one. –  Rafflesia arnoldii Aug 15 at 17:25
    
@900sit-upsaday I am primarily focusing on most blatant cases of passer-by voting, those from first day users. I was also thinking about making certain amount of days-visited or something like that a requirement (see note about citizenship level). That would be interesting, for sure –  gnat Aug 15 at 17:40
    
@900sit-upsaday ...an important point to consider is, the higher we set the bar, the higher we get the risk of abuse from those trying to hop over it. Think about multiple users currently hopping over "infinite" q-ban at Stack Overflow - or, in the context of this discussion, about my example of the guy who hopped over the bar in protected question at Programmers. Cases like that hopping at Programmers are now rare and I would really wish these to remain rare. If that takes softer requirement for voting, so be it –  gnat Aug 15 at 18:13

What users from other site want is important but I think the main questions that we should answer are the following ones:

Is voting because of association bonus beneficial for a site or not? What does the site's community think about it?

After all a site is mainly for its community: users who actively participate on that site. I think this is far more important than what users from other sites might like to do. I think the decision whether users from other sites can vote because of association bonus should be decided mainly by each site's community.

In response to wax eagle's answer, I don't think we need to reward every useful resource on the Internet. If it is helpful for you that is great. But it doesn't mean you know what that particular site is about or even if the question is on-topic on it. Knowing about how SE software works does not imply we are familiar with what is considered on-topic or off-topic on another site. Take for example the famous batman question on MSE. I think the active users of MSE do not think it is a very good question. However, it is highly voted because of votes from users active on other SE sites. This can send a wrong message about what a site is really about. If we are not actively participating on a site do we have a right to define what that site is about? It seems to me we don't. And the batman question is not an exceptional case. Many questions with very high number of votes are like that: the site's community does not think that the question is good enough to represent their site. But because of votes by users from outside the community, those questions are on top of the list of highest voted questions (and answers).

SO has a significantly larger number of users and this can create a heavy bias in the votes for questions that attract lots of views from outside a site's community (e.g. when it is mentioned on HN) and this doesn't seem helpful for smaller sites who might agree what questions and answers should represents their site. (ps: it might make sense for the trilogy sites which share a large portion of their users but between sites where this is not true.).

On the other hand, I think the association makes sense for other privileges. I guess they are the main reason why association bounce was put in place. It doesn't make sense to forbid posting links because one has not earned enough reputation on a particular site yet. I have never seen people complain because they can't vote on another site. However, I have seen several times that people got frustrated because they were not allowed to post comments.

I don't know if it is viable to distinguish between reputation earned on a site from the association bonus but it seems to me that the software already supports this: a protected question can be answered only by those users who have earned reputation on that site. So it seems it should not be difficult to put something similar in place for voting.

share|improve this answer

You touched on something interesting here:

If I voted there, my votes would not provide any actual value as I can only judge the answers in a very superficial way.

It's difficult to discuss the premise in a matter-of-fact way, because it's not really possible to ascertain why someone voted on something the way that they did without outright asking them. However, I'd argue against the premise, because as far as I can tell people up-vote mostly for the following reasons:

  1. They're acknowledging the technical accuracy of something because they know it to be correct
  2. They learned something that they didn't know before
  3. They were entertained, in some way, by reading the post

This only becomes undesirable when something is wrong, inordinately up-voted because of reason number 3, and now falsely sitting in the number 2 bracket. I can only think of one instance where this happened, and it was on Gaming when someone brought up extreme pig riding in Minecraft:

Porkchop, of course!

... yet the association bonus really can't take credit for the votes, people found it valuable because it made them feel good.

Yes, it happens. But I don't think it happens nearly enough for us to make the level for participating in a new community any higher than it currently is for our seasoned, trusted users.

Put differently, there's litter on every major street, but the majority of people hold onto their rubbish until they can find a suitable place to deposit it. Most people that earn this bonus accept that it's also a responsibility. I see a lot more harm than good coming from taking this away, even if the problem was significantly worse than it is right now.

share|improve this answer
    
what's your take on delaying granting the voting privilege for a day or two? would this do more harm than good, just like you believe it would happen in case of taking this away completely? –  gnat Aug 15 at 6:32
    
That's an interesting question because it's very difficult to measure. If the question was just something in a realm where pretty much anyone could understand it (and thus feel confident voting), but most of the rest of the stuff wasn't, would we really be putting off a potential contributor? I think it's one of those 'it depends on the site' propositions. –  Tim Post Aug 15 at 6:36
    
putting off a potential contributor - how's that? I don't quite follow. I mean, it's not about asking nor answering –  gnat Aug 15 at 7:09
    
...as for measuring it, a first step / upper estimate would be simply to count how many votes users cast in their first day (no matter with bonus or not) and compare against amount of votes cast in subsequent days, wouldn't it? (following your note in prior comment, such an estimate would better be per site, because otherwise it would be heavily dominated by SO) –  gnat Aug 15 at 8:47
    
this ode to indiscriminate voting (mostly coming from SO users?) sounds particularly funny in the light of recent whining of these very SO users about Meta Effect voting. Funny how they seem to change their mind to opposite when things turn out to impact SO instead of smaller sites. "Meta effect is actively detrimental" wow. Let's lock everything, it hurts (when it's at SO)! –  gnat Aug 20 at 7:46

What if only votes from active users of a site affected what was shown on the front page of the site? This would remove the problem of the “easy” question setting an example of what the site is about.

Also the system that chooses the “Hot Network Questions” should not take into account any activity a question gets when access var the “Hot Network Questions” list.

share|improve this answer
    
as far as I can tell, SE team believes that only plain straight vote count is feasible for hot questions performance wise –  gnat May 13 at 10:46

The first thing that I can say is that the association bonus only allows UPvoting on a question on a site. There requirement for DOWNvoting is 125, greater than the association bonus. You need to have earned 24 rep on that site to downvote, even starting from 101.

I'm a member of many sites. There have been a number of times when I started by voting on a single hot question, and then returning to the site to become active.

One idea mentioned here that appears to have merit is the "time lag" proposal. Perhaps one shouldn't "associate" and be allowed to vote the same day. Maybe the same should apply to someone getting an upvote privilege the "natural" way. S/he shouldn't be allowed to vote on the first day on the site even after earning 25 "rep" if that happens; maybe only on the second day on the site.

share|improve this answer
    
bounty awarded for an interesting idea to get rid of first day voting completely. Not that I agree with it mind you (at least not yet) but it certainly made a good food for thought. Have to admit, it initially shocked me (wonder why it has got so little downvotes:) but the more I ponder about it, the more it feels, why not? Bonus-shmonus, upvotes-shmupvotes, edits-shmedits, no matter how does one gets rep, wanna vote? stick with us! (for at least a day or two) –  gnat Aug 21 at 12:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .