What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

Vote fraud happens on Stack Exchange sites, it is a natural consequence of the reputation system and one of the duties of a moderator is to deal with cases of suspected vote fraud. There are various tools to help moderators investigate vote fraud, and they work very nicely (especially after they were updated a while ago). But the process begins to become somewhat fuzzy and complicated once the vote fraud is confirmed and it is time to clean up the mess.

The old way was to just merge the sock puppets into the main account, this invalidates votes automatically. This is no longer done in many cases, a bad merge causes a lot of work for SE staff and has quite a few very undesirable side effects.

Deleting sock puppets is the second way, but that can only be done for blatant low-rep sock puppets with hardly any original content to their account. More complicated cases of voting irregularities cannot be handled that way.

The final option is to ask the community team to invalidate the specific votes between users. This usually involves grabbing a random member of the community team in chat and asking them to invalidate votes.

My idea would be to streamline the process of invalidating votes and give moderators the power to do so, though with review by the community team. It would work similar to merging users, which mods can initiate but which has to be reviewed by a dev for users above a certain rep threshold.

Ideally, we mods would get a page where we can add all the involved users and which would then display the voting statistics between them and easy links to the relevant tools. It should also contain a free-form comment field where we could write up the evidence we have. The page would serve as a paper trail for other mods to quickly check up on what happened there and for the community team to be able to see at a glance all the relevant information. This should also be useful if the user writes a mail to SE and they check up on what has been done. Once the mod is confident in the collected information, he should be able to invalidate votes, which would submit the whole report to a place where mods and SE staff can check it out.

I personally think it would be safe to give mods the ability to invalidate votes, as long as such a paper trail exists that makes it easy for SE to see what happened in each incident (and undo it). But one could also require either a second mod or an SE employee to review and validate each case.

share|improve this question
    
+1 Good idea. I wonder why I didn't think of it... –  American Luke Aug 17 '12 at 17:12
17  
Yes! Then I can break up that votering Skeet has going. –  casperOne Aug 17 '12 at 17:13
3  
I thought we were all just sock-pupets for Skeet. Should I be acting independantly? –  Dave Aug 18 '12 at 17:57

4 Answers 4

I support this proposal.

As noted, we deal with a number of people who try to cheat the voting system here. I've personally handled ~50 such cases on Stack Overflow over the last month. Every one of these instances can take some time to deal with, and ways to make this process more efficient can save a lot of effort on the part of moderators and the community team.

For obvious sock puppets from a central user, merging or deleting those accounts is currently an option. However, we've had enough problems with incorrect merges that we now only use that in the most clear-cut cases, and deletion is a fairly destructive act.

Some of the most harmful behavior around vote fraud comes from rings of friends and coworkers who vote for one another. Beyond the reputation boosting, this can lead to wrong or harmful answers being voted up well beyond correct ones. I've had more than one person point to incidents of this and say their trust in the site had been reduced as a result. These are clearly separate users, so we can't merge them, and deleting their accounts seems far too extreme. Directed vote invalidation is needed in those cases to eliminate the harmful effects of these fraudulent votes on the many questions and answers involved.

For those concerned about giving moderators this ability, I'd point out that we currently have the capability of performing far more destructive actions. This tool would in fact allow us to avoid having to employ some of those actions while keeping users honest. All of our current moderator actions can be reviewed right now by other moderators or SE employees, and I would expect no less from this.

share|improve this answer
4  
This would definitely make vote-ring busting a lot easier. I'd also request that reversing vote invalidation be a part of the feature, just in case we need to fix any mistakes. –  Bill the Lizard Aug 17 '12 at 17:19
2  
@BilltheLizard Agreed, as well as some internal understanding of at what point we should invalidate votes. –  George Stocker Aug 17 '12 at 17:28
    
I would support this if it required two moderators (or perhaps a majority of the mods) to perform the action. As a moderator from a smaller site, we don't have to deal with this as much as you mention here and emailing the developers for assistance works for our needs when voting irregularities arise. Alternatively, it might be nice to have a moderator queue where a more aggressive version of the current tool could place "potential" problems for a human to give a thumbs up - this is a problem, please invalidate those votes now. Over time, the choices we make could tailor that tool. –  bmike Aug 26 '12 at 14:12
    
I am having a problem at work and ask a question on SA, then email the question to the team. I expect my co-workers to read the questions, some of them will up vote the questions; one of them may answer the question. In a few days time one of my co-workers does the same. (Given it is my co-workers job to help me do my job, it could be said that is it their job to up vote any clear and answerable work related question I ask.) Is this fraud? –  Ian Ringrose Mar 1 at 18:31
    
@IanRingrose - In that case, the voting is coming based on the content of the question, not necessarily that you're the one behind it. Now, if you and all of your coworkers got together and started voting for every single thing each one of you posted, that's a problem. We still have to make a judgment call on this, and tend to step in only when we see obvious fraud (multiple accounts by the same person) or clear voting coordination that is pushing wrong or poor quality content higher than it should be. –  Brad Larson Mar 1 at 18:34

This is a terrible idea. So now we want the moderators to be able to change our minds for us?

If there is abuse, deal with it in the prescribed manners. If not, don't keep giving more power to abuse the users to the moderators.

share|improve this answer
10  
Using this for anything other than obvious cases of vote rings or sock puppets would be a clear abuse of moderator power, and I would hope that it would be dealt with appropriately. As I said, we're already entrusted with far more destructive powers by the community. –  Brad Larson Aug 17 '12 at 17:35
3  
+1: I completely agree it is a terrible idea. It seems that this also has the potential to break vote secrecy, which I was under the impression that developers went out of their way to keep secret, unaccessible and unleakable. –  carlosdc Aug 17 '12 at 17:59
1  
@carlosdc This doesn't have the potential to break vote secrecy. It's already broken in cases of vote fraud. –  Bill the Lizard Aug 17 '12 at 18:43
8  
@BilltheLizard Maintaining voting privacy is not broken. When we analyze vote patterns, we are talking about aggregated proportions of votes going between users. None of the private, specific "user X voted for post Y" information is leaked. –  Anna Lear Aug 17 '12 at 18:48
3  
@AnnaLear Yes, that's true. Thanks for the correction. We only see that votes flowed in abnormally high proportion between users, not what specific posts anyone voted for. –  Bill the Lizard Aug 17 '12 at 18:57
1  
Moderators have been known to consistently violate the trust of their powers. Notice the dissent by moderators. It is telling. –  GEOCHET Aug 17 '12 at 21:57
    
@AnnaLear: I don't think you can assure me the there won't be false positives. For example there are two or three users that are the top answerers in some obscure tag. They do not know each other, talk to each other or anything, are not friends but without each of them knowing it they really appreciate each other's answers. The secrecy of votes in that "ring" will certainly be broken by any algorithm. –  carlosdc Aug 17 '12 at 23:34
    
@carlosdc Currently, votes are invalidated by comm team anyway. This proposal will not change false positiveness, I think.. There still will be a human reviewing it. –  Manishearth Aug 18 '12 at 11:45
4  
@GEOCHET - "Moderators have been known to consistently violate the trust of their powers." That's a pretty serious allegation. If you have proof to back that up, I suggest you provide it to the community team. In regards to dissent, I see none on this proposal. If you are referring to previous times that moderators have disagreed with one another in public, we're individuals with our own opinions on topics. The fact that we can have such arguments out in the open is a sign to me of a healthy system. –  Brad Larson Aug 18 '12 at 14:16
4  
@Manish: that's not strictly true. When we review requests for vote invalidation, we tend to do a lot of sanity-checking that moderators don't (can't) do. That's not ideal either, but it does prevent false positives now and then. Hyperbolic perhaps, but this is a legitimate concern: you should be made at least a bit uneasy by the notion of voting moderation. –  Shogging through the snow Aug 18 '12 at 17:36
1  
BTW, @carlos: the sort of "top user voting" you talk about is relatively easy to detect and ignore (see also casper's joke about the "Skeet voting ring" above). There are edge cases that turn up false positives (for both the script and the moderators), but in the majority of cases won't happen (there was the one guy who claimed he simply queued up his votes throughout the day so he could cast them all at once... on one person... But that's definitely an outlier). –  Shogging through the snow Aug 18 '12 at 17:42
2  
I agree that this idea is not ideal. I don't agree that abuse of privileges by moderators is rampant, or even remotely close to common. The cases I have seen of unjustified action were actually mistakes that human beings make from time to time, and were corrected. When has a community elected moderator ever taken unwarranted action on your account? I'm not talking about Jeff, or any other employee. I'm referring only to community elected moderators which defines the scope of this proposal. –  Tim Post Aug 19 '12 at 3:12
1  
It is telling that there are moderators who are arguing FOR more power. That should be the big red flag here. –  GEOCHET Aug 20 '12 at 12:59
6  
@GEOCHET I'm sorry that you see it that way. In reality, the interface between community elected moderators and community coordinators needs some .. love .. to say the least. This proposal is an artifact of a frustrated mod trying to make things work better than they do .. nothing more, nothing less. Nobody is grappling for power here, gaming of the voting system is (quite recently) a very serious problem we face. The process to deal with that just needs to be re-thought. –  Tim Post Aug 20 '12 at 14:58
2  
+50 l bounty for challenging the opinion that "it would be safe to give mods the ability to invalidate votes" –  gnat Aug 23 '12 at 17:04

First off - I like this idea, but I don't like either of the options you put forth.

There's definitely room for improvement in the current system. But not by creating more bureaucracy, and not by making the current process - which is difficult in part to reduce the danger inherent in the task - more dangerous.

A philosophy of voting: privacy and freedom

I want to emphasize that normal votes are private on Stack Exchange. This isn't an accident, or something we're cavalier about - it is important that anyone is able to vote according to their conscience, to rate each post according to how helpful they honestly feel it is to themselves or others... and to do that, they must be protected from external pressure to vote a certain way.

Normal users, and even moderators, cannot see who has voted on a particular post, or see where a given user has cast their votes. Even developers have to go well out of their way to find out who has voted on their own posts (developers and some employees do have access to the databases in order to do their jobs, so it's not really practical to block this information entirely... But that doesn't mean we have to make it too tempting to abuse).

While it is possible, in very limited circumstances, to make an educated guess at who voted, this is information we're obligated to protect.

All that being said, there are situations where it is necessary to remove votes that we have strong reason to believe were not cast in good faith:

A definition for voting fraud and abuse

The history of tracking and reacting to unusual voting patterns on Stack Exchange goes back to late '08:

I was pleasantly surprised to find that the vast, vast majority of Stack Overflow users do not engage in revenge downvoting. Unfortunately, for those users that do, the pattern was quite clear.
...
While working on this code, I also realized that we should be checking for the inverse of this behavior — unusual upvote patterns. To game the reputation system, unscrupulous users might create alternate user accounts that vote up their main account.

Over the next few months, this system was refined into a script that periodically checks for, and reverts abusive and fraudulent voting. And along with it, the core philosophy of voting moderation:

it will always be easier to earn reputation legitimately — by asking good questions and providing great answers — than by gaming the system.

The score of a given post should always reflect the number of members - real people - who've found it useful. Revenge voting, sockpuppet voting, and meat puppet voting all hurt that - and often worse than their actual impact on post ranking is the effect they have on the perception of post score. After all, why waste time writing a good, detailed answer when someone else can dash off something - anything - and get a larger reward for their (lack of) efforts?

We've never stopped improving the automated tools for detecting abuse, but sooner or later you have to face up to the realization that - like all moderation of human activity - doing it effectively requires some human input.

The only safe way to moderate voting is to moderate patterns, not votes

The vote examiner

Dealing with voting fraud then, is a constant balancing act between keeping individual votes confidential, and allowing trusted members enough oversight to catch and correct anomalies. As Anna notes, only broad patterns - which must involve a significant number of votes - are exposed, even to moderators. This goes a long way toward preventing abuse - but it also obscures certain information necessary to properly evaluate some voting patterns.

The actual mechanisms for resolving anomalies are the easy part - as you note, moderators can easily delete or merge sockpuppets to handle blatant abuse (with merging strongly discouraged except when it is a certainty that both accounts belong to the same person).

Vote invalidation is technically simple as well, but isn't currently available to moderators for one reason: it is intended for situations (like revenge down-voting) where there's a pretty good chance the accounts are legitimate. And if the accounts are legitimate, there's a decent chance the votes are too - even if the patterns look a bit suspicious. The fraud script - by far the largest vote invalidator - has access to sufficient information to distinguish between, say, a naive meatpuppet and top users. Well, most of the time. Being a bot makes him free from temptation, but also more than a bit stupid.

Non-solutions and arcane magic

Adding more logging and sanity-checking after the votes have already been invalidated, users messaged, accounts suspended... Doesn't really solve anything. Indeed, it stands a good chance of catching more people in the crosshairs, upsetting innocent users, and creating more work for everyone involved. Adding more signoffs without also adding more information similarly creates more work - and also a false sense of security. If you're unsure about a given situation, nothing stops you from asking for another set of eyes right now - if you're not already doing that, forcing you to probably won't help.

If there's a solution to be found here, it will be in making the script - that cold, incorruptible robot that everyone loves to hate and hates to love - do more work for you. In other words, rather than us signing off on the patterns you detect, or you signing off on the patterns that you detect, have you signing off on patterns the machine detects.

share|improve this answer
2  
interesting. That sounds like permanently tuning the script to recognize more and more sophisticated voting rings. For this to work as mid- / long-term solution, one would likely need some way for moderators to feed the data of new abuse patterns into some sort of "knowledge base" that would be useed by script. Now that would probably be the missing tool –  gnat Aug 18 '12 at 17:45
    
Half of my idea is having a better overview of all important information about the case, making it easier for you to decide if the votes should be invalidated. And I'm really not a fan of the "ping random member of the comm team" workflow. For ideas about improving the script I'll better reply in private. –  Mad Scientist Aug 18 '12 at 19:28
    
I had originally been working on a proposal to improve the vote fraud script, but was concerned about possible false positives. This feature request was attractive because it requires a human to look and identify these trends, which allows us to rule out many odd-looking but legitimate voting cases. Also, silent invalidation by the voting script currently doesn't give us a notification that someone is trying to game the system, which we might want to have in order to explicitly warn someone about their behavior. That said, I'd be glad to send script improvement suggestions offline. –  Brad Larson Aug 18 '12 at 20:26
2  
I like this Shog. One of the reasons I did not propose the 'two human initiated keys needed to launch missiles' idea is that I found it imperfect in many ways, some of which you've discussed in your answer. It is better to have the script do the heavy, mentally taxing work of analyzing the patterns and then have a human turn the second key. –  Tim Post Aug 19 '12 at 3:00
    
I think you're on to a good idea with the script (or a more aggressive version) sending problems to a moderator queue where a human has the chance to speed up the invalidation or rule it's probably OK at present. Over time, statistics on how moderators ruled in that queue would help direct changes to the automatic reversal criteria. –  bmike Aug 26 '12 at 14:18

To prevent abuse, how about:

  • notifying involved parties (or even all moderators, as in the case of moderator messages) of the vote change, and/or
  • requiring a suspension or moderator message to be issued in order to invalidate votes

Plus everything would obviously go in the usual log.

I think some combination of those restrictions or features would go a long way in making sure that the process is pretty transparent while still making it useful for moderators. The process right now is… not a process.

It’d also discourage deleting entire users to invalidate votes, which is kind of what I do right now if their answer, reputation, and other participation count is sufficiently low – and I was mistaken once on that. (And I am really sorry for it.)

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .