TL;DR: We are interested in finding 2-3 Stack Exchange sites willing to volunteer to test lowering the reputation required to upvote and downvote to 1 so that we can understand how this change impacts participation on sites. This test will be run with direct communication between the company and the moderation teams and we will be monitoring user behavior during the tests for specific indications that adjustments to tooling or ending the test may be necessary. If successful, we'd like to expand this change to the network.
What is the background of this experiment?
We've long understood that many users find it difficult to participate on Stack Exchange in some ways. We have collected user research over the years indicating that many people have struggled to participate because they feel the restrictions about who can vote are too limiting and prevents them from indicating when questions or answers are good or bad. While one of the core concepts of Stack Exchange is that anyone should be able to freely access the content and have the opportunity to ask, answer, or edit, this openness doesn't extend to other privileges on the sites.
We’ve historically used reputation as a way to prevent misuse, but in doing so we’ve made it hard for people to participate at all without creating content. By removing or changing some of these barriers to participation, we hope to spur participation and engagement - including posting, voting, and curation/moderation. The goal of this particular project is to understand the impact of removing the reputation required to upvote and downvote.
Reputation is a simplistic way to prevent abuse, but it doesn't reflect actual risk or a user's experience with the site or subject. By understanding the impact of removing the reputation barrier, we can find more targeted ways to address abuse for users of all reputation levels rather than sticking to the rather simplistic solution of relying on a reputation barrier to prevent wider-spread misuse of votes.
We have these restrictions in place for a variety of reasons, but the core concept behind all of them is to prevent misuse and protect the perceived value of the content on the sites. We do not want to make it more difficult for mods and community members to moderate content on the sites, nor do we want to see people inappropriately voting on content and thus reducing the value of the library of information.
As such, while removing the reputation required to vote is a five minute change to our privilege settings, we've spent the past few months understanding the potential risks of this change and adjusting our plans for this project. Based on voluminous thoughtful feedback from moderators across the network, we've made improvements to some moderator tools and have enhancements in the works based on their recommendations. While these changes aren't going to prevent all issues, they should make identifying and acting on them simpler.
Before we build additional moderator tooling and before we expand this test beyond these initial test sites, we want to see this change implemented on a few sites to see how it impacts engagement. It's important for us to hear from sites directly and see these changes in action to learn how changing the reputation to 1 changes voting behavior or if it adds an unreasonable amount of work for moderators and CMs to investigate. Our goal is to use this information to guide our future efforts, identify any necessary additional moderation tools, and validate whether these voting updates lead to increased engagement.
What would change for sites participating in this test?
There are three main privilege changes happening here as part of our test:
- Reputation required to cast upvotes is changing from 15 to 1
- Reputation required to cast downvotes is changing from 125 to 1
- The 1 reputation cost to downvote answers will be removed
We will notify logged-in users with <125 rep to let them know that they can vote with a popup.
To understand motivations for new downvoters, the first few downvotes from a <125-rep user would require them to explain their downvote - this information will not be posted anywhere, but will instead be collected by our research team and bucketed into categories so that we can better understand reasons people downvote. While we expect that many voters may just enter gibberish, we feel like we'll get at least some usable information.
While we have considered various options for this test, since our goal is to be as permissive as possible, we want to start with opening voting to everyone. Regarding removing the reputation required to downvote and the cost to downvote answers, this is a situation where we want to balance the system. Users don't like receiving downvotes, but they are vital to a quality-based system. When we're looking at things like an aging content base, we recognize that giving even passive users the opportunity to say "this answer doesn't work" may help combat that to some extent.
One major thing is not changing - having a registered account will be required for voting. Unregistered accounts and logged-out users will not be granted the ability to vote. Additionally, users who are suspended will still not be able to vote during their suspension.
What are the risks and challenges of this test?
Over the last few months, moderators have shared their feedback and concerns about this experiment, which have been related to changes to vote quality, disproportional increases in vote fraud, and difficulty identifying handling sock puppets with the existing moderation tools. We understand those concerns and have done our best to adjust our test plan to find a path forward that allows us to test our hypothesis without risking negatively impacting the entire network by suddenly changing voting practices. Much of our test plan is designed to catch cases the moderators have identified and give room for communication throughout the test.
To address their concerns about identifying vote fraud and sock puppetry, we've made various improvements over the last three months to these tools. While we generally avoid speaking too publicly about moderator tooling, here are some of the general improvements we've made or are planning to make:
- We have overhauled a moderator dashboard that draws attention to potentially suspicious votes and made it easier to find (live on site for use).
- We have overhauled the Moderator Messaging and CM escalation UI to show mods and CMs more history about the user in the message context (live on site for use).
- We are making it easier for moderators to find and act on sock rings (in design phase).
- We are making it easier to escalate groups of suspicious votes for CM review and invalidation (in design phase).
We're still working out some bugs in these tools but the feedback we've gotten indicates the updates are well-received and they're looking forward to further improvements.
What is the test plan?
We want to find 2-3 established sites that are willing to volunteer for this test. Preferably, these will be larger sites that have moderators willing to work directly with the members of the Community Enablement team, the team dedicated to this project whose primary responsibility for the past year has been creating and improving moderator and community management tooling.
Our specific plans for data collection include:
- Reviewing existing voting and participation data and analyzing the impact of the changes in collaboration with site moderators and community members
- Conducting user research to assess perceptions of post quality before and after this change
- Analyzing voting differences based on various factors including:
- Account age
- Final outcome of the post - e.g. if a question is closed or deleted and why, or an answer is deleted
- Keeping an eye out for changes in users going undetected by tools that use reputation as a signal to block participation
- This includes both built-in tools like SpamRam, as well as community-created ones like SmokeDetector.
- Setting guidance before the test that will help us identify cases where rolling back the changes or making other adjustments are necessary.
We intend to run this project as transparently as we can, considering it relates directly to voting, which is something that we try to keep as private as possible. Due to the sensitivity of this data, some of the communication will be on a Private Team with the moderators of these sites. We will ensure there is open communication of publicly shareable information with the sites directly through the moderators and staff working on this project.
Because our test includes regular monitoring and clear guidance on when reverting the change is the best course of action, our plan is to retain these changes unless we see evidence they're causing negative impacts that we can't address quickly by making changes to tooling or automations.
After the test has been running for a month, we will create a report to share with here on Meta Stack Exchange before we determine the next steps for this project.
Which sites will be chosen?
This is a big change, and we would really love to find sites willing to volunteer - we don't want to force any sites into this test group. While we have sites we think would be good options and we'll be reaching out to them directly, we encourage members of sites to discuss whether they're interested in participating in this test on child meta sites. We would like to have some of the larger sites participate and will likely lean towards them as the impacts of these changes will probably be seen more quickly than on very low-activity sites.
Conclusions / asks
Removing the reputation barrier to voting for all users is a change we feel will lead to more users being able to indicate post quality and usefulness, but we recognize that it will require a lot of care to ensure that we are educating voters about when to vote and encouraging them to follow acceptable voting practices while also preventing or removing votes that were cast inappropriately.
We recognize that you may have many questions or concerns about this project and that this post may seem like it doesn't include a lot of specifics about the risks of this change. In writing this question, we're trying to focus on the test to keep it short(er) - the version of this that was posted on the Stack Moderators Team three weeks ago was significantly longer, as it responded to many of the mods' specific concerns.
- If you think a site you participate in would be willing to test this, let us know in a comment so that we can reach out on the child meta site or start the meta discussion and flag the question to request mods to feature it on the site. We can create an answer with links to site discussions network wide.
- If you have questions, please ask them in answers so that we can respond to them individually. It really helps if you can try to limit answers to one primary concern.
We have spent over three months reviewing and re-evaluating our plans for this project based on a huge amount of very helpful feedback from moderators over the course of two questions on the mod team to get to where we are now. We really appreciate the feedback they shared with us because we want to investigate this change in a way that works to avoid negative impacts to the sites.