This experiment has been graduated. The new styling for vote arrows is now live across the Stack Exchange network.
About a year ago, the Product team, focused on improving the general experience, conducted an experiment to update the styling of the voting arrows on Stack Overflow in order to improve accessibility and meet Web Content Accessibility Guidelines (WCAG) compliance. We shared positive results from the experiment: a net 28% increase in overall votes.
At the time, we decided to not move forward for a few reasons:
We wanted to take a broader approach to looking at the question page instead of one particular element and decided to roll this into a different initiative.
Additional accessibility issues were raised that needed to be re-tested in dark/high contrast modes
We’ve decided to revisit our decision and to graduate the experiment on Stack Overflow and the Stack Exchange network. Before we proceed, we acknowledge there are still a few outstanding accessibility issues and bugs that we need to address first before we graduate.
Here is what the arrows look like before and after the change:
We’ve heard your feedback from the previous posts about this change, and are not moving forward until we address them. We are prioritizing issues specific to accessibility compliance and bug reports. This might mean that any drastic visual changes to the styling will not be within the scope of this first iteration, but that does not mean we won’t iterate the design.
Moving forward, we are committed to:
Prioritization of bug and accessibility reports to be addressed before we go live such as:
- Any future iterations will be adequately communicated earlier in the process (i.e. designs, experiment plans).
Monitoring accidental voting and other relevant data points
- In the previous post, we saw feedback from users with >1k rep who flagged the risk of accidentally voting when checking the post score. We took a closer look at this report and pulled the following data:
- Last month (Apr 2023), we saw 175k users who voted 870k times. Of those users, only 8% (14k users) had undone their votes.
- The number of posts that had votes undone only accounted for 3% of total votes. This is the benchmark we will use moving forward.
We will continue to monitor these numbers to see if there is a noticeable uptick, and then determine if it’s a widespread issue to prioritize.
This change is part of a holistic exploration of the voting mechanism and what it means for everyone. Voting is an essential part of the mechanics of the platform that makes this repository of knowledge so successful. It is how we acknowledge the good work of our contributors, and we want to really explore different ways in which we can make this process more streamlined and engaging for everyone.
Year over year, we are seeing 28% fewer votes (upvotes + downvotes) and 21% fewer users voting. During our research, we also found that voting is a pain point for new users as well. In April alone, we saw:
- 72k unique users who attempted to vote 166k times (avg 2.3 votes per user).
- Of those 72k users, 17k users were new users which is 24%
- This means that 1 in 4 new users attempted to vote, but could not due to system limitations.
Because voting is one of the lowest barriers to participation, we felt that this was the best place to start to help users engage more on the platform. In the coming weeks, we have plans to roll out more ideas that we have to iterate on voting and it’s part of the user story of this site to increase engagement, so please stay tuned for those updates.