As part-sociologist, I have been thinking about this for a while. It seems to me that, while voting is superficially a good way to measure community influence, its prominence on the questions and answers they prefix tends to lead to self-reinforcing, illogical voting patterns--which of course undermines the whole point.
The reason is something called pre-condition bias (a related term is anchoring). Essentially, the current score of a question or answer influences how people receive it in a positively correlated way.
The practical upshot is that downvoted posts automatically go further down and upvoted posts automatically go further up, neither with much bearing on posts' quality.
Let's look at some examples, and then I will present a proposed solution.
Here's an answer of mine on english.stackexchange. Analysis (blockquoted for clarity):
My answer has 46 words, of which 21 come from a dictionary definition and another 7 are quoted from someone else. To write this answer, all I had to do was remember a term, look it up, and copy+paste. It took me all of five minutes. As of today, it has 61 upvotes.
After I posted it, I got 10 upvotes all pretty much at once, and from there, from its cushy location at the top of the answer pile, it just kept gathering more. There's no way I deserved that for 46 words.
The fact is, this is just an average post. It's a simple answer to a simple question, and a fantastic answer it ain't.
Contrast this with my recent answer on stackoverflow. Analysis:
As of today, it has 19 downvotes (and 10 up). How it got to be this way is fascinating. I posted the first answer and got an immediate upvote. Then, someone thought I ought to mention the effect on object-oriented code, downvoted, and left a comment. Though the answer fluctuated around 0 for a few hours, the moment it passed -2 on its Gaussian walk, everyone started downvoting. What started out as a fairly even balance of 8 up vs. 10 down suddenly became 10 up vs. 19 down!
What this says to me is that past -2 votes, the negativity of the average voter "just happened" to increase by 279%. Of course, then came the comments decrying the answer. And at least one of the comments I wrote in my defense "somehow" got deleted.
Meanwhile the question, inundated with attention, acquired 47 upvotes. But this question clearly had no research effort. If you Google its title, the first three hits are
exit(int)(the call I suggested), a post from 2008 which advocates
exit(int), and a duplicate on stackoverflow whose accepted answer suggests
By the way, the accepted answer on that duplicate question is functionally identical to mine, and yet it has 43 upvotes to my -9.
The extreme examples are amazing:
Look at this one sentence question with more than two thousand upvotes. Is that really three orders-of-magnitude better than this nice question? The first user has posted just 33 times in total on the entire StackExchange, but because of that one sentence he now has moderation tools.
Look at this one which states a question on line 1, describes steps taken to solve, and provides a simple example, yet somehow has -44 votes (spoiler: it got some negative publicity on meta). Is that post, denigrated for grammar and sass, really forty times worse than this semi-incoherent drivel?
What's happening in all these examples is that votes have an inflationary effect. If once a post goes positive, it gets higher faster. If once a post goes negative, it gets lower faster. I know we're supposed to judge posts on their merits, but this sociological precondition bias effect is very, very powerful--and it is a sad reality that votes on StackExchange are almost never merit-based beyond the first couple of votes in one direction.
I believe this is a serious flaw in the way StackExchange operates. Posts on StackExchange are almost invariably direct functions of their initial votes multiplied by pageviews. I challenge the community to find a majority--or even a significant minority--of posts that defy this.
Towards a Solution
But this wouldn't be much more than a complaint unless I had an idea for how to fix it.
Reputation comes in several varieties:
- On users, it serves to delegate or restrict privileges. The higher the reputation, ostensibly the more trust the community places in them.
- On users, it is a rough measure of how good the quality of an answer or question is expected to be (and how much the user understands the StackExchange system). It's a point of respect. It helps you know how invested the user is in the community.
- On questions and answers (via votes), it (theoretically) serves to order posts by quality.
All of these are well-thought-out, important aspects, and any solution should preserve them.
But there's an important thing to note in the above points: reputation is nothing more than imaginary internet points--a proxy for worth to the community. When a question is posted or an answer is given, the differential value readers assign to it should not have any bearing on other readers, nor should it unduly and recursively influence the poster's privileges. What I mean is that reputation changes are supposed to affect standing in the community, not bias readers in reading that user's content.
So here's what I propose:
- At the very least look into this problem with an eye toward possible fixes. I believe this is a serious flaw that at minimum deserves attention from the community.
- Consider my pet idea: make votes on questions and answers on non-meta sites completely invisible to all but the poster and moderators.
As a longtime StackExchange user myself, I understand that this would unquestionably be a radical change--probably one that would have to be tested in a limited rollout on one or a few networks. Actually, I think it would fundamentally alter the way the site works.
But think about what we stand to gain. It solves the majority of the problem. Votes still are accrued by post, and affect the poster's value to the community. However, the reader would be unable to ascertain the degree to which the post is endorsed or reviled beyond the difference one vote makes to relative ordering of answers. This would leave the reader to judge the content primarily on its own merits.
And isn't that what we want?
- Regardless of the solution ultimately chosen, the StackExchange voting system is utterly broken because of this biasing effect. This needs to be fixed yesterday.
Thank you for your consideration.