Indicate How Trustworthy the UpVotes Are

Question

Takeaway: Are 4 inexperienced users more reliable than 3 experienced users? A high rep doesn't ensure you are smarter, but it makes you the safer option.

---

I went to SuperUser to see what suggestions can be found for different Malware-removal Applications. Question #19217 was right on target, and sported five answers, with a total of 9 up-votes.

All of the answers were provided by low-rep users, and that leads me to recognize that the up-votes may be a similar case. Just as answers aren't always equal, neither should up-votes be. The top answer currently has 4 up-votes, and the second most popular answer currently has 3 up-votes. The people who voted in the 3, may be more reputable than those who voted in the 4. Am I communicating this clearly?

I'm not asking that we be told who voted for a particular question, but I am asking that we be told what type of reputation they had. If the 4 votes come from 4 users with less than 300 rep, and the 3 votes came from two users with 300 rep and 1 with 10k rep, the 3 votes should appear stronger by the standards of the system.

Right now I'm worried about accepting the top-voted solution simply because I fear the up-votes may be from people with similar rep to those providing the answers - that doesn't make them wrong, only less-reliable by the standards of the stackexchange engine.

Can we see the sum of rep for the votes? (Data in parenthesis would not be visible)

4 Votes (Ted • 72 | Bill • 183 | Carl • 423 | J • 100) = Power of 778
3 Votes (Bob • 19,123 | TJ • 283 | Mike • 8,923)       = Power of 28,329

So in this case, you wouldn't see who voted, only that their rep-sum was 778 and 28k.

4 Votes : 778
3 Votes : 28k

I'm not tied to this way of indicating the vote-power, but some method is better than no method.

Another concern is that higher-rep users (having been around longer) participate in voting less-frequently than new-comers do. After all, we're not trying to gain badges, build rep, etc. As a result, it's more likely that a low-rep users will cast a vote than it is a high-rep user. Does that follow logically?

---

Rebuttal

Reputation (From the SO-FAQ)

Reputation is a (very) rough measurement of how much the Stack Overflow community trusts you. Reputation is never given, it is earned by convincing other Stack Overflow users that you know what you're talking about. (emphasis added)

1. "Rep has nothing to do with credibility, competence or knowledge!" - John Smithers
   

   I agree. But it does have to do with "how much the SO community trusts you." Granting that, some votes are more trustworthy than others. Because they come from users who have convinced "other SO users that they know what they're talking about."

2. "The purpose of anonymous voting is that all votes are valued equally, whether they come from Jon Skeet or from some new person." - devinb

   That is clearly not the case though. If SO operates by labeling some users more "trusted" than others (which it does), then that would mean some votes are trusted more than others. I would trust Jon Skeet to suggest a good programming book than I would two freshmen. It's irrational to suggest Jon Skeet will offer the best recommendation but that isn't what is being argued here. Only that Jon Skeet is the safer choice given his reputation.

3. "Just because a person has reputation does not mean they have more technical knowledge." - yshuditelu

   I agree, but it does mean they are the safer source than somebody with no rep. That is, according to the SO-system.

Answer 1

This is a terrible idea. The purpose of anonymous voting is that all votes are valued equally, whether they come from Jon Skeet or from some new person.

A few reasons for this.

1. We have no way of knowing if they have expertise in this particular area. Marc Gravell has no knowledge (as far as I know) of anything to do with Lisp. But that won't stop him from voting on a Lisp question. If he, and the leading SO Lisp expert put votes on different answers, this counter would claim that the MG voted answer is 'more correct'.

   • People are attracted to high numbers. It won't matter if one answer has ten upvotes, and a total score of 2000, they'll choose the one with two upvotes and a total score of 17000. The ten people who voted will be overruled by the two people with a higher reputation. This spits in the face of the purpose of the site, which was designed to have the best answers bubble to the top based on community acceptance not elitist acceptance.

   • Reputation is NOT expertise! Reputation is at best a measure of some part quality answers, and mostly time spent on SO. I've been aroudn SO for a while, so I have a decent reputation. I gained that reputation by waiting a long time to find simplish questions that are within my limited knowledge, and then answering them in a stellar way. It doesn't mean I'm a technical expert. My reputation means I've been here for a while, not that I'm smart. Therefore, your new methodology would bias answers based on whether or not the old people like them.

   • Another barrier to new people. New people deal with the following:

     1. Fewer powers. They cannot do certain things
     2. There are people who have admitted they don't answer questions by new people.
     3. They will likely have a low acceptance rate, because they don't understand the system. So people will not answer their questions for that reason.
     4. They are trying to integrate themselves into a community with no consensus and a lot of abuse of newbie problems.
     5. We're telling them that their votes don't matter either.

The voting system is the one part that is even and fair to everyone. Let's keep it that way.

Answer 2

Just because a person has reputation does not mean they have more technical knowledge. Ideally it means they have provided more to the system of SO/SU/SF and the system reciprocates by allowing them to do more, but it definitely does not say they know more and therefore should be trusted more on a technical issue. They are trusted more on a issues of how to maintain the system (voting to close, editing, etc). Furthermore, someone could spend all their time answering every windows question that comes along, gains lots of rep, and then go vote lots on linux questions, where they certainly have not proven any knowledge.

Answer 3

Isn't this simply SO-flavored PageRank?

1. Copy any Google product
2. Call Paul Graham, or Joel Spolsky if Paul is on tour
3. Profit!

Seriously, I think it's a good idea to weight votes, perhaps even with different weighting schemes for up and down votes.

Think about this: how much better online product review would be if there's some kind of trustworthiness-based weighting? (Yes that's a billion times more difficult to implement.)

But reputation alone is NOT a good weight, as most answers correctly argued against.

However, some other measures can be part of a good weight, for example up votes on a tag, which is a very good (and actually the only) indicator of domain expertise.

I would suggest that SO shows two vote counts, one raw "popular vote", one weighted "electoral college".

There could even be a 3rd vote count where you can use your own weighting algorithm.

Answer 4

Reputation clearly is the only measure we have to trust an answer or a person - and it is thoroughly flawed: you can earn tones of reputation by just asking masses of low value questions.

This said, does it really help to know whether the people upvoting an answer have a lot of reputation or not? The next I would like to know is: how did they earn it? And that would clearly take it too far.

If the answer comes from a high rep user, I can search his rep building pattern on the user page and even see whether he repeatedly answered questions in the field my question is, looking at his top tags.

Yet, if an answer has upvotes, it means that this many people found it to be a valuable answer (or at least this many more people found it to be good as there were people that found it awful). This should be enough.

Answer 5

I would say this is the worst idea ever, however there is a few other examples that are far worse. It is the best way I can decribe how strongly I disagree with this.

It doesn't matter what the repuration is of the people that upvoted. If the answer was credible enough to recieve more then one upvote, and the answer works for you, then it is a valid answer. Just because someone has low reputation today does not mean they won't be in the top 2000+ users by the end of the next week, since they may have just joined, yet have the knowledge to vote on a correct answer.

If you asked a subjetive question, this is irrelevant, since this by nature will not have a right or wrong answer.

Reputation is not neccessarily an indication of experience or knowledge, but rather a guide towards finding the correct answer based on individual experience. I have a decent amount of reputation on SU, but have a lot of answers I haven't recieved a single vote for because I was outright wrong.

Answer 6

As has been mentioned before... high rep != experienced in subject material (programming or otherwise)

And vice versa... experienced != high rep.

Answer 7

Reputation functions to generate and maintain interest in the site. Aside from showing how interested a person is in the site, it's pretty much useless as a basis upon which to form opinions.

Answer 8

Ok, here is the simple answer:

No. Voting is meant to be simple and anonymous, not complicated and predictive.