What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 133 Stack Exchange communities.

In real life, if 10 unskilled people give their opinion on a specialist topic, like medicine or law, their 10 "votes" count nothing. Infinite amounts of it still count almost nothing. A single professor weighing in is worth many unqualified opinions.

In chess, when somebody reviews a game they have an ELO rating but their ELO rating says something about their skill, "Reputation" does not, here on StackExchange.

Even worse, if 10 "nobodies" start voting up their top nobody discussing trivial matters among themselves, this selected "nobody "becomes "somebody", that does not make sense at all.

share|improve this question
8  
Does this refer to a particular question of yours? How can you be sure you've been downvoted by numerous idiots and not by those who are actually "qualified" as you say? How can you be sure that it is nonsense? Or are you merely trying to address the "weight" of a downvote based on reputation? If that is the case, make sure you clearly state it. Now it reads as a rant. –  Bart Jun 11 '12 at 14:26
    
This does not relate to anything I said, or any judgment cast on me or anything I do. This is about knowing which votes count, in general. I tend to participate to the forum in which I have most to learn from so I don't mind "downvotes" at all. I visit sites like this to learn. My question is about qualifying opinions on a topic by actual knowledge and not by "masses of numbers". e.g. if many people like a new "craze" (e.g. Britney Spears, Magic The Gathering card games, etc) it does not mean a lot "per se". Think medicine and comparing the opinion of 10 GPs to 1 subject matter specialist –  Robottinosino Jun 11 '12 at 14:31
2  
Trying to reduce that to a question, is it accurate to say that you're asking: "Should we perhaps consider a user's reputation as a weighing factor for his downvote?". If so, if you phrase it like that you would not have had your question closed I guess (just my opinion though, no guarantees), and the pile-on downvotes would have been less. Try to be constructive, even if you're upset about something. –  Bart Jun 11 '12 at 14:34
    
Hmm, the comments and answer explain why we downvoted, no need to reiterate the same points. –  Manishearth Jun 11 '12 at 14:35
    
Let me rephrase the question as recommended by Bart.. I'll open a new one. BTW, I meant relative weighing of opinions both in the positive sense and in the negative.. –  Robottinosino Jun 11 '12 at 14:36
5  
@Robottinosino: Edit this one instead, it will be reopened if we feel that it has been edited satisfactorily. Don't keep asking new ones. –  Manishearth Jun 11 '12 at 14:38
1  
@amanaP Why the rollback? –  Bart Jun 11 '12 at 14:48
    
@Bart we don't need the snidyness, that's why this was getting downvoted before.. Comments should be in the comments, not in the OP –  amanaP lanaC A nalP A naM A Jun 11 '12 at 14:49
    
@amanaPlanaCAnalPAnaMA Hmm, the edit IMO got closer to the question the OP intended to ask. The original question below it should have been removed instead. Still not a great question (far from it), but closer to the point. –  Bart Jun 11 '12 at 14:50
    
I did. Is it worthy of being re-opened now? I accept suggestions, of course, provided my intent has been clarified now... –  Robottinosino Jun 11 '12 at 14:51
    
@Robo: It has already been reopened. –  Nicol Bolas Jun 11 '12 at 14:52
2  
Note that downvoting on meta is not the same as on a regular exchange--a downvote just means the voter doesn't think it's a good idea. It may not need/warrant specifics. –  Dave Newton Jun 11 '12 at 15:02
    
Equal voting is one of the (many) reasons I left everywhere else. -1 –  U2744 SNOWFLAKE Jun 11 '12 at 15:06
3  
Sorry, here's the question: "Is it a problem that everybody's vote is worth the same?" And my answer is "no". So that's what a downvote means on a discussion. –  U2744 SNOWFLAKE Jun 11 '12 at 15:08
3  
@Robottinosino If someone downvotes a suggestion on meta, it usually just means "no". That could mean "bad idea", "stupid idea", "impractical idea", etc. If the voter feels it warrants explanation, they'll add an answer or a comment, otherwise they won't :) –  Dave Newton Jun 11 '12 at 15:15

9 Answers 9

No, not everybody's opinion is worth the same.

Without a mechanism to accurately reflect that, however, The Masses is the best we have.

This accuracy would be tricky, and a slippery slope. How could it be implemented?

Give people with a higher rep "extra credit" and nudge them up? What if they're wrong?

What if their answer is outside their domain of expertise–start basing on a user's tag stats, potentially across tags? Grade on a curve? How is the curve defined? Fuzzy likelihood of correctness?

Yikes.

share|improve this answer
    
Why "barring"? What has been researched already for us to come to the conclusion "barring that"? If feels like there might be a possibility to do better. I am not surprised that there is at least somebody thinking along these lines: (quote) "No, not everybody's opinion is worth the same." –  Robottinosino Jun 11 '12 at 14:48
    
@Robo: "If feels like there might be a possibility to do better." OK, so what do you suggest? You say that you don't think everyone's vote should count equally and that this creates problems (though you don't actually cite examples). So... what do you want to do about it? –  Nicol Bolas Jun 11 '12 at 14:54
5  
@Robottinosino I think you misunderstand the use of the word "barring": it means "given that we don't have a mechanism to accurately reflect that, The Masses is the best we have." It's obvious on the face of it that some opinions aren't worth the bits they're stored with, but determining who, and of those who, which opinions matter, is non-trivial. –  Dave Newton Jun 11 '12 at 14:57
    
I did misunderstand it, you're right. I am not a native speaker. So you are (yet another) one who agrees that some opinions aren't "golden". Voilà. I agree it's non-trivial to fix. Lemme ask you then: if you thought about this for a second, would you have any ideas of your own? Hearing these ideas is why I asked the question... –  Robottinosino Jun 11 '12 at 15:09
1  
@Robottinosino I have all sorts of ideas ;) I'm just not convinced they'd be worth the effort to investigate. The simplest, a rep and/or tag-rep approach, would likely be the most-practical. –  Dave Newton Jun 11 '12 at 15:14
    
Would love to hear about that, how about a separate answer with some ideas... they'll be better than the stupid one I had about "validating diplomas", that was wacky and really stupid... –  Robottinosino Jun 11 '12 at 15:36

While this is a problem in principle, it is not one in practice.

The basic reason is that the site was seeded with a group of relatively serious, relatively responsible, relatively professional users (and me, of course), so there has always been a large and active contingent who would subject self-congratulatory circles to a significant set of outside opinions.

share|improve this answer
4  
It is sometimes a problem in practice. Information Security has this problem more than most sites, where questions that are widely publicized outside the site attract a lot of votes, and bad but popular advice tends to flow to the top, drowning good advice. –  Gilles Jun 11 '12 at 14:35
    
Hmmm...yes. My argument relies on having a large enough resident and expert user base to overwhelm such effects. –  dmckee Jun 11 '12 at 14:37
    
I also feel that this is sometimes a problem. e.g. users says "I don't know why that other answer is up in votes given that.." I would even go as far as saying "each one who casts a vote should justify why!" (the comment would be only visible by "clicking on the vote" or something) –  Robottinosino Jun 11 '12 at 14:45
3  
@Robottinosino The "justification for downvotes" idea has been discussed here many times and has been consistently downvoted equally as much because of its impracticality and the anonymous nature of downvotes. You might want to have a look through this site before suggesting it. ;) –  Bart Jun 11 '12 at 14:56
    
Scrap the idea of "justifying downvotes" then! You may be right and it might not help at all here... in fact the issue I am framing is: unqualified people voting count just as much as real-life experts. That is... "counterintuitive". –  Robottinosino Jun 11 '12 at 14:59
    
We have this problem a lot with SQL/Database questions as well. A lot of people know a little about the topic, and I and a few others have noticed bad answers getting pushed to the top recently. –  JNK Jun 11 '12 at 15:23
    
Here we go, somebody else calling the current system "a problem". The more specific/specialist the domain is... (SQL/security) the more this happens. It's not an absurd issue to raise, it seems. And I do not say so myself, some other opinions come in with that view... Why does nobody recommend a fix? :) –  Robottinosino Jun 11 '12 at 15:25
    
@Robottinosino Well to me this sort of issue is a large part of why there are subsites popping up. We have Database Administrators for database questions and IT Security as well. Consolidating topical experts will mitigate it somewhat - if you get 10 upvotes on DBA there's a good chance it's a good answer. –  JNK Jun 11 '12 at 15:27
    
Agree that compartmentalisation mitigates. A very skilled CS generalist may know less about network administrator that an intermediate-level specialist. So this is part of the answer... which other factors could contribute to evaluating an answer apart from "vote/popularity"? –  Robottinosino Jun 11 '12 at 15:33

The theory behind the reputation system is that over time, your reputation will become a rough estimate of:

So this way, lower-rep users are on the same playing field and face no disadvantage when it comes to gaining reputation. If they couldn't participate fairly, they would leave.

share|improve this answer
1  
"How much time you spend participating": does not seem related to skill to me. I can play chess for a lifetime in the park with my friends and still know less than somebody who studied it for 6 months. "How much you know": in my point of view now this is not factored in, just "popularity" is. "How well you express your ideas": agree this is absolutely a factor but I could be expressing well a "not-so-bright" idea... "How much the community trusts you": I don't know enough to comment on this, seems very reasonable.. –  Robottinosino Jun 11 '12 at 14:50
    
@Robottinosino: "I can play chess for a lifetime in the park with my friends and still know less than somebody who studied it for 6 months." Try it sometime. If you do something for a long enough period, you will get better at it. Especially since your friends will also get better at it. –  Nicol Bolas Jun 11 '12 at 14:52
1  
@Robottinosino I can tell you that this is definitely true with me. Over time, by participating, I've learned to ask good questions to get good answers. Feel free to look at the progression of my SO questions. My first questions were, I confess, stupid beginner problems caused by errors in other areas of my code, but if you participate, you learn, and your reputation reflects that. –  gobernador Jun 11 '12 at 14:56
    
I used to play chess at national level and no, parks are full of mediocre people mostly (with the exception of those in which people bet and are retired pros) and chess clubs with tuition make you improve very quickly. I (respectfully) disagree very much with this: proper training is the key, not just "practice". Deliberate intent to improve is different from "practice", it has supporting material, exercises, etc. –  Robottinosino Jun 11 '12 at 14:57
    
@Robottinosino @Dave You are both correct. If you make no effort to improve, you will continue to be downvoted. Perhaps I chose the wrong words. I do not mean that simply by earning the Fanatic badge you have by definition become a better programmer. This is a community full of experts and not-quite-experts. We work together to learn together. When I said participation, I meant an active application of the answers you receive and, as you say, deliberate intent to improve. –  gobernador Jun 11 '12 at 15:03
    
So the question remains: relative value of one's opinion. I have discovered since asking the question that a non-zero amount of people feel like there are opinions which are "sub-optimal" but still count and sometimes offset outcomes... and views of people reading those outcomes... and... –  Robottinosino Jun 11 '12 at 15:11
    
as dmckee said, it's an idea in theory, but I don't see this in practice. –  gobernador Jun 11 '12 at 15:40

Is it a problem that everybody's vote is worth the same?

It can be a problem. But it seems to work out well enough.

Perfect is often the enemy of good. We have a system that functions. Sometimes, things fall through the cracks. Sometimes, bad answers win out over good ones. But overall, it works well enough.

Any solution you propose must be at least as good as what we have now.

share|improve this answer
    
I agree that this site is exceptionally successful and I admire the voice you give us by keeping it up. No question about that. "Perfect is the enemy of good" is a comment that can be applied anywhere no matter what, I agree with it in general. What I'm saying is: you do have good now, so you are past "nothing" already, how about researching "better"? It should be clear I don't have a proposal, I am possibly unable to craft one on my one otherwise I would have written an article on it, not a "question".I'm trying to elicit a discussion researching a better solution, highlighting what to fix. –  Robottinosino Jun 11 '12 at 15:03
    
@Robo: In order to do actual research on the subject, we would have to design and build such a site, then see what happens. That's not going to happen within the confines of this thread. Otherwise, it's just people speculating. –  Nicol Bolas Jun 11 '12 at 15:06
    
Well, you may not develop such a thing but people reasoning about this may! The fact that this kind of development would not commence after reading this thread does not make all of this "just people speculating"! Every invention starts with "people speculating". Arguably executing and implementation are just minions of good ideas, details of making concrete a new concept. Concepts are worthwhile in their own right, in my opinion. Somebody could develop an algorithm and open-source it for you just to thank you for creating SE! Don't stifle investigation just'cause you are not gonna implement it –  Robottinosino Jun 11 '12 at 15:20
    
@Robottinosino: How am I stifling anything? You asked if it was a problem. I said that it doesn't appear to be, or at least, it isn't a major problem. –  Nicol Bolas Jun 11 '12 at 15:24
    
I got that impression. Fine if that's not what you meant. It just seemed a "rush" to say "That's not going to happen within the confines of this thread." I would think an idea worthy of development could in fact come from this thread... –  Robottinosino Jun 11 '12 at 15:28
In real life, if 10 unskilled people give their opinion on a specialist topic, 
 like medicine or law, their 10 "votes" count nothing. Infinite amounts of it 
 still count almost nothing. A  single professor weighing in is worth 
 many unqualified opinions.

Very nice subject to talk about.

Who is building the house, the ten people that they live inside by voting how they think they want it built, or one person that is the architect?

However here are some main differences. All persons that ask and answer are on the same playing field (eg programmers) so they can have an opinion. And it is good to say what you have to say with arguments about the problem itself, and not with arguments like "I have more reputation,- so I am right and you are wrong"

The vote is like "check this answer - may help you" and the accept is something like "Its work with me"

The total votes is something like how much help you have given, how they appreciate it, and the reputation is a result: the more you give, the more you recieve.

You can see it in exams, some one makes the problems, some others try to solve them, and get some score towards earning their degree.

And at the end all this seems to work. People ask here for help, and they get it.

So its not a court here where we have subjects that must be judged, but a help place, a community where one helps another.

share|improve this answer

Every user on stack exchange is equal, but some are more equal than others.

share|improve this answer
    
I will work harder, and Jon Skeet is always right. –  gobernador Jun 11 '12 at 15:39

I think in the long run this balances out because there are enough true experts on the site to judge what's a good answer and what isn't. I also don't think it's typical for new, "rookie" users to vote on answers they don't really understand, so I don't think they are biasing the answers too much.

If you pick a question randomly on SO, the top rated answer will be inline with what "experts" would say, so I think this is proof that the system works in light of non-experts voting.

share|improve this answer
    
Well, actually by definition experts are less than non-experts. :) Even if I understand your point is "non-experts vote less", or something based on that. –  Robottinosino Jun 11 '12 at 15:41

Your question is essentially asking whether "the wisdom of the crowds" is a valid concept. There are plenty of arguments against it, but it is still a reasonable measure of whether a question or answer is broadly accepted within some statistical demographic.

Examples

  1. There was a time when it was broadly accepted that the earth was flat. That didn't make it true, but it would certainly have been upvoted on Stack Overflow at the time because it was a widely-held belief.
  2. An answer of sed -r 's/a{2,}/b/' will probably get less upvotes than sed 's/\(a\)\1\+/b/' because not everyone has GNU sed, and so it might be less widely-accepted as a canonical answer.

Ultimately, crowd-sourcing is not a synonym for peer-review, but it's a reasonable proxy for the common case.

share|improve this answer

If there were a correlation between Stack Overflow reputation and technical expertise, then you might have a valid point. However, since there are plenty of skilled people who don't have an SO account, it's unlikely at best that such a correlation would hold (and that's assuming that one could reliably and meaningfully quantify levels of technical expertise).

share|improve this answer
2  
And plenty of unskilled people who do, and plenty of both who are sometimes useful, sometimes not. –  Dave Newton Jun 11 '12 at 14:58
    
Well, there are fields in which you can, say "language skills" and the unified European framework of reference. One could say, probably a bad idea but I am brainstorming here, "those who fax in their CS diploma start out at level X". Disregard the advice, it's the intent that counts: finding a new idea to fix a problem some people here acknowledge but hurry to dismiss as unsolvable. –  Robottinosino Jun 11 '12 at 15:05
    
@Robottinosino - So, you're assuming that everyone who has a CS degree is more technically skilled than everyone who does not? Do I need to dig up counterexamples to point out just how absurd that assertion is? –  Jack Maney Jun 11 '12 at 15:13
1  
That assertion was wildly absurd and absolutely incorrect. Completely agree and chuckling at myself thinking "even if brainstorming is done suspending judgment, you just took it one notch too far man". Sorry. That idea is BS. –  Robottinosino Jun 11 '12 at 15:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .