11

There are two observations on Stackoverflow:

  • Some topics naturally receive more activity than others - for example a question about PHP/Javascript would get much more activity (in terms of views, votes, answers etc) than a question on Scala/Erlang.

  • Esp within the high activity areas, voting on some answers becomes a popularity contest rather than a vote on the quality of the answer. For example, a question on which Javascript library can be used to perform a certain task becomes a popularity vote on JQuery vs Dojo etc, so answers supporting any get more up-votes depending on the respective popularity rather than answer quality.

Here is a fundamental question:

If a Scala/Erlang expert has provided 100 good quality answers, do they roughly accumulate the same points as a PHP/Javascript expert who has provided the same number of good quality answers?

In the current system the answer I think is no. This kind of skewed reward system might result over the long run in a winner-takes-all situation, where few popular areas would get lots of users and interest while most unpopular areas get very few users.

Here is a proposed solution:

A weighted score can be used, and the factors that affect this score would be:

  • the relative total activity in the tag/tags area of the question.
  • the relative activity for the given question.

Based on these two factors a relative scoring factor can be generated that would be used to weight the the points generated for activity in the given question. This weighted points score is then used to accumulate the total weighted score per user which better reflects the quality of the user activities. So basically, the total points figure is like your net worth, and the weighted score is like your worth value relative to your demographic group etc.

Not all details are sorted out here of course - like whether the popularity measure is a snapshot or changing over time.

Such weighted score can then be displayed next to the user name as a mark of credentials, and can even be used in the permissions system.

Update:

Some clarifications:

  • The popularity contest above refers to the popularity of the languages/platforms/apps/libs etc not the popularity of the poster. There is no issue w.r.t poster popularity.

  • The proposed weighting system does not give higher weight to less popular topics, rather it tries to achieve fairness by giving equal relative weight. For example if Javascript or an aspect within it is decided to be 5 folds more popular than Scala, then 5 up-votes in Javascript would be equivalent to one up-vote for Scala. Or maybe a factor of that like 60%, or an effect that comes into play after certain threshold - say after 4 up-votes. All these are details on how this balance can be achieved.

    Another devilish detail is how to decide the popularity factor - I proposed factors based on the popularity of the tags on the question and the activity in the question. Others might suggest other factors or a different take on these.

Improve this question
2
  • 1
    I really like this, but I think it's too big a paradigm shift (and too much work) to be considered by SE right now.
    – John
    Nov 21, 2011 at 5:49
  • 2
    I fully agree with the concept
    – Andrey Rubshtein
    Jan 9, 2012 at 21:29

1 Answer 1

Reset to default
3

I disagree that it's a popularity contest, I don't see where you got that from. I rarely ever see answers getting upvoted just because someone has more reputation. In fact, I quite often see people point out things that higher-rep users do wrong. I think it's more the opposite of a popularity contest. "They have a lot of reputation, gotta find everything they do wrong and point it out."

If a Scala/Erlang expert has provided 100 good quality answers, do they roughly accumulate the same points as a PHP/Javascript expert who has provided the same number of good quality answers?

Well you're being a bit hypocritical in a way. Just because an answer is posted in a less popular topic doesn't mean it's any better than an answer posted in a more popular topic. So why should it get more weight? Honestly, if a certain language or feature, whatever, is a lot less popular around the world, it doesn't make much sense to credit users more just for using it. If you tags are appropriate, all the users who find it relevant to them will look at it. That's the point, to get people who can help you to look at your question. The reputation system may be a game for a lot of people, trying to get as high as they can, but really that's not the focus of the site. Not to say I don't love getting reputation, but I spend a lot more time commenting and helping others improve their questions/answers, and more recently, a heck of a lot of time in the review panel.

This particular line bothers me: the relative activity for the given question. The activity of a question completely depends on the wording in the title and the tags that it is given, and maybe a mix of upvotes that it's received so far. If it's poorly worded and tagged, a lot less people are going to look at it.

This would be like giving students who take Japanese in high school extra credit because the subject is less popular than say Spanish or German, even though all the students put in an equal amount of effort and time into their work.

Update:

For example if Javascript or an aspect within it is decided to be 5 folds more popular than Scala, then 5 up-votes in Javascript would be equivalent to one up-vote for Scala.

That still seems entirely unfair. A vote is a vote is a vote. We're just giving specific users more attention for their equally helpful answer, merely because they're posting in a less popular topic. A post in JavaScript will receive more upvotes because it's more useful to the majority of people than a post in Scala. You're trying to say that Scala is equivalent to JavaScript, and other languages, when it's not.

The base of the problem is you're shifting the "fairness" from one side to the other, not really creating an equal medium. Reputation is a rough measurement of how much the community trusts you. The community, not the peers within the specific tags you work with. If the greater majority of the community works with JavaScript, then it would stand true that a greater majority of the community trusts a user posting in JavaScript than a user posting in Scala. Note as well how it says "rough measurement."

Improve this answer
5
  • I disagree with the last part; in school, there is only one source of credit (the teacher). On SE, there is a variable amount of people who can give credit.
    – S.L. Barth
    Nov 20, 2011 at 10:18
  • @S.L. Barth: Not really, at least at my school they had tutor rooms available where students could help in the subject areas they were doing well in, so you'd have the student base for each language available to help, which would be much smaller in Japanese than in Spanish. Even without tutor rooms, you can still ask other students.
    – animuson StaffMod
    Nov 20, 2011 at 17:08
  • That's about the available sources of help, not about the available sources of credit.
    – S.L. Barth
    Nov 20, 2011 at 17:14
  • 2
    @S.L. Barth: Just because they're a student doesn't mean they're not a source of credit. If they have good reputation for providing excellent help, more students will go to them. But of course, a student in Japanese will have a lot less reputation than a student in Spanish because there are so many fewer students that need help.
    – animuson StaffMod
    Nov 20, 2011 at 17:18
  • One could argue that the world needs more Westerners speaking Japanese than we need one more Spanish (or German, or other semi-important indoeuropean language) fluent person. On the other hand, not so good to reward obscurity for it's own sake.
    – Prof. Falken
    Oct 22, 2012 at 15:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .