87

Please disregard. This proposal has been abandoned in favor of a new proposal.

Please disregard. The new proposal has been abandoned in favor of just plain not doing it.

Based on:

Could we put a cap on reputation earned by asking questions?

I am starting to wonder if we should cap reputation gained for questions.

I have historically resisted this, but:

  1. It does seem nonsensical that users can get 10,000+ reputation purely on the basis of asking hundreds of questions.
  2. We already cap reputation for suggested edits -- you can never get more than +1000 total reputation from suggested edits.
  3. It would de-incentivize asking questions for the sake of asking questions which I think is a very good thing.
  4. 99.99% of users never even come close to this limit, so in practice it won't change the experience for users who ask a reasonable, typical number of questions.

I wonder if a limit of max +2000 reputation from question upvotes would make sense?

Not many users would be affected; here's a relevant data.stackexchange query:

https://data.stackexchange.com/stackoverflow/s/1057/users-with-more-than-400-upvotes-on-their-questions

That's about 226 users on Stack Overflow, and almost none everywhere else in the network. However, some of these "ask a zillion question" users would indeed be affected severely. You can roughly calculate how much rep they'd have after this change, by taking the sum of all their answer upvotes and adding 2000.

29
  • 16
    Interesting, your own employee would be the only one affected on Server Fault ;) Commented Mar 6, 2011 at 22:38
  • 17
    Clicking through I notice some of those users have been philanthropic with their rep - awarding many bounties - would that rep be affected? Commented Mar 6, 2011 at 23:23
  • 2
    Hey I'm in that list! With +600 :(
    – Earlz
    Commented Mar 10, 2011 at 0:40
  • 5
    On some sites, we need more askers than answerers, this would hurt those sites. Commented Mar 10, 2011 at 15:14
  • 1
    Questions maybe, answers, absolutely not.
    – Ghost User
    Commented Mar 10, 2011 at 20:02
  • 2
    In some ways questions are more important than answers, because an answer cannot exist without a question. <ducks> Commented Mar 10, 2011 at 20:16
  • 5
    There goes my hope of ever getting 10k on SO :-(
    – Ivo Flipse
    Commented Mar 10, 2011 at 20:19
  • 2
    Are you planning on making this retroactive? (which seems a bit unfair)
    – Andomar
    Commented Mar 10, 2011 at 21:29
  • 1
    @mark for those sites, they're so low volume that it would take forever for them to hit this limit Commented Mar 10, 2011 at 23:50
  • 8
    @jeff atwood - that's a good point, but I think that it's possible it could have chilling effect on people who are asking solid questions. I mean when I read this suggestion, its makes me feel like I'm going to eventually get jipped out of reputation from my high-quality questions. While that may not ever happen, my gut reaction is to become less involved in Stack Overflow, and I doubt I'm the only one who will feels this way. Commented Mar 11, 2011 at 0:11
  • 4
    @Mark completely agree. Many times I try to give back to the community in my own way by asking questions even if I know the answer. Up voting valid answers, and if no one posts my answer, then I also post my answer. This would make method of giving back much less worthwhile.
    – Earlz
    Commented Mar 11, 2011 at 1:10
  • 5
    No, please don't do this. I think there are so many different ways to participate in Stack Overflow that don't include always answering and answering questions. I've been working on a couple of projects since I've been using the site and having the abililty to ask questions without feeling like I'm being punished because I don't hunt for things to answer. Now that would be different. Maybe one day when I have more time I'll give back, but I can't right now. Asking questions is my contribution.
    – TLCo
    Commented Mar 13, 2011 at 1:12
  • 4
    One problem is that the implications of this suggestion are that this proposal is something of an attack on Question-askers (budding experts), suggesting that they are somehow less valuable to the community than answerers (hopefully experts). Essentially this proposal implies that it is ok to hurt legitimate question askers in the pursuit of taking out gamers, because question askers are not nearly as important to the sites as answerers. But I don't think that it's has been empirically established that Askers are less-valuable overall to the community than Answerers. Commented Mar 13, 2011 at 17:25
  • 1
    The greatest learning often comes through teaching, but what if there are no great students. It seems like alienating great students would decreases the quality of what the experts had to teach, because they would have to field lower-quality questions. Commented Mar 13, 2011 at 17:32
  • 7
    @Jeff Is this a theoretical or actual problem? Do you have any example of people with "too many questions" misbehaving with their "fraudulently earned" privileges?
    – tiago2014
    Commented Mar 14, 2011 at 12:07

37 Answers 37

94

I wonder if a limit of max +2000 reputation from question upvotes would make sense?

Absolutely not.

As I've said before, asking good questions provides value to the community, not only through posing interesting problems, but also because tremendous insight can be gained from the answers -- answers cannot exist without questions. If a given question has no value whatsoever, then it simply should not exist. Maybe moderators need to start culling more questions.

In any event, why is reaching 10k from question upvotes alone a problem?

I've yet to see any evidence there's something wrong with this. Please cite cases of these users abusing the system, or otherwise behaving in a way that is unacceptible to the community.

Is it possible these users can actually add value to the community having access to the 10k tools?

If we're concerned about certain users reaching 10k in a certain way, then it's time to change the relationship between reputation and privileges. For example, to reach 10k privileges, perhaps we should require a balanced mixture of participation on the site, i.e., 10k+ rep and 80 edits and 500 total votes cast, or something to that effect. The same approach could be applied to the lower tiers as well, although less aggressively.

I see no reason to artificially cap the reward for providing value to the community.

In fact, I think it would be anti-community to do that.

We've already gone to the length of reducing the weight of question upvotes, but where will this end? Eventually these users will meet the criteria for the 10k tools. Reputation is cumulative: it's inevitable. At that point, are we going to change the rules again to keep them out?

8
  • 6
    @Jon the problem is a few specific users that have learned the game and are, in my opinion, abusing the community. There are huge lists of them, but here is an example: stackoverflow.com/users/68183/… ... these users have a bunch of rights they should not, they have never given back.
    – waffles
    Commented Mar 7, 2011 at 0:00
  • 19
    @waffles: If the whole problem is not giving back to the community, then require more than just rep to attain a privilege. Or if, in your opinion, a user is asking garbage questions which have no value, DELETE THE QUESTIONS and do a rep recalc.
    – Jon Seigel
    Commented Mar 7, 2011 at 0:04
  • 1
    @jon please read blog.stackoverflow.com/2010/03/… from a year ago, particularly the bottom 3 or 4 paras above the strikeout Commented Mar 7, 2011 at 3:05
  • 7
    @Jeff: I don't entirely disagree with that reasoning, but how extreme will it get? We've already cut the amount of reputation in half for questions. As I said in my last paragraph, are we going to keep changing the rules every time these users get close to 10k?
    – Jon Seigel
    Commented Mar 10, 2011 at 2:21
  • 6
    @jon it's typical to rebalance over time as the sites grow, to properly reflect the right incentives -- that is, we want the things that make the community and its users great to be incentivized. Asking thousands of questions and contributing no answers does not make a community great in 99% of cases. Commented Mar 10, 2011 at 3:30
  • 6
    @Jon - Deletion of content should be a last resort, not something to be used on borderline questions by users we don't approve of. The problem isn't from users that ask truly terrible questions (they are already stopped by the bad questions filter), it's those who spam the system with mediocre questions and rack up individual upvotes here and there. This is like those mobile developers who slap together 500 "applications" and spam stores with them because they can sucker in a few consumers here and there. It works for them, but it harms the community. Commented Mar 10, 2011 at 15:35
  • 1
    @jon - I agree, I think that this is a case of over-engineering, at least, at this point in SEN's history. When you add complexity to anything without much pay-off, you have to be very careful you are making the correct decision. Commented Mar 10, 2011 at 16:54
  • @Jon Seigel: +1 I agree. Askers should be welcome. Bad questions should be improved or closed. Duplicates would be closed, but works as alternative entry point. Compulsive askers with low accept rate will loose community attention. Privilege abuse will conflict with community and rise moderator attention. It's not consistent with the new policy of letting everyone to edit pending for approval. Special cases needs special measures: moderators will never be replaced by automation. How many questions in average one should make to reach... say 3000 reputation points?
    – user150068
    Commented Apr 7, 2011 at 4:41
67

I am for this system provided there is an adjustment:

After your first 500 question reps, you can only gain more question reps when your answer rep is higher than your question rep.

  1. Cap question rep at 500
  2. Allow for another 500 rep more from questions per 2000 rep gained.

Question rep means, all rep generated by question upvotes, downvotes and accepts.
Answer rep means, all rep generated by answer accepts, upvotes and downvotes.


Overall this change will affect we less than 1000 users on Stack Overflow. There are a handful of exceptional question askers that are affected but the vast majority do not fit into this mold. As long as you answer a total of half the the questions you ask you are unlikely to be affected.


This system would cause a 0 reputation change for:

It would eradicate reputation and make it close to 500 for:


Overall I would be very happy with this change, as it encourages our high rep users to keep on asking the great questions they always ask, provided they give back.

I dislike the idea of punishing users, even slightly, for what we all believe is great behavior.

47
  • 11
    Well, you're the dev, but this sounds even more complicated to implement or even understand than what Jeff's been complaining about above. Maybe easier to set the threshold at 500 * average question upvotes? That way people who ask few, great questions get all the rep they want, but people who ask mediocre questions in volume will get practically nothing.
    – Aarobot
    Commented Mar 6, 2011 at 23:49
  • 6
    One suggestion is to exclude "Great Question" badges from the cap as these represent genuinely well-asked questions (as recognized by the community).
    – JoseK
    Commented Mar 7, 2011 at 5:47
  • 8
    @Jose, I am not sure about that ... stackoverflow.com/questions/2193953/flash-cs4-refuses-to-let-go
    – waffles
    Commented Mar 7, 2011 at 5:56
  • 7
    @waffles: Moreover, I think that the 500/2000 ratio is rather tough. We have a much valued member in the tex.sx community who according to the current data dump has has 36.9% of his 4012 rep out of questions, so the change you propose would affect him quite a bit. I wouldn't like that at all since he asks many nice questions and is active in many positive ways on tex.sx (e.g. guiding new users). Commented Mar 7, 2011 at 15:56
  • 7
    IMHO, a ratio of 500/2K is a little low. Perhaps 1K/2K?
    – John
    Commented Mar 7, 2011 at 17:08
  • 28
    Maybe this makes sense on Stack Overflow, Sever Fault and Super User. I don't use any of them, so I can't speak. However, on any other site, this is an horrible idea. On a nascent site, users who ask great questions are extremely valuable. There should be an incentive for asking great questions, even if you don't give back. Jeff's proposal is just micro-optimization. Yours is damaging. I strongly oppose it.
    – Borror0
    Commented Mar 10, 2011 at 6:14
  • 9
    Cool. One "Stack Overflow Valued Associate" asks a question about community rules and accepts the answer of another "Stack Overflow Valued Associate". Is it your way to say This is not a democracy?
    – tiago2014
    Commented Mar 10, 2011 at 20:33
  • 3
    @Borror, as far as I can see this algorithm change will only affect a few users on Stack Overflow, the SEs are pretty much all immune to this change
    – waffles
    Commented Mar 10, 2011 at 21:03
  • 9
    @Borror, no I do not think you are a bad user, not at all. However, there is a question of motivation, if your only motivation is to get awesome answers, why would you stop asking questions? Why do you think you should have the rights to edit other answers and close questions if you do not think you have the expertise to provide any answers? I think the ability to answer stuff needs to be codified into the rep system, if your rep is 10,000 we should know you are able to provide awesome answers.
    – waffles
    Commented Mar 10, 2011 at 23:48
  • 5
    @bor I view this as incentive for you to provide some answers as well, which is healthy in a Q&A community. It's not a Q-only community. Commented Mar 10, 2011 at 23:52
  • 3
    @waffles: I think I deserve the right to close questions because I know whether a question is on or off-topic. Isn't the quality of my questions and my activity on meta a proof of that? I think I should be able to edit questions because I think I can make valid improvements. Again, the quality of my questions and my many approved edits should be good indicators of that, no?
    – Borror0
    Commented Mar 10, 2011 at 23:55
  • 5
    @borr so just provide some answers on scifi: problem solved. You will be "limited" to 601 rep on scifi under the new regime, which I don't feel is actually that limiting. As I said, these are Q&A sites, not Q-only sites. Your other profiles demonstrate this, so scifi is the outlier. Commented Mar 11, 2011 at 0:49
  • 3
    @Jeff: You have yet to explain why good questions are not worthwhile contributions. Andrew perhaps best explains my position: good questions are the sand most likely to create pearls. Good contributions should allow you to unlock the proper privileges.
    – Borror0
    Commented Mar 11, 2011 at 0:55
  • 8
    Waffles, I'm seriously worried about the false positives this would catch (e.g., Drake would lose 2k rep from this change). How about turning the cap around? "You can't gain more than 2000 rep from questions for every 500 rep from answers."
    – badp
    Commented Mar 11, 2011 at 18:05
  • 6
    I'm very very strongly against this. Good questions attract many search hits, and have much needed quick answers for those that need them. Asking good questions is almost as important as getting good answers. Commented Mar 15, 2011 at 12:28
48

This is another case of Stack Overflowism; since something is broken for the 0.0001% of the population on Stack Overflow, we must bend the rules everywhere and make the reputation system "even more complex" (complaint Jeff threw in my general direction in reply of an unrelated suggestion, see chat for that).

10k rep from just asking means 2,000 question upvotes. Two thousand of them. Okay, let's say 1.5k and a few hundred accepts. It's still outstanding community participation, for the relatively measly powers having 10k reputation on Stack Overflow unlocks (= a few lists, but I take it those users don't close anyway, and inline retagging, but I take it those "problem users" aren't editing anyway).

I don't think any action is required, or necessary.


If action is required, I don't like the plan of placing a hard cap of 2,000 reputation, or the plan of capping at 500 question rep per 2k rep, for they will also affect users who are not abusing the community. Even under Waffles' more permissive plan this means your question reputation may not be more than 25% of your total rep, or in other words, you need 2 answer upvotes per question upvote to not be capped.

Pardon my horrible, horrible SQL:

  • Users with more than 400 question upvotes: SO vs Gaming (bonus: SF)
  • Users with more than 100 votes and more than one question upvote per two answer upvotes: SO vs Gaming

If we really do need to address abusive users with an ad hoc rule, why not turn it around? Let's say you cannot gain more than 2,000 question reputation per 500 non-question reputation, or in other words, you need more than 8 question upvotes per answer upvote to trigger the cap.

  • Users with more than 400 votes and more than one question upvote per eight answer upvotes: SO (no other site is affected that I can see).

I think this query matches actual problem users more accurately than the other does. It also stops users at 2,101 reputation instead of 601; on many sites, that only means gaining edit privileges (and chat gallery creation privileges). I see no issue with this.

7
  • 7
    This is a good point (although it's less Stack Overflowism than normal development in any kind of community. Look for example at tax regulations 500 years ago, and tax regulations today.) However, the problem of users gaining rep through mediocre questions is nevertheless real and needs action IMO. I could live with removing reputation from question upvotes entirely, too - that would be a much simpler alternative
    – Pekka
    Commented Mar 7, 2011 at 11:33
  • you should really read blog.stackoverflow.com/2010/03/… for context Commented Mar 7, 2011 at 19:55
  • Er...2000 upvotes or accepting 5000 answers. Or a linear combination of the above. Not that 2000 isn't a whole heck of a lot'a votes or anything. Commented Mar 9, 2011 at 6:27
  • 2
    The problem is that is misrepresents a person's actual reputation. Somebody who floods the site with questions like "Is VARCHAR like totally 1990s?" can seem god-like due to their massive rep score.
    – Gabe
    Commented Mar 14, 2011 at 6:39
  • @Gabe 9k is hardly god-like on SO...
    – badp
    Commented Mar 14, 2011 at 8:25
  • @Gabe: is that even an on-topic question?
    – SamB
    Commented May 1, 2011 at 4:05
  • @SamB: I hope not, though it probably was when it was asked. @badp: That user now has 15k rep, just a one privilege away from being a "trusted user" (as god-like as mortals can become).
    – Gabe
    Commented May 1, 2011 at 4:12
26

As I offered in my original question, I think we should scale the cap with the user's answer-rep. For instance, a user's cap for question-rep could be 200 points or 20% of their total rep from answers (whichever is larger).

The reasoning behind this is that, with a fixed cap, every high-rep user would eventually lose some of the incentive to ask questions. Even users trying to game the system can sometimes ask perfectly good questions, and if they leave, those questions would be lost. Also, obtaining an answer is not always its own reward.

If we scale the cap with answer-rep, on the other hand, we now give them an incentive to start answering questions. But what if the gamers decide to start posting lots of constructive answers to up their answer-rep? M.F.A.

6
  • the main problem is that this makes the reputation calculation hideously complex, even more so than it already is -- massive time-based dependencies for each question vote that have to be calculated at that point in time, relative to other users. Whereas a simple cap is .. simple. Commented Mar 6, 2011 at 22:54
  • 3
    @Jeff: You don't have to scale it to answer reputation, you could just scale it to answer upvotes. That way you only need to do one extra COUNT, at the very beginning of the rep recalc.
    – Aarobot
    Commented Mar 6, 2011 at 23:42
  • @Jeff: nonono, you wouldn't need any complex calculation, as long as the answer-rep was cached (in the same way that total-rep is). When someone upvotes a question, just check if askerAnswerRep/askerTotalRep > 0.80 - if it's not, don't give the asker any points. This would be an even easier calculation than waffles' answer above, and IMHO is more fair (and its effects take place more immediately, rather than requiring 2000 rep first). (cont.)
    – BlueRaja
    Commented Mar 7, 2011 at 17:22
  • (cont.) The only problem I see - which also applies to waffles' answer - is that if user-with-many-questions later becomes user-with-many-answers, he wouldn't get the retroactive-points for his questions until the next recalc. However, considering that this only applies to a minority of a minority of users, it doesn't seem like a big issue.
    – BlueRaja
    Commented Mar 7, 2011 at 17:23
  • @Jeff The implementation details are SO's developers problems. Do not strive for the easiest to implement.
    – tiago2014
    Commented Mar 10, 2011 at 20:18
  • 1
    @tiagoinu: Development/running speed should be a consideration, especially for a high-volume site like this... but, my suggestion would not be hard to develop, and would not negatively impact the site.
    – BlueRaja
    Commented Mar 10, 2011 at 20:36
23
+250

Let me get this straight — users ask questions, of a quality level sufficient to garner upvotes from other users. We have a problem with this because __?

  • If it's because they're earning privileges, why not target the privileges directly? With all the new sites opening up with much smaller communities, that's a system that's due for an overhaul anyway.
  • If it's because they "outrank" a few other users who spend more time answering questions, perhaps the thing to do is help those answerers "be more awesome" by comparison.
  • If it's because that's all they do on the site, I think we need a clearer reason for why that's a bad thing. If they were low-score questions that would be one thing, but this will only target users with lots of questions that people have voted for.

The only reason I see to do this is when you look at rep in it's role as a gauge for trust. Some people feel like certain users are using question-rep to game the system, and if they're gaming the system in this way we have an inherently untrustworthy user. But again, this goes back to point #1: in light of the all new sites with smaller communities and no high-rep users, it may be time for a general re-examination of how some of these privileges are conferred.


Update:

I think my whole beef here is that just because questions are very low value, especially compared to answers, it doesn't mean that their value is 0 or negative. There must be some net-positive, or the best solution to the problem would be to do away with them entirely and morph Stack Overflow into some kind of wiki+voting blog service. After all, the questions weren't adding anything. Obviously I don't want to do that, and I don't think anyone else does either. However, that is the natural result if you really apply the low-value philosophy for questions consistently.

So sure, people may be asking questions as a game in it's own right, but, as long as the quality level is high enough to earn votes, is this actively harmful? Is Stack Overflow really worse off for having these questions?

I suppose it might be, if you consider these questions to be nothing but "noise" on the site. Each question individually might be okay, but add up thousands of them and suddenly you have a background layer of fluff that makes finding what you really want just that much more difficult. The problem here is that these questions still tend to generate good answers, and so there's little evidence that they really make searching or answering harder. And I don't think there are enough of these that they pull significant attention of answerers away from users with legitimate issues we could be spending time resolving.

In the end, I think when you have a question with a positive score the burden of proof switches back away from the poster to administration to prove that this particular question is noise and has no value, versus just assuming that's the case, or a very least a clearer explanation of how, exactly, these questions from frequent askers are actively hurting either answerers, searchers, or other askers.


Update 2:

I see now another reason why some feel so strongly that this is harmful: it dilutes their efforts at answering. I'll put it in the context of the career site. Two users with extremely similar resumes are both up for the same job via Stack Overflow careers: one with 3000 rep from answers and one with 4000 rep from questions. Who gets the job?

Personally, I think that's kind of silly. If the hiring manager is just looking at the number without digging deeper, are you really sure you wanted to work for this person in the first place? It does, though, illustrate the problem. How does the user who worked hard for 3000 rep feel that there are 4000 rep users out there who don't contribute?

So if we want to make it easier on the manager and other outsiders to distinguish the two individuals, perhaps the solution here is some kind of indicator on a profile that says, "This person has earned more than X percent of their rep from asking questions." The notice would only need to be displayed for users above a certain score when X is above a certain value to be determined based on meaningful examination of the the real data. Like a big asterisk next to a sports record.

I know Jeff is generally against such scarlet letters, but we already have at least two of these: the student flag and the accepted answer percentage. Another option is to just set that flag automatically on the assumption that if you're asking so many questions, you must still be a student rather than a pro.

It's tempting here to think that I may be too divorced from the issue because no one on Stack Overflow who earns rep by only asking questions has even 1/5th of my score, but my ServerFault and SuperUser profiles are much closer to the main stream. Nearly 20% of my ServerFault rep is from questions, and my total question count there is roughly half of my answer count. This does effect me, especially as I work in IT support now rather than as a programmer.

Really, I think this whole issue is rooted in the idea that reputation is an approximation of one's ability or usefulness. It's a mistake to believe this is possible. That said, if this is part of the goal I also know that just because it's not possible to be perfect, it doesn't mean we should just give up. Nevertheless, I think that if there is to be another adjustment that changes rep scores, before it happens we need a way to objectively quantify the value of questions verses answers.

The root of the problem is that 5 points for a question verses 10 for an answer is completely arbitrary, just as 10 each was before the last recalc. If there is to be another change, to support it I would want to see an object criteria for determining the value of questions compared to answers that can be supported based on data from the monthly dumps. The purpose of this request is to ensure that this is the last time such an adjustment needs to be made. Support the request from data in a way that tells us exactly where the adjustment needs to be.

17
  • 3
    it's quite simple: questions aren't that valuable, in the big scheme of things. Every question asker thinks their question is super-mega-important, but the reality is that every questions are far from unique (see: umpteen zillion dupe q detection discussions here) and questions will get asked no matter what. Questions are merely the sand that produces the pearl in the oyster, and you don't optimize for sand. Commented Mar 10, 2011 at 5:50
  • 4
    @Jeff - but, again, what problem are we solving? You're saying that we have a lot of low-value questions? Fine - then target those questions that through voting are shown to be have lower value, not those that are well-voted. Commented Mar 10, 2011 at 5:56
  • 2
    @joel we are disincentivizing asking questions for the sake of asking questions. In other words, you will get no rep beyond a certain point (if you ask only questions), so you better really need an answer. Commented Mar 10, 2011 at 6:11
  • 2
    @Joel the problem in my view is this - hundreds of not-totally-bad, not-totally-great questions leading to huge rep counts
    – Pekka
    Commented Mar 10, 2011 at 9:20
  • 5
    @Pekka: So what? Why is that a problem? It can't be that it's too common. According to Jeff himself, his proposal would only affect 226 users. Of those, the few that misbehave can handled bu moderators. That's what they are for: Moderators are human exception handlers. So, what are you trying to solve exactly?
    – Borror0
    Commented Mar 10, 2011 at 20:24
  • @Borror0 those users are not misbehaving. There is nothing for moderators to handle.
    – Pekka
    Commented Mar 10, 2011 at 20:42
  • 3
    @Pekka: ...so, what is the problem?
    – Borror0
    Commented Mar 10, 2011 at 20:44
  • @Borror0 The question is whether it's okay that users with hundreds of mediocre questions can accumulate the same reputation levels as an expert in their field who occasionally provides a great answer. I think it's not. Plus, this is about SO only. I don't think anybody is talking about applying this to other SE sites. What's the problem?
    – Pekka
    Commented Mar 10, 2011 at 20:44
  • 11
    @Pekka: If that bothers you, it also should bother you that someone can earn more reputation than an expert by writing tons of lousy, but not wrong, answers. Look, reputation is the sum of two things: quality and quantity. Someone might write answers that, on average, are better than Jon Skeet's but Jon Skeet writes more answers so he has more reputation. Are you angry about that too? If not, why not? Sure, there are outliers but they are just that: outliers.
    – Borror0
    Commented Mar 10, 2011 at 20:52
  • @Borror0 Sure, that's a problem too. But that doesn't make the issue at hand less valid, does it?... what exactly is your issue with this anyway? If you're saying you don't want to see this on StackExchange sites, you have my total support. In my eyes, this is for SO only, and maybe SF and SU if they want it
    – Pekka
    Commented Mar 10, 2011 at 20:55
  • Also @Borror0 on a different note, it stands to reason that mediocre questions are more destructive than mediocre answers: Questions demand peoples' attention and input. Mediocre answers can lead to huge rep gain, but they are always subject to peer review and somebody else is most often likely to provide a better answer.
    – Pekka
    Commented Mar 10, 2011 at 20:58
  • 3
    @Pekka: If this is about punishing bad questions, how about making downvotes more potent?
    – Borror0
    Commented Mar 10, 2011 at 21:06
  • 1
    @Borror0 @David puts the issue excellently in his answer: "Pity votes given to counteract downvotes is something for another day. But the reputation gained by those +1 pity votes adds up when a user is dumping half a dozen pitiful questions on the site each day. Changing the questions to only +5 reputation only slowed this down." Re making downvotes more potent, that is an interesting idea but would not solve the problem of the pity upvote. The problem is that on SO, anything that even remotely resembles a question often gets upvoted
    – Pekka
    Commented Mar 10, 2011 at 21:07
  • 3
    @Pekka @Borror0 from my analysis, pity votes are much less of a problem than what people think. Most of these asker ask average questions that over time will gain votes, they are also very aware of the bike shed effect and know how to ask general questions that are likely to get upvoted. "What is your favorite project tracking tool?" ... The problem @Joel is that as some point people are just asking for the sake of asking, asking has become a game. The system should send a clear message that we expect you to start answering at some point - if your goal is asking for the sake of rep.
    – waffles
    Commented Mar 10, 2011 at 22:18
  • @Jeff Atwood: What's the source for your italics? you don't optimize for sand? It sounds good at first, but at second thought I couldn't disagree more. I think some sand is free, would be more accurate. Commented Mar 15, 2011 at 13:05
21

It would hit some users heavily; I feel bad for some, for example Michael Stum who has a record of asking consistently excellent questions. Edward Tanguay's track record is well known as well. There are others.

Still, I say do it. Some discrimination starts becoming necessary when such large reputation figures can be reached only through asking questions.

8
  • 2
    Disclaimer: According to the simplified formula outlined above, I stand to lose about 6k points from this
    – Pekka
    Commented Mar 6, 2011 at 22:12
  • 11
    my feeling on this is that after a certain point asking the question and getting great answers, should be its own reward. Commented Mar 6, 2011 at 22:15
  • 18
    I was drafting an answer that pointed this out for a small number of users. Edward Tanguay - 964 questions, only 1 negatively voted. I'd rather see a limit that scaled with your answer upvotes. Commented Mar 6, 2011 at 22:17
  • @Jeff I tend to agree. Plus there are still badges for good questions.
    – Pekka
    Commented Mar 6, 2011 at 22:17
  • 2
    Maybe we could work it out differently, like a cap on number of questions, beyond which they don't generate rep anymore.
    – MPelletier
    Commented Mar 6, 2011 at 22:25
  • @Pekka: Not according to my original suggestion. I don't think Michael Stum would be hit hard (or at all) either, though Edward Tanguay might, I'm not sure.
    – BlueRaja
    Commented Mar 6, 2011 at 22:30
  • 4
    @bill this is hideously complicated and adds n-squared time dependencies to all rep recalcs for every question vote. Commented Mar 6, 2011 at 22:58
  • 1
    @Jeff see my comment on BlueRaja's answer. There's no O(N²) or even O(N) if you just look at answer upvotes independently.
    – Aarobot
    Commented Mar 6, 2011 at 23:46
21

I'm reluctant to support this. While answering questions is all well and good, there would be no answers without questions.

Further, we have a glut of bad questions. A quick perusal doesn't appear to demonstrate that those that ask questions beyond the 2k limit are asking bad questions. Penalizing those that are feeding good questions into the system, which in turn draw good answers from the community, which in turn gets more Google juice (The pagerank must flow!) seems to be poor form.

To some degree, it has a feeling of killing the golden goose.

However, I don't honestly think that it'll hurt the situation - we'll still get answers. Surely we'll lose a few people who will leave in disgust, but if you publicly state questions aren't very important (which is what this is) then you should feel no loss for those hurt by this change.

So the question you should be asking is:

What is the point of reputation, and how does this change make reputation better or worse?

Ostensibly reputation is a measure of how much the community trusts a given user and appreciates their contributions.

In reality it hasn't met its lofty goals, but it is used for two important tasks:

  • Determining a user's ability to participate in the community by voting to open and close, editing, and performing other tasks.
  • As a relative measure, against the other users on the site, of expertise in terms of potential employees to SOIS employee seeking clients - in other words a ranking system that may make hiring decisions a little easier.

Honestly, the first is merely moderation, and I don't think that asking good questions has any more or less impact on whether a user should be able to make such decisions or not, so this change doesn't matter in that regard.

The second point is more interesting. This change is in stackoverflow's interest primarily because the job engine now acts as a significant revenue source for SOIS. While being able to ask good questions is an important skill, already having the knowledge required to fill a position is arguably better, and in my experience, those that can answer questions well already have the skills to ask good questions where needed.

I still don't like the feeling of it - this is a significant change to the reputation system, and since questions do count up to a point I can't see that we're really re-defining what reputation is. I'd feel more comfortable if we redefined reputation to dis-include questions altogether, as it seems to be more clean and self consistent. However, we do need to consider that questions bring new programmers here, and getting them hooked on reputation is also important.

So... after some rambling, I say go for it.

6
  • 2
    Your assumption is that this discourages asking questions. But you have to ask quite a lot of questions to hit that amount of rep (which Jeff has backed up with data), and I would argue that people already have enough motivation to ask questions (i.e. they need answers). People asking questions for the sake of asking questions probably isn't going to get good results, even if they're well-intentioned - cf. the CF at Area 51.
    – Aarobot
    Commented Mar 7, 2011 at 1:49
  • @aarobot True .
    – Pollyanna
    Commented Mar 7, 2011 at 2:15
  • but if you publicly state questions aren't very important (which is what this is) Didn't I cover this a year ago with blog.stackoverflow.com/2010/03/… ? We're just moving the slider a bit farther over toward answers. Commented Mar 7, 2011 at 3:00
  • also, if answers are pearls, the questions are the sand that produces the pearl. You don't optimize for sand. Commented Mar 7, 2011 at 3:06
  • 13
    @Jeff: You don't optimise for sand, but you do optimise the sand as it is what you have direct control over! Also, you don't optimise for pearls either, you optimise for sand most likely to produce pearls. Commented Mar 8, 2011 at 12:26
  • @Andrew - nice counter-sand analogy, I think Jeff was sandblasting us with that argument <ducks, again>. Commented Mar 14, 2011 at 16:06
17
  • A few names in that list are known to me as good contributors (my list looks rather like Pekka's in that regard).

  • Rather more than that are on my list of users I think are abusing the system.

  • But I'm surprised to see how many I don't recognize. Maybe I haven't been as active on SO lately; I'm spending more time on real world stuff and on some new SE sites.

Anyway, how costly would it be to use a limit like max(2000,PointsFromAnswers)? (Working on the assumption that people who generate a lot of points from answers ask better questions than those who do not.)

3
  • 1
    Hey, that's an interesting idea as well! Especially if implemented in "real time": When you reach your limit for question upvotes, it's like you hit the daily cap: Further upvotes don't gain rep. You need to increase your answer rep first.
    – Pekka
    Commented Mar 6, 2011 at 22:21
  • 2
    Good suggestion. Maybe make it max(2000, PointsFromAnswers / 2) since answer upvotes are worth twice as much? (I know the numbers are pretty arbitrary at this point...) Commented Mar 6, 2011 at 22:24
  • 1
    @pekka just FYI that would require global recalc for all users, in addition to being hideously n-squared complex on each question vote. Whereas a simpler cap would not require a global network recalc. Commented Mar 6, 2011 at 23:00
13

dmckee already just suggested what I wanted to say, but I have one more thing to add:

Don't apply it here on meta, for the same reason that you still give 10 rep instead of 5 rep for question upvotes. Good questions (i.e. bugs or feature requests) generally have a higher importance attached to them on meta than on Stack Overflow.

Other than that, go for it.

2
  • well we could set the cap at something really high like 100k Commented Mar 6, 2011 at 23:04
  • 5
    @Jeff: Has anybody got that much from questions here? If this is some practical, technical thing that'll make it easier to implement then sure, I see no practical difference between a sky-high limit and an infinite one.
    – Aarobot
    Commented Mar 6, 2011 at 23:41
13

If the primary objective is to punish bad questions, how about making downvotes on questions more potent? Make it so that a downvote on a question is worth -5, rather than -2. Like this, bad questions will not earn a user any reputation and perhaps will even cost him some.

11

I respect and believe in the intent of this proposal. I agree with the problem present. But I feel that this may be going about it in a very inefficient manner.

Yes, this would stop the reputation flow of the problem users in question. But it takes a long time to do it if they aren't immediately affected. And in the mean while, the people who are most easily affected by it are the people who provide good questions that receive many upvotes.

In essence, if the cap is +2000, it will take hundreds upon hundreds of mediocre questions to reach it. In the case of waffles' suggestion, it won't take nearly as long but it'll still take around a hundred posts, give or take the luck of whether upvotes are earned or not. That's a lot of junk questions!

On Stack Overflow, perhaps the speed is high enough that you'll see this more readily. But on the smaller sites, this will take a long time. These users will just be sitting around, "not giving back" just as you call it, and nothing gets done about for months upon months. Eventually, they do get stopped, but not after all of this has piled up and festered on the site.

Meanwhile, while all of these problematic users are still left to their own devices, users who actually do contribute by providing great questions that receive great answers will be halted much earlier. According to the numbers, there aren't that many affected. But they're the ones who will be affected a lot more easily on the new sites. It takes far fewer excellent questions to reach these caps than it does with mediocre ones. Losing 500 reputation might not sound like a lot, but it is when you've only got 1200 reputation for around 30 posts in total.

The queries that people have run tell us how many people will be immediately affected. But how many budding "ask-for-the-sake-of-asking" users are just sitting there underneath these limits, just spewing out questions without anything to stop them? We stop those with a history of low quality, but we don't do anything if they just have a sheer volume of mediocrity. Nothing until it's an overwhelming volume, it seems.

Is the problem that they're not giving back to the community, or is it that they're earning reputation in the process of such? If the problem we want to target is the former, I don't think this affects much more than the most high-profile of examples.

5
  • I have asked for a SEDE query about users who are piling on questions but stagnating in that spot between a low-quality asker that gets banned, and the limits that the proposals here impose. I'm curious to see how it'll turn out. I just feel that this proposal has too much impact on the now, and too little impact for the future.
    – Grace Note StaffMod
    Commented Mar 10, 2011 at 21:50
  • 1
    I agree completely that this probably won't do anything about the problem of "question factories". OTOH, it should reduce the number of "very high-rep question factories". Frankly, I think this is something that should have been done within the first year of SO's existence - it's a relatively low-impact change that would have been almost painless at that point... But of course, there's no helping that now, and the longer we wait, the more trouble it'll be for high-traffic SE sites as they grow up.
    – Shog9
    Commented Mar 10, 2011 at 22:29
  • well this isn't really accurate; per Waffles' proposal they'd actually get capped at 500 rep for asking nothing but questions, which I consider to be pretty low. Commented Mar 10, 2011 at 23:49
  • 1
    Well, I got the query, which I adapted into this one. It yields 73 results, excluding 5 users who seem to have been suspended at the time. They range from a small handful of answers to some who do decently contribute.
    – Grace Note StaffMod
    Commented Mar 11, 2011 at 7:51
  • @Jeff Ultimately, I'm aware that any sort of "better" algorithm, however it is constructed, is likely to be inefficient or too complicated to implement. I avoided posting an answer to this for a while, because I know it's mostly pointless to try and gauge "question quality" towards this measure. The query I wrote gives me numbers that show me that the number of 'budding' factories is not that big. I'm not fond of the fact that nothing is yet done about them, but as Shog points out, perhaps it is important enough that we cut out the high rep. They are the target of this suggested cap, as it is.
    – Grace Note StaffMod
    Commented Mar 11, 2011 at 8:04
11

If the problem is that some users ask hundreds or thousands of low-quality questions garnering significant rep due to the sheer number of questions asked, the solution isn't to cap reputation gained from asking questions, but to cap the number of questions used in calculating reputation. If you limited question rep to the amount gained for the top 200 questions, then you'd have to average 2 upvotes per question in order to amass 2000 reputation. Further limiting it to the top 100 questions would mean that you'd need 4 upvotes on average to reach 2000 reputation.

One possible solution to question "spamming" as a reputation gathering tool: only count max(100,answers_given_count) questions for reputation purposes. This would still provide an incentive to ask good questions, even if beyond the 100 question threshold since you might add a question that moves into your top 100 and increases your reputation, but it would remove any incentive for continuing to ask crappy questions for purely rep purposes. It wouldn't punish anyone for asking good questions at all as really good questions would still earn significant reputation.

Though this query doesn't take into account the answers, you can see that there are people who do ask very good questions who aren't what we would call problem users. It would be a shame to remove the incentive for people who do ask good questions to keep doing it just to corral a relatively few problem users.

9

The site needs questions just as much as it needs answers. Why shouldn't people who ask good questions be rewarded for doing so?

The problem is people asking lots of weak questions just for the random upvotes. If a question gets a lot of upvotes then it's a good question and the person who asked it deserves the rep. So what you'd want is to disincentivise people from asking lots of questions just for the sake of it, but not limit people who ask good questions. The current proposal seems too indiscriminate. The punishment should fit the crime: punishing all question askers because some questioners are abusing the system is misguided.

So here is a proposal to do that (or rather, some proposals):

  • After your first 20 questions, questions "cost" 10 rep to ask [Numbers negotiable]
  • After you've hit 2000 rep from questions, the first two upvotes on new questions don't earn you rep [This way, there's no incentive to ask weak questions. If a question is getting several upvotes, then it's worthy of a reward]

I don't know how easy this would be to implement however.

In the interests of full disclosure, I have earned a large proportion of my reputation on tex.stackexchange from asking questions. But at least a dozen of those questions have more than 10 upvotes, so I feel I am contributing to the community, even if I don't yet know enough to answer many questions.

4
  • 3
    rep cost for asking questions is really really bad. I do like your second suggestion though. Basically, means that weak questions with sympathy upvotes don't count.
    – Earlz
    Commented Mar 10, 2011 at 1:04
  • @Earlz could you explain why you think rep cost for asking questions is a bad idea?
    – Seamus
    Commented Mar 10, 2011 at 11:47
  • 1
    Not rewarding people for asking questions is one thing. Taking something away from them is another.
    – Brant
    Commented Mar 10, 2011 at 21:11
  • 1
    Without users who would ask questions, there would not be users who would answer to such questions. I don't understand why users who answer could get reputation points, but who asks questions would risk to not get reputation points. I think that the difference between the points given for every up voted question, and the points given for every voted answer is already enough.
    – avpaderno
    Commented Mar 10, 2011 at 21:56
8

I'm against this proposal, for reasons more than me being on this list(I had decided it sounded bad before I found out)

The reason: I think a flat +2000 rep cap is ridiculous, and very low at that. I suggest something more like only 1000 rep from questions per 500 rep from other forms(answers, answer accepts, edits, etc). Otherwise, after hitting the rep cap, what use is there in trying to form good questions? I could ask anything I want no matter how stupidly formated, negative votes on my questions won't matter, since I'm about 1000 rep over the rep cap in questions.

Bad idea, you're just suggesting people make spam answers instead of spam questions. I'm sure it's a bit more effort, but not much.

Also, this doesn't change the real problem you're trying to fix. I'm pretty sure you're trying to fix question spam: A user posting 50 questions a day so that they can get 5 or 6 upvotes per day. What you should be doing is counting the amount of bad questions(or questions period), not the amount of votes received just because someone has asked an overall moderate amount of good questions.

I really like Seamus' 2nd suggestion, which perfectly does what Jeff wants. Basically, making it so that a question doesn't count toward reputation until the score is more than +2(after say 2000 rep, or 100 questions, or some similar measure). This would thump basically every question-spammer because now instead of asking 20 questions a day and getting 2 or 3 sympathy upvotes a day, they must ask at least one good question to get an amount of reputation. This does hurt the niche tags however which do not commonly get very many votes on questions or answers. This is a minority of a minority though, so I think it doesn't matter much.

2
  • I do not think your rep would be impacted much by my proposed change. I agree there is a big conundrum here cause as a community we want lots of great questions, however not answering anything and just asking is a not giving back or paying forward, so we should not reward that behavior
    – waffles
    Commented Mar 10, 2011 at 1:27
  • 1
    @waffles how is asking good questions not giving back to the community is my question? For instance, "How do I do fizzbuzz in Ruby" may seem like an extremely open ended rep gaming question with useless answers. But within, there is new knowledge created. For instance some comments to the best answers may be "wow, I didn't know you could use recursion/arrays/lambdas like that". I think the rep "gamers" that ask good questions shouldn't be punished. We should instead make it more desirable for them to answer questions
    – Earlz
    Commented Mar 11, 2011 at 1:15
6

So what is the root of the problem you are trying to solve?

Obviously the symptom is that some people are able to gain moderator access and devalue the reputation system without actually contributing anything - they just post any endless stream of very low value questions.

A hard cut off (2000 rep from questions) would stop the symptoms, but it also affects users who make good questions, which we want. I don't think full effects of this yet known - most of the sites that are defined by the quality of their questions have not reached the mass of SO, where the quality of answers is the main qualifier.

I would argue that the cause of this problem is the pity vote. For some reason people are inclined to add an up vote for:

  • A question with zero votes
  • A question or answer with 1 or more downvotes that may be wrong, but is not outright 'offensive'.

Pity votes given to counteract downvotes is something for another day. But the reputation gained by those +1 pity votes adds up when a user is dumping half a dozen pitful questions on the site each day. Changing the questions to only +5 reputation only slowed this down.

To solve this can we change how much reputation is awarded for a question upvote based on how many upvotes that question already has. Under the new system a question with only 1 upvote would give zero reputation. Further votes could either give +5 reputation, or a curve could be introduced so that truly good questions (>5 upvotes) would start receiving the original +10 reputation for each subsequent upvote.

1
  • 1
    Well, waffles implies that it's actually an endless stream of very popular, but low value questions. For instance, Bikeshed type subjective questions. Most of this has been offloaded to programmers.SE though, so I'm not sure why they are so worried about it
    – Earlz
    Commented Mar 11, 2011 at 1:16
5

My first thought, which I do not yet have data to support, is that a better solution might be to cap reputation earned from awarding the accepted answer checkmark. Right now, you can get +2 for each question you ask by doing that, regardless of whether your questions are good or not.

In theory, bad questions won't get upvotes but good questions will. If the system works like the theory suggests it should, this change would minimize the damage to good askers while continuing to penalize the bad ones. In practice... I need to figure out how to generate the relevant data.

EDIT:
Looks like Jeff was right. According to this SEDE query (developed with assistance from @drachenstern), not a single user has earned 2k rep purely from accepting answers, and only ten have earned over 1k rep that way.

Of course, 1836 rep is still a lot of rep to have gotten just from a +2 here and a +2 there (especially when, thanks to bounties, that's 224% of the user's total rep) but it's not a useful metric in the context of this question.

4
  • 3
    Good point; this alone accounts for a base amount of 2k rep for somebody with 1000 questions.
    – Pekka
    Commented Mar 6, 2011 at 22:23
  • 1
    This suggestion is rather independent from the main one (at least if we take Jeff's query as representing the plan). We could have one, the other, or both. Commented Mar 6, 2011 at 22:30
  • 2
    I seriously doubt this is a practical concern, it's at best a micro-optimization Commented Mar 6, 2011 at 22:36
  • 2
    @Jeff: And limiting the amount of rep gained by questions at 2000 isn't? Is there really a major problem to be solved here? By your own admission, it only affects 226 users on SO and almost no one elsewhere.
    – Borror0
    Commented Mar 10, 2011 at 20:17
5

I'm for the spirit of this proposal but against the methodology of employing it. A few users have suggested solutions in the neighborhood of what I think is best, but haven't nailed it.

The goal here, as I understand it, is to mitigate the reputation gain for people who ask tons of mediocre questions. Jeff has said a few times in a few answers that we're optimizing for great answers (pearls) rather than for the questions that produce them (sand).

However, Andrew makes a good point as well. We should optimize for great questions that have the best chance to produce great answers (sand most likely to create pearls). If we have a user who posts 1000 questions and each one of them brought the site thousands of views, had answers with tons of upvotes, and solved problems not addressed elsewhere, the user who asked them is doing the site just as much good as those answering the questions.

While I realize that we want to "de-incentivize asking questions for the sake of asking questions", I think that we need to take the quality of the questions into account rather than the number of questions asked by a person.

With that in mind, my proposed solution would be to scale up the value of rep gained from upvotes to questions as the question received more votes. An example of how this might work:

  • Before "Nice Question" (votes 1-9): 1 points per upvote
  • Before "Good Question" (votes 10-24): 2 points per upvote
  • Before "Great Question" (votes 25-99): 5 points per upvote
  • At/After "Great Question" (votes 100+): 10 points per upvote

I think it's very easy to see how people who ask a lot of mediocre questions would not see a lot of movement in their reputation scores, whereas people who ask awesome questions would be rewarded in kind.

My suggested point values per upvote are not something I'm dogmatic about. You could alter this such that the first 9 votes rewarded no reputation at all (thus further promoting that asking a question for the sake of rep means asking a good question), reduce/increase the value of each tier based on the traffic of the particular site (to normalize for upvotes given simply because of a site being a site with higher traffic/question views), etc etc etc.

I would say, however, that you receive a vote that sends a question to the next tier, the increase in point value would only apply to that vote and future votes. Otherwise, rep gain would see a lot of spikes.

4
  • 1
    The problem with this is that questions with silly amount of upvotes, there's an even larger explosion of reputation. Sure, it's an SO only problem. I still don't see why you'd want to reduce question upvotes to 1 rep for pretty much everything else though.
    – badp
    Commented Mar 10, 2011 at 21:35
  • @badp You're concentrating too much on the numbers and not enough on the general idea. You could easily tweak those numbers higher and I'm not suggesting that '1' is the solution for early votes. The overall idea behind the increasing scale is to incentivize great questions as they become great while mitigating the rep gained from questions that the community had deemed as "not high value" via voting. You could easily make this scale 1-5, 5-10, or take out a few tiers as you played with the results of test queries.
    – Shaun
    Commented Mar 10, 2011 at 21:40
  • 1
    The problem is that I'd rather see a law of decreasing returns on question upvotes, if anything. Here's an example Waffles used which should make the issue with increasing returns easier to see: stackoverflow.com/questions/2193953/flash-cs4-refuses-to-let-go
    – badp
    Commented Mar 10, 2011 at 21:53
  • @badp: That sounds like you're attempting to address a separate issue which my solution would exacerbate depending on the numbers selected: People getting thousands of rep just from one question. The solution to that can be implemented separately from my solution and would essentially be one of two options as I see it: Start decreasing past a certain number of votes on the other side (250 or so) or just cap the rep you can gain per question. A decreasing scale overall (rather than after 250 votes or so) fixes the problem you posit, but doesn't really incentivize asking good questions as much.
    – Shaun
    Commented Mar 10, 2011 at 21:58
4

We have some users that ask lots of lots of bad questions and just by asking so many questions get up votes and the 2 rep from accepting the answers Then we have other users that ask good questions and therefore get lots of up votes.

I was about to say

The 2 rep for each question you accept the answer for should be capped

However I think it would be better if.

It cost 2 rep to post each question after someone posts their first 50 questions – therefore it will not be possible to get more than a 100 rep gain from accepting questions.

So as to cope with people that ask lots of good questions, maybe something like

The max rap from asking questions is limited to 1000 unless you got on average more up votes then the number of questions you ask.

3
  • the data does not support this, and changing accept +2 rep is not being discussed. You can easily get +10 or +15 from every single question on upvotes, whereas an accept is a MAXIMUM of +2, ever. Commented Mar 6, 2011 at 22:39
  • @Jeff, I think park of the issue is that asking a quesions is a "one way bet", you may get rep but are very unlickly to loose rep. However each question has a small cost to everyone else Commented Mar 7, 2011 at 9:33
  • 1
    my initial answer on this question was quite similar to yours, but I just updated it with data that prove that Jeff is right.
    – Pops
    Commented Mar 10, 2011 at 19:39
4

I don't like the idea that people who've asked a lot of questions lose any reputation incentive to ask good questions in the future. This is a problem both with the proposed straight reputation cap and the reputation cap adjusted by answer votes.

I propose to reduce the rep gained from question upvotes after a certain threshold (the figure 100 may need to be adjusted):

  • The first 100 questions upvotes get you 5 rep each.
  • The next 100 get you 4 each, then 3 each, then 2 each.
  • After 400 question upvotes, each question upvote only gives you 1 rep. Question downvotes still cost 2.

With this proposal, “pity upvotes” no longer work if you've asked many questions, but there's still an incentive to receive upvotes.

I don't know how hard it would be to gather all the necessary data, but it doesn't seem to require more information than waffles's proposal.

5
  • I dislike this idea, it discourages users from asking good questions. I kind of like the idea of reducing the maximum amount of rep you can get from one question, but affecting everything is counter productive. I think the message we need to send is "asking great questions is a very "good" thing (tm) ... ONLY asking questions and not answering anything is anti community"
    – waffles
    Commented Mar 9, 2011 at 23:39
  • @waffles: I don't understand how my proposal discourages good questions, compared to Jeff's: I'm proposing to always make upvotes count. If you also want to penalize people who ask without answering, you could combine your proposal with mine. (But I don't like penalizing non-answerers, it makes me want to rename the site to stack-overflow.) Commented Mar 9, 2011 at 23:46
  • however ... your proposal has side effects ... people who answer TONS of questions and answer TONS * 2 questions, will get reduced reputation for their upvotes, this causes a side effect where you are telling these people their new questions are not as important as the old ones. The threshold I suggest is boolean which, I guess, makes you uneasy. But from our tests this drastic measure only applies to users who have crossed the line.
    – waffles
    Commented Mar 10, 2011 at 0:24
  • @waffles: Then combine the two (i.e. use round(answer_upvotes*ratio) instead of 100). I don't mind the boolean threshold, what I dislike is 1. the message that you're not supposed to only ask questions, 2. that asking good questions makes no difference after a point. Commented Mar 10, 2011 at 0:47
  • if answers are the pearls produced by the oyster, questions are the sand. You don't optimize for sand. Commented Mar 10, 2011 at 5:24
4

I really don't think that this is a good solution. The SO community depends on questions. Without questions, you can't earn any reputation by answering them.

I understand that some people who earn a lot of reputation by providing outstanding answers think that it is unfair that users earn a lot of reputation by "only" asking questions. But you seem to forget that asking good questions which will increase the reputation is also a very time-consuming and not an easy task.

If an user does not put any effort into asking good questions, he want earn any upvotes for this question in the first place.

So this suggestion to set the reputation cap wouldn't improve the quality of questions. In contrast, if anything changes, user won't put as much effort as now into asking good questions since this isn't rewarded anymore.

4

In a comment Waffles said

ONLY asking questions and not answering anything is anti community

I am opposed to this sentiment. Let's not pretend that SO is not about questions. I hazard a guess that above 90% of questions if not above 95% are mundane drivel of trivial questions. SO is clearly enjoying the traffic, so are you now saying that the majority of your question askers are "anti-community"? A very strong sentiment if you ask me.

As for any cap between question and answer, the question is, if you can churn out a high rep from volumes of moderate answers (sympathy vote et al included) over 2 years since inception vs a newbie with 2 months of good answers, why can one not gain a high rep from churning questions? Should a person who churns mediocre answer now also be labeled as "gaming" the system by logging in often and answering easy questions early for petty rep?

The crux of the above paragraph is "why the difference"? churn is churn, mediocre is mediocre, gaming is gaming - whether question or answer - same difference.

What I would rather see is a proper split between questions and answers. Is a project manager a highly reputable technical person? Possibly. Is he a good programmer. Heck no (mostly). Is an avid questioner a "reputable" person on SO (by question sense)? - Yes, or at least insofar as a mediocre long-term answerer is.

I am arguing for a reversal of question rep point decrement. Bring it back up to 10 and have a separate (from answer) rep count for it.

Because I don't care if you are a good questioner and at the same time good answerer (crux of Waffle's ratchet system). Only answers should count - period. Questions count for something, yes, but not for answer-rep per-se. Give it its own count.


Missed one point. Should anything be capped, it should only be the +2 for accepting answers. Even 200 is a generous enough cap to train people into accepting answers, after which, it should be a no brainier that such an action is not "reputable" at all.

3
  • 1
    Fair points, especially about the equality of gaming. Splitting rep counts - and giving users access to mod tools based only on answer rep - has crossed my mind, too.
    – Pekka
    Commented Mar 14, 2011 at 0:01
  • 3
    a flood of poor questions is the easiest way to destroy a Q&A system, as it pollutes the front page and incoming questions on tags. Mediocre answers are easier to deal with and, at worst, will bump an already decent question for review which is a net positive. Splitting rep between questions and answers would make the system needlessly more complex. Commented Mar 14, 2011 at 0:40
  • 1
    @Jeff Removing question rep from the current rep tally. A new field to accumulate it - displayed at selective places.. this is "needlessly more complex" than placing ratcheting caps? Color me unimpressed with the technical skills of the team then <disclaim>not that this is what I feel, but this is my response to the comment<disclaim> Commented Mar 14, 2011 at 2:11
3

I wasn't going to comment here because, frankly, I don't care what you do. It's your website, and you can do what you like with it! I don't mean that in a nasty way, I mean that in a "I'm delighted that it's not my problem" way. But my comment about optimising for pearl-producing-sand rather than optimising for pearls has gotten 10 votes so I'm going to expand on it a bit.

It feels as though you've just discovered something that every educator knows in their bones. If I assign my students a mental arithmetic test, then line them up according to height, and finally give the 5 shortest an 'A' grade in French, then someone might complain about my grading system.

Welcome to the world of criterion-based assessment! Relative assessment, which is what the reputation system currently is, has its place, but is very crude and not easily justified when trying to claim that a certain score relates to a certain ability.

As others have said, there's a problem stemming from the fact that reputation is used in more than one fashion: both as a way to unlock moderation tools and as a crude measure of their worth to the SE network. So changes to how reputation is calculated that are designed to improve its use in one of those will have an effect on its effectiveness for the other.

Also as others have said, if anyone is just looking at the reputation as a number and not digging further then that's actually a mark of a poor assessment system of the employer. Sadly, this happens a lot (search for "impact factor" and "academic journal" to get a measure of one place where this happens), but SE has very little control over this. The best that it can do is to make the more refined data clearly available so that job seekers can do more than just list their reputation, and can point to (say) number of accepted answers, number of "excellent" answers, or other relevant features. Again, which pieces are needed will depend too much on the actual situation, so SE should just make all the data easily available so that employers and candidates can easily find what they need.

That leaves us with reputation as a way to unlock moderation tools. Here, SE has total control and here the relative-based method of simply saying, "User X has N reputation so they can now do Y" is shown up to be completely daft. It really ought to be criterion based.

I completely agree with the hypothesis that a user who only asks questions should not have access to the higher moderation tools. But I would say the same for a user who only answers questions. Both users have not shown evidence that they have a good understanding of how the site works. Over on TeX-SX we have a couple of "mini Jon Skeets" who power through questions, answering all that they come across, but never asking anything. (A quick glance at the top page of maths-SX shows that this phenomenon is even more pronounced over there!) These are highly valuable members of the site, but I would not trust them with the moderation tools (if anyone of them ever read this, I'm overstating to make a point). Indeed, none of these stood for moderator in our recent elections and I would not have voted for them if they had. Others, who also haven't asked many questions, have nonetheless shown a greater commitment to the site by voting, participation in meta, writing tag wikis, and so forth.

So if the purpose in this proposal is that people who only ask questions should not have the power that their reputation brings with them, then perhaps that should also apply to people who only answer questions! Of course, I would prefer a proper criterion-based system over this crude measure, namely that there should be criteria that a user has to satisfy to get access to each tool.

Incidentally, the key phrase here is: shown evidence of. You aren't ever saying, "User X has ability Y" but "User X has shown evidence of having ability Y". It's much easier to justify later.

While I'm on this, as far as moderation tools are concerned, I agree with the proposal that only recent activity should count, or at the least that there should be some decay. Suppose someone gains 10k, goes backpacking for a year, and then comes back. SE could have changed a lot since they were away and the culture shifted to a point that they need to relearn the norms before taking back their powers. (So actually maybe the formula should be (significant reputation in the last month) x (total reputation), or something. I'm a mathematician, we can dream up formulae to compute anything.)

In short (I've recently learnt what TL;DR means and while I despise the phrase, I can understand the sentiment behind it): come up with a criterion-based method of assessment and then implement it. Of course it will have parameters that will need tweaking, but by being criterion-based it will be a lot easier to justify changing those parameters than just saying, "Answers should accrue more reputation than questions so we're making it 10 to 5". But the criteria themselves will not be parameter based and will be phrased along the lines of "To gain access to moderation tool X, a user has to have shown evidence of Y". Or, for the other use of reputation, "To be listed as a 'knowledgeable user' (made-up term) in area X, a user has to have shown evidence of Y". Once you've done that, you can discuss how you will actually gather that evidence and that's where the discussion about weights and measures comes in.

8
  • This isn't just about moderation tools; far from it. The bottom line is that questions are simply not that valuable. The world is awash in endless questions, what it is not awash in is quality answers. Not to put too fine a point on it, but we want users who only ever ask questions to GO AWAY. More optimistically, capping their rep at 500 is a way of encouraging them to balance with answers. Commented Mar 18, 2011 at 10:48
  • @Jeff: You're halfway there! You just need to listen to yourself. Yes, there are endless questions. Yes, most of them are complete dross. But a quality answer does not exist in and of itself, it has to be an answer to a question. So there are quality questions: they are the ones that lead to quality answers. The problem is that votes does not show any evidence that a question will lead to a quality answer. And, Hey Presto!, you're in the land of criterion-based assessment, only this time of questions, not people. But it's the same principle. Commented Mar 18, 2011 at 11:28
  • (ctd): So you first need to decide on the criteria for a "quality question". Then figure out how you will assess that. Lastly, implement it. Problem solved. Commented Mar 18, 2011 at 11:29
  • "So there are quality questions: they are the ones that lead to quality answers" I can point to literally thousands of answers that are stunningly good which happen to be on mediocre to outright bad questions. It is, dare I say, common. There's no real relationship between 'quality of question' and 'quality of answers'. It is essentially random which grains of sand will produce the pearl. Commented Mar 18, 2011 at 11:32
  • I would also argue that the people who provide the best answers are so very good that they don't need to ask questions often -- if at all. So a constant supply of mediocre, mundane, garden variety sand (NB: I am NOT saying poor quality is acceptable here) is all that is required for pearls to form. You're basically telling me something like "you must figure out how to optimize for the best pornography!" when really, most (all?) pornography is plenty good enough for purpose, with no real quality whatsoever ... Commented Mar 18, 2011 at 11:37
  • @Jeff: Okay, so there's a philosophical point there that we disagree on and possibly always will. I say that a "quality" answer to a naff question isn't an answer at all. It may well be "quality", but it isn't a quality answer. It could probably stand by itself on a wiki page somewhere. However, that's orthogonal to the point of this post which is that criterion-based assessment is the Right Way to approach this sort of question. Commented Mar 18, 2011 at 11:45
  • @Jeff: (thinking a bit more) As regards questions I think we broadly agree: there are a lot of junk questions out there and it would be great to not have them cluttering up the system. But the key is to focus on the questions not the questioners. If someone asks a lot of truly great questions, that's to be encouraged! Truly great questions are a fantastic way of learning and are worth as much as a truly great answer, often better. Commented Mar 18, 2011 at 11:51
  • @Jeff: Last word on this. If you think that pearls are worth anything, you need to read "The Pearl" by John Steinbeck. Pearls are only worth something if people are willing to pay for them. Commented Mar 18, 2011 at 20:48
2

I would support this proposal, with a caveat:

In the way that an accepted answer is immune from the rep cap, perhaps a question is immune from the rep cap once it reaches a threshold (say, +10), because then you can be fairly sure that it is a good question.

However, the threshold would have to differ for sites, because great questions on Server Fault almost never reach +10. I'm not sure about Super User but I'm guessing the same is also for the smaller Stack Exchange sites.

2
  • well, on Server Fault the only person even close to this limit is, ironically, our very own Kyle Brandt.. Commented Mar 6, 2011 at 22:33
  • @Jeff, yes I just noticed this independantly ;) Commented Mar 6, 2011 at 22:39
2

Good Question. Ultimately, there are multiple factors in play here:

  • People want Reputation for their privileges and fame
  • People want Answers for their questions
  • Asking questions - even excellent ones - is relatively easy because they come directly out of real world problems
  • Providing Answers - especially excellent ones - is a lot harder and in the end the real value
  • However, people who provide good answers maybe the people who had good questions at the beginning

So rep on questions is a great way to "suck people in", to give them another reward along with good answers and hopefully turn them into return customers that eventually transition into giving answers as well.

On the other hand, there is nothing wrong with saying "Feel free to ask as many more questions as you want and get great answers, but you now have to work harder for points. If you received 2000 Rep on questions, you should know a bit about the stuff you're asking, so maybe you can help others?"

2000 is a good limit because it gives access to the pending-edits-indicator, so it's a soft nudge towards participation.

I am at the point where reputation became sort-of meaningless: I won't catch up to Jon Skeet ever, yet I do have access to the 10k rep tool which is just a great source for really good questions (highest votes) and some great comedic relief (the low voted stuff). For me Stackoverflow is now a giant saw-sharpener, informational tool (just browsing random questions) and a great source to ask my own real world problems with confidence of getting a good answer.

Go for it if you feel this is really solving an issue that SO is having or will have in the future. I'm not sure if this could discourage further participants though as reaching rep is constantly harder, but the weekly/monthly/yearly top lists seem to work against that perception.

Edit as per waffles comment, I wonder if the cap could be a sliding cap? For each 1000 rep you gain through answers, your question cap raises by an additional 500? rep. Details would have to be worked out (e.g., should this work retroactively or is reputation "gained" while at the cap lost even when your cap increases later on?).

7
  • 3
    The issue I have, is the principal. I do not think it is fair to punish you with a 4000 rep penalty for the pure reason that it is more simple for us to implement. Also I think that if you give back you should not be punished. On the other hand, I also feel that some of the serial askers are not punished enough, I have always been uncomfortable with people getting "approve edit" and "close" rights due to questions alone
    – waffles
    Commented Mar 6, 2011 at 23:56
  • @waffles Very true, but these people we want to hit don't seem to be hit by this. I wonder if the Question Cap could be a sliding cap? For each 1000 Rep gained through Answers you can gain an additional 500? Rep through questions? Commented Mar 6, 2011 at 23:59
  • @Michael, exactly my suggestion :) see my answer
    – waffles
    Commented Mar 7, 2011 at 0:01
  • @waffles I suggest looking at Seamus' second suggestion. I think this sounds the most fair way to stump the serial askers. Basically, make it so that questions only count toward rep after getting +3 on a question. Basically make it so that the mass amount of +1 sympathy votes don't get counted on serial asked questions
    – Earlz
    Commented Mar 10, 2011 at 1:18
  • @Earlz sympathy votes are not the problem data.stackexchange.com/stackoverflow/q/94853/… , the bigger problem is that people are too precious with downvotes AND a lot of the serial askers figured out that asking soft broad questions gets you lots of upvotes, see meta.stackexchange.com/questions/3486/…
    – waffles
    Commented Mar 10, 2011 at 1:22
  • @waffles I still highly disagree with a straight rep-cap. Perhaps the question/answer ratio thing is a better suggestion then. A straight rep cap is nothing but harmful for a multitude of legitimate users(I'd lose I believe 1,500 rep) who do contribute back
    – Earlz
    Commented Mar 10, 2011 at 1:33
  • @Earlz totally agree with you on the straight up cap, I only support a system that encourages some sort of balance
    – waffles
    Commented Mar 10, 2011 at 2:09
2

I'm strongly opposed to a question cap. Instead I believe that questions should attract larger down vote penalties.

I already lost a significant amount of reputation when question votes were devalued, and reputation whoring aside, I believe that good questions are critical to the value of the site external to its more hardcore users. Stackoverflow has a very good hit rate on Google, and higher question rates enable searchers to more often hit questions specific to their problem.

Furthermore, many questions remain unasked, especially in the more niche areas that heavy question askers such as myself lurk. I believe that fresh questions in those areas should attract more reputation if possible. Wading through pages of C#, Java and ASP minutae and trivia is pushing away many potential users.

2

No limit

A question is a question. If it is not you have to trust the community to moderate itself.

Isn't that the definition of StackExchange websites?

1

I am inclined to agree with this measure, although I do have a decent amount of questions on TeX-SE.

However, I'd like to propose one change: Add all bounties awarded to this number. I don't know how resource-intensive this could be, but I'd guess the reputation from bounties is much easier to calculate than doing a full rep recalc.

For example, if I have awarded 500 rep worth of bounties to questions, my effective limit should be 2500. In this way, I still have an incentive to ask questions (up to a limit) and I'm not going to accrue ridiculous amount of reputation from questions alone as I am distributing it to others.

2
  • A very poor complexity-to-usefulness ratio-- who would understand this?
    – Andomar
    Commented Mar 10, 2011 at 21:23
  • @Andomar: I honestly can't see why this is any more complex than a sliding cap for question reputation. You have a fixed number, and you add another easily calculated number. And the usefulness should be pretty obvious -- ask as much questions as you want, but use your reputation for bounties to prove you don't do it for the rep alone. If a user don't earn reputation in other ways, their reputation will stay effectively the same -- but they will be able to ask more questions and get bounties. Some of the great question askers will still have an incentive to continue and give back. Commented Mar 11, 2011 at 6:35
1

Okay, my other answer was a general scheme. Here's a specific recommendation.

So questions are only valuable in so far as they are the raw stuff of which quality answers are made. So votes on questions are similarly pretty worthless, until it has been shown that the sand is pukka-pearl-producing sand. Except that votes are a good indication to an answerer that their answer is going to reach more people. I think that I would be more interested in answering a highly voted question than a lowly voted one (all other things being equal). So the votes for the question give an indication of what the answers are worth. Thus:

  1. Make the reputation available for the answerers dependent on the votes for the question, and divide it up according to the votes for the answers.
  2. Reverse this as well, and make the votes for the answers feed back to the reputation of the questioner.

Then a question can accrue reputation for the questioner, but only if it garners quality answers, namely if it proves to be pukka-pearl-producing-precipitate. Meanwhile, the votes for the question are merely a measure of the questions potential rewards to anyone brave enough to venture an answer.

1

If you want to stop bad questions, don't just take away the rep.

Simply be far more aggressive in deleting bad content. That ultimately should increase the overall quality of the site, take away even backdoors for gaining rep (for everyone) and makes it very clear to everyone: if you don't put some effort in your post, it will get deleted.

If those questions are really worth that little, then there's no reason to keep them around in the first place. Surely, if you'd apply that logic to most of the users with outrageous amount of answers, that number would be a lot smaller. And you could stop accepting their questions (as you do for downvoted stuff) after a very bad streak. The easy solution is for users to use their downvotes more (don't you get them back if the content gets deleted?) and filter out the bad content the way it's intended. Everybody is being far too nice for such bad content.

Honestly, I think your proposal takes care of the symptoms, but ignores the actual cause.

1

I think the law of unintended consequences comes into play here.

From reading the comments of the cappers it seems they want to dissuade people from building high rep cheaply by asking questions for the sake of asking questions.

But if that's your motivation you can fairly easily switch to building rep by answering cheaply - by cheaply I mean as in answering easy duplicates quickly. A brief easy answer to a duplicate is not a high-value contribution.

I don't think you'll get rid of them, and you can't turn them into high-value users.

1
  • answers are far more competitive and tougher to get upvotes for; good luck with this strategy -- you'll need it. Commented Apr 5, 2011 at 8:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .