164

Update

Thank you everyone for the feedback you have provided. At this time, we will work through the feedback we have received to help us determine where to take our research on reputation next.

Please remember that this is just research and does not impact any product decision now or in the immediate future. We look forward to returning to the community to discuss more related and adjacent topics before embarking on any product development.


Having incentive systems to participate in online communities (like Stack Overflow) is not a new concept. Basically every Q&A site or forum has had some type of points or karma system, and they all work to varying degrees.

But as we’ve seen, our system and community in its current state has a high barrier to entry. Asking or answering a question back in 2009 was a lot easier as we were building the library of knowledge that exists today. There were a lot more questions yet to be asked!

We still want users to be able to participate on our site and feel as though they are a part of this community. But unfortunately, we’ve heard from so many of our users, particularly newer ones, that the learning curve is steep and it’s difficult to find ways to engage on the site.

In order to combat some of that, we’ve been experimenting with the Staging Ground work and with better onboarding for new users (currently only on Stack Overflow). These are just a few early attempts, and we know there could be additional solutions out there.

However, as the network has grown and changed, we’ve started thinking about our reputation and privilege system and how we might want to evolve it. Our Research and Community teams internally have been doing a lot of work to understand incentive systems more broadly, but we also wanted to hear from you all about what you think works really well with Stack Exchange’s reputation/privilege systems, as well as what is broken with reputation. We’d also love to hear about other rep/karma/incentive systems from other online communities that you think are exemplary that we could learn something from.

We don’t have any concrete plans yet. This is really the first part of a long process to see how we could potentially rethink reputation and privileges on this site and our larger network. We’ll be doing several rounds of research as a part of this larger exploration, and will update you on our findings as we go, so please be on the lookout for future updates.

So, for now, our questions for you are:

  • What is really great about our reputation & privilege system? What do you think makes it great, and why is it worth keeping?
  • What is broken about it, and why? Are there any solutions to it?
  • What other systems work really well that we could learn from?

Since there are many answers already written now, if you'd like to add one but want to check first that someone hasn't already said what you want to say, try using the searchbar with inquestion:387356 is:answer ....

48
  • 19
    Possibly related: Help us identify new roles for community members (some of them are also related to reputation/privilege) Commented Mar 9, 2023 at 15:23
  • 132
    How is reputation connected to there being an entry barrier? The only privilege that newbies frequently miss is the ability to post comments, they mostly don't care about moderation tools. Commented Mar 9, 2023 at 15:31
  • 124
    Don't try to fix something that is not broken. You will just break it, without any shred of doubt. Commented Mar 9, 2023 at 15:35
  • 44
    FWIW, I joined about three years ago, and a few minutes with the site tour was plenty to make the workings clear. The rep/privileges/badges system was and is, for me, a great incentive system.
    – Aaron
    Commented Mar 9, 2023 at 15:37
  • 237
    "its current state has a high barrier to entry" - This is good. Quantity dillutes quality. The problem is frustration, not the barrier. You minimize frustration putting good explanation in the barriers BEFORE any interaction, so people understand why this barriers are protecting and promoting knowlege and trust. Currently the network says "c'mon, you can do everything", then slaps the rules at the user's face after his actions with close and downvotes. Somewhat like a carnivore plant. Also, you need to choose between qty of users, or the original network mission (so we know what to follow).
    – Largato
    Commented Mar 9, 2023 at 16:14
  • 14
    Also remember: views is not tied to # of registered users, but content. The recent inflation of registered user numbers maybe was very good to sell the company, but we are past that, so maybe we can focus on content again to atract views, help people (mostly in read-only mode) and keep the high reputation the network still has only because the "tough old times".
    – Largato
    Commented Mar 9, 2023 at 16:32
  • 116
    I agree with @Bacco's comment: "you need to choose between qty of users, or the original network mission (so we know what to follow)". Decide once and for all what you want. The official position is still "quality", but all actions in the last years favors "quantity". And we've seen how incompatible those two things are. This is one of the main reasons behind new users frustation, BTW. The site says they can ask anything, but users worried about quality says: "wait, there are rules" - and then the latter are called "toxic" by the former.
    – hkotsubo
    Commented Mar 9, 2023 at 16:32
  • 38
    @ShadowWizardChasingStars Slow down there. There are no plans to remove reputation or privileges. This is quite literally the opening question to the community before we have even engaged in serious research on our reputation system. We want to hear what people consider pain points, opportunities, and things they love.
    – SpencerG StaffMod
    Commented Mar 9, 2023 at 16:53
  • 13
    @CypherPotato well, I don't think SE are doing it out of greed, they just see so many people complain about reputation, how broken and toxic it is (try to have a comment discussion with a new user who got a downvote, their vocabulary of bad words is amazing) and since the amount of those people, who rant and complain and send tickets is x1000 than those few who trust the reputation system, they are doing what they deem right for more people. Commented Mar 9, 2023 at 17:00
  • 38
    @SpencerG The most crucial point is how hard it is to instruct new users to understand what our community is actually about. It's not clear enough for them to understand, and that is why thousands of posts are closed every day. Also, the entire community's effort to filter content is decreasing because we don't have the right tools or the community entry point is not effectively doing its job in showing what Stack Overflow is for. Commented Mar 9, 2023 at 17:03
  • 21
    @SpencerG it's clearly mentioned in the question here: "as well as what is broken with reputation" - not "if something is broken" but rather "what is broken", word choice means a lot, in this case it means SE thinks reputation system is broken, due to "we’ve heard from so many of our users, particularly newer ones, that the learning curve is steep and it’s difficult to find ways to engage on the site" which means great many new user keep hammering you with contact forms and emails after getting downvotes, complaining how the system is broken. Commented Mar 9, 2023 at 17:04
  • 35
    @ShadowWizardChasingStars I can understand how that can seem to indicate that; however, that is not a position of SE. We do not internally believe that reputation is broken and needs to be replaced. We think there are opportunities for improvement, like any system as old as ours. This is a healthy process for us to hear what the community thinks/feels on the matter. Our only intention is to collect and consider feedback to help guide our research to see if suggestions made here could help improve the system.
    – SpencerG StaffMod
    Commented Mar 9, 2023 at 17:21
  • 21
    This SE's positioning leads us to understand that SE intends to attract both Q&A and social media audiences to their products. Q&A users and social media audiences interact differently, behave differently, and seek different things. Commented Mar 9, 2023 at 17:28
  • 28
    BTW, I believe the reputation system isn't a problem (or at least not the biggest concern right now). There are other things you should focus on. Ex: IMO, if you do things like this, the user experience would improve a lot and it would solve (or at least minimize) the problems reported by new users (regardless of the reputation system).
    – hkotsubo
    Commented Mar 9, 2023 at 17:44
  • 9
    @OverclockedSkid that doesn't make sense: it takes no rep to answer but there is a minimal rep before you can comment, so how were you "answer[ing] things as comments" when you did not have reputation?
    – terdon
    Commented Mar 10, 2023 at 10:30

71 Answers 71

212

What's broken: uncapped rep per post. Long-time users have had the luck to answer basic questions that are very popular to search, and gather hundreds, thousands, or 10k+ upvotes. This rep gain is completely meaningless - it doesn't reflect greater skill, knowledge, or any other trait that should reflect 'authority'. Meanwhile, newer or more specialized questions are inherently less prone to such a boost, yet at the same time demand much greater skill. From my observation, this is very pronounced. A hard-earned +2 post isn't worth x50 less because it didn't make it to HNQ and get +100.

Solution: cap rep gain per post, and apply retroactively. I don't know where exactly, but +100 net votes seems reasonable.

Solution 2: every upvote after +50 earns 5 rep; 2 after +100; 1 after +200. Catches up with the first solution at +250 while affecting more posts but keeps a permanent incentive. Also retroactive.

It doesn't have to be these exact solutions. I think this should be its own featured Meta Q&A.

47
  • 40
    the rep gained per post is mostly garnered through quantity, not a single post receiving 10k upvtes. There's certianly a few edge cases, but posts with 100+ upvotes isn't where long-term users are getting their rep. A very large portion of the rep from these posts get voided due to the daily rep cap.
    – Kevin B
    Commented Mar 9, 2023 at 15:32
  • 68
    @KevinB That's the point, for most users this won't matter much. But I've encountered cases where users have earned a lot of rep asking a couple of basic questions without research many years ago, and still ask and bounty basic questions without demonstrating research effort every now and then that can't get closed when bountied. It won't affect most users, but will fix a rare but annoying issue imo.
    – Erik A
    Commented Mar 9, 2023 at 15:36
  • 12
    @ErikA right, but, what i'm saying is the users who gain rep from these posts are users who have 1-3 posts at most. Users like me, with 2k, getting 3k upvotes on an answer gave me only maybe 3k rep, not the 30k that the score would seem to indicate. It's not about the singular high scored question, it's about over time receiving 1-5 upvotes on the other 2k answers. This long-tail of receiving rep for old contributions encourages quanitity over quality, the opposite of a single post receiving 10k upvotes.
    – Kevin B
    Commented Mar 9, 2023 at 15:46
  • 4
    I mean, they don't tend to receive downvotes. correct answers tend to continue being correct.
    – Kevin B
    Commented Mar 9, 2023 at 16:04
  • 4
    @VE7JRO Is that meant to indicate all that user's rep is from one question? Because it's not; you could delete that question and that user would still have over 15,000 reputation. You may notice, if you bother to look at their profile, that they have numerous answers near or above 100 score, for example.
    – TylerH
    Commented Mar 9, 2023 at 21:55
  • 6
    @TylerH - This answer is about limiting the total rep limit per Q+A. Your comment ignores the fact that the top answer here: stackoverflow.com/questions/276927/… received 20550 rep. I guess you missed the point of OverLordGoldDragon's answer.
    – VE7JRO
    Commented Mar 9, 2023 at 22:03
  • 7
    @VE7JRO You are incorrect. The top answer there is a Community Wiki answer and has been since 2012, so no rep has been awarded beyond the initial 100 or so upvotes that were cast. Maybe do a little more homework before dismissing someone's criticism of a vague comment with no explanation or justification
    – TylerH
    Commented Mar 9, 2023 at 22:38
  • 16
    I've seen one case of a user who posted a question 10 years back, didn't check in since, and now has all sorts of moderator privileges they don't know what to do with. Someone who doesn't know the site should not be able to collect so much rep, if the rep is tied to moderation privileges. Either decouple the two, or limit rep gain to weeks where the user has logged on or something like that. Commented Mar 9, 2023 at 23:25
  • 10
    @VE7JRO there's no indication that the answerer got the maximum of 117710 reps because there's a 200-rep daily limit. But then, it's Jon Skeet, and it's really a revelation for someone who has been bitten by timezone quirks. Commented Mar 10, 2023 at 3:06
  • 31
    If you want to argue about 'worth' of posts, this reminds me mostly of current economic circumstances: An old fart used his first few paychecks to buy a cheap home for himself, and over the years that increased in value, to a point where if he had to buy it today, he wouldn't be able to. Young people nowadays can't buy homes because they're all too expensive too. Capping whatever income the old fart has had over his lifetime, making him pay back large chunks of it, and limiting what income he still gets isn't magically going to make his home worth less, nor any new homes worth more, though.
    – Tinkeringbell Mod
    Commented Mar 10, 2023 at 8:46
  • 22
    Similarly, capping reputation gain (or decaying it over time) from an old post isn't magically going to make new posts worth more.
    – Tinkeringbell Mod
    Commented Mar 10, 2023 at 8:48
  • 8
    @Tinkeringbell "lowering old won't raise new" - if everyone but me earned 0.01 per upvote, I'd top Jon Skeet. As all measures, rep is relative, so deflating old/popular posts does prop up everything else. Commented Mar 10, 2023 at 12:14
  • 9
    If your main problem with SO is posts with lots of upvotes, new questions in your frequented tags must look vastly different from the garbage dumps in python and java. I agree that it's not great if some users gain tons of rep from popular but basic answers or questions, but it's a niche edge case compared to everything else that's wrong with the rep system, so IMO this is a really low priority.
    – l4mpi
    Commented Mar 10, 2023 at 12:45
  • 13
    @Tinkeringbell I really don't follow. False impressions are my exact quarrel with the current system. Newer/unpopular posts aren't worth less just because fewer people in total looked at them. What people see at end of the day is the total rep of a user, and that shouldn't be so heavily controlled by popularity. Any given post has finite worth, and if you maximize your rewards on it, good for you - smile and move on; demanding more is pointless greed. Putting some down does prop others up. Commented Mar 10, 2023 at 13:04
  • 9
    I'd like to see quantification on this. There are certainly edge cases, but that's the case for most systems. A system should be tuned to work well in the majority of cases, rather than penalize the majority for a few odd cases. As an example, I have one very simple answer in C++ that has gotten me an ungodly amount of reputation -- though not as much as the score would indicate, due to the daily cap -- and at the beginning it felt unfair... but I've come to terms with it: it compensates for all the elaborate answers to complex questions I've only ever got 1 or 2 votes for... Commented Mar 11, 2023 at 15:28
166

This is something I was hoping you all would explore sometime soon, and I'm excited that you've posted about this now.

For awhile now, I've felt that tying moderation privileges to reputation... Doesn't really make a whole lot of sense. Whether one is capable of utilizing particular moderation privileges to better the community should not necessarily be determined by the quality, or quantity, of their answers on a given site. Rewarding users who know how to use particular moderation privileges well with more power in those particular areas is, in my opinion, the better way to go. You guys currently do this with the flag posts privilege, where you get 1 bonus flag per day per every 10 helpful flags, up to a maximum of 100. That's great, but you stop there. No other moderation privileges are expanded upon when you've shown that you're a "good" user of them. Giving close votes and delete votes this treatment could prove to be difficult (the question of "What is a good close vote?" is a difficult question to answer!) but there are some nice folks I know on Stack Overflow who would appreciate the ability to do more without committing to something like becoming a diamond moderator.

I'll take your questions one at a time and give my 2 cents:

What is really great about our reputation & privilege system? What do you think makes it great, and why is it worth keeping?

For the privilege system, super simply put, the biggest contributors get more access to help guide the site the way they see fit. The ability to unilaterally edit, vote to close, review posts and edits by lesser-privileged users, see deleted content... All of that empowers a veteran user to just do more with the site. The reputation levels that are currently chosen for the more powerful privileges (such as access to moderator tools and trusted user) are intentionally high to ensure that community members that have contributed a lot, and likely over a long period of time, have access to them.

This model has worked for a long time, and for good reason. Reputation as a number alone provides incentive to post, and tying the ability to do more with the site provides even further incentive. I would never suggest that we do away with reputation because it's a core part of the Stack Exchange model.

When we're just looking at reputation the number... I mean, it's just overall a great way to incentivize good contribution. There's a degree of accomplishment one feels when they answer a question that leads to resolving a problem someone has, but getting "paid" for it? That never gets old.

What is broken about it, and why? Are there any solutions to it?

Well... As you noted, the more saturated a given site becomes with good quality Q&A content, the more difficult it is for newer users to find a place where they can contribute meaningfully. Someone who is interested in helping with site cleanup efforts has a long way to go to get access to the ability to review, edit posts without review, and vote to close. They do get access to the ability to flag quite early, which is quite important and quite powerful, but the march towards getting anything beyond that is long and, sometimes, difficult when attempting it on a site that's quite saturated with good content. You will eventually experience a situation where a user has thousands of helpful flags, a great suggested edit history, a great knowledge of how the site works, and a good head for user-level moderation... And they simply can't contribute in the way someone who has posted more questions/answers can. I have one particular user in mind as I type this who is an absolutely exceptional example of someone I'd like to see be able to do more. And... That kind of sucks.

This sort of touches on a personal note for me, as well. I've got 500 reputation on Stack Overflow, which allows me to see some review queues, but... I frequently find that there are a lot of situations where I wish I had the privileges I do on this site. Unilateral edits (and the ability to help out with Stack Overflow's suggested edit review problem), the ability to vote to close, the ability to help to delete posts that need it... I mean, if I review any post, I frequently find myself skipping because the post needs an edit but the suggested edit queue is full. I've got thousands of helpful flags and plenty of approved suggested edits, and I'm trying to help improve a question that was presented to me in review. Why am I being stopped here?

The inability for newer users to find questions they can post answers to on saturated sites also encourages answering questions that are either known duplicates or that are duplicates but the duplicate just isn't found (yet?). The thing is, there's no incentive to find a duplicate, whereas there's plenty of incentive to just answer a duplicate. Better duplicate discoverability would be good, but some incentive for finding a good duplicate would be great. However, if you explore this, I would exercise caution, as it may incentivize incorrect duplicate closure, which is an extremely frustrating experience for a new user.

What other systems work really well that we could learn from?

There are not a lot of sites that reward user participation with (moderation) privileges like Stack Exchange does. From a reputation/content scoring level, there are a fair few sites that allow upvoting but remove downvoting, which... I don't know if I particularly care for, since there's no way for old content that may have become obsolete to be reduced in "usefulness-level", but I suppose it does make for an experience free of the typical derision that downvotes receive here on SE.

While I don't feel like you'd particularly appreciate the mention, Codidact does feature an abilities system, all of which reward users who have a good user-level-moderation track record with the ability to do more, which is in addition to a reputation system. Competitor site or not, there's something to respect about users who created a whole platform (out of their love for this platform!) rebuilding the privilege system in their own vision, and in a way that makes sense.

I think you're essentially going to be paving your own way here. My suggestion would be an implementation of something that's side-along to reputation, rewarding the good curation work that many users are interested in performing without invalidating reputation's longstanding meaning. A good idea you've been exploring is roles, but I feel like none of the suggestions tackled a particular pain point that can be useful across the network: The idea of opening up the full edit privilege to more users who have shown to be good editors. That's a great privilege to start this sort of exploration with.

22
  • 12
    Re: Abilities system, I wrote a blog post which included a part where I give a brief overview of what's wrong with SE's rep system. I've quoted the relevant part in chat.
    – Mithical
    Commented Mar 9, 2023 at 18:01
  • 20
    I think users with a good track of suggested edits should just receive the full edit privilege. I don't know how many approved suggested edits is enough, but maybe after 500 you get the full-edit privilege.
    – Dharman
    Commented Mar 9, 2023 at 18:13
  • 13
    @Mithical: That's a great writeup – I'd encourage you to post it (or some version of it) as an answer if you can :)
    – V2Blast StaffMod
    Commented Mar 9, 2023 at 19:02
  • 16
    Regarding the Suggested Edits queues being full, I proposed a role for that here. On Arqade there is a user who almost exclusively has marked duplicates. I would love to see them being rewarded for that, as it becomes increasingly important as a site grows (as well as harder).
    – Joachim
    Commented Mar 9, 2023 at 21:04
  • 7
    @Joachim I'm pretty sure I'm familiar with that user, and yes, absolutely. The work they've done in cleaning up old duplicates and ensuring newly-posted ones have a good duplicate proposal has been really impressive.
    – Spevacus Mod
    Commented Mar 9, 2023 at 21:08
  • 6
    I think gating certain privileges behind reputation works, but we should scale up the frequency of using those privileges based on usage of those privileges. Just like how you get more flags by casting useful flags, and similar to how you get more delete votes per day as you rise in reputation beyond 10k/20k, you should get more close votes or more review 'rounds' the more helpful actions of those types you perform. This would allow for hybridization of things like suggested edits: you can suggest an edit any time, but starting at 500 rep, you get 1 unilateral edit per day; at 600 you get 2...
    – TylerH
    Commented Mar 9, 2023 at 21:48
  • 3
    If users still had to suggest edits at 2k reputation, for example, but could make 15 edits per day before having to start submitting them to the review queue, I think they'd be a lot less frustrated due to running into the "the queue is full" issue so often. And once you've done 1,000 reviews in a queue (assuming some additional metric is met as well, like < n review suspensions during that time), you earn additional daily reviews. Doesn't have to be a lot, but something like 80 or 100 max instead of 40 would make a big difference and go a long way toward encouraging users to review more.
    – TylerH
    Commented Mar 9, 2023 at 21:51
  • 3
    RE: Incentivizing duplicates - maybe only award reputation if a proposed duplicate is accepted by the Asker or a Moderator, not just because enough people suggested the same. This skews towards under-rewarding when a moderator doesn't get to it in time or the asker is stubborn about their question being unique, but I think that's a good thing.
    – David K
    Commented Mar 10, 2023 at 13:20
  • 3
    @reirab Of course! But there must certainly be better metrics we can gauge a user on when determining whether they're ready for certain moderation privileges than reputation alone, and that's the idea I think is worth exploring.
    – Spevacus Mod
    Commented Mar 10, 2023 at 20:02
  • 26
    fantastically well written and insightful post 👏 Commented Mar 11, 2023 at 19:06
  • 10
    @JeffAtwood That means a lot coming from you, Jeff! Thanks!
    – Spevacus Mod
    Commented Mar 11, 2023 at 19:43
  • 3
    "the more saturated a given site becomes with good quality Q&A content, the more difficult it is for newer users to find a place where they can contribute meaningfully." 100% yes. For a microcosm of this issue, look no further than small Teams sites, e.g. the Moderator's one. Most of the general questions have been answered years ago, such that new mods (when they get access) do not have any rep and can't do stuff like downvote. (Incidentally, the mods worked around this by having a question where the sole purpose is to post answers to farm rep so we can downvote!)
    – Robotnik
    Commented Mar 13, 2023 at 6:20
  • 3
    I'd also agree with this suggestion. Just as an example, on SO, I have 434 rep right now. I also have 1,373 post flags (spam/rude flags excluded), of which 1,189 of them are helpful, four are declined (and 1x disputed, 145x aged away, 33x retracted, 1x waiting for review). My most recent declined flag on SO was about six months ago. IMO, I have a pretty good idea of what constitutes a good post and what needs to be closed, and having CV privileges + queue access would be useful, as I spend vastly more of my time on SE doing moderation-related tasks than posting questions/answers
    – cocomac
    Commented Mar 13, 2023 at 8:07
  • 5
    Upvoted esp. re: new-user rep-farming to try to get to mod abilities encourages answering duplicates, and little-to-no incentive for correctly identifying and citing duplicates instead. (I've seen myself being aware of target dupe[s] but answering instead to try to get to required-rep threshold, esp. because the 'suggested edit queue full' discourages me from trying to improve existing content. In combination with new-question-askers being frustrated with their question closed as duplicate, and struggling to figure out how to adjust the target dupe code to fit their scenario, bad experience.) Commented Mar 14, 2023 at 13:57
  • 2
    As an anecdotic evidence, I also don't really trust this system. I have only ever asked one single question on Lifehacks.SE, and don't recall any other contribution; however, that question went viral and I got more than 1k reputation from it, granting me some basic moderator-ish permissions (review queue etc.). For asking one single question, that wasn't even that good, only relatable for many.
    – Neinstein
    Commented Mar 16, 2023 at 10:24
101

Just a friendly reminder that these proposals exist:

11
  • 14
    You basically say that instead of asking for new ideas SO should "do their homework" and look at already posted ideas in the past first? Commented Mar 10, 2023 at 8:09
  • 41
    @Trilarion I'm not against asking for new ideas (where'd you get that idea?). But I also hope that old ideas aren't being forgotten or neglected- both by SE Inc. and by other community users.
    – starball
    Commented Mar 10, 2023 at 8:20
  • 2
    I got the idea from you posting old ideas to a post that seemed to ask for new ideas and accounting for that the staff likely cannot do all those things at the same time (thinking about old and new ideas) but I didn't want bother anyone. New and old ideas - the more, the merrier. Commented Mar 10, 2023 at 8:39
  • Strong -1 for these proposals. I regularly see users blindly close vote or leave 'possible duplicate of...' comments on questions that absolutely aren't duplicates, but perhaps have some of the same words in the title.
    – dauphic
    Commented Mar 14, 2023 at 19:04
  • 2
    @dauphic the dup-closure system requires 3-5 users with the close-vote privilege to agree that the dup is a dup. The first listed proposal provides zero incentive for pile-on close-voting. The second listed proposal has a lesser incentive for non-first dup-votes and a punishment mechanism for wrong duplicate-closure. Don't those address your concerns?
    – starball
    Commented Mar 14, 2023 at 19:09
  • No: the problem is that many users close vote without reading or understanding the question content. It's not just a lone close vote here and there, and I suspect others are willing to blindly throw their vote in when they see 'possible duplicate of [similar wording]' from a high reputation user. That aside, it's also a bad new user experience, even if the question isn't ultimately closed.
    – dauphic
    Commented Mar 14, 2023 at 19:21
  • 3
    @dauphic how big is that problem compared to askers and answerers not searching for duplicates and the resulting fragmentation of information for all users of SE? I'd wager not as big. I've heard my fair share of complaints about wrong dup-closure, but consider that rarely does anyone praise a system like SE when it works (invisibly for most users) for their good. We just hear the complaints when things happen wrong, and I'm not convinced they happen wrong more often than right (data needed).
    – starball
    Commented Mar 14, 2023 at 19:32
  • The reopen system is a thing and I'm pretty sure mod-flagging patterns of bad closure is valid. To users with dup-vote privileges, closure is public, non-repudiated info, discussible on meta sites. Not that I don't agree that wrong dup-votes is bad though.
    – starball
    Commented Mar 14, 2023 at 19:32
  • 4
    @dauphic in my experience as a curator, it's relatively rare to see questions closed that shouldn't be. It's much more common to see questions closed with the wrong canonical, and this often happens because the right one is hard to find, largely because of previous failures to close questions that should be closed. Commented Mar 15, 2023 at 1:02
  • I've seen more than a few questions closed on other Stack Exchange sites because users either didn't like the question content or just piled on. Question: "How do I do X?" comment: "it depends, what is your Y?" question closed as needing information comment: "hang on, all X's are mostly the same regardless of Y, this is already answerable with different Y's being a footnote" Commented Mar 16, 2023 at 16:06
  • Duplicate questions are a minor annoyance, essentially an offense against taxonomy. However incorrect identification of "duplicates" is a much bigger problem, creating hostile unhelpful environment, and in extreme cases leading into irreversible mistakes, such as merging of unrelated questions, and subsequent mass downvoting of (now retroactively off-topic) answers from the source question. On some sites (particularly SO) its a plague. Encouraging more of this kind of behavior is a horrible idea, and the fact that you've turned up 6 proposals for doing so is proof of the problem.
    – T.E.D.
    Commented Mar 17, 2023 at 13:39
101

It’s better than nothing… but I have many complaints regardless.

The following is a rough sketch listing some observations and ideas about the reputation system I had accumulated over the years. This was initially written in some haste, and even as I refined this post over time, I sometimes avoided writing explanations too elaborate, since just the number of problems mentioned can be overwhelming enough. As such, please forgive some simplifications and leaps of logic below. (I can also recommend @Karl Knechtel’s answer with a similar list, but I think much better written, and it manages to cover a few issues I overlooked writing this one; yet another case of the FGITW problem.)

A number of other answers here express the sentiment that “oh, reputation doesn’t really mean all that much”; suspiciously often, it seems to come from users who have accumulated tens or hundreds of thousands of points. That makes it sound just as believable as a billionaire telling you that money doesn’t buy happiness; well, I am not falling for this trite so-above-it-all and holier-than-thou humblebrag countersignalling. This post takes the stance that reputation influences at least some people’s behaviour (especially those nearer the bottom of the ladder), and that’s enough for it to matter; this impact better not be overlooked.

So here goes:

  • Reputation concentration (the early-vs-late adopter problem): in early stages of a site, users can easily earn reputation by asking (and answering) basic questions, then earn “passive income”, as those questions accumulate views and duplicate links. This means reputation is awarded to early users disproportionately to their level of expertise, merely by virtue of them having asked or answered popular, low-hanging-fruit questions in the past.

    Meanwhile, for latecomers it can be hard to accumulate enough reputation to use a site comfortably, as their questions get closed as duplicates of the former… even though the older questions aren’t necessarily all that good themselves once you look closely: they implicitly mix unrelated concerns together, fail to state background assumptions or define circumstances in a sufficiently concrete way, are ambiguously phrased, attract redundant answers and ones addressing completely unrelated circumstances to the asker’s (the “Long Tail of Crap” problem) due to poor wording, and sometimes have bad titles. What is worse is that the administrators, moderators and the wider community tend to be apathetic towards those problems and sometimes actively thwart attempts to remedy them.

    The dearth of opportunities to earn reputation is very visible on Stack Overflow, where the only way to gain enough points to meaningfully participate and not be written off as a newbie is by collecting bounties; which is not all that easy, as bountied questions tend to be ridiculously specialised and demand complicated set-ups to even begin replicating the issue at hand. And even those are slowly drying up at this point: I remember there used to be 16 or so pages of unanswered bountied questions on SO. Now it’s just 9.

  • Perverse incentives: Stack Exchange is basically Goodhart’s Law: The Website.

    • Fastest Gun in the West: The well-known problem where early answers get the most upvotes and the most visibility, both of which reinforce each other; but usually not in proportion to quality. This encourages writing answers hastily and sloppily.

    • Beating dead horses to undeath: Highly-viewed questions attract redundant, poor-quality answers long after they have already been exhaustively explained. (“Protecting”/answer-locking a question ameliorates this, but is not always available.)

    • Encouraging unscrupulous answerers and selfish askers: Answers to poor questions can nevertheless earn the answerer some reputation, which encourages wild-shot answers, discourages voting to close unclear questions, and removes askers’ incentives to put some effort into improving the question, especially if they don’t care about their own reputation score (since they are likely to get the answer they want anyway). It also adds another aspect to the Fastest Gun problem: users voting to close a poor question have to race against unscrupulous answerers who hope to gain reputation from undiscerning upvoters (more on the latter below). Additionally, if the question is eventually closed regardless, the fact that it has answers makes it ineligible for automatic deletion, which contributes to the “late deletion is too late” problem below.

    • Making crap sticky: Because downvotes are so underpowered relative to upvotes (more on that below), even a negative-score post will often on net earn the poster reputation. This means that posters, especially those at the bottom of the ladder, are disincentivized from removing posts judged as bad quality by the community. Privileged users may cast delete votes on negative-score answers, but negative-score questions are harder to get rid of, as they need to be closed first – and that can be hard, since a question can be bad without falling under any of the rather narrow general closure criteria. Although a write-in reason can be entered, (i) it will not have the same air of legitimacy the stock criteria have, which makes the vote less likely to succeed, and (ii) it is posted in the voter’s name instead of anonymously, which encourages retaliation.

    • Central Meta groupthink: There is also one problem (mostly) exclusive to the central Meta site, which is to say, the very https://meta.stackexchange.com/ where I am posting this. On meta sites, votes tends to indicate subjective agreement instead of vouching for objective quality. This is fine, or at least not as bad, on site-local metas where reputation on the associated regular site is used for display and determining privileges, but the central Meta has no regular site to be associated with, so the reputation on it is derived from scores of posts on the central Meta itself. Now, any kind of voting/karma system will to some degree promote groupthink and discourage the posting of unpopular opinions, even if only at a psychological level, but tying post scores to on-site privileges pushes this much further.

      The only people who can get away with writing negative-score posts on a regular basis here are the staffers, and boy, do they ever run with it.

  • Stacking the deck against downvotes: A number of mechanics rig the voting system so that more upvotes are cast than downvotes, and the impact of the former on the recipient is disproportionately larger. This was apparently meant to prevent and mitigate various potential misuses of downvotes, but there was no similar consideration of the harms from poor use of upvotes.

    • Asymmetry of suffrage: The reputation threshold to be able to cast upvotes on answers is very low (15 points on most sites), much lower than the threshold to cast downvotes (125 points on most sites), which means the ratio of upvotes to downvotes is skewed by upvotes cast by low-reputation users, who have not been vetted by the site’s community as having expertise, and are less likely to have internalized how the site is supposed to operate.

    • Asymmetry of reputation costs: It costs reputation to cast downvotes on answers, which means the few users who can cast downvotes are disincentivized from it anyway. The cost of downvoting was meant to entice users to carefully consider casting downvotes and discourage casting them indiscriminately, but there is no similar countermeasure for indiscriminate casting of upvotes. There is not even a penalty for upvoting a spam post.

    • Asymmetry of reputation gain and loss: It takes five downvotes to nullify a reputation gain from a single upvote. It takes three more to nullify a reputation gain from answer acceptance alone, even though the asker is often a newbie, and therefore the least qualified person to judge how good an answer is. This means answerers are rewarded for posting superficial answers, and are not incentivised to correct mistakes discovered by outsiders that may be easily missed, yet important.

    • Ballot stuffing from HNQs: Hot Network Questions and the association bonus distort voting further. Answers to HNQs get much more visibility and lots of upvotes from passing visitors who may not even be experts in the particular site’s subject area, or familiar with its house rules, but happen to have the +100 bonus from participation elsewhere and upvote based on superficial plausibility.

      I often see this dynamic on Retrocomputing. On ELU, answers to single-word requests are an embarrassment to read, also, I suspect, primarily for this reason. But the real poster child for undue HNQ interference is probably the LangDev site: it used to attract a lot of opinion-based, bikeshedding topics, still kind of does, and did so even before it launched its public beta. The tragedy of LangDev has many aspects, and not all are related to reputation, but to the ones that are, the association bonus has certainly contributed a lot.

    • Killing not just democracy, flawed as it is, but plutocracy as well: The skew towards upvotes manages to spoil even the bounty system. Because of the large number of users who are not discerning with their upvotes, it’s not that uncommon that a mediocre-but-not-outright-awful answer (or worse) accumulates a score of 2, which is enough for it to get the bounty automatically, regardless of the bounty poster’s wishes. This means that bountied questions are subject to popularity-based perverse incentives against answer quality just as much as (if not more than, because of the higher stakes) normal questions, with little possibility for a bounty poster to correct for that. There used to be a way to decline awarding a bounty to anyone else, but now this can be accomplished only through sockpuppetry.

    • Late deletion is too late: Imagine you encounter a question posted several years ago. It has one answer with a score of 8 or so, reasonably decent and detailed, written soon after the question was posted by someone with hundreds of thousands reputation points. There is also another one with a score of 2, focusing on one particular aspect of the question, written a few days later by someone with just a thousand reputation points. There is only one problem: the question is awful. The title is “Please help debug my program I am beginner”, it had a score of −1 before you found it, and the body is just a large code dump followed by a messy explanation of five unrelated issues, two of which turn out to be trivial typos. It should not have been posted in this form in the first place, and even if editing could have salvaged it, the asker is long inactive. It should have been closed on the spot, as unfocused, or as based on a typo, or maybe as a duplicate of three separate questions addressing the non-typo aspects. Perhaps there even was a vote to close it, but the question only received four of five required votes, so the closure did not happen before the votes expired.

      Suppose you manage to get this question closed and deleted today. There is an obscure mechanic that skews reputation further: although normally users lose reputation earned from deleted answers, if the answer is older than 60 days and has a score of at least 3, the reputation is preserved – even though if a question is deleted, then it means it should not have been asked and its answers should not be rewarded, no matter how late deletion happens. In the example above, neither answerer deserves their reputation, but the one with hundreds of thousands points will keep it all, while the one with just a thousand will lose what little they got, thereby exacerbating the “reputation concentration” problem. (Now imagine this happening several times.)

      On Stack Overflow, I run into this kind of situation with some regularity. Except most of the time I cannot even get the question closed, never mind deleted.

    • Toxic positivity: The nerfing of downvotes that was probably naïvely meant to prevent toxicity and encourage positive interactions has non-obvious deleterious psychological effects in the long term. Readers who are unaware of the quirks of voting mechanics and their long-term effects may have an inflated perception of a given post’s worth, while the posters may similarly have an inflated perception of their own expertise; the latter can breed arrogance. Users who notice problems with content feel powerless to act on their observations as their downvotes are penalized, drowned out by indiscriminately cast upvotes and therefore ineffective; they become frustrated and lash out. The lack of systemic solutions creates the feeling that nasty comments is the only kind of negative feedback that will be noticed. (I am no saint here myself.) And so, despite this ostensible effort to prevent toxicity, Stack Overflow manages to be incredibly toxic place regardless.

    Some particularly egregious examples of vote skew:

    It’s the wiki whack-a-mole: invite a lot of newbies to add to the site without first teaching them about research and quality standards, then leave overwhelmed and frustrated curators to drown them in negative feedback, at best wordless or passive-aggressive by virtue of being impersonal canned boilerplate, at worst passive-aggressive by being handcrafted for the occasion. I suspect we are in the worst of both worlds now: newbie askers feel attacked because they see negative numbers next to their questions, while curators feel ignored because their negative feedback, even when thrice as numerous as positive feedback (which is rare), ends up being ineffective and swamped by the latter on the reputation balance sheet anyway.

    The number of people who say they never cast downvotes no matter what convinces me that discouraging downvotes does not really correct for a claimed bias, but it exacerbates an actual opposite bias that would have been problematic even without this measure. The Java fanatic downvoting everything .NET is nothing but a contrived bogeyman – who actually votes like that?

    There is also the bullshit asymmetry principle to take into account: it has always been easier to invent claims that are wrong, but sound plausible to an untrained audience than to conclusively debunk them in a way that is convincing to that same audience (which usually means: from first principles, without relying on shortcuts an expert would take for granted and easily fill gaps within); and with the advent of LLMs, generating large amounts of bullshit has become easier than ever.

  • A measure of popularity, not quality, nor information value (aka the dev.to problem): An answer in a highly specialised area where very little information is otherwise available will probably receive just a couple of upvotes. Meanwhile, an answer about basics one might as well look up in Wikipedia can receive hundreds. In itself, it may be argued that this is intended: if an answer is useful to more people, then it’s more useful to the knowledge base, and the author should receive more credit for it. But given that votes are not necessarily cast by experts (see above), this means that posts will receive upvotes merely for covering a popular subject area, without necessarily covering it correctly. And someone actually knowledgeable in a niche area can have a hard time getting their expertise recognised; having barely a couple hundred of reputation points makes them easily equated with a newbie.

  • Naïve scoring of individual posts. Computing post score as upvotes minus downvotes is nonsense. An answer having no votes at all is not worth the same as one having two upvotes and two downvotes: the latter is likely (given the previous point) to be something highly misleading which happens to have been upvoted by a couple of newbies who just don’t know better.

  • No reputation for moderation or curation. There is no reputation to be gained from moderation tasks (raising flags, reviewing posts, identifying duplicates) or updating outdated answers, which discourages people from performing those otherwise rather dreary tasks. Only low-reputation users can earn reputation from suggesting edits to posts, and it’s a measly 2 points per approved edit. This also encourages writing redundant answers (see above), as there is more reputation to be earned from writing your own answer than from editing an existing one, even as a full answer may be not warranted.

All those problems mean that reputation is not a very meaningful or accurate measure of either expertise or contribution to the site, and not a very good incentive towards meaningful participation.

My proposed remedies:

  • Make separate downvote and upvote counts on answers visible to everyone, instead of making it a privilege. Especially that a userscript can circumvent this anyway: "View Vote totals" without 1000 rep. There is a good reason people hate YouTube for removing the dislike count.
  • By default sort answers based on some kind of Wilson score (shameless plug), or other more meaningful statistic, not naïve upvote minus downvote score.
  • Raise the reputation loss from receiving a downvote on an answer. Perhaps introduce an incremental penalty: first downvote subtracts 2 reputation, second downvote subtracts 6, third and all subsequent ones subtract 10. (Not making them equal from the start should soften the blow from tactical downvotes cast for spurious reasons, which I expect to be rare.)
  • Raise the reputation threshold to cast upvotes on answers to the same level as for casting downvotes.
    • At the very, very least: make the reputation threshold to upvote answers not take the association bonus into account. The bonus should only grant privileges like commenting and flagging, that are subject to moderation review.
  • Implementing the previous point will likely incentivise now-“disenfranchised” users to upvote questions more instead (which we might want to encourage anyway: questions tend to be under-upvoted), which may skew question scores upward; to counter this, lower the reputation gain from a question upvote back to 5 points. Maybe to a bit higher level, but probably no more than 8 points.
  • Lower the reputation reward from answer acceptance to just 5 points. Unpin the asker-selected answer from the top on all sites.
  • Implement sub-linear reputation gain from upvotes, e.g. logarithmic (the first n upvotes give you 10 reputation each, the next n upvotes give you 102 = 5 reputation each, the next n upvotes give you 103 reputation each, etc.) or square-root (getting the first n upvotes gives you 10n reputation, the next 2n upvotes give you 10n reputation, the next 3n upvotes give you 10n reputation, etc.). This should solve the “too much reputation from just one very popular post” problem better than the daily reputation cap.
  • Deny all reputation to answers on negative-score questions; possibly also on positive-score closed ones.
  • (Much later edit:) In the answer author’s “signature box” at the bottom, display the user’s highest absolute score and relative rank in the particular question’s tags, preferably more prominently than overall reputation score.
  • (Even later edit:) Bring back the option to decline awarding a bounty. Automatic awarding should only be used as a measure against inattentive bounty posters.
  • (Much, much, much later edit:) Introduce a reputation penalty for upvoting a post that is later deleted, whether by community vote or unilateral moderator action: a smaller one (say, −2) for questions and larger one (maybe −4 or −6) for answers. Or perhaps a smaller one for community-vote/automatic deletions, and a larger one for moderator deletions. Or perhaps have the penalty scale with the number of downvotes instead of having a fixed one per post. This should make the system fairer by taking into account that if you upvote a post that turns out to be useless or downright harmful, it reflects poorly on your judgement and expertise. It would balance out the cost of downvoting, and it should also disincentivize bleeding-heart, indiscriminate upvoting and encourage users to review posts more thoroughly before upvoting them.

Not sure what to do about the moderation bit. Also, perhaps there should be separate reputation scores for subject-matter expertise and for moderation tasks, as those are definitely separate kinds of skills, but I only have the vaguest ideas how that would look like.

(Even later edit:) Also, we could somehow encourage closing earlier questions as duplicates of later ones. After all, just because a question was asked first doesn’t mean it’s a better one: questions on Stack Overflow are often formulated badly and mix unrelated concerns in one post, which sometimes gets unnoticed right after it’s asked, and only becomes apparent when answers accumulate. This could also transfer some attention, and therefore reputation, to newer users, instead of having it accumulate with long-inactive users, thereby ameliorating the “reputation concentration” issue.

These are, I admit, rather bold proposals. Do I expect any of them to be implemented? No, I don’t. I expect Stack Exchange Inc. is going to do with them exactly the same thing as with all community proposals: ignore completely, or perhaps even implement their complete opposite (given that they attempted to force through the 1-reputation voting experiment, despite immense pushback to the contrary). In fact, even if they were to be implemented exactly as laid out above, I would expect them to be pretty unpopular (despite the rather high number of upvotes this answer has received); they would turn lots of people who feel entitled to their meaningless Internet points very upset. The best case, I think, is that they will be read and taken into consideration by whoever builds the next Q&A website from scratch.


0 Since then, as of late October 2023, the answer has earned three more upvotes, only one downvote, while the comment received five more upvotes. If the meta effect has made any difference here at all, it seems to have made the problem worse.

30
  • 3
    "Make separate downvote and upvote counts on answers visible to everyone" This is done for performance reasons. While it is possible to get the vote counts, I would not make it obvious to everyone how to do it as it would lead to more load on the servers.
    – Dharman
    Commented Mar 9, 2023 at 18:22
  • 19
    Implementing logarithmic reputation gain for upvotes would incentivise me to post a few thousand mediocre answers, rather than a couple of good ones.
    – wizzwizz4
    Commented Mar 9, 2023 at 18:34
  • 7
    @wizzwizz4 Upvotes measure popularity, not quality. An exceptionally good answer can be rewarded with a bounty instead. Whether you will get one is another matter… Commented Mar 9, 2023 at 18:41
  • 5
    This is true – but incentives are as much about people's perceptions as reality.
    – wizzwizz4
    Commented Mar 9, 2023 at 18:44
  • 18
    @starball Not always, but quite often, and even with how-to questions on a technical site. A newbie asker can accept FGITW solutions that neglect multi-codepoint Unicode grapheme clusters, or fail on file names containing spaces, or are susceptible to SQL injection, because it didn’t even occur to the asker that such problems exist. And often enough, the asker posts their question precisely because they’re a newbie. More experienced people can notice such subtleties and react accordingly. Commented Mar 9, 2023 at 19:04
  • 2
    @Dharman That's right, but I feel only because they are hidden to begin with, and every view needs to fetch that information. If the score is transparent from the beginning, this might not be the case (my reason for thinking this is that the result of the upvotes gets parsed (almost) straight away as votes come in, and I can't imagine fetching the amount of votes will be that hard on resources).
    – Joachim
    Commented Mar 9, 2023 at 21:33
  • 2
    @Joachim No, that's not it. Score is cached. You can check the SEDE schema and see that Posts has a column Score. But upvotes and downvotes need to be calculated separately from Votes table.
    – Dharman
    Commented Mar 9, 2023 at 21:39
  • 12
    @Dharman that's my problem with hidden votes. How difficult/costly is to store ups and downs instead of caching a total? Can't see a reason to not store/cache both.
    – Largato
    Commented Mar 9, 2023 at 22:11
  • 13
    @starball I don't know that I agree with that. Newer users often accept the solution that works, or the first one that works. Just because an answer works doesn't even mean it's a good answer, never mind the best one. All too often I have seen people spend much more time on elaborate and better answers, and even post comments indicating why the accepted answer is inferior, but the OP got their quick answer and is gone. Commented Mar 10, 2023 at 5:52
  • 3
    There's actually a badge for good answers on negatively-scored questions. Commented Mar 10, 2023 at 10:36
  • 12
    @starball "But if it's a fairly simple "how" question on a technical site, usually answers either work or don't work," not necessarily true. Often there are nuances. An O(n^2) solution vs O(n) for example - both work but one works worse than the other. Also you seem to overlook partial solutions. Where an answer may not solve the general case but only works for the sample input in the question. And we know it's a partial solution also because OP accepts it...and then complains it doesn't work for other input. So "how" is more than a boolean "works"/"doesn't work".
    – VLAZ
    Commented Mar 10, 2023 at 14:15
  • 24
    By removing the downvote count, YouTube killed the value of the upvotes as a quality reference. Nowadays, it's just a bonus that we give to the channel owner, we have to gather the quality scanning the comments section.
    – brasofilo
    Commented Mar 11, 2023 at 2:34
  • 4
    @NotThatGuy Maybe it wouldn’t. But it might actually address at least some sources of frustration: with downvotes so de-powered, easily brushed aside and swamped by upvotes from people who don’t know better, people may feel leaving nasty comments is the only kind of negative feedback that will be heard and acted on. Seeing downvotes do something might actually calm them down. But even this said, I don’t think reforms of the reputation system alone can fix this problem. Commented Mar 11, 2023 at 9:19
  • 3
    @user3840170: The Wilson score corrects the uncertainty of a few number of votes, it does nothing to solve the issue of a witless mob. What about weighing votes based on the log10 of the reputation of the voter at the moment the vote is cast? The single downvote from a 1000+ user would nullify 100s of upvotes by 10+ users. Commented Mar 11, 2023 at 15:37
  • 6
    '"Make separate downvote and upvote counts on answers visible to everyone" This is done for performance reasons' - Are there any tests showing the performance degradation if this feature is enabled? As pointed above by previous comments, there are ways to do it without sacrificing performance, and unless I see a good test showing I'm wrong, I simply don't buy it.
    – hkotsubo
    Commented Mar 12, 2023 at 22:02
73

What is really great about our reputation & privilege system? What do you think makes it great, and why is it worth keeping?

You need some way to sort the noise and votes are a decent way to do that. We also want to reward good posts and posters. You can earn privileges this way. The system does well here.

What is broken about it, and why? Are there any solutions to it?

How long you got?

There's some odd distortions the system creates with regard to reputation. The first would be the false dichotomy of "high score/rep means good quality". I've seen users in the last month who insisted that their decently scored post indicated it was good quality. I've seen terrible posts that got lots of upvotes. I've even seen rep used to feed posts that are blatantly off-topic (the worst would be service outages).

Then there's the people for whom reputation = life. I've suspended sock puppet masters, only to have them come begging for the lost rep back. One literally told me that his getting a job in India depended on having high rep on Stack Overflow. Wat?

There are too many people who want reputation for... reasons. As I've noted elsewhere, I have seen a direct pattern of people who get into vote fraud, get caught, and then switch to plagiarism (or vice versa). Maybe they like the big number. Maybe they want privileges. ChatGPT has just piled onto that list. It's hard to say. There's not always a clear motive there.

I'm not sure there's any solutions that don't exacerbate the problem.

What other systems work really well that we could learn from?

  • Non-rep privileges. Want to do some rep-based things? Maybe take a test. We have no shortage of people who earn privs and then lose them through bad usage. it's not a great model. Maybe create a way to train anyone who wants to review
  • Tag badges - We give you a nifty privilege for earning a gold one. What about silver and bronze?
12
  • 7
    My pet peeve in this regard is that a lot of users seem to judge post quality by weight (size of the post). I'm not sure what could be done (reasonably) to fix that though.
    – T.E.D.
    Commented Mar 9, 2023 at 21:41
  • 9
    Statistically speaking, at least on a site that isn't purely opinion-based like Meta, it's vastly more likely that a long post will be of higher quality than a very short one. Obviously, there's no shortcuts to actually reading the post and directly assessing its quality, but as far as heuristics go, length is a pretty good one, hardly anything to pet your peeve about.
    – Cody Gray
    Commented Mar 10, 2023 at 0:16
  • 8
    "high score/rep means good quality" I don't think that is a false dichotomy. It's definitely a positive correlation between score and quality, just not working all the time. For people who disagree: just use random sort (date created might be a good proxy) instead of highest score first for answers for example and see if you get better information faster from the site. Commented Mar 10, 2023 at 8:22
  • 1
    The motive could be fear. Maybe the reputation points requirement (or online presence in general) for getting a job is a myth (not actually true in most cases). The plagiarism may be a means for getting the reputation points by the least amount of effort (enter words from the question into a search engine (or ChatGPT) and blindly enter the result as an answer). Commented Mar 10, 2023 at 11:02
  • 1
    "Then there's the people for whom reputation = life" - I have been paid six-digit dollars because of my Stack Overflow reputation. What happened to Stack Overflow Jobs, anyway? Commented Mar 10, 2023 at 23:28
  • 3
    I have significantly more reputation than you, @user253751, but I've been paid a grand total of zero dollars because of my Stack Overflow reputation. What am I doing wrong? (Seriously, I don't understand these claims people make about reputation having some value for getting a job or whatever. I've had a high reputation on SO for years now, even while actively looking for jobs, but it hasn't helped me at all. No one cares. It didn't/doesn't even help me to overcome my lack of a CS degree by, oh I don't know, proving that I know what I'm doing even though I majored in something else.)
    – Cody Gray
    Commented Mar 11, 2023 at 11:47
  • 5
    @CodyGray What you are missing is the very good luck to get targeted by someone who is recruiting through Stack Overflow. Perhaps you even still think we live in a meritocracy. Commented Mar 11, 2023 at 14:55
  • re: badges. I would think this has the same problem as rep, which is a sort of "grandfather" issue. One really good question or answer, many years ago, got you a nice badge. Even finding that kind of Q or A in a long established site today is much harder to do. Maybe kind of boils down to an issue of "what have you done for me lately".
    – JamieB
    Commented Mar 13, 2023 at 15:34
  • @JamieB are you aware that you should have several post in the tag to be able to get the badge? And it's the aggregated score of all posts under the tag? And that questions doesn't award badges?
    – Braiam
    Commented Mar 15, 2023 at 16:21
  • @T.E.D. I think that's true for answers, I'm not convinced it's true for questions. I've found that a succinct question often gets better answers than a long-winded one. That's why we include "minimum" in [mcve] too. Commented Mar 17, 2023 at 10:19
  • 2
    @simonalexander2005 - That's an interesting perspective. I'd generally go the other way. On History we routinely ask question authors to provide details of all the reasearch they've already done (so 50 users don't waste t*50 time running down the same leads the asker already looked into), and where they got the info in their question (so 50 users don't waste t*50 time double-checking their interpretation), and by the time they've done that a good question will have a decent amount of meat in it. Some short words about their goals in asking can be helpful too.
    – T.E.D.
    Commented Mar 17, 2023 at 13:47
  • @simonalexander2005 minimum doesn't mean shortest, it means that it doesn't have extraneous irrelevant information. The problem is that most askers don't know what's relevant and what not, so I prefer them to default to giving too much, so we can polish and clean the diamond, rather than having a chunk of graphite.
    – Braiam
    Commented Mar 22, 2023 at 21:14
66

I was asked to contribute this as an answer, so here goes. I'm going to directly quote from a blog post I wrote in 2020:

The problem with Stack’s approach

Stack Exchange — which Codidact largely grew out of — awards various privileges based on your reputation score, which is calculated largely by votes on your posts.

This… is a bit problematic, in a couple different ways.

First off, reputation itself is wearing way too many hats. It’s simultaneously supposed to be acting as:

…a general indication of a user’s proven expertise on the site’s subject matter. (More simply put: It’s supposed to give you a rough idea of how much that user knows about whatever that specific site is about. A high reputation score on a site such as Math shows that you probably know what you’re talking about with math-related stuff.)

…a general indication of a user’s familiarity with the site mechanics. This is why reputation unlocks privileges, and why users who have 200 reputation on another site are automatically granted 100 reputation when they join another site on the network.

…a gamification incentive. Also known as “Pointz”. People love earning points for their contribution; reputation serves as a gamification incentive by awarding people points when they get votes. (See also: Reddit karma; Imgur Internet Points.)

As a result of trying to do all these things at once, it winds up doing all of them… less than ideally.
Anyone who’s paid any attention to Stack Overflow Meta will notice that there are users with thousands of reputation points who have absolutely no idea how site mechanics or policies work. Sure, they’re probably good coders, but they don’t know how to edit a post or what the heck a “review queue” might be.
You can also earn up to 1,000 reputation by suggesting edits on a site. I’ve personally earned a bunch of reputation across various sites that I know nothing about, just by suggesting edits… but if I decide to post, my reputation score might lead people to think I’m more familiar with the subject than I actually am.
It works for gamification pretty well, I suppose. Not perfectly, but good enough… and getting into that is really off-topic for this article.

Circling back up to what I just said, I want to emphasize the important part: “…there are users with thousands of reputation points who have absolutely no idea how site mechanics or policies work.”
That is the problem… because reputation points give privileges.

As users with thousands of reputation points, they have access to moderation tools such as voting to close posts, voting to delete posts, unilateral edits, and more… because they proved they know how to answer questions.

They don’t have those privileges because they’ve showed they know the rules about when to vote to delete answers. A lot of them have no idea what the policies for deleting posts are, or how the privilege (that they have) works. They earned reputation through answering questions, and suddenly have the ability to vote to delete posts.

There’s another way

So now we’ve identified the problem: Users earning privileges through actions that are entirely unrelated to the privilege itself.

And now we have an obvious solution.

Users should earn abilities and privileges through actions that are directly related to the ability they are earning.

The rest of the post goes on to detail how we went about implementing this idea over there, but since we're talking about changing things here on SE, I can contribute a few more thoughts in that direction.

Rep is wearing too many hats; let's take at least one of them off.

Privileges, for the most part, should not be tied to reputation.

This is a point others have raised, mostly focusing on edits. I'd like to expand on that somewhat.

Perhaps the best example in the system we have on SE today is flags. You earn the privilege to raise a flag at 15 reputation[1]. When you earn the privilege, you have a maximum of 10 flags per day[2]. As you raise flags, you earn more flags to raise per day if your flags are being marked helpful, while you run into warnings and flag-bans if your flags are being declined.

This is how the system should work; as you prove that you know how to properly use each system, you should be trusted more within that particular system.

Extending that thought, if enough of your suggested edits are being consistently approved, you should earn the privilege to edit directly. If your close flags are being marked helpful and the posts that you flagged are getting closed, you should be able to vote to close directly. If your NaA and VLQ flags are consistently helpful, you should be able to VTD directly - and as your close votes and delete votes result in consensus, you should earn more per day.

While it makes sense to set a baseline level of reputation to be able to begin earning those abilities, in order to ensure some level of familiarity with the site and not to overrun people with poor review items[3], we're currently locking a bunch of people out of helping with moderation just because they aren't prolific contributors to Q&A, without taking into consideration any effort they've put into other aspects of the site.


[1]Aside from custom mod flags on your own posts, which you can raise at any rep, IIRC.

[2]Well, 10 flags for posts and 10 for comments, since they're counted separately.

[3]Although it's a fair argument that that ship has already sailed.

8
  • Who does the approvals in this system? I'm imagining a full system reset where we all start off with flagging right and....presumably some poor moderators beset with handling them because no one else has that privilege yet? Maybe we unlock the whole shabang. Everyone can flag. Everyone can answer flags. But the bars are all raised. If I VTC then it goes into the queue, which anyone of any rep can handle, but it needs more agreement (and/or a certain ratio) in order to succeed. Hmm. Just spitballing here really. But I'm wondering how to launch from 0 with an already popular site.
    – JamieB
    Commented Mar 13, 2023 at 16:04
  • 3
    @JamieB: Mods are the only ones who can actually see flags anyway (and that wouldn't change as part of Mithical's suggested system) – and certain flags are automatically handled by the system (e.g. by causing the post to appear in a review queue). But also, the proposed system wouldn't have to "start from scratch", with everyone limited to suggested edits again regardless of their rep; we could potentially use each user's previous actions as a metric by which to recalculate folks' access to privileges, based on the same metrics by which we'd determine that access in the future.
    – V2Blast StaffMod
    Commented Mar 13, 2023 at 22:35
  • 3
    (...For instance, if we were to automatically grant the ability to edit posts directly after a certain number of approved suggested edits, that same calculation could be done to the users that are here now, based on the actions they've already taken – we wouldn't have to start with no one having any privileges at all.) That's just one way to address your concern, if we were to implement this.
    – V2Blast StaffMod
    Commented Mar 13, 2023 at 22:37
  • 8
    Well said. I believe that I am in agreement with this. For some time,. I've been wondering if there was a way to assign privileges based on demostrated need for the tool and ability to use it correctly. And unassign them when no longer needed.
    – Philippe StaffMod
    Commented Mar 14, 2023 at 6:57
  • 1
    And what of the consequence of removing so many votes and users from the community moderation effort? How is that being quantified and qualified here? How many users with thousands of reputation truly fall into these broad generalizations?
    – Travis J
    Commented Mar 14, 2023 at 22:08
  • @V2Blast "the proposed system wouldn't have to "start from scratch", with everyone limited to suggested edits again regardless of their rep" except that we want a system that works well for freshly created sites and established ones. The full reset scenario is any site that comes out of Area51.
    – Braiam
    Commented Mar 15, 2023 at 16:27
  • 1
    @Braiam - Similarly to how private beta works now, I'd say. At the moment, during private beta anyone can VTC and edits have a very low rep bar. CMs handle flags. At the end of private beta, mods are appointed and public beta begins. I'd think that anyone who voted to close or reopen a certain number of times which resulted in the action happening would then keep that ability, along the same lines with edits. I don't have specific numbers worked out for SE right now.
    – Mithical
    Commented Mar 15, 2023 at 19:30
  • @Mithical I am mostly worried about the ability to upvote. I see that anyone can upvote during the private beta, but I'm still worried about the participation of those users during the open beta or full site.
    – Braiam
    Commented Mar 16, 2023 at 15:54
65

What is really great about our reputation & privilege system? What do you think makes it great, and why is it worth keeping?

I think it's a great system for encouraging participation. If there was nothing to signify your long-term participation on the site, a lot of people who are here simply to provide answers would grow bored with it pretty quickly.

What is broken about it, and why? Are there any solutions to it?

What's broken about it, particularly on Stack Overflow, is it encourages users to answer posts that should be closed as a duplicate or off topic rather than voting to close because one is rewarded and the other isn't and is increasingly difficult to use with each additional one that is answered. It encourages users to ask for upvotes or to beg users to accept their answers, and it encourages users to fight against any moderation that goes against that gain of reputation. The incentives need to be balanced out to counter this conflict. I don't think simply adding reputation to moderation actions is the solution, rather, remove the negative rep of receiving and giving downvotes and reduce the incentives of accept votes and upvotes. Adjust how quickly users are introduced to these systems and allow feedback from anonymous users to be visible, perhaps through a new helpful/not helpful UI element that's available to everyone and doesn't impact reputation (not a replacement of voting, an additional feedback source without barriers).

What other systems work really well that we could learn from?

A lot of other systems leave out downvotes entirely. They certainly foster a more... "friendly" atmosphere, but where that hurts is with quality and the ability for the avg user to indicate that a given post is wrong/out of date. Limiting that kind of feedback to users leaving comments is a barrier that stops most people from leaving feedback at all. I think we could learn from these systems and better capture feedback from users, possibly in a way that isn't perceived so negatively but still accomplishes the goal.

32
  • 5
    "reduce the incentives of accept votes and upvotes" how can you do that, when those are the only sustainable way to gain reputation? If we change the "+10" to "+3", all scores will be about a third of what they are now and the incentive will be the same. (Bounties will of course also be reduced by the same amount because the capital for bounties also comes from upvotes and accepts).
    – wimi
    Commented Mar 9, 2023 at 20:42
  • 2
    ideally, this would be combined with some of the higher reputation privileges moving to an alternative method for attaining them, see the "abilities" or "roles" suggestion in Spevacus's answer. the point of all this would be to reduce the gamification slightly while removing barriers and the perceived negatives of moderation actions. going to +3 would be a bit extreme, I'm not certain it'd need to actually change from +10, but I'd definitely want to see the -2 and -1 removed.
    – Kevin B
    Commented Mar 9, 2023 at 20:49
  • 13
    I don't hang out on SO as much as I used to, but back when I did (and it was using the same rep system as today), the userbase was far far too quick to close questions as dups. All it takes is a few users who don't quite understand the question, but think they do, and SO has thousands of users. I once had the fun nightmare scenario of suddenly starting to get downvotes on a well-recieved answer years later because it had wrongly been closed as a dup, then merged with the supposed (but actually unrelated) "dup", where my good on topic answer was made retroactively completely off-topic.
    – T.E.D.
    Commented Mar 9, 2023 at 21:19
  • 5
    So no, last I checked, SO had the exact opposite problem to the one related in this question, IMHO as a side effect of it having orders of magnitude more users than the reputation system's designers anticipated.
    – T.E.D.
    Commented Mar 9, 2023 at 21:22
  • 53
    I wasn't happy about the rep cost of receiving a downvote being reduced from 10 to 2. A downvote should fully cancel an upvote. I'm even more not happy with the idea of removing the rep penalty altogether. There's nothing wrong with penalizing people for posting bad content, especially when the penalty can easily be canceled by deleting the content. On the flip side, I'm fine with paying rep to cast a downvote, but I'd also be ok with removing that cost, I think. Consider that the rep cost for downvotes offers some protection against wanton downvoting. Commented Mar 9, 2023 at 21:25
  • 5
    The primary penalty of downvotes at the moment is the user loses the ability to post entirely if they do not improve. I think that's penalty enough, generally
    – Kevin B
    Commented Mar 9, 2023 at 21:28
  • 17
    "What's broken about it, particularly on Stack Overflow, is it encourages users to answer posts that should be closed as a duplicate or off topic rather than voting to close because one is rewarded and the other isn't" - I completely agree with this.
    – Mark
    Commented Mar 9, 2023 at 22:18
  • 19
    I agree that +10 vs -2 for up/down votes is a huge issue. There's a looot of users who leave a stream of crap, but if you get +2/-2 on a post you're in the net positive. Considering that it takes almost zero effort to get enough reputation to upvote, and that clueless masses will upvote literally anything that has an up arrow next to it, this is a pretty bad premise. Commented Mar 9, 2023 at 22:41
  • 5
    On the raspberrypi.stackexchange.com site the +10 -2 causes anomalies. There is a serial upvoter (practically every question gets upvoted, even those with multiple close votes). I don't particularly care up reputation but this defeats the need for users to have reputation to Comment.
    – Milliways
    Commented Mar 10, 2023 at 0:41
  • 8
    I fail to understand WHY downvoting a poor answer penalises those who downvote. This is a disincentive to flag poor answers.
    – Milliways
    Commented Mar 10, 2023 at 0:48
  • 10
    @Milliways See Why do you lose reputation for downvoting answers? for some explanations. I have no strong feelings either way, but I've always been more puzzled by the inconsistency that downvoting answers is penalized, but downvoting questions is not.
    – skomisa
    Commented Mar 10, 2023 at 2:26
  • 3
    @JohnBollinger "A downvote should fully cancel an upvote" - good idea in theory, but it's problematic in practice. People downvote for all sorts of petty reasons. Getting downvoted for seemingly no reason already sucks, that'll make it suck more. The unequal reputation probably isn't ideal, but most users are human, so we should be considerate of that. If downvotes were more limited, this may be a better idea. The bigger issue, perhaps, is that one can have a negative scoring post that still gives you reputation (this might be easy enough to fix by just ... not having it give you reputation).
    – NotThatGuy
    Commented Mar 10, 2023 at 10:21
  • 4
    @NotThatGuy, I am well aware of downvotes occurring for petty and / or arbitrary and / or just bad reasons. I have received my share. I have seen even more posts getting upvotes despite being poor quality or downright wrong (and I've received one or two of those, too). I think the DV / UV asymmetry is more problematic. Commented Mar 10, 2023 at 15:21
  • 5
    I've seen too many people say they don't downvote because it removes rep from the user too many times to want to increase the penalty. The penalty is a punishment when we should instead be encouraging the user to improve. Penalties should really only come into play once it has been determined that the user isn't improving. Seeing a -1 on their question score is plenty enough to get users who would be impacted by a loss of rep to want to do something about it. Those who don't care won't be influenced by anything we do.
    – Kevin B
    Commented Mar 10, 2023 at 15:49
  • 3
    @JamieB eh, i think you're jumping through a few unnecessary assumptions there. question downvotes are free, so a close vote instead of a downvote makes no sense there, in terms of it being a reputation issue.
    – Kevin B
    Commented Mar 13, 2023 at 15:04
46

Reputation is meant to be a gamified system - do a good, get a shiny. That already motivates some people to play the game to get shiny, not because they want to do good.

Add to that, reputation is meant to reflect the quality of action a user has taken, most significantly in the content provided for the sites, and it seems clear that the shiny you get for doing good must be balanced by the shiny you lose for doing bad.

Upvotes and downvotes must have the same impact on reputation, otherwise it becomes meaningless at any scale worth discussing.

Consider some situations that are extremely common.

A user who posts on a small site with a +1/-1 will have a net gain of 8 rep. Is this post really contributing the same value as four distinct approved edits, or most of the same value as a post with +1/0 score? A post on a larger site with a score of +10/-10 will have a net gain of 80 rep. Do we really believe such a post is at the same level of quality as one with +8/0 score? I absolutely don't think so. This only encourages controversy and appealing to a popular interest instead of fact-based informative content.

A user with enough rep earned off historic posts has cleared any barriers to posting. They won't get Q-banned without a moderator intervening, which most are loath to do to someone who's been around for a while. They have a silly idea pop into their head, and they ask a question. +1/-3 and, oh, wait, they've gained rep for posting a bad question?! And that's just a small site. Getting +3/-7 on some sites isn't hard, but that person is still gaining half the rep of a post at +3/0 score, despite being clearly of the opposite quality by the standards of the site.

A user finds a question that is definitively off-topic by the consensus policies established. They disagree and they want to answer it. +1/-4 because the unhelpful question asker is happy they got what they want, site standards be damned, and that's a net gain in rep for going against the community. Even larger if the answer was accepted.

If we're meant to be about using rep to gauge value created, value added quality generated, then we have to be doing that in both directions. We can't even get people into negative number reps anyway, so it's not like we know anybody below a certain threshold is a net drain on content and quality (and an increase in curation and moderation workload). The only way we can see a user's net contributions are not valuable, is if the only corresponding public measure is reduced.

And that requires losing the same shiny for doing bad as one gains for doing good. Anything imbalanced as we have now, makes rep into a certificate for participation, not quality assurance.

17
  • 10
    A small problem with this is that it will also make it easier and more worthwhile for users, who are, for some reason, upset about another user's action, to affect that person's contributions. With the small loss of the current downvote such an action of revenge simply doesn't have such a dramatic impact.
    – Joachim
    Commented Mar 9, 2023 at 21:16
  • 22
    I'm probably not a typical user, but my experience with getting multiple downvotes on non-technical sites is that its about 30% because my take was generally judged to be horrible (no or almost no upvotes), and about 70% because some people don't want what I wrote to be true (net positive vote result). Giving downvotes the same weight as upvotes would seriously deter people from weighing in on subjects like religion or politics, and would empower some of the worst actors on our sites. Perhaps the ratio could be adjusted some, but making them equal is a seriously Bad Idea.
    – T.E.D.
    Commented Mar 9, 2023 at 21:38
  • 4
    Equalizing the change between up and downvotes would be a big mistake. Instead we should undo the change from a few years ago that made question upvotes worth 10 rep and instead make them 5 again.
    – TylerH
    Commented Mar 9, 2023 at 21:57
  • Writing comments on a post that makes clear the reason to do something is a pretty bad place to argue for doing a bad thing in multiple places instead.. @TylerH
    – Nij
    Commented Mar 9, 2023 at 23:58
  • 6
    That's one form of abuse of the system which we already address in several ways. It is a terrible excuse to build in other ways to abuse the system, some of which are described in this very answer, and have no solution beyond not building abuse into the system. The number of unfixed revenge downvotes is tiny compared to the amount of rep unduly earned by crap posts that get sympathy and/or popularity-based support. @Joachim
    – Nij
    Commented Mar 10, 2023 at 0:01
  • 1
    I think I agree, if the main thrust here is that we should lean more on "the overall community" and less on "reputation is king". +10/-10 being better than +7/0 is a great highlight of community input being of secondary concern to reputation gain, but this extends across many aspects. 5 people doing a VTC on a popular topic because they, personally, have a different interpretation of the rules than the rest of the community is a wrong I frequently see, too. Community input needs to count for more, and reputation needs some counterweight.
    – JamieB
    Commented Mar 13, 2023 at 15:51
  • 1
    This would deter asking 'expert' questions that require a domain expert to answer. These are frequently downvoted, either because readers don't have the level of expertise to understand the question and assume it's bad, or assume that the person asking must be doing something wrong if they need that information.
    – dauphic
    Commented Mar 14, 2023 at 19:19
  • 1
    @T.E.D. Kudos. This is the first comment here that truly gave me pause and made me consider that the policy surrounding reputation should perhaps be different for different network sites. (Aside, perhaps, from the issue of smaller sites needing more curators and thus desiring a lower threshold to assign those privileges; but the entire rep=privilege system is broken in my view anyway.) Commented Mar 15, 2023 at 1:37
  • @Nij I disagree. Lesser reputation penalty for downvotes certainly doesn't make the system meaningless and here's why: 1- If the goal is to gauge individual content quality, the post's score is enough to some extent. 2- As an incentive to improve, for new users it's also enough to see a negative value, as evidenced by the frustration felt (enough for some to leave and not return) despite rep not going below 1. 3- If the purpose is to value a user's overall contribution, not getting points where others do is also enough to leave one behind, in relative terms.
    – DystD
    Commented Mar 16, 2023 at 0:13
  • Furthermore, a higher loss in rep would make the decision to downvote even harder than it already is (as mentioned in Kevin B's comment in his answer above). Since a negative vote has a higher signal than a positive one, why should they be equal in terms of reputation? A small loss is indicative of wanting it to work as an incentive to improve, not as a punishment or stigma (why would anyone want users with negative rep?). The point shouldn't be about making the system mathematically symmetric, or even fair for that matter, but to make it work in a way that encourages good content production.
    – DystD
    Commented Mar 16, 2023 at 0:13
  • Interesting perspective. Not a useful one, as far as what can be actually observed to happen on the network in general, but thanks for adding your 2¢.
    – Nij
    Commented Mar 16, 2023 at 3:51
  • Indeed it is my 2 cents, although I won't take credit for all of it. It's also the perspective of a relatively new user, I am sorry you didn't find it useful.
    – DystD
    Commented Mar 16, 2023 at 23:27
  • @T.E.D. Would it be fair to catagorize such sites as "more opinion based". I'm not quite sure the word "technical" paints the line in the right place. The meta sites themselves would fall into this same characterisation. Purely fact based sites tend to suffer less with what people want to believe. Commented Mar 20, 2023 at 12:59
  • @PhilipCouling - No it would not. Asking a question that is asking for opinions is a damn good way to get your question closed within the hour on History (to be fair, there are a lot of good ways). I believe the same could be said for Politics, Physics, SciFi, RPG, English, etc., but I don't moderate there so perhaps I'm wrong on one or two of those.
    – T.E.D.
    Commented Mar 20, 2023 at 13:45
  • @PhilipCouling - Good History is also based on verifiable / verified evidence (and the analysis thereof). There are differences of course (we largely have to rely on "natural experiments" and can't just go make our own) but there's really much less difference there than you appear to think there is.
    – T.E.D.
    Commented Mar 20, 2023 at 15:11
40

Two things to add:

  • A few users on my site (I'm a mod on Academia) have done a lot of community moderation work (retagging, posting on meta, voting to close, editing, etc.) and would be good candidates for moderators. But, they have relatively low reputations, which effectively makes them unelectable. I realize "incentivizing" moderation actions is potentially problematic, but our current solution (not offering any reward at all) is perhaps a bit extreme.
  • There is a huge first mover advantage -- answers posted in the first few hours will usually outperform answers posted a day later in terms of votes (and therefore visibility, and reputation). The green checkmark counteracts this to some extent, but this introduces its own tyranny -- askers sometimes make non-sensical selections, or do not select anything at all. There are several possible improvements here, the simplest of which is to introduce a post sorting order that will rank posts based on a combination of newness and upvote-to-view ratio, and allowing alternative ways to award the checkmark (or killing it entirely).
18
  • 12
    Or kill entirely the checkmark. One user chooses now, you are proposing exchanging "one" for "a few". Same problem. Votes are enough. We can still have a checkmark that OP can choose, but without any consequences, points and less visible.
    – Largato
    Commented Mar 9, 2023 at 16:36
  • 2
    Good point, I generalized a bit, don't want this answer to become a referendum on the checkmark solution.
    – cag51
    Commented Mar 9, 2023 at 16:38
  • BTW, I'm currently a mod too, nothing against mods, but the concept itself.
    – Largato
    Commented Mar 9, 2023 at 16:38
  • 2
    But, they have relatively low reputations, which effectively makes them unelectable. I can relate :( voters will not vote for users who have low reputation despite being active, simply because new users need a very long time to reach a high reputation score Commented Mar 9, 2023 at 16:43
  • 9
    "the simplest of which is to introduce a post sorting order that will rank posts based on a combination of newness and upvote-to-view ratio, and allowing alternative ways to award the checkmark" that was actually implemented recently on SO, research "trending sort." I wonder if it'd have value outside of SO?
    – Kevin B
    Commented Mar 9, 2023 at 16:56
  • @Minh-LongLuu - agree. On SO especially, I use SO everyday and would be willing to contribute, but virtually all of the questions/answers I have are already there and so there is nothing to do except upvote. The reputation system definitely favors a certain type of user.
    – cag51
    Commented Mar 9, 2023 at 17:43
  • 6
    The checkmark could be attached to the question, just to indicate that OP has the answer they were looking for, without pointing at any specific answer, and without any rep gain for the answerers. Commented Mar 9, 2023 at 23:38
  • Well maybe that's appropriate. There's so many very good questions closed as it is already we don't want a flood of noobs coming in and thinking they have to close as many questions as they like.
    – NeilG
    Commented Mar 10, 2023 at 5:54
  • @Bacco: I would be on the losing of that one. I've seen too many cases of a +50 bad answer and a +5 good answer that got the checkmark. It's not a great tool but it's the best tool against the first mover advantage.
    – Joshua
    Commented Mar 10, 2023 at 16:24
  • related feature-request that I'm thinking of eventually "re-hashing" with much more detail: Why there isn't reputation points for successful flagging?
    – starball
    Commented Mar 10, 2023 at 19:35
  • @Bacco The checkmark is useful because the highest-voted answer is not always the correct one. Often, someone posts an answer that is usually useful but actually not useful for the question where it was posted because of some detail in the question the answerer and voters did not consider. Commented Mar 10, 2023 at 23:35
  • 1
    @user253751 you described a situation that don't represent the majority of cases, we can't evaluate something based on this. Usually when the check don't match votes the check is wrong because some "helpdesk" factor. Answers are not "for the OP" only, the check usually promotes undesired helpdesk. I'm not saying that what you describe isn't a problem, but it probably deserves a real solution (maybe pre-a filter of questions before letting someone answer).
    – Largato
    Commented Mar 11, 2023 at 12:49
  • 1
    @bacco Getting rid of the checkmark is a great point. One of the purposes of questions is to be generally useful, not just to that individual asker, but to a broader spectrum of future searchers. Knowing which answer worked best for the asker is often irrelevant. Certainly in Stack Overflow, I frequently find the answer I used further down, because I had the same issue, with slightly different circumstances, so his checkmark served no purpose.
    – JamieB
    Commented Mar 13, 2023 at 15:55
  • 1
    "We can still have a checkmark that OP can choose, but without any consequences, points and less visible." Or we could track them separately from reputation, and have the system know that certain people offer answers that OPs like without giving a rep reward. Of course, what the system does with that information is up in the air a bit... Commented Mar 15, 2023 at 3:45
  • "and would be good candidates for moderators. But, they have relatively low reputations" This has been debated quite a bit on SO. The sound reason for requiring rep from moderator candidates is to ensure that they actually have some foothold on the actual site and don't spend 100% of their time in some "meta-reality", spending 100% of their time doing meta tasks instead of participating on the actual site. There are lots of people like this. The problem is that they lose touch with the actual site culture and what the site is actually about - doing moderation is not the goal of any site.
    – Lundin
    Commented Mar 16, 2023 at 15:07
35

Make downvote criteria the same as upvotes.

Downvotes are not worse or any less important than upvotes. On the contrary, with the current system, downvotes are more important than upvotes, as they can only be cast by more experienced users and there are fewer of them.

Remove the -1 rep cost for downvoting answers.

Increase the reputation necessary to upvote to 125.

The only thing which I would leave the same is the reputation loss from receiving downvotes. With -10, users could feel like participation is too difficult. With the abundant revenge downvotes, this problem would be bad for psychological health of SE's users.

14
  • 9
    I agree with almost everything here, but I do want to point out that new sites should have a different (lower) threshold for upvoting as otherwise no user will ever have the privilege, and some thought needs to put into what that should be and when to increase it.
    – Tinkeringbell Mod
    Commented Mar 9, 2023 at 19:36
  • 1
    I'm not particularly concerned about revenge voting, but for those who are, one could consider increasing the rep cost for downvoting. I would be happy with a -10 rep penalty for being downvoted (like in the good old days :-)) and a -5 or even -10 cost for casting DVs. Commented Mar 9, 2023 at 21:38
  • 18
    @JohnBollinger I would be left with 0 reputation points if every downvote cost -10. Downvotes should not cost rep as this discourages voting.
    – Dharman
    Commented Mar 9, 2023 at 21:42
  • 2
    It would be interesting to know how "-1" was arrived at. Did someone actually do a study and found -1 was the sweet spot for encouraging needed dv's, but discouraging flip ones, or was it just a SWAG?
    – T.E.D.
    Commented Mar 9, 2023 at 21:52
  • 3
    @T.E.D. This is where it would be helpful to still have Shog around...
    – TylerH
    Commented Mar 9, 2023 at 21:59
  • 3
    @TylerH - Well, he's not dead. I think I follow him on Twitter still. However, this number probably goes back to the Jeff Atwod days, no?
    – T.E.D.
    Commented Mar 9, 2023 at 22:02
  • 3
    @T.E.D. Maybe? I've seen some "OG" people talking about downvotes costing -10 before though... And a lot changed between late 2008 and ~2010/2011, after which point a lot has been more or less the same... Plus I mentioned Shog because, aside from having been here since around then, he was always particularly adept at translating technical details into user-friendly speak, not to mention usually excellent at the justification part, too.
    – TylerH
    Commented Mar 9, 2023 at 22:24
  • 3
    One consequence of the discrepancy between upvote and downvotes is that any HNQ post tend to get skewed in terms of votes. HNQ attracts people from across the network who can upvote thanks to the association bonus but cannot downvote. Moreover, people attracted from HNQ also seem to upvote on what seems most fun (on some sites, at least) not what actually answers a question of what is useful. Thus further skewing the results.
    – VLAZ
    Commented Mar 10, 2023 at 14:04
  • 5
    @VLAZ Oh how many times I wish I could downvote with my 100 bonus.
    – Dharman
    Commented Mar 10, 2023 at 14:13
  • @Dharman discouraging downvoting was precisely the point. Commented Mar 10, 2023 at 23:38
  • "The only thing which I would leave the same is the reputation loss from receiving downvotes. With -10..." We don't need to make downvotes have less effect than upvotes in order to soften the blow. IMX questions at +1/-1 are more likely to be worse than ones at 0/0, not better (partly because the upvoter could be someone in the 15-125 rep range), but currently they net +8 reputation. Several other proposals suggest treating the net score on a post in a non-linear fashion, and this could also be used to limit the reputation damage of downvoted posts. IMX it doesn't mean much past -3 anyway. Commented Mar 15, 2023 at 1:22
  • 1
    I wish it took a CV Reviewer badge to down-vote without comment.
    – greybeard
    Commented Mar 16, 2023 at 6:45
  • @T.E.D. it's easy to figure out. The cost of downvoting was thought of by Jeff to dissuade people from downvoting and be more judiciously. What he didn't count was the effect of people trying to hoard as much rep as possible (I'm sure it has a name, fear of <something>) so downvoting, which was something people avoided, now had a objective reason to avoid.
    – Braiam
    Commented Mar 17, 2023 at 21:02
  • 1
    Loss aversion explains a lot of behavior surrounding voting and the greater reputation system here, @Braiam. And yes, Jeff was very much counting on this effect - what has ended up causing more issues is the disproportionate growth in posting vs voting, which has resulted in a "vote" being worth a lot more (+ and -) in 2023 than in 2009.
    – Shog9
    Commented Mar 20, 2023 at 14:19
34

(This answer is primarily driven by my experience with Stack Overflow, and my understanding of the Stack Overflow tour page; but I have confidence that the same principles should apply more or less network-wide.)

Whoo boy. I have so many complaints about the reputation system generally (and even more about how voting impacts on the Q&A, as opposed to users). I actually have a blog post planned about it for the somewhat near term, but I stumbled upon this Meta post so I might as well take the opportunity to gather some thoughts.

Perverse incentives

  • Getting your question or answer upvoted awards 10 reputation; getting it downvoted only takes away 2. That incentivizes people to be controversial rather than boring. Boring is usually higher-quality and more encyclopedic.

  • Getting your answer accepted awards an additional 15 reputation on top of the +10 from OP's upvote. This incentivizes tailoring answers to OP's specific circumstances, rather than trying to write in a broadly-applicable way that will be helpful to people who find the Q with a search engine. It also incentivizes tolerating questions that are clearly too broad (if you help OP with multiple issues, and make code with multiple issues work, that increases the likelihood of OP being positively overwhelmed with the helpfulness). Also, anything with that effect, has the ripple effect of discouraging editing (since Q&A are now more tightly coupled, the minimum unit of coherent editing increases).

  • Accepting the answer also gives 2 reputation to OP. This incentivizes OP to accept an answer even if none of the provided answers actually get it right.

  • Reputation is awarded the same way for answers even if the question is downvoted and/or closed. This incentivizes answering questions that are clear duplicates, because of the likelihood that OP will upvote and accept that answer. Such answers interfere with the deletion of low-quality signposts (although I am personally of the opinion that keeping around duplicate questions is useful because it helps establish the importance of their canonicals). It also incentivizes trying to mind-read OPs who don't express themselves clearly.

  • There's an especially obnoxious cumulative effect here: the first upvote on a "fastest gun in the west" answer, along with the accept vote, nets a total of +25. It takes thirteen downvotes from concerned curators to make it unprofitable for unscrupulous reputation-seekers to answer questions that they ought to know shouldn't be answered. I've interacted with users like that many times. They often have 10+ user accounts and hundreds of thousands of rep; and their response to comments explaining the damage they're doing to the site ranges from a shrug to extreme indignation.

  • There's a lock-in for votes on Q&A that weren't edited, which doesn't respect the fact that sometimes people genuinely change their mind. (One thing that's happened to me a few times is that I upvoted something because I thought it was the best canonical available, then found something much better and actually discovered serious issues with the first option.) This incentivizes 2k+ users to make dummy edits in order to reflect that changed viewpoint. (I try to make my edits substantial, but there have been times where I improved a post while also changing my upvote to a downvote... because I really don't know what I was thinking with the original upvote. Sometimes the upvote was literally years ago, and I still had to edit because of the lock-in.)

  • Getting an edit approved awards reputation, but making unilateral edits does not. This means that people who reach the 2k threshold have a sudden shift from extrinsic to intrinsic motivation for improving the site. (This makes me want to launch into a whole separate diatribe about the edit queue, but I don't think it's on topic here.)

  • Downvoting costs a point. I doubt that many people are seriously discouraged by this, but it bears noticing that the stats show a heavy upvote bias among logged-in users. Last time I checked the statistics, votes from users were more than 90% up, while "anonymous feedback" is close to 60%. (This is, admittedly, skewed by the fact that users between 15 and 125 reputation can upvote normally but have attempted downvotes logged as feedback.)

  • Meanwhile, there is no reputation incentive for most curation activities. As above, that includes unilateral edits (2k+ users), and also:

    • closing duplicates

    • casting close votes on things that should be closed

    • casting delete votes on things that should be deleted

    • using the review queues (in any capacity whatsoever) - the only thing one can get from using the review queues is a ban from the review queues

(There are finer points I could make about the curation experience, but again I think they drift off topic.)

  • Comments can be upvoted, but not downvoted. While this doesn't award reputation, it has psychological implications. I've NLN-flagged quite a few old comments now that had a significant vote tally on them - typically, comments on an answer complaining that the problem shouldn't exist or should be simpler to address, and typically ones that show a serious (but common) lack of insight or understanding.

  • I'm not entirely sure how this works, but it's clear from my experience that moderators are disincentivized to delete duplicate answers on old questions if they're contemporary with the post. I don't get why, because this apparently would not significantly impact on the answerer's reputation - but I do think it bears mention in this section.

  • Currently, Q&A get rewarded the same regardless of the topic matter (or the inherent difficulty of the problem). This incentivizes people to stay in already popular tags and allows less popular tags (where answers are often more needed) to languish in obscurity. While the "no sense of urgency" culture is a net positive, good questions really should get answered eventually. As it stands, our only workaround for this is an extremely clumsy "bounty" system. Which segues nicely into...

Linearity

In my mind, it makes very little sense to count each upvote and downvote on a Q or A the same, and it makes barely more sense to tally per-post reputation to give an overall score to a user.

  • In my experience, both Q and A aren't really any higher quality past a score of +10 or so; they're just older. Often they're lower quality because a) the information has become deprecated or b) it's an A for a Q that used to meet site standards, but now it doesn't. There are any number of psychological barriers to downvoting things that already have a lot of upvotes, and even with "trending sort" it can be quite hard to lower them on the page. Personally, my highest-voted answer is at +346/-2 on a Q that's +444/-5 - and I've thought more than once about deleting it, because the question is still blatantly two unrelated questions, even after extensive editing to make sense of a barely coherent mess.

  • In my experience, questions below -3 aren't really any lower quality than ones at -3 (except for spam/vandalism/trolling, which we do an excellent job of cleaning up). Even from -1 to -3 is a rather shallow gradient. A -3 score makes posts eligible for manual deletion; there's little point in going further than that.

  • On the other hand, in my experience, answers at +2 are much higher quality than ones at +1 on average - because the +1 category is swamped by things that were upvoted by OP and nobody else. A +2 typically entails some kind of confirmation from someone with relevant knowledge and understanding. Most people asking don't have that (or they would have self-answered).

  • Summing reputation gained across multiple questions doesn't make a lot of sense. For people who didn't get in early (and thus have the ability to gain "dividends" on old questions that come up a lot in search engines), the easiest way to game the system is quantity over quality. The first upvote on a question ought to count less (per the previous point), but it counts more (because of the accept system). But being able to write a lot of answers each day isn't evidence of expertise; it's evidence of addiction to a particular mode of using the site. (I shudder to think what would happen if the daily reputation cap had not been implemented.)

One-dimensionality

Even if the system worked as intended in the above regards, and incentivized people to write high-quality Q&A (and put A only on Q that should have them), and take positive steps towards curating the site:

  • The ability to be helpful or insightful has little to do with the ability to be responsible - but we use the reputation system to award moderation-like privileges. (This has a knock-on effect that off-site critics mistakenly describe non-diamonds as "moderators".)

  • Clear communication is totally separate from expertise, but we reward both on the same scale. A reputation score can't tell you who's an X guru vs. who is obsessed with X and wants to find a lot of things to do with it vs. who is skilled at creating simple demonstrations for common X gotchas.

  • The site already has a separate system for recognizing expertise: tag badges. Why is, for example, editing of a tag wiki tied to overall reputation, and not to experience with that tag? Which leads me to...

Quirks of the privilege tower

  • It takes 15 reputation to upvote, but 125 to downvote. I mentioned earlier that this contributes to a bias in the voting/feedback stats. It's also inconsistent with the widely-held view (at least on Stack Overflow meta) that downvoting is a vital curation tool. In particular, that makes it that much harder to sink old content that is deprecated (assuming it can't be salvaged; but trying to add up-to-date information to old As that aren't CW often gets you yelled at) or which no longer meets site standards (which changed since the content was posted).

  • The site association bonus is enough to allow people to create chat rooms and edit CW posts, but not enough to downvote. This strikes me as very backwards. It shouldn't take subject matter expertise to recognize that a question or answer is unclear or unhelpful (or, say, politically inflammatory - and it's usually very hard to get site moderators to agree with R/A flags for that); but it might well require an SME to recognize a technical inaccuracy or an opportunity to rephrase something in proper domain language, and it definitely requires SME to chat about the subject matter of that SE site.

  • It takes 50 reputation to comment on other users' questions. This means that a lot of new users who know a little and want to be helpful, end up trying to use the answer section to give feedback on questions (that should be closed). Even flagging takes 15 reputation, and doesn't let these new users explain specifically how they think the question should be improved. (Speaking of which, the flagging system is also a mess, insofar as low-rep users have additional flagging reasons that copy the close vote reasons, but work differently and independently from actual close votes, and are hidden in that interface.)

  • It takes 3000 reputation to vote to close questions. That's absurd. It's saying that "I think this question has issues that need to be fixed and shouldn't be answered in its current state", and getting two other people to agree with you before anything is actually done about it anyway, is somehow a greater and more dangerous responsibility than unilaterally attempting to fix it yourself (2000 reputation). (It's especially absurd in an environment like Stack Overflow, where the large majority of incoming questions are terrible and the site would be much better off if new questions started in a closed state and had to petition to get opened.)

  • It takes only 1500 reputation to create new tags, but 2500 to synonimize them. This is another bias towards creating junk and away from curation.

  • It takes only ten reputation to answer protected questions. This is definitely not enough, at least for Stack Overflow. Lots of old, famous questions have way more answers than they need - and we don't close them because they meet site standards and closure is seen as a black mark, but we still let almost anyone contribute additional redundant garbage to them. There have been times when I had something useful to contribute to a question that already had dozens of answers - but it's accompanied by a sincere wish that half those existing answers would just disappear.

12
  • 4
    "That incentivizes people to be controversial rather than boring." what does it even mean for something to be controversial on a tech site about facts? If anything, I'd expect a controversial post to get more downvotes on SO.
    – starball
    Commented Mar 14, 2023 at 21:28
  • 1
    "But being able to write a lot of answers each day isn't evidence of expertise; it's evidence of addition to a particular mode of using the site. (I shudder to think what would happen if the daily reputation cap had not been implemented" Note sure I understand what you're saying here. The daily rep cap is geared more toward limiting "passive income": "Bounty awards, accepted answers, and association bonuses are not subject to this daily reputation limit."
    – starball
    Commented Mar 14, 2023 at 21:34
  • To your point about 2k edit and 3k CV, some close-reasons are not fixable by anyone other than the asker, such as needs detail and (on SO) needs debugging details. And edits are not for anything that violates the post owner's intent, so what you're really left with is mostly cosmetic changes. Granted, you can often (with non-trivial effort) fix things like opinion-based and needs-focus, so I see your point there.
    – starball
    Commented Mar 14, 2023 at 21:39
  • 8
    "(It's especially absurd in an environment like Stack Overflow, where [...] the site would be much better off if new questions started in a closed state and had to petition to get opened.)" – This is basically what the Staging Ground is (which is currently in testing on SO, as mentioned in the question post).
    – V2Blast StaffMod
    Commented Mar 15, 2023 at 2:05
  • 12
    IMHO that first point about incentivising controversy is wrong. What it does is incentivises sloppiness: an answer that is one-third right is worth posting, because its upvotes will outweigh its downvotes. Commented Mar 15, 2023 at 3:39
  • 7
    Answers on SO can be "controversial" by injecting opinion on the "right way" to solve a problem based on some subjective factor - even if the question didn't ask for such opinions. But thinking about it more, I think @SteveBennett has the right take on the issue there. Being controversial might be more incentivized on other sites though, like the ones to do with politics or religion. Commented Mar 15, 2023 at 3:55
  • 2
    @starball What I mean about the rep cap: there are a handful of people on Stack Overflow who answer way too many questions, seemingly just hoping to get 20 +1s and 20 accepts a day. These are often people with 6-digit rep totals who seem to be motivated purely by making that number go up. It would be even worse if the 21st answer gave them +25 instead of +15. Any time that a site supports user-generated content, you will end up with a long tail of a very few users who spend way too much time generating content. (Incidentally, thanks for highlighting the typo.) Commented Mar 15, 2023 at 3:58
  • 1
    @V2Blast the staging ground seems like a splendid idea. I hope this time, the shiny "get new users to actually ask questions properly" toy actually functions as advertised. The current "Ask Question Wizard" does basically nothing except demand that users fill out the form sequentially (trivially circumvented, and often counterproductive anyway) and split the body in two (in a way that is often unnatural and leads to a bunch of questions where substantial portions are just duplicated, while doing absolutely nothing to improve clarity or focus). Commented Mar 15, 2023 at 4:01
  • What do you mean by "if they're contemporary with the post"? Commented Mar 15, 2023 at 7:22
  • 2
    Re "motivated purely by making that number go up": There could also be a financial incentive that we don't know much about. Or altruism ("I just want to help."). Commented Mar 15, 2023 at 7:34
  • "answer contemporary with the post" is straightforward: i.e. posted near the time that the question was originally asked. Commented Mar 15, 2023 at 8:45
  • 1
    you illustrate a lot of potential exploits in the system, which I have never even thought to investigate in this way, so thank you! is there evidence for the degree to which this are exploited? I tend to have a simplistic response of "well couldn't SO detect and cap such-and-such exploit" which is possibly unrealistic
    – lofidevops
    Commented Mar 17, 2023 at 13:19
32

Reputation should not be gained from closed (non-duplicate) questions and their answers.

A closed question means that it should not receive answers. If it has received answers nonetheless, then these answers should be treated as invalid. Users should not be rewarded for asking popular off-topic questions. Users should not be rewarded for answering questions that should not be answered.

The only exception to this are questions closed as duplicates that serve as useful signposts.

61
  • 4
    @KevinB They have been mostly all been deleted now. All remaining ones should either get a historical lock or be deleted too.
    – Dharman
    Commented Mar 9, 2023 at 19:06
  • 4
    I support this, rep cap the canonicals and rep lock de dups.
    – bad_coder
    Commented Mar 9, 2023 at 19:08
  • 9
    "Closed questions should have all answers hidden from public view."...What would this accomplish?
    – CDR
    Commented Mar 9, 2023 at 19:31
  • 6
    @Heartspring-CDR It would hide potentially unuseful answers from normal visitors of the site.
    – Dharman
    Commented Mar 9, 2023 at 19:35
  • 6
    This seems to be blurring the distinction between closed and deleted questions. It reads like you'd like all closed questions to be deleted.
    – T.E.D.
    Commented Mar 9, 2023 at 21:58
  • 7
    @T.E.D. The second part of this proposal is less important. The main part is about not getting reputation from closed questions.
    – Dharman
    Commented Mar 9, 2023 at 22:00
  • 8
    I don't know if I agree with this. Closure can be a temporal thing and the mob can get things wrong. Why should someone's reputation growth be stunted because a group of people decided to close their question, and it took some time for that to be reversed?
    – Makoto
    Commented Mar 10, 2023 at 0:00
  • 8
    @Makoto Then it will motivate that person to get the question reopened. Win win.
    – Dharman
    Commented Mar 10, 2023 at 0:08
  • 8
    I disagree. I answered too many questions that later got closed for bad reasons.
    – Joshua
    Commented Mar 10, 2023 at 4:00
  • 15
    @Joshua "I answered too many questions that later got closed for bad reasons" - Were those really "bad" reasons, or were those just reasons you don't approve of?
    – NotThatGuy
    Commented Mar 10, 2023 at 11:56
  • 4
    @NotThatGuy: The ability to answer the question and get upvotes on the answer shows more ability to understand it than the ability to close it. I've had to fight hard to get questions reopened so I could answer them more than once. Most people simply don't have the energy to overcome a bad close reason on most questions that get closed long after answering them.
    – Joshua
    Commented Mar 10, 2023 at 15:37
  • 4
    @Joshua If you believe you understand a question perfectly, but other people don't understand it, rather than fighting to reopen it, it probably makes more sense to edit it so other people can understand it. Not to mention that having the "ability to understand" the question only really relates to one close reason, whereas knowing the site rules and what's on topic is more relevant to closure. You being able to answer a question doesn't mean it belongs here.
    – NotThatGuy
    Commented Mar 10, 2023 at 15:52
  • 5
    @Dharman: Reopening a question can be a gargantuan effort. I mean, there's a whole lot of people who are just chomping at the bit to play rules laywer on why specifically your question was bad enough to not warrant your reopen vote, for instance. There's also this whole fatigue of just...not wanting to touch a question which has been super downvoted. Maybe if those kinds of questions were "forgiven", I could agree with your suggestion. But otherwise, I don't think this proposal makes sense.
    – Makoto
    Commented Mar 10, 2023 at 17:29
  • 4
    My Brother in Christ, you hang out on Meta Stack Overflow. How do you not know that question reopening is near impossible with the parade of people who ask about it there?
    – Makoto
    Commented Mar 10, 2023 at 17:47
  • 8
    Regarding the problem of questions being closed too much or reopened too little: Mind that one reason for this is that many people who answer questions do not care enough to improve questions, engage in reopening, or Meta discussions to better delineate what should be closed or not. I have even witnessed (and worked against) site cultures where the same people answer and vote to close certain types of question. Denying reputation for answering closed questions creates an incentive to solve this.
    – Wrzlprmft
    Commented Mar 11, 2023 at 9:30
30

Well, at least in my own experience, reputation as a motivator is a bell curve. You'd need to get pulled into being an SE regular for reputation to matter. I turned up on Super User because a site about computer stuff seemed interesting and branched off from there. And the last time I actively cared about reputation was when I was trying to work my way to 10k to see a deleted answer of mine (the system changed halfway through me trying to get it... but I got my 10k anyway!).

Most of this isn't actually about reputation. Bear with me; I'll come to a point eventually ;)

TLDR: Reputation is overrated.

Unfortunately, some of the actions taken to try to fix new user onboarding - I think Joel spelled out the problems really well 10 years ago but to be really blunt, quite a lot of what was done was rather hamfisted and... didn't work quite well. He is right about the community dynamics here, just not how it was handled.

To quote Joel

After a few years, an insider group of old-timers forms. They get to know each other. They know the rules. They know the history and the legends of the community. And it’s only natural to get little bit irritated when newbies show up who don’t know the rules.

And there's so much more here than the reputation. Even if a new user gets past the reputation hump - growing that circle of 'insiders' is essential. The newbies see the number and think that's important. It’s an entirely arbitrary number, and frankly past a point gets you nothing.

The problem isn't as much 'reputation' as much as how newcomers being onboarded. A new user with quality posts, or maybe even an intuitive understanding of the system is going to have an easier time than someone who just walked in via Google and/or is expecting a forum. There's different user stories for either as a new user and... neither is going to go "Hey! How do I make this number bigger?"

They are going to remember helpful comments, people editing/improving their posts and guiding them through their fledgling stage.

But as we’ve seen, our system and community in its current state has a high barrier to entry. Asking or answering a question back in 2009 was a lot easier as we were building the library of knowledge that exists today. There were a lot more questions yet to be asked!

Not entirely true. Our standards were much lower and we ended up often needing to change scope as time went on. Even as a very experienced user, there are new technologies I'm messing with (I'm currently working on and having trouble with Nebula VPN for example) and software and software UXes evolve over time. That there are fewer questions to ask, or we'll ever be done with new questions is a misnomer.

The definition of a good question and answer , and the barrier to that has gotten higher. As an oldtimer, working/asking mostly about personal projects on my techie sites, I'm happy to refine a question in my head for weeks before asking. I go back and work on old answers and write new ones over several writing sessions because I've had years to work this out.

I'm pretty sure if I started a new account, picked up a few projects, and started using the same tricks I used as a new user, I'd probably be able to hit 10k in a year. It’s not a reputation learning curve; it’s a skills and spoons learning curve.

We still want users to be able to participate on our site and feel as though they are a part of this community. But unfortunately, we’ve heard from so many of our users, particularly newer ones, that the learning curve is steep and it’s difficult to find ways to engage on the site.

I don't think anyone comes in here worrying about the depth of the mechanics and reputation. Much of the friction is not mechanical and about reputation; it’s cultural, with a mismatch of expectations between new users and experienced one.

The 'learning' curve is as much about learning to deal with how to ask 'good'/better questions, dealing with criticism (textual and votewise) and the mechanics of the site. Worrying about votes is very much an early/middle lifecycle thing for a user—when you're wondering 'can I ask my next question' or 'can I unlock the next privilege?'

I've often said that SE builds communities in spite of itself. Healthy communities need to go beyond the utility of knowledge sharing. As much as main site use, things like chat and meta are also really good, neglected onboarding tools.

Reputation isn't broken other than the attention put to it. 5 reputation vs 10 for a question upvote didn't 'break' the network - the reasonings behind it did.

Reputation as a number isn't 'unfriendly'; it’s the actions and meanings behind, say, a downvote that matter more. There are folks who tend to be a little too focused on reputation, but people are going to come here to find answers and stay for the community.

By the time you 'worry' about reputation, you're already hooked, so any new user who complains... isn't really new, just someone who's engaged already, but who is still finding their feet. It’s better to teach them where the ground is, than move the ground to their feet.

As an old user - well, what's the value in my 145k reputation here?

Meme from Jurassic Park. Man in colourful Hawaiian shirt and a man in polo T sitting together. Text in first panel "LOOK THIS  GUY HAS 145K rep! Panel 2 "See no one cares!"

At this point I have name recognition on the site, so my reputation isn't just reputation. I've not unlocked any additional benefits from being a high rep user. My motivation isn't reputation for most parts - it’s what I can do to try to, depending on the time period, protect my other sites (where its the same story), push (what I think are) positive changes...

Reputation isn't as important as people think it is. Reputation is broken because people focus on it too much. It’s also a handy, mildly arbitrary way to gauge how well people know the system, even if it’s how to break it.

And because of that, well, I don't think 'major' changes are needed. Our privilege system, tied to reputation, could use some consideration though. What would be positive actions that could be rewarded outside reputation gain? What are actual pain points for new users? (Downvotes? Not getting the structure of a good post? Not being able to find help and mentorship?)—these feel more important than trying to 'fix' the reputation system.

15
  • I 90% agree with this IMO.
    – Rishon_JR
    Commented Mar 10, 2023 at 6:39
  • What's the 10%?
    – Journeyman Geek Mod
    Commented Mar 10, 2023 at 6:41
  • That this site is full old-timer forums.
    – Rishon_JR
    Commented Mar 10, 2023 at 6:51
  • 4
    I have been here... 12 or 13 years... I have noooo illusion of not being an old timer. And that's a 9 year old quote 😅
    – Journeyman Geek Mod
    Commented Mar 10, 2023 at 7:34
  • But that's just my opinion and thats how I view the site as.
    – Rishon_JR
    Commented Mar 10, 2023 at 7:50
  • 9
    I sense it's mostly high-rep users who claim reputation doesn't really matter, though. I think that says a little about the relativity of the relativity of reputation :)
    – Joachim
    Commented Mar 10, 2023 at 8:38
  • 2
    something like..., "Reputation doesn't matter" -- high rep users @Joachim ? :D
    – bertieb
    Commented Mar 10, 2023 at 16:38
  • I think you get it. Reputation is just a number. But for those of us with six digits worth of numbers we can say that. For those with less orders of magnitude of numbers, it's everything. But it's also the center of everything, which plays against your perception here (and mine a bit). But it's the one I agree with.
    – Makoto
    Commented Mar 10, 2023 at 17:49
  • Welcome to Stack Exchange Journeyman Geek, thank you for posting here! Definitely agree with your answer, not entirely sure on the name recognition thing.
    – Travis J
    Commented Mar 10, 2023 at 18:42
  • 2
    "Our privilege system, tied to reputation, could use some consideration though." – That is part of what our question is trying to touch on here – not just reputation for reputation's sake, but the way in which reputation touches other parts of the site as well. You may want to elaborate on that part of your post (or link to an existing post where you've already elaborated on it). :)
    – V2Blast StaffMod
    Commented Mar 10, 2023 at 21:01
  • 1
    @Joachim I'm not high rep everywhere - and I still don't really care about reputation on those sites too :D
    – Journeyman Geek Mod
    Commented Mar 10, 2023 at 23:18
  • @TravisJ If I started a new account as "Journeyman Geek" - people would recognise it, and probably flag it as a fake. Least here and hopefully on my site metas, people 'know' me by more than the size of my... reputation.
    – Journeyman Geek Mod
    Commented Mar 10, 2023 at 23:19
  • @Joachim: It's been years since I cared about reputation. Until 50K there's the next batch of privileges to unlock, then 100K is a nice round number, and you get swag so that's cool too. After that, though... I mean, what's the incidence of a +1 (or -1 for that matter) on my reputation these days? It'll take 100 upvotes for the "abbreviated" score next to my name to change anyway... Commented Mar 11, 2023 at 15:44
  • 2
    @MatthieuM. My point exactly. And the swag is no longer a thing, right?
    – Joachim
    Commented Mar 11, 2023 at 16:01
  • @Joachim: I got the swag a long time ago, and to be fair I didn't even know I would get it ahead of time ;) Commented Mar 12, 2023 at 10:53
29

In my not so humble opinion (active Stack Overflow user for 10+ years, active forum user for 20+), you're coming at this from entirely the wrong direction. Paraphrased, this is your concern:

"We want more users to like our site, so they stick around and make our numbers bigger, because eternal growth is totally not a capitalistic fable and we're not yet big enough and still don't make enough money for our shareholders".

You can then tweak the reputation system, but people (most notably the ones you need to keep the site alive) don't come nor stay for the reputation system. People stay because they have time to spend and want to hang out with likeminded people, and/or they want to display their knowledge, and/or they want to help people who run into the same challenges they have.

And then there's people who use Stack Overflow as their resume, so they think they have to have a profile full of badges and points so potential employers will notice them.

Just accept that there are only so many people wanting and capable of giving meaningful, helpful, constructive, qualitative answers that actually improve the value of your Q&A site, and you're not scaring those away with any reputation system, and that there never has been and never will be a lack of people daring to ask questions.

11
  • 4
    This is wrong because you need to renew the userbase, and it's now become unreasonably harder for the next generation to establish themselves. Your fundamental premise is also wrong: devs come here because they need to find solutions, if there were be a better repository elsewhere that's where people would be. I'm also not enthusiastic about the vets who got lucky rep-wise in the early days and never contributed anything afterwards (be it Q&A or curation) it might just be time to level the field and return some meaning to rep.
    – bad_coder
    Commented Mar 10, 2023 at 12:38
  • 21
    People needing help can't be bothered about reputation points, they just want answers. The people that need to be retained, are people who can write, research, explain, and they generally don't care about reputation either. The premise of this question is how to retain more users overall by tweaking the reputation system, which is in my opinion a fallacy.
    – CodeCaster
    Commented Mar 10, 2023 at 13:07
  • 7
    @bad_coder Why do you need to "renew the userbase"? And your claim that "if there were be a better repository elsewhere that's where people would be" is also obviously false, since it entirely ignores the effectiveness of SEO and name recognition.
    – Ian Kemp
    Commented Mar 10, 2023 at 16:00
  • 2
    A large number of users can write, research, and explain that care a lot about reputation and wish to have more. Both for “number go big” gamification and for “I need/want that privilege. Commented Mar 17, 2023 at 14:00
  • @James then why do I see so few? People don't need gamification, they generally want recognition. Big numbers and lots of badges are but one way to provide that. See also: Where's the new boatload of experts who can explain stuff to me like I'm five?.
    – CodeCaster
    Commented Mar 17, 2023 at 14:12
  • I don’t have 20k helpful flags because I’m cool. It’s due to gamification. If I could have 20k rep by answering or asking I would game that. But it’s next to impossible for a user only active for 7 months to ask or answer non-duplicate questions that are clear and have no answers 3 minutes after being asked. Commented Mar 17, 2023 at 14:18
  • @James and how is a change to the reputation system going to cause an influx of non-duplicate questions and going to motivate you to write those or answer them? My point is that the focus on reputation and growth is wrong to begin with.
    – CodeCaster
    Commented Mar 17, 2023 at 14:21
  • 1
    That might be your point, but it doesn't show from the answer. It seemed to be based on what I believe is a misunderstanding of the motivations. While true, I may be inclined to wish to play the "number go up" game with rep I need to ask/answer to get to 10k to be a better moderator: I could then see deleted posts. Commented Mar 17, 2023 at 14:38
  • 1
    @JamesRisner expect that by posting here, on this site, you have selfselected yourself into a group of users that isn't the 99.99999% of the people that use SE sites. So, your argument while valid and correct, is meaningless. CodeCaster here actually hit something in the crux, reputation is not the main motivator for people asking questions nor it is for people answering questions with well through-out answers. They former has a extrinsic motivator that the site can't control while the later usually have an intrinsic one. See JourneymanGeek's answer for an example.
    – Braiam
    Commented Mar 17, 2023 at 21:28
  • @Braiam seeing as in moderation chats it seems to be a common discussion on how reputable seems to be so overpowering that it makes moderation difficult. I guess your point is for the non-reputation seeking and non-moderators make up 99.99999% of the user base. You might be right. I’ve just never witnessed someone with your viewpoint on reputation until now. Commented Mar 17, 2023 at 21:33
  • At least, someone with more than 100 rem. I’m not counting the 1 rep people. Commented Mar 17, 2023 at 21:41
24

As a mature system, the problem with the reputation system starts with the name, as "reputation" is related to trust, but the current reputation system is actually a points system. It's a single bucket for points that might represent different things, such as effort (among other things). A high reputation value doesn't always mean that someone has a high level of trust from the community (understanding community as the real people that use the site, not just as an abstract concept).

Also, the current system assumes that there is a direct relationship between these reputation points and values, skills, and knowledge. It's ok that the points system give access to privileges/benefits, but tasks and especially moderation responsibilities deserve to be handled in a better way.

The designation of site moderators already have defined things for this and experience alonge more thang 10 year on multiple sites. There might be things that require to be improved there things that could be extended for certain tasks an perhaps not necesarily on a per-site basis.

Tasks requiring skills

For tasks that requires skills, one way to handle them better might be to limit the rate of tasks that one user that can do according to the their experience level.

Examples:

gold tag badge holders

Those at the beginner level might use the dup hammer once a week, those at the intermediate level might use it once a day, and those at the advanced level could use it without restrictions.

To be promoted to the next level, the posts they closed by using the dup hammer should not be reopened within one month.

tag creators

  • At the beginner level, they might create one tag.
  • At the intermediate level, they might create up to five tags.
  • At advanced level, no restrictions

To be promoted to the next level, the tags they create should have a tag wiki approved by an advanced tag creator. A diamond moderator might also approve tag wikis, but only when there are not advanced tag creators available.

Trust

For tasks that trustiness is a very important element, I think that we should elevate trust above a mechanical points system. It might be related and might be required to have access to have earned a certain level of reputation points, but privileges/benefits strongly promoted as being related to trust, IMHO, should be reserved for very specific privileges and preferably avoiding "mechanical" access to them (i.e. they might be assigned/removed by an election process and/or by site moderators/CMs).

One example might be participation on disputes of moderation actions. Anyone might start a dispute and anyone starts with the privilege to participate on disputes According to participation history in disputes their participation rate of participation in current disputes might have some limit. Let say, someone that derails conversations / present deceiptful content frequently in disputes might be limited to make one post a day instead of being allowed to post freely.

Gap

The current system already has several features that allow moderators to suspend specific privileges – i.e. when a user is caught repeatedly approving evidently very low-quality posts, that user could be suspended from doing further revisions, but could continue posting questions and answers, which is fine.

Something that might need to be reviewed regarding trust is the way that moderation disputes are handled. They might deserve their own trust point scale, as failing in conducting a dispute probably should lead to a whole site participation ban.

9
  • "privileges/benefits strongly promoted as being related to trust [...] assigned/removed by an election process and/or by site moderators/CMs" which privileges are you referring to, and how might this scale with large sites like SO?
    – starball
    Commented Mar 10, 2023 at 6:15
  • 3
    There isn't really a "beginner level" to having a gold tag badge, it is extremely difficult to obtain, and in that process most users become intimately familiar with the site and its functions.
    – Travis J
    Commented Mar 10, 2023 at 18:30
  • 10
    @TravisJ Someone that recently earned a gold tag bagde might be SME in the topic related to the badge but that doesn't make them experts in the use of the dup hammer... actually a gold tag badge holder might never voted to close a question of duplicate before getting the dup hammer privilege.
    – Rubén
    Commented Mar 10, 2023 at 18:47
  • @Rubén - Let's not get too far off the point here (with regards to reputation), but that isn't really the case. As I said, there isn't a beginner level. Read more on the insights and debate around gold tag badges here: meta.stackexchange.com/questions/230865/…
    – Travis J
    Commented Mar 10, 2023 at 23:09
  • 1
    @TravisJ The examples here are not descriptive (they are not about the current state) they are imaginative / innovative (they are about a posible future state) .
    – Rubén
    Commented Mar 10, 2023 at 23:16
  • 1
    @TravisJ the idea is that, having received a gold tag badge, one would hypothetically gain "beginner level mjollnir" access, which promotes to greater levels of mjollnir access after it has been established that previous hammerings were approved by the community. Also, it's just spitballing one of uncountable ways of doing it. Commented Mar 14, 2023 at 23:59
  • @KarlKnechtel - Yeah, I get it. It's just not based in any actual analysis, and highlights the sort of kneejerk reaction aspect of this suggestion. Then again, maybe we should require people to actually have well received questions, before allowing them to judge how to produce them; that would give a much needed insight into using these tools. "Tasks requiring skills"... Skill: Ability to create well received questions, Task access: Question closure.
    – Travis J
    Commented Mar 15, 2023 at 5:59
  • "It's just not based in any actual analysis" - I don't think analysis is required for the proposition "let's consistently make it so that new privileges are awarded on a trial basis and access to them expands when the user demonstrates a capacity to use them properly". Commented Mar 15, 2023 at 6:02
  • Re the dupe hammer: I have it in the Java tag, and there are cases where I wish I didn't. Some things are clearly dupes, and I'm happy to wield the hammer. But there have been subtle cases, where I wished I could submit a vote like a normal person, and see if there's agreement among other high-rep members.
    – yshavit
    Commented Mar 16, 2023 at 4:20
24

The core mechanics work; don't change them.

  • Upvotes and downvotes to earn and lose reputation.
  • Earning basic privileges: it's like a fizzbuzz test.
  • Downvotes cost something. This makes them meaningful.
  • A culture of learning by doing. (But not being an asshole about it.)

There are lots of comments here about reputation economics. If this needs fixing, please don't change the core mechanics to do so.

Illustrative examples that don't change the core mechanics:

  • OverLordGoldDragon suggests a decay mechanism

  • Multiple answers hint at separating "moderation rep" from "quality rep". I don't think this conflicts with my suggestion.

  • Multiple answers suggest adjusting particular values (e.g. value of a question upvote).

Based on comments on this answer, maybe I should clarify that I believe the core "gameplay loop" provides necessary and sufficient short-term rewards for someone already committed to quality questions and answers. In my opinion, onboarding and longer-term economics are separate concerns (which I am eager to get involved in, e.g. when Staging Ground becomes available).

11
  • 5
    Upvotes cost nothing, so they mean nothing? How is needing to earn privileges like a test? Why do you need reputation to learn by doing? Forums and newsgroups haven't had reputation or karma or kudos or likes for ages, yet people matured pretty well there.
    – CodeCaster
    Commented Mar 10, 2023 at 19:19
  • 4
    I think that certain parts of the current system work well, but I feel like it'd be helpful to elaborate on which parts of the system you consider "core mechanics" that "work", and what you think works well about them. (I don't think anyone's advocating getting rid of voting or reputation entirely.) But if we better understand which aspects are fundamental to ensuring that the site/network continues to function well, and which aspects aren't working well (and what is not working well about them), that'll help us (i.e. staff) research and develop solutions to those issues.
    – V2Blast StaffMod
    Commented Mar 10, 2023 at 21:07
  • @V2Blast my point is that the short-term reward structures I listed actively work on a daily basis; how many (non-zero) points are involved is a detail, what happens over a multi-year engagement is an elaboration ... my suggestion is that the core mechanics (listed above) should not change, even if you tweak numbers and/or introduce long-term mechanics; is that clearer? if not I'm happy to expand my answer
    – lofidevops
    Commented Mar 11, 2023 at 20:40
  • @CodeCaster I don't think forums engender maturity, I think they're an environment where mature, robust contributors can succeed -- I would argue that the SO approach (a meaningful reward system) enables short-term gains for new users and high signal:noise for long-term contributors
    – lofidevops
    Commented Mar 11, 2023 at 20:49
  • @CodeCaster fwiw I think we agree that utility is the primary incentive!
    – lofidevops
    Commented Mar 11, 2023 at 21:06
  • 1
    @V2Blast I can clarify further that I'm talking about existing incentives for writing accurate, informative questions and answers; in other answers here I read sentiments about improving moderation tools, perhaps even a separate reputation system for moderation, and I defer to users with experience on those issues
    – lofidevops
    Commented Mar 12, 2023 at 14:52
  • No, the core mechanics really don't work very well. There are several other answers now going over this. They collapse several categories onto a single axis and create countless perverse incentives. Commented Mar 15, 2023 at 0:52
  • @V2Blast If I'm really blue-skying it, I't like to see more voting and multiple forms of reputation. Clicking to give feedback on a post should allow for marking it with several positive and/or negative qualifiers. Users should get separate scores for being helpful/trustworthy/active/good communicators/etc. etc. etc.; and those scores should separately feed into relevant privileges, and make their votes count more in ways that make sense. Commented Mar 15, 2023 at 0:54
  • 1
    Instead of displaying numeric scores (for most users), posts should just sort by various mostly-opaque algorithms and have pop-ups with messages like "many [trustworthy] users found this answer [technically accurate]". Commented Mar 15, 2023 at 0:55
  • @KarlKnechtel as I mentioned in a comment above, I think numbers could be tweaked, new categories added, long-term numbers dampened, as long as the basic "gameplay loop" outlined in my answer remains; maybe I need to clarify this in my answer
    – lofidevops
    Commented Mar 15, 2023 at 18:12
  • 1
    RE "Multiple answers hint at separating rep..." There's this which I opposed at the time. I think the reasons I outlined are still relevant, though it's a reasonable debate to have. Commented Mar 16, 2023 at 18:12
22

What is broken about it, and why? Are there any solutions to it?

It might not be "broken", but reputation (and voting, to a lesser extent) have an ongoing identity crisis where it is not clear what they mean. For example:

A user having a higher reputation than another user could mean any of these:

  • Has answered more questions
  • Has given more useful answers
  • Has answered fewer, but more popular questions
  • Has given more answers, but on tags that are not relevant to the current question.
  • Answers questions that are easier to understand (and therefore, more likely to be voted on)
  • Was first to help someone fix a common syntax error 15 years ago

The fact that every question and answer puts the user's reputation in your face encourages people to notice and interpret that number in some way. The problem is that within any random Q&A thread, this number has no reliable, coherent interpretation. I think most would agree that if you rely on reputation numbers to make choices about voting, you are using the site wrong, and maybe even harming the community. And yet, the question/answer page is designed as though we want you to notice to them.

There is nothing wrong with having cool Internet points for fun, but we have an incoherent blend of a merit system and an engagement system, with magical numbers displayed everywhere. In my opinion, this seems to be dragging against the goal of having high-quality, self-moderating information.

7
  • The idea in this answers seems to be that rep should be printed smaller or later or not even displayed at all? Commented Mar 10, 2023 at 8:12
  • 1
    @Trilarion Hiding rep on the Q&A page could be an improvement. Really though, the idea of my post is to try to describe the bigger picture of why reputation feels broken. Commented Mar 10, 2023 at 8:39
  • I read your post and agreed with it. Just thought about possible fixes to this particular way the reputation might be or feel broken. The obvious direct solution would be to not display rep. Maybe there are different ways to mitigate the problem. Commented Mar 10, 2023 at 8:43
  • 1
    @Trilarion There's so much to say on this topic, as it goes to the core of the site's design. I may open a new question for that purpose later. In short many common use cases fall onto this blended vote/rep system, simply because that's the only tool available. Sometimes a tool feels broken because you're using it for the wrong job. We need more tools that support how people use the website, rather than piggy-backing everything off the same system. Commented Mar 10, 2023 at 9:38
  • 3
    The voting system is good at sorting answers on a single question, and the reputation system is good at delivering dopamine and generating engagement. Apart from that, we just take what we can get from those systems and make it work. Those two factors were important enough and done well enough that they made SE as big as it is, but they are not a be-all and end-all. Commented Mar 10, 2023 at 9:48
  • 1
    There is one coherent thread through all of these: "has used the site a whole bunch". The purpose of a system is what it does. Commented Mar 10, 2023 at 23:40
  • 1
    here is the entire body of one question: "How do I set, clear, and toggle a bit?" stackoverflow.com/questions/47981/… the top answer has 4000 upvotes. assuming each vote it +10 that is 40000 rep. assuming 5 that is still 20000 rep. Most users of C++ with more than 10 years of experience could answer this but one guy gets the crazy rep bonus.
    – user752782
    Commented Mar 17, 2023 at 18:20
19

We still want users to be able to participate on our site and feel as though they are a part of this community. But unfortunately, we’ve heard from so many of our users, particularly newer ones, that the learning curve is steep and it’s difficult to find ways to engage on the site.

That's quite hard to parse for me. What "learning curve" does one need to search for and read answers, or post a question? Or is it the "learning curve" for gaining reputation points while not adding any value to the site?

Also, "find ways to engage": does one not know how to answer questions or edit posts to improve their quality? Or is it about, again, gaining rep points, for... what?

I guess my point is, how are the modifications of the SE reputation system can help "find ways to engage" and/or the "learning curve"?

Perhaps we would want to ask those users who think the learning curve is steep and can't find ways to engage to tell us more?

2
  • 2
    It's hard to parse because it's extremely condensed. Every post that spends like only 2-3 lines on the problem will very likely misrepresent it or simplify it so much that it could be all and anything (e.g. "learning new things is hard" is true but doesn't help anyone). I fully agree that more time should be devoted on finding out what exactly the problems are before asking for a solution. Commented Mar 10, 2023 at 8:16
  • One of essential limitations for new users is a restriction to comment. When they try to put their comment as an answer, they receive an unrelated advice “consider to answer other question” Commented Mar 14, 2023 at 10:40
18

I want to add that in my main site, Earth Science, many of the users with privileges to moderate are not participating or doing moderation tasks (neither upvoting). Earlier users have answered the good questions and left the site, and the new users find it difficult to obtain the reputation that the users that have been there since the beginning and are still active (a few) have (they see they obtain a few upvotes for their detailed answers, while top users have 23K).

So as well as a loss in users, we have moderation troubles. For example, after we banned id-my-rock questions, it took months to completely delete the existing questions. I needed to open a post asking for delete votes so that the site would stop appearing first for searches as "Help identify my rock" in Google, whose status took two months to complete (while we continued receiving a flood of id-my-rock questions). There is also an accepted proposal of merging synonym tags from 2016, but the tags were never merged and we do not have people with privileges to vote to merge active in the site (you need to have answered some question with the tags).

I would say then, at least for Earth Science, the reputation gains for upvotes on answers and accepted answers should increase, or the moderation privileges reputation needed should reduce, if the site wants to survive (it is giving signs of dying).

2
  • 12
    A site might be dying regardless of the reputation system, but in this special case I'd say that maybe network sites should be able to set their own moderation privileges thresholds. That way the mechanics would be familiar to people (you get the same reputation for an upvote, lose the same for a downvote) but small sites with less questions to answer could adapt to that and give privileges faster to their experts. Commented Mar 10, 2023 at 8:25
  • 5
    I'm definitely in agreement that faster rep/privilege gain is more important to low-traffic sites. Some of my low-t sites I've been in since beta, so I got that early lift & can now participate fully. Others I'm far too late to the party, & realistically, I'll never achieve those levels, nor will anyone coming in behind me. Even if I answered every single question & gained some rep from it [which of course I would/could never do]; there's just not enough in total to ever get there. Actual rep I'm not particularly interested in, most of my frequent sites I'm 'trusted' or thereabouts.
    – Tetsujin
    Commented Mar 10, 2023 at 9:39
17

One of the problems on Stack Overflow and many other sites is that it rewards with reputation answers to questions that instead should be closed as duplicates. To fix this:

  • Provide reputation gain to the user who correctly marks the question as duplicate, for example +10 ("correctly marked" means that the question is finally closed as a duplicate). The reputation gain should be comparable to that from answering a question, because finding the duplicate requires non-trivial effort.
  • Provide badges for users who correctly mark many questions as duplicates ("correctly marked" means that the question is finally closed as a duplicate).
4
  • I'd support a reputation loss to people that post an answer to a question that is later closed as off-topic (not when closed as dupe, it's impossible for newer people to know what has already been asked, and having to do a search before answering is counter-productive IMO). Commented Mar 10, 2023 at 15:31
  • 5
    did you see my answer post? maybe you should get a small reputation loss for writing a duplicate answer? (jk)
    – starball
    Commented Mar 10, 2023 at 18:26
  • 4
    It should not always be a loss - see the "signpost question" concept. If a duplicate helps more people find answers that's a good thing. Perhaps if the question is not a good signpost question, then simply having it deleted and losing any upvote rep is enough? But we definitely need positive reputation for dupe-finders. Commented Mar 10, 2023 at 23:47
  • @user253751 Thanks. Edited to remove this suggestion. Commented Mar 11, 2023 at 22:19
16

A more specific proposal for handling per-question reputation

Working within the reputation system generally, several previous answers (including my own) seem to have highlighted some points of agreement:

  • +10/-2 for up/downvotes is bad, and has negative consequences
  • However, downvoting things into oblivion isn't that helpful in terms of correcting user behaviour, and tends to feed resentment
  • Super-high-upvoted questions aren't really that much better than sort-of-high-upvoted ones
  • However, we should incentivize answerers to aim for quality, rather than mass-producing answers that get +1s

Some citations:

Answer By Quote
Me [Too much to quote here]
Lundin Reputation is also a measurement of 'I happened to answer some question that went viral'. Some who've asked and answered one particular up-voted post can gain ridiculous amounts of rep just because of that.... Reputation encourages answering duplicate questions over and over again.... Up-voting costs nothing and yields +5 or +10 rep. Down-voting costs 1 rep and yields -2 rep. This is ridiculous and leads to rep inflation.
Minh-Long Luu However, I have seen some long-time users who just asked a single, yet basic question, then disappeared forever, still gained thousands of points.... My proposed solution is to consider implementing a decay system. [Although this seems to propose a time-based decay instead]
Starship is go for launch ...rep is supposed to be about trust. Posting something with 1 upvote and 4 downvotes gains rep, but certainly loses trust.
SQB ...the rep cap. That should be per question instead of daily.
user3840170 Answers to poor questions can nevertheless earn the answerer some reputation, which encourages wild-shot answers, discourages voting to close unclear questions, and removes asker’s incentives to put some effort into improving the question.... It takes five downvotes to nullify a reputation gain from a single upvote. It takes three more to nullify a reputation gain from answer acceptance alone, even though the asker is often a newbie, and therefore the least qualified person to judge how good an answer is.... Implement sub-linear reputation gain from upvotes [admittedly, along with some other ideas that don't align well with this proposal]
Ne Mo For that reason, I strongly believe that for new users (not me, real new users) should have a reputation "floor" on their questions." [although this also seems to propose a voting floor]
OverLordGoldDragon Long-time users have had the luck to answer basic questions that are very popular to search, and gather hundreds, thousands, or 10k+ upvotes. This rep gain is completely meaningless - it doesn't reflect greater skill, knowledge, or any other trait.... A hard-earned +2 post isn't worth x50 less because it didn't make it to HNQ and get +100.... Solution 2: every upvote after +50 earns 5 rep; 2 after +100; 1 after +200.
Nij Upvotes and downvotes must have the same impact on reputation, otherwise it becomes meaningless at any scale worth discussing.

There are also some pertinent comments as part of the discussion, particularly on Kevin B's answer.

To me, the conclusion is clear - we should count upvotes and downvotes the same, but scale per-post reputation in some non-linear way based on the net vote total. Specifically, the curve should be steeper in some low-positive total range, and shallower elsewhere.

I wanted to spitball some numbers and see how people feel about them:

Net post score Total reputation impact Change from previous
-3 or below -10 0
-2 -9 1
-1 -6 3
0 0 6
1 10 10
2 30 20
3 55 25
4 85 30
5 120 35
6 155 35
7 185 30
8 210 25
9 230 20
10 240 10
11-30 245-340 5
31-50 343-400 3
51-100 402-500 2
101+ 500+ 1

Of course, it could be done with a mathematical formula of some sort rather than a table lookup, but I think this illustrates the sort of curve I have in mind.

The choice of 25 as the maximum marginal/delta of 35 is partly to compensate for re-introducing upvote-downvote parity and partly to compensate for the rep withdrawal on super-popular questions; but mainly to make the numbers nice and have more resolution to work with. I'm assuming that other numbers could be scaled at the same time if need be; but I see how that would have ripple effects on an established userbase.

7
  • 2
    I have no idea how that table is supposed to be read. The total reputation impact and change from previous don't seem to align with each other or the existing reputation system. This looks like it's intending downvotes to have a massive impact since you're suggesting something based off of net post score and not upvotes and downvotes separately as it is now.
    – Laurel
    Commented Mar 15, 2023 at 22:35
  • 2
    @Laurel "change from previous" means the difference between "total reputation" in the current row and the previous row. It hadn't occurred to me that it could be interpreted any other way. The purpose is to show the cumulative effect of each incremental change to the score. Of course "total reputation impact'" doesn't align with the existing system; the point is to change the existing system! As for downvotes, not really; I don't want them to have any impact past -3, because scores below -3 don't IMX signal worsening quality. Commented Mar 15, 2023 at 22:37
  • I would opt for a lower maximum marginal/delta than 35. Personally, I don't see a problem with a max marginal/delta of 10 and having a wider bell curve. I'm not yet convinced either way whether zero-rep-loss downvotes past a certain point is necessarily good. Other than that, I like the idea.
    – starball
    Commented Mar 15, 2023 at 23:35
  • I think I understand. But downvoting an answer would usually cost the answerer 10–35 rep per downvote? (A single user could tactically downvote you to lose an entire bounty's worth of rep with two votes! Who's resentful now?) The severe penalty for downvotes is very hard to swallow, even knowing that reputation would be greatly inflated from what it was previously for most users.
    – Laurel
    Commented Mar 16, 2023 at 0:00
  • Hmm, yes, well. I guess we've all had an experience or two with inflation lately. Commented Mar 16, 2023 at 0:03
  • Not sure how I like tying net reputation gain it to mere net score. A +3 / −3 answer is quite likely to be strictly worse than a +0 / −0 one. I think I could write down a table of my own… Commented Mar 18, 2023 at 15:13
  • @user3840170 if you think downvotes should count more than upvotes, well... I can see the argument for it, but many people think moving from -2 to -10 reputation adjustment is already radical. (It also occurs to me that maybe the effect on reputation should still be divorced from the effect on page sort order.) I welcome you to write the proposal, though. (I really don't know why I marked this answer CW. It's not like I intended for people to just replace my numbers with their own preference. This is the particular type of discussion that the Stack software is especially unsuited for.) Commented Mar 18, 2023 at 20:20
15

Allow low-rep post feedback to be useable

When talking about limits on participation for new users due to low reputation, an important one is the ability to vote. Low-rep users are allowed to leave feedback, which could signal question askers and answerers that posts could be improved.

However, this feedback is nearly entirely inaccessible to authors of posts, which means new users can't properly indicate that posts are unclear or low quality to the author. Comments don't work but open an avenue for spam, but votes don't really allow for spam.

I understand that allowing 1-rep users to cast votes that affect reputation is undesirable since it could enable voting fraud, so:

Please make post feedback easily visible to the author of posts.

Currently, the only way I know of to view post feedback on your own posts is through SEDE. It would be great if your user profile could reveal post feedback by low-rep users, so it could be used to adjust questions. Ideally I'd have it split between logged in and logged out users, but I understand that might be asking for a little much.

6
  • 4
    It could also be worth looking into if the anon feedback can be converted to normal votes, e.g. at 1/10th value.
    – Dharman
    Commented Mar 9, 2023 at 18:39
  • 4
    The anon feedback would be subject to so much bot activity that it would be practically useless. Once people figured out they could anonymously game a metric that was available everywhere, they would.
    – Travis J
    Commented Mar 10, 2023 at 18:37
  • 2
    @TravisJ I'm advocating to make it visible for the authors, not public, and also do not think it should pop up notifications, and it already is public through SEDE. The main thing I think that should change is to make it more accessible to the authors. That should make the chance of abuse small, as you can only affect what a single user might or might not look at. However, you may still be right, which is why I'd prefer a split between logged in and anonymous.
    – Erik A
    Commented Mar 10, 2023 at 20:35
  • 1
    Other than SEDE, where is it accessible? Is this the "you do not have reputation to cast a vote but it has been recorded" message? Commented Mar 10, 2023 at 23:41
  • 1
    @user253751 Indeed, I'm talking about those. They can also be viewed in the 10K+ rep moderator tools under meta.stackexchange.com/tools/post-feedback, in an unhelpful overview that mainly reveals voting anomalies, at least for larger sites. Afaik that's it, SEDE + overview pages for >10K rep users.
    – Erik A
    Commented Mar 11, 2023 at 7:23
  • 1
    It is also possible to see it on the main site using a userscript or plugin (shameless plug) by performing SEDE queries automatically but that's ... not a solution for everyone.
    – dan1st
    Commented Mar 14, 2023 at 10:36
14

Something to call out here before we dive into it:

...[W]e’ve heard from so many of our users, particularly newer ones, that the learning curve is steep and it’s difficult to find ways to engage on the site.

This is orthogonal to reputation. Being able to participate on a site is a different problem than what the numbers mean next to your avatar and what you get for having your numbers grow.

Getting users to participate and feel like they're engaging on the site means that you have to give them an incentive to engage. A lot of people use Stack Overflow as a means to an end - ask a question, get an answer, move on with their day. The folks who want to engage in a community may stay for longer. The folks who want to share their knowledge may stay for longer. None of this has anything to do with reputation.

With that bit out of the way:

What is really great about our reputation & privilege system? What do you think makes it great, and why is it worth keeping?

A lot of this boils down to familiarity, but it's really the sentiment that the system we have accomplishes a few things:

  • Strong bands/divisions based on either participation levels or quality of answers
  • Placing tools in the hands of power users, should they choose to use it, when they've had enough winters under their belts
  • Something to strive for; while no one will really come close to Jon Skeet, they can at least shoot for the stars

What is broken about it, and why? Are there any solutions to it?

It is quite fragile, since it's the main measure of two things in my mind:

  • How active you are, and
  • How expert you are

To that end, these are the things that have broken it in the past:

  • Giving questions 10 reputation, which inflates the amount of reputation that users actually get/earn from participating in the site
  • Previous efforts which rewarded reputation in an unchecked fashion which were never recalculated (looking right at you, Documentation)

What other systems work really well that we could learn from?

While it's not perfect, I think the sentiment is that it's a more social variant of an Elo rating for everyone here. That is to say, the total reputation you have is a factor of how well your content is received against the wave of everyone else. This does make participation slightly daunting (which fine, you've caught me, I've come a bit full circle on this haven't I?), but it does mean that the system has a very explicit goal in rewarding those who are participating and whose content is comparable or better to others.

You don't explicitly state what your goals are for reputation, so it's not really feasible for me to say that something needs to change here. But I really don't think the system is particularly bad.

We just have to be very judicious when giving reputation out.

7
  • 5
    Maybe we should introduce a review queue to review reputation awards. :-p
    – Cody Gray
    Commented Mar 10, 2023 at 0:23
  • While I agree rep and engagement should not be conflated, the rep system design does have a big effect on engagement. Primarily that the current system ties possible engagement types to rep level. But also the sociological effect of stratifying people into "haves" and "have-nots". It's easy for a 50k user to shrug off downvotes or advocate giving bounties, but if you're sitting at the bottom, those point losses hurt, especially if the site then keeps you from participating. -- Perhaps the issue is "you had a popular answer" and "we trust you to downvote responsibly" are orthogonal.
    – R.M.
    Commented Mar 10, 2023 at 15:52
  • 2
    @R.M.: This thing about "haves" and "have-nots" implies something about downvoting or rep gain that hasn't changed in the ten-some-odd years that I've been on the site. Everyone has the same opportunity to get reputation. The experts are the ones who get more of it. To get more rep, you must become more expert. Things like downvotes become less of a factor (like, at all) if you focus on good answers and getting things right, without succumbing to the sensationalism of answering one-liners or unclear questions.
    – Makoto
    Commented Mar 10, 2023 at 17:04
  • @R.M.: So again I posit, reputation is still based on both how active you are and how expert you are. In that vein it is very much identical to an Elo rating in that, just because you think you're "good" at a game doesn't mean you can reasonably compete against someone who is at the upper echelons of the game (with a ranking in the 0.01-percentile).
    – Makoto
    Commented Mar 10, 2023 at 17:06
  • 1
    @Makoto Sorry, I was unclear on my point - I'm talking mainly about your "call out". I'm just stating that the construction of the reputation system has an effect on engagement & participation. If your ability to interact with the site and your social interactions are determined by the number next to your avatar, a new person's willingness to become active and gain expertise with the system is also going to be affected by that. Reputation system design doesn't happen in the absence of onboarding/engagement criteria (and vice versa, given the "gamified" nature of the site).
    – R.M.
    Commented Mar 10, 2023 at 18:07
  • 4
    @R.M.: You're making my point here - just because the number is big or small next to your account, that doesn't really mean you can't participate. You're limited in what you can do on the site because there are sensible limitations to prevent bad actors from creating hordes of puppets to unduly influence things, but everyone started there. If you're expert and you're able to consistently show your expertise, you don't typically run into this barrier of "omg numbers need to go bigger" that you seem to be honing in on.
    – Makoto
    Commented Mar 10, 2023 at 18:11
  • 1
    "Giving questions 10 reputation, which inflates the amount of reputation that users actually get/earn from participating in the site" - you mean upvotes on questions; and we absolutely should make it at least as easy to gain reputation for good questions as good answers, because good questions are hard to ask. In many cases, it's about as hard as answering them, plus most of it is attempted by people who are uniquely unqualified. The few who rise above that adversity are the heroes of the site. Commented Mar 15, 2023 at 1:14
13

I think that the reputation penalty for downvoting answers is (became) a broken part of the system.

This was hinted to in one of the prior answers, but I would want to explicitly focus on this. To start with, I believe it worked reasonably well until the recent few months - maybe not quite perfect, but not worthy of raising serious concerns about either. Until recently, I could not even imagine that I would ever complain about this.

As it turned out though, this has been working based on a certain implicit barrier to posting answers. First, you had to have a certain level understanding in the topic of the question and second, you had to put some effort into well... just writing the answer - you know, figuring what to write, finding words, typing them, and stuff like that.

This barrier is no longer with us now that one can just copy the question into ChatGPT and copy its response into the answer field.

This has led to an unprecedented flood of low-quality answers that are in the sore need of downvotes to rate, and rep penalty makes this impossible - at least at the scale corresponding to that of the ChatGPT flood.

If needed, refer more detailed discussion at MSO

This looks like the most basic and natural way to establish trust model, proven to work reasonably well on questions and well integrated into the system (rate limits etc). This is what we miss for answers...

answer downvotes should be free unless these are cast on competing answers...

...to start with, multiple downvoted answers will throttle low quality posters (bot or not) - this is embedded in the system of rate limits. Besides, all the usual quality mechanisms associated with downvotes and negative score will start working at last: option of 20K-deletion votes, roomba etc etc. The last but not the least, people will just start feeling better getting the ability to rate content as they want and there will be less of bitter feelings (I believe this is very important)

4
  • 16
    I think ChatGPT answers are a bad example because you get the lost reputation back once the post is deleted. It doesn't actually take that long for mods to handle a ChatGPT flag.
    – Dharman
    Commented Mar 9, 2023 at 21:09
  • @Dharman this was covered in comments under linked discussion at MSO, I can only quote self "there is no knowing that post will be deleted in this case, not for a casual reader / voter unfamiliar with our obscure meta discussions" - this means moderators have to do loads of routine work via flags instead of letting rep/rating system work as intended
    – gnat
    Commented Mar 9, 2023 at 21:14
  • 3
    Imagine the impact if downvote rep loss weren't refunded when the posts were deleted... *shudders*
    – TylerH
    Commented Mar 9, 2023 at 22:08
  • 2
    and some months after... imagine if suddenly something happened that made deletion of those AI generated answers far more difficult to achieve Commented Jun 7, 2023 at 14:36
12

First of all, thanks for bringing this up to discussion with the community before jumping the gun and implementing some change.

  • What is really great about our reputation & privilege system? What do you think makes it great, and why is it worth keeping?

As noted many times, some people care a great deal about imaginary internet points and so it works well for attracting those kind of people. They might be people who contribute with high quality content and/or people who just answer endless duplicates, but overall the system works somewhat well for attracting users. Essentially it is the same thing as receiving a golden star sticker from your teacher back in elementary school - it appeals to some basic human need to be recognized.

The rep system works quite well for discouraging low quality and incorrect answers. In fact down votes work much better than up votes for filtering out quality over low quality.

What works somewhat well is that moderation tools and privileges are gradually unlocked. Things like review queues should obviously not be unlocked immediately. However, there is not much of a relation between high reputation and moderator suitability.


  • What is broken about it, and why? Are there any solutions to it?

So many things.

  • As noted above, reputation is a very poor metric for moderator suitability. It is also a somewhat poor metric for community trust in one particular user. If anything, reputation is a metric for how active someone is on the site.

  • Reputation is completely out of proportion for Q&A under popular tags. Looking at the top users at SO, pretty much everyone in top 50 are active in the most mainstream, active tags. So besides being a metric of how active someone is, it is also a metric of how popular tags the user is following.

  • For these reasons, reputation is also a very poor metric of technical knowledge. The bronze, silver and gold badges are somewhat more relevant.

  • Reputation is also a measurement of "I happened to answer some question that went viral". Some who've asked and answered one particular up-voted post can gain ridiculous amounts of rep just because of that.

    As a perfect example from SO for those who know programming: What is the "-->" operator in C++? This is the single-most up-voted C and C++ question and only because it is about a feature which is curious, intriguing and exotic. Not because it is about relevant technical knowledge or a canonical duplicate. Looking at my own by far most up-voted answers on SO, the top 2 are also about such curious and strangely behaving code - while they are far from my highest quality answers posted on the site.

  • Reputation encourages answering duplicate questions over and over again. Some users purposely do this to "grind rep" instead of closing obvious duplicates. It is always the same users.

    This in turn encourages low research effort by the posters. The easier it is to find the answer to a question yourself, the higher the probability that it will also get answered even though it is very likely a duplicate.

    And well, this problem has been debated endlessly over the years and a pretty solid, recurring argument from the answerer's perspective is that "it takes me less effort to answer the question than to find a duplicate". This is true, the lack of working on-site FAQ systems or other easy ways to find high quality duplicates remains an unsolved problem. Various sub communities tend to solve this by implementing their own tag FAQ, either on-site or off-site. I brought this problem up here as far back as 2016.

  • The system is terribly inconsistent. Up-voting costs nothing and yields +5 or +10 rep. Down-voting costs 1 rep and yields -2 rep. This is ridiculous and leads to rep inflation. As noted earlier, down-votes are a much more efficient way to ensure quality than up-votes, so they should weigh as much and be encouraged.

    What about those users who do heavy moderation work in tracking down spam and close voting bad content? Why are those users penalized for making the sites a better place? They will no doubt down vote far more frequently than they up vote.

    A stray thought: maybe implement a system where a user must remain within a certain ratio between how many up and down votes they have cast, so that they'll eventually get blocked from up-voting until they start to cast down votes (and maybe vice versa).

  • The rep system as well as the Q&A format in general is a terrible one for the meta sites. When these sites were designed, the devs were far too much in love with the Q&A format and thought it was the universal format for any kind of site. It is not. It is a system which encourages elitism, populism and discourages discussions. Instead everyone is supposed to be holding their own monologue and then people should silently vote on that monologue without explicitly arguing for/against. Don't you dare raise any unpopular arguments or counter arguments, or you may be down-voted into meta hell.

    And also the system kind of encourages having the same "beat the dead horse" discussions over and over, since it's far too hard to find older discussions, and even if you do find one and add an answer there, nothing will come of it since old posts don't get much attention, they aren't in the "feed".

    This is not a healthy system to use for meta at all. Good systems encourages diverse opinions and debates, bad systems put a lid on such things.


  • What other systems work really well that we could learn from?

The badge system isn't ideal either, but it is the best working one in terms of measuring activity and technical knowledge both.

As for unlocking moderation tools and site privileges, it should probably be disconnected from the rep system entirely.

Also, while SE isn't social media, the Q&A format discourages users from hanging out with like-minded people with similar interests, or for discussing things in a less formal manner, without all the rules against subjective topics. There are the chats, but they are well-hidden and very inactive. A place to "hang out" in parallel with the Q&A (which is definitely not the place for such) would give people more reasons to stick around and also ease "on-boarding" of new users, if there was a place encouraging informal mentoring.

One example of such a "hang out place" could be blogs. There is the SO official blog, which rarely ever holds contains anything of interest to me, and on top of it the bloggers have various company associations and likely hidden agendas, making it even less appealing. I'd be much more interested in blogs posted by the site users, maybe attached to the tags. There used to be such a system in place even, though it was so well-hidden that I didn't even know of it until I read the message that it was closing down due to lack of interest.

8
  • 1
    "Things like review queues should obviously not be unlocked immediately." yes, but when I gained my review privileges, I still did not know how to use them properly. I had to go read the Help Center and FAQs get advice from others and made plenty of mistakes. I don't think having any amount of rep or having written the answers that gained me that rep contributed to knowing how to review at all. Perhaps I had a weird case though. I gained all my early rep on SO from answering bounties, so I had little overall site experience at that point.
    – starball
    Commented Mar 15, 2023 at 15:35
  • "It is always the same users." - This is worth emphasizing WRT rep grinding. "Various sub communities tend to solve this by implementing their own tag FAQ, either on-site or off-site." Unfortunately, the off-site Python tag FAQ is still not very good. "The rep system as well as the Q&A format in general is a terrible one for the meta sites." Also an excellent point. A major part of the Stack system design is trying to emphasize that the main sites aren't discussion fora, but Meta needs to be a discussion forum. Commented Mar 15, 2023 at 20:43
  • 1
    "A place to "hang out" in parallel with the Q&A (which is definitely not the place for such) would give people more reasons to stick around and also ease "on-boarding" of new users" - chat, perhaps? It's horrendously undiscoverable, though, and diamonds seem to hate ideas like that due to a perceived expectation of them moderating a lot more text. Commented Mar 15, 2023 at 20:47
  • @starball Good point about the bounty system. I had forgotten about it but it is completely dysfunctional. Both for the reasons you mention here and because it locks bad posts from getting closed. This system should just be removed IMO.
    – Lundin
    Commented Mar 16, 2023 at 7:33
  • @KarlKnechtel Regarding the FAQ, that's why it needs to be implemented on-site and ideally in an exposed way so that people asking questions can easily find it. Some tags implement the FAQ below the tag wiki. This allows for maintenance and review by anyone, but the info is very much hidden away for new users. Such tag wiki FAQs are more of a collection of canonical dupes for the benefit of gold badgers etc.
    – Lundin
    Commented Mar 16, 2023 at 7:35
  • 2
    @KarlKnechtel I think the chats would be used a lot more indeed if they were advertised better. That diamond mods are supposed to maintain chats is ridiculous - I'd rather see a system where diamond mods are given the privilege to hand out "chat mod" rights to trusted users. "Chat mods" would then have the power to remove inappropriate content and kick out troublesome users from the chat. Diamond mods would only be called in as an escalation - "this user is a spammer/this user keep causing trouble". Or they can be called in (through meta) to mediate in case someone disagrees with chat mods.
    – Lundin
    Commented Mar 16, 2023 at 7:40
  • 1
    Alternatively if chats were tied to tags, then maybe give anyone with lets say a Python gold badge chat mod privileges to the Python chat.
    – Lundin
    Commented Mar 16, 2023 at 7:40
  • RE the up/down vote ratio to allow more voting. I understand the sentiment but I feel whatever the limits are set to would be too strict. One can very reasonably use the website for a decade and never need to cast a downvote (as opposed to abstaining from it). Imagine the user mostly uses it for research and upvotes content that helped them. Introducing a mandatory downvote count for them to continue doing that is not really useful for anybody, IMO.
    – VLAZ
    Commented Mar 16, 2023 at 12:54
11

The one thing I would absolutely change, is the rep cap. That should be per question instead of daily.

Take someone like Jon Skeet. He doesn't have to answer a single question anymore to still hit the rep cap every day for years to come.
I've met him once and he's a nice guy. I think he deserves the rep he earns. But I still think it's weird that one would earn rep for answers given long ago.

Or take my "home stack", SF&F. It has received an answer to a question about Groundhog Day by the screenwriter of the film. Great answer, don't get me wrong. I'm not linking it here, though. That would exacerbate the problem.
But he has earned enough rep from that single answer to cast close and reopen votes, without ever interacting with any other aspect of the site.

This would be remedied by having a rep cap per question. Jon Skeet wouldn't get "rep for jam", Danny Rubin would've gotten 315 rep (200 from votes + 15 for the accepted answer + 100 bounty) instead of 4000+.

In general, people would need to interact and keep interacting to gain rep and earn privileges.

8
  • 4
    This seems to be a duplicate of OverLordGoldDragon's idea
    – starball
    Commented Mar 12, 2023 at 20:18
  • 1
    @starball darn, missed that one. Thanks. Edit: I do think I add more justification, though.
    – SQB
    Commented Mar 13, 2023 at 10:23
  • Don't you think it would remove the incentive to go back to your old answers and edit them? Why fix old answers if you can no longer get reputation from them?
    – Dharman
    Commented Mar 14, 2023 at 12:16
  • You'd still get upvotes and bragging rights.
    – SQB
    Commented Mar 14, 2023 at 13:04
  • Re "I still think it's weird that one would earn rep for answers given long ago", I think it would be weird if that was not the case. If you wrote an answer 5 years ago that was upvoted today because someone found your answer helpful to them, what on earth is "weird" about that?
    – skomisa
    Commented Mar 14, 2023 at 18:13
  • 1
    A related issue though is the undesirable "superstar" effect, where I'm convinced votes are granted mindlessly simply because of the name of the person posting, though I have no solution for that. (How else to explain why the nice but vanilla comment "fantastically well written and insightful post" to an answer posted here here received 11 fawning upvotes?)
    – skomisa
    Commented Mar 14, 2023 at 18:19
  • What's this about jam? I think I'm missing something about SO lore. Commented Mar 14, 2023 at 22:39
  • @KarlKnechtel I've seen the term used for "automatic" rep, but I'm not sure where I've seen it.
    – SQB
    Commented Mar 15, 2023 at 7:22
11

Simple answers to common questions get lots of reputation.

If you are lucky or, old enough to answer a question that is easy to understand and commonly asked, you'll likely get lots of reputation. Even if your answer is simply correct rather than the best possible. This can be true if the question is a (near) duplicate.

To game this system, there is a race to get a valid answer up quickly and improve the answer while the votes accrue.

Difficult answers to well written but specific questions get almost no reputation.

If you read a question that others can't quickly understand and provide an expertly written answer that covers all the angles and makes world leading suggestions that answers the question perfectly, you'll probably get 25 points of reputation.

Once these questions are answered, others will rarely invest the effort to understand them and consequently, appreciate your answer.


Giving a better answer to an established question gives unreliable, slow rewards.

If an existing common question already has a few well "up-voted" answers, there is a disincentive to answer, you feel like it is too late to get reputation. If the world has moved on and there is now a better answer, is it worth adding it to the question?

If the answer is good, it will attract votes, slowly. So slowly that it is unlikely to usurp the outdated previous leader. That old answer got all its rep in flash when the question was new, but now, floats at the top misdirecting the impatient.

6
  • 1
    for your second point, that's true, but I feel like it would be hard for the system to know what answer posts demonstrate more subject-matter-expertise just by votes alone. We do have the bounty system to give extra rewards to answers.
    – starball
    Commented Mar 21, 2023 at 17:30
  • @starball, bounties can be good when the asker chooses. I have only problems, not solutions.
    – Jodrell
    Commented Mar 21, 2023 at 17:32
  • Well commonly asked questions affect a lot of people and an answer therefore helps a lot of people. Conversely your well written answer to something obscure may only help that one person. Commented Mar 21, 2023 at 17:42
  • @RobertLongson, its a good point that I had considered. I guess my counter question is, should rep reward the effort of the answerer or the answers accumulated usefulness to the world? I don't know the answer. As a poor metaphor, should we reward the academic or the celebrity?
    – Jodrell
    Commented Mar 21, 2023 at 17:53
  • 3
    @Jodrell IMO, it's not a good metric for either. As of now, rep mostly correlates with how much time a user has spent on a site and how popular the questions they answer are. It's very sketchy to derive any other information. The next Alan Turing can come in and post something so detailed and insightful that it would single-handedly push computer science a decade forward in advancement. Yet if nobody sees it then no rep for poor new Mr Turing. Similarly, there are plenty of users who have five and six digit reputation yet only from prolonged exposure over a decade, not pure contribution.
    – VLAZ
    Commented Mar 21, 2023 at 17:59
  • "Simple answers to common questions get lots of reputation." This isn't inherently a problem. Common questions are more valuable exactly because they are common - having them properly asked and answered is beneficial to more people. Commented Mar 21, 2023 at 23:47
10

I see several long term users award bounties for good answers, nice in itself, but they award 500 rep bounties to new users, often on first answers, sometimes repeatedly.
This creates a user who has access to tools and actions they have not had time to get to know. And I have seen some that did act with that access in a way that I did not see as good. (Not bad enough to hand on to staff but it took a lot of comments, and even some e-mails to the user, to teach them how not to use those options.)

I would suggest that no bounties can be awarded to a user who has fewer rep than the bounty is. (Maybe put it in hold somewhere, to award when they reached a next level of reputation.)

3
  • If it is more than a few bounties, it is probably reputation points laundering (from voting rings) and/or selling reputation points for real money. Commented Mar 10, 2023 at 10:28
  • 1
    The people offering the bounties are long standing high reputation good users several of whom are in the top 10 of reputation level. The awarded bounties can go to other long standing members or newer (to very new) members. There is no reason to expect faul play.
    – Willeke
    Commented Mar 10, 2023 at 11:19
  • 6
    Yeah, I can foresee situations where the users who are starting bounties are just doing so because they have a lot of rep and want to support good answers – they're just not thinking about the fact that the rep they're giving out might end up giving the recipient a bunch of new privileges that they didn't have before, due to the way our reputation and privilege systems are connected. (And to be fair, I don't think this should be something they should need to worry about, either – the system should be designed in a way where that isn't a potential problem in the first place.)
    – V2Blast StaffMod
    Commented Mar 10, 2023 at 21:17
9

This is a very isolated example and not a full-blown "what to improve" post. But it should be a guide when you improve things.

When you think about changes, please always make sure that the user receives constructive criticism.

Okay, a downvote is not constructive per se. But it is a pretty clear signal that this content as is is not wanted on this site. And as bad as it sounds, you could still "constructively" delete your own post, if it gets too many downvotes. Not great... not terrible. At least it stops the bleeding and improves the site as a whole.

However, once your question received an upvoted answer, you cannot. The system does not allow you to delete your own question.

So now there you are, with a question that is genuinely bad and maybe cannot be improved (lets say it was lacking research, you should not retroactively add the research that would invalidate the answer).

A downvote comes in. Nothing you can do. Another. Nothing you can do. Another. Nothing you can do. Just sit there and take the "punishment" with no way out. This is no longer about curating the site, since the obvious option to delete the bad content is banned by the system itself. It is only about revenge. Revenge for having done the bad thing in the first place.

Whatever you change, make sure that there always is a way forward for users, other than sitting there and "take it". There should be no traps, where users cannot react to negative feedback in a way to make it stop.

6
  • However there should be a limit - if there are enogh upvotes then you can't delete it., or perhaps you can disassociate yourself from the question. On other sites I often see my answers but to a deleted original - this is not good.
    – mmmmmm
    Commented Mar 13, 2023 at 12:05
  • 3
    You can always mod flag, argue why the content is better deleted and cannot be improved, and hope they agree. Especially if the question is closeworthy, you stand a good chance, and questions that aren't closeworthy can usually be improved.
    – Erik A
    Commented Mar 13, 2023 at 14:27
  • At a certain (pretty early) threshold, the post is no longer shown in the post listing. I believe that threshold is something like -4. There's the current mercy mechanism. And what about the people who took the time to write an answer? (yes, I know answering bad questions isn't considered good practice, but it's not outright banned). Should they still lose rep when their answer gets deleted with the question? The current mechanism is that they do unless the post existed for 5 months already (or something along those lines).
    – starball
    Commented Mar 13, 2023 at 16:45
  • 3
    Add to the fact that the asker gets an answer to their question- even despite it being a low-score (probably bad) question. That's pretty constructive to me. I've heard that a lot of askers don't even care about rep and just want answers.
    – starball
    Commented Mar 13, 2023 at 18:20
  • 1
    I'm not sure if that maybe did not come across: I am not advocating for a specific solution here. I have enough of those answers, that I understand that deleting it would not be cool either. I am just advocating to stop the punishment if there is no constructive way forward for the punished. Our site is about improving the post, not punishing the poster. If improving the post (even as a last effort by deleting it) is not possible, punishment should stop.
    – nvoigt
    Commented Mar 14, 2023 at 6:40
  • “ you should not retroactively add the research that would invalidate the answer” imho, if the asker knows, how to improve the question, they definitively showed ( maybe with “UPDATE” mark). Commented Mar 14, 2023 at 11:04
9

A frustration for me (I currently have ~1300 reputation on Stack Overflow) is that when I want to edit posts, I usually get an error saying the review queue is full. I can't do anything about this because I don't have sufficient reputation to review things in the queue. I usually avoid making edits as a result.

Most of my edits are obvious changes (spelling, fixing broken links). I think it's sad that I've been contributing to Stack Overflow for over 11 years and am unable to make these simple changes.

1

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .