18

In the discussion on design-independent graduation, which stemmed into a more general discussion on the graduation process, it occurred to me that there's an obvious solution to the reputation-change problem: make reputation thresholds scale automatically to meet demand.

This isn't a solution without its problems, though, so I'd like to see if we can figure out how to make it work. This isn't a working feature-request quite yet; it's a discussion on how to make (or whether it's possible to make) the feature request work.

When I say 'automatically scaling reputation thresholds,' I imagine a system wherein edit, close vote, review queues, tag wiki, trusted user, possibly even downvoting, etc. privileges scale automatically to meet the demands of the site. This is a complicated proposal with a lot of moving parts, so bear with me for a bit.

Why do we want this at all?

Yes, it's a complicated proposal. However, it solves a couple major problems:

  • It improves site graduation. Instead of sites experiencing a huge privilege shock when they graduate, they'd experience a nominal transition. (Remember that when a site graduates, a lot of users lose the privileges they've already been granted.)
  • It improves the process of site growth. From private beta into public beta, there are (often long) interim periods where the community moderation needs are not being fulfilled. This can result in some bad community standards which need to be cleaned up later, but would be better addressed at the outset.
  • It brings on active community moderators only as necessary to fill the needs of the site. When too many people are afforded unrestricted privileges, things can occasionally get a little awry, and it can be hard to spot. By having a smaller influx of privileged users, not only will those users have more experience but they'll also have an easier time learning by example from existing reviewers.

Obviously these things are a little less effective on larger sites, but in a lot of respects, larger sites are a lot more chaotic than smaller sites, so that's probably to be expected.

Here are some of the major points about such a system, and some of the unsolved problems I've been pondering:

  • My initial suggestion was that once a privilege is granted, it isn't revoked. The reason I had suggested this was to avoid confusing users.

    However, it was pointed out by HDE and Scimoster that this might not be the best approach, as the base of users able to review should change to match who's currently active on the site. Scimonster suggested a notification when you're about to lose a privilege, which makes sense to me.

    Consequently, if you stop being active on the site, you'll lose the associated privileges that are afforded at your reputation level.

    That also avoids the problem wherein there is a potential to have users late into the site's existence with long-term privileges and very little reputation. Granted, the only way they could end up there is by being inactive, which is a self-solving problem.

  • Heuristics need to be developed for what the "demand" for each privilege actually means. For review queues, this is easy: lower the threshold until enough people have the privilege to cover the queue. For voting, closing, and edits, it's a little more complicated, and I'm not sure how to do it.
  • Consideration of inactive users: As sites age, they develop a larger base of less-active users. A general metric for the activity of each user with regards to each automatically-scaling action needs to be created and balanced. This has to feed into whatever algorithm is determining how many users to add; it's not just about how many users there are, but how active those users are at moderation.
  • Minimum thresholds need to be set. There has to be a minimum reputation for each privilege, otherwise a site like Stack Overflow would set the close-votes queue to 10 reputation. Those could probably stay the same.

Finally, this last one is the most significant and hardest to solve:

  • The raw reputation thresholds need to be presented in a non-confusing way. I'm not sure how to accomplish this, and it's a major holding point on this idea.

    A couple ideas which occur to me are to hide reputation thresholds for privileges entirely, presenting solely their order, or alternately to show that the top M users have been granted a privilege.

I do realize this needs work, which is why this isn't a feature-request yet. I'm trying to figure out how to solve these problems here, not proposing it as a feature request or full solution.

The task is a bit too much for me to piece together in my head, so please feel free to answer with whatever feedback and information you think is necessary to make this work.

|improve this question
  • 5
    "Once a privilege is granted, it isn't revoked." - If you slip beneath the reputation threshold for a certain privilege on Stack Exchange, you lose it. It would also make things weird - why should one guy with 1k rep who is now inactive have more privileges than a newer guy with 1k rep, who missed out on the lower privilege levels? Overall, though, I really like the idea. – HDE 226868 Jul 27 '15 at 20:19
  • @HDE226868 Maybe some sort of inactivity expiration would be helpful, then? – user206222 Jul 27 '15 at 20:33
  • That works, so long as it's hard to dodge the minimum activity requirements. – HDE 226868 Jul 27 '15 at 20:35
  • What are your suggestions on the algorithms to be used? Details do matter. – Deer Hunter Jul 28 '15 at 0:27
  • @DeerH I'm still mulling it over, and will likely update the post later, but I don't have very firm ideas at the moment. – user206222 Jul 28 '15 at 0:53
  • 1
    Statistical information that may be helpful: I've observed that the reputation scores of users on a site tend to follow a Power Law distribution. – 200_success Jul 28 '15 at 9:29
11

Once a privilege is granted, it isn't revoked. That would make things horrifically confusing to users ("wait, I could do this yesterday, and I can't today?"). Besides that, we expect sites to grow, not shrink. If they're shrinking fast enough that it's an issue for the algorithm, it's likely the site has other issues which need to be addressed.

I disagree with this one, for the same reason as HDE 226868:

If you slip beneath the reputation threshold for a certain privilege on Stack Exchange, you lose it. It would also make things weird - why should one guy with 1k rep who is now inactive have more privileges than a newer guy with 1k rep, who missed out on the lower privilege levels? Overall, though, I really like the idea.

I think that users' personal privileges should roll with the site's. Once you earn a privilege, you have to stay active to keep it. Otherwise, you might lose it when the site grows.

I do think it's mean to just pull the rug out from them though. What can be done is to notify the users. Just like you get a notification when you earn a new privilege, we could send a message to any users who will lose a privilege, "The privilege ___ has just changed from xxx rep to yyy. You're only zzz points away from re-earning it!" This a) lets them know that they (temporarily) lost the privilege, and b) encourages them to regain it.

Overall though, it's an interesting idea, definitely worth some serious consideration.

|improve this answer
  • 1
    I'm convinced. I've edited this into the main post. – user206222 Jul 27 '15 at 21:51
6

I appreciate and respect what you're trying to do here. However, I have some reservations about this idea.

I think it could get a bit confusing. I can just imagine telling users: "once you get to 1632 rep, you can edit other people's answers and have it take effect immediately", which would immediately raise the question "why 1632?" -- not to mention that two months later it might actually be 1636 instead of 1632, so earlier explanations might become confusingly wrong. Also, we'd lose the ability to talk about 10K-users (who can view deleted posts and review queues) in a simple way.

Probably those are tolerable: a minor annoyance. Here's the bigger wildcard, for me. Humans are not computers. A threshold like 1K or 5K or 10K is a meaningful threshold -- it is an accomplishment that is noteworthy and memorable in its own right, regardless of any privileges that it brings. A number like 1,000 might be arbitrary in principle, but in practice you notice when the thing increments and you're now in 4-digit territory. I think there's something to be said for aligning the awarding of privileges with those inherently-significant milestones. It creates enthusiasm and provides a kind of recognition of this milestone -- it's not just me sitting there feeling pretty awesome about having made it to 1K, the site is celebrating my accomplishment and showing recognition for the contributions I've made, at a time that already has some inherent meaning to me. That has value, I think.

|improve this answer
  • The problem of 10k users is mostly that no one came up with any actually good names for the privilege, so we're stuck using one that is merely least terrible... it already breaks on beta sites, where it's called 2k instead. Anything that pushes the network into coming up with a legitimate name that doesn't self-reference rep here is arguably a good thing. – Nathan Tuggy Jul 28 '15 at 14:47
  • I've been thinking about this a bit, and perhaps the solution is to limit the automatically-scaling thresholds to meet only extraneous/smaller features, like review queues. Everything else could stay the same. – user206222 Jul 29 '15 at 21:01
2

You might make gaining privileges easier than losing them. For example, if you gain a privilege at 2000, you lose it if you fall below 1000 or if gaining the privilege goes above 4000 and you're still at 2000.

This would avoid someone who is growing right at the site's growth curve getting a series of messages:

  • You have gained privilege Foo for having a reputation of 2000!
  • You have lost privilege Foo as the requirement has gone up to 2100. You need 10 reputation to regain it.
  • You have gained privilege Foo for having a reputation of 2200!
  • You have lost privilege Foo as the requirement has gone up to 2300. You need 5 reputation to regain it.

So on and so forth. One of the nice things now is that these messages are rare. There aren't that many privilege levels and you only get one or two notifications per level. Under this system, you could potentially get an unlimited number of notifications.

This still handles the case of an active user becoming much less active. They'll eventually lose their privileges, just a bit slower. But it doesn't make users lose privileges regularly in the way that having a single moving threshold does.

Someone who is active should rarely lose privileges, not gain and lose them on a daily basis. Just my opinion.

|improve this answer

You must log in to answer this question.