Benfords law as it relates to SO reputation [closed]

Question

This isn't really a question but I thought that it may interest people here to see that SO reputation seems to follow Benford's law

Here's the data I got from statoverflow (I omitted accounts with reputation = 1)

Compare this to Benford's predictions

Answer 1

I think I have an explanation for the anomaly at 3. I queried not just based on first digit, but also on number of digits.

Here is the query I used:

SELECT LEFT(reputation, 1) AS first_digit
     , LENGTH(reputation) AS num_digits
     , COUNT(*)
FROM users
GROUP BY first_digit, num_digits
ORDER BY first_digit ASC, num_digits DESC

After massaging that a bit, you get this as your raw data:

1st Digit   5 digits   4 digits   3 digits   2 digits   1 digit
        1        190       2504       7049      12838     52486
        2         30        876       2973       7180         0
        3         11        485       2150       4634      2623
        4          7        274       1578       3074         0
        5          0        157       1114       2336       301
        6          1        128        899       1972         4
        7          1         84        687       1481       146
        8          0         62        618       1305         0
        9          1         40        446       1141       447

The 1-digit reps throw things off. In particular, there is a large number of users with rep of 3, because that is your rep if you ask one question, get no upvotes, and accept some answer. Also, due to a bug, if a new user receives a downvote, and then the downvote is revoked, they will have rep=3.

Let's look at that in chart form. First, here is all the data. Clearly the users with rep of 1 are anomalous.

---

Now, let's filter out those users with rep=1. Now we can see that there are anomalous spikes for rep=3 and rep=9. (Rep=9 happens with 1 upvote and one downvote.)

---

Finally, lets filter out all users with one-digit rep. We get a result that is much closer to an ideal Benford's Law distribution now.

Answer 2

Could the excess number of 3s as starting digits be related to the fact that 3000 is the required rep level for closing/reopening posts, which is kind of the last stepping point before 10k. If people are motivated by achieving certain targets I would imaging that motivation to "level up"/gain rep falls off after 3000 as the next target is so far away leaving a larger number of people in the low 3000s.

Answer 3

May I direct everyone reading/contributing to this question to "How addicted to StackOverflow are you?"

The answer "I download rep data and compare it to Benford's law" still hasn't been given as an answer yet.

Answer 4

If you haven't seen it before, Benford's law is pretty interesting, and seems on the face of it rather amazing. It takes a bit of thought to see how it arises, but once you do, it isn't at all surprising to see it appear in the Stack Overflow reputation graph. I highly recommend this well-explained article that goes through the details with a minimum of maths, and with some great examples.

Quoting from that article:

We can conclude that data from any distribution will tend to be 'Benford', as long as the distribution spans several integers on the log_10 scale - several orders of magnitude on the original scale - and as long as the distribution is reasonably smooth.

We have reputation spanning 1 - 100,000. That's 6 orders of magnitude. If we didn't see something close to Benford's law, that would be really interesting!

Answer 5

This suggests that log(reputation) is more-or-less uniform. We can test this hypothesis directly since I understand the data's available.

The next question is then: why is log(reputation) uniform? It makes sense that fewer people would have high rep that low rep, of course (since rep takes effort), but a uniform distribution of reputation seems equally likely. Or is there some mechanism through which the people who have reputation are more likely to gain more rep than people with fewer to start with? A sort of Matthews effect?