5

Does the iron law of oligarchy apply on Stack Overflow and other Stack Exchange communities? How do we measure it?

From Laboratories of Oligarchy? How the Iron law Extends to Peer Production:

Robert Michels’ “iron law of oligarchy” ([1915]) [is] an influential sociological theory which proposes that as membership organizations become large and complex, a small group of early members consolidate and exercise a monopoly of power within the organization as their interests diverge from those of the collective.

In Shaw and Hill 2014, Laboratories of Oligarchy?, the iron law was confirmed on hundreds of large MediaWiki wikis beyond Wikipedia:

Our findings show that peer production communities tend to follow Michels’ iron law of oligarchy [...] We note that some wikis in our dataset appear more robustly democratic than others.

They looked at some data which depends on MediaWiki-specific bits, such as the sysop group, page histories and reverts:

Leadership roles remain concentrated among an extremely small group of elites as organizational membership grows, affirming H1. This group of administrators is increasingly active in administrative activity, controlling for the total amount of such work, consistent with H2. Finally, we observe that as membership grows, the number of reverts of experienced users by administrators tends to rise, supporting H3.

On Stack Exchange, it's less important which version of a post prevails. More impactful is probably where the flow of votes is directed, e.g. by closing questions or bringing an answer to the top. Some voting patterns are private.

What data could be tested on Stack Exchange to verify the iron law? Maybe some of the following?

  1. Does the reputation produced in any given day become increasingly concentrated in a few hands? (Maybe a few oldtimers "own" the questions and answers which get more traffic.)
  2. Do the review queue votes become increasingly concentrated?
  3. In general, do the upvotes and downvotes become increasingly concentrated (both in their origin and target)?
  4. Does some kind of "minority" group of experienced users emerge, who are constantly sidelined in votes (e.g. close votes)?
  5. Does it become increasingly rare to achieve reputation increases with new posts (for new users in a site or in a tag)?
  6. In any given tag, do the gold tag badge holders increasingly channel reputation towards few users, e.g. by closing questions as duplicates of questions with accepted/upvoted answers by a specific group of users (themselves or others)?
  7. In any given tag, does a group of users or posts emerge which mostly gets upvotes (or downvotes) from a certain group of low/high reputation users?
  8. Can we calculate a "social mobility matrix" of users? What would be the analogous of one's parents' wealth? Does the rank change in the user reputation league tell us anything about mobility?
  • Cf. @jon-ericson jericson.github.io/tag/race_to_1k.html – Nemo May 5 '18 at 7:20
  • some of this can be answered with a query. – rene May 5 '18 at 7:24
  • @rene yes, that's the final goal. But first I'd like to hear what data is useful to extract. – Nemo May 5 '18 at 7:25
  • 1
    What is the follow up if the iron law is confirmed? – rene May 5 '18 at 7:26
  • 3
    Do you expect the potential oligarchs on Meta Stack Exchange to provide you with data to prove the existence of an oligarchy? </jk> – Glorfindel May 5 '18 at 7:29
  • 3
    @rene If we establish the consequences before the premises, we may bias our findings (e.g. search only what confirms the need of what we'd like to see). But I can imagine two things: 1) some of the patterns observed in other communities are confirmed in SE, therefore SE can study some learnings from those communities and see what can be applied here; 2) some patterns happen in some SE sites/tags but not others, and by looking closer the SE community identifies some best/worst practices worth looking into. – Nemo May 5 '18 at 7:31
  • 2
    I'm not going to facilitate a witch hunt that will have me as an outcome but this comment is also evidence the law is confirmed. – rene May 5 '18 at 7:34
  • 7
    @rene Why would it be a witch hunt? When this was studied on Wikia, nobody was demonized. We might also happen to find out that the iron law doesn't apply to 99 % of StackOverflow tags but a tiny minority of tags have some odd dynamic, for instance. – Nemo May 5 '18 at 7:41
  • 1
    Note, Jon Skeet's average is 5 upvotes per answer on the SO. He has so many rep because he wrote an extreme mass of answers. My direct stat results has shown for me that the system is in fact not so bad as it seems. – peterh says reinstate Monica May 6 '18 at 0:57
  • 1
    @peterh I guess that depends on your starting point for "it seems". To me, SE's democracy seems very robust on paper (one persone one vote and whatnot, reputation caps etc.), so my expectations are rather high. But I want the data! – Nemo May 6 '18 at 5:15
  • 5
    Just to be clear, the 'iron law' as you quoted it explicitly claims that the minor group with major influence are the old-timers, but I don't see that in all of your specific data queries. I think this 'old-timer factor' is very relevant, as otherwise the effects we see is merely the 20-80 rule. (the minority of users do the majority of work). So, I think it is important to stress that aspect a bit more. – Discrete lizard May 6 '18 at 8:26
  • 1
    @Discretelizard thank you for pointing it out. Feel free to edit my question. My example research questions mock those that the paper found feasible to analyse. There are certainly better ones I have not thought of! – Nemo May 6 '18 at 8:28
  • 1
    This query shows the entropy density of the answer upvotes on month basis in the history of the site. It shows a slow decrease since the beginning. If you show interest, I am ready to make it parametrizable with tags. – peterh says reinstate Monica May 7 '18 at 18:23
  • 1
    Oh by the way, I see there was a Joel blog post on how reputation/gamification can end up being non-inclusive joelonsoftware.com/2018/04/13/gamification – Nemo May 12 '18 at 12:52
  • 1
    @Nemo I've made another entropy density-based calculation, the entropy density of the used tags. It shows a slow increase. Thus, the used technologies are becoming slowly more and more diverse on the site - compared even the (typically, growing) count of both the questions and the tags. – peterh says reinstate Monica May 12 '18 at 19:34
9

I will try to answer some questions, but for the most of your though it’s just that some members are more active, thus giving more time back to the community by being active and helping users, calling them an oligarchy can be a lack of respect.

Don’t forget that without some active members some sites would just die. SE brings the background, but it’s up to the community to make a site viable.

  1. Does the reputation produced in any given day become increasingly concentrated in a few hands? (Maybe a few oldtimers "own" the questions and answers which get more traffic.)

Like I said, the more active you are or were in a community, the more points you will have. It’s a simple rule.

  1. Do the review queue votes become increasingly concentrated?
  2. In general, do the upvotes and downvotes become increasingly concentrated (both in their origin and target)?
  3. Does some kind of "minority" group of experienced users emerge, who are constantly sidelined in votes (e.g. close votes)?

On some community, there are like only 5 to 10 active reviewers. Don’t like their ideas for closing questions? Then be active and do reviews in the review queue, it’s that simple.

  1. Does it become increasingly rare to achieve reputation increases with new posts (for new users in a site or in a tag)?
  2. In any given tag, do the gold tag badge holders increasingly channel reputation towards few users, e.g. by closing questions as duplicates of questions with accepted/upvoted answers by a specific group of users (themselves or others)?

Yes and no, you are an expert in your category/tag, then yes rep will be like a magnet on you. Some users do very well writing answers and yes, such answerers get more upvotes, but some old questions will always get upvotes, but remember that if a member was active and helped start the community, you can’t blame him.

  1. In any given tag, does a group of users or posts emerge which mostly gets upvotes (or downvotes) from a certain group of low/high reputation users?

Certain tags are watched by third party, like Microsoft was with Internet Explorer, I don’t know if still true, but a third party specialised team following a tag, you understand that their answer might prevail, and will not call a new comer stupid too, so yes, certain tags can get special attention.

6

Old-timer here. I'm not good enough with SEDE to crunch some numbers, but I'd like to share my experiences and observations.

Does the reputation produced in any given day become increasingly concentrated in a few hands? (Maybe a few oldtimers "own" the questions and answers which get more traffic.)

Probably not. There are two big effects I've noticed - HNQs which result in a lot of reputation at once, and folks accruing passive reputation over time. Without a certain degree of consistency though - except on the biggest sites - someone who consistently posts good answers can easily catch up with old-timers. One of our mods basically got to 30K and modship within 2-3 years.

In general, do the upvotes and downvotes become increasingly concentrated (both in their origin and target)?

Fresh questions get attention by design, so... unless its a frequently referenced question, it's unlikely an old question will get a boost. On the other hand, a good new answer to an older question gets it attention and may help.

There is no inherent advantage to tenure, outside eventually working out what works in a community. I have lots of reputation because I post a lot of answers.

Does some kind of "minority" group of experienced users emerge, who are constantly sidelined in votes (e.g. close votes)?

I'm not aware of it, and I don't think I consciously vote against people. I try my best to guide folks who're 'doing it wrong' since if someone is consistently having a bad time, they'll end up getting post-banned or have other issues.

Does it become increasingly rare to achieve reputation increases with new posts (for new users in a site or in a tag)?

I don't have data. I'll leave this to someone else. I personally look at what I know, and what I need to know, and not who posted it. Unless it's a new user who posted a spectacular answer and positive reinforcement helps.

In any given tag, does a group of users or posts emerge which mostly gets upvotes (or downvotes) from a certain group of low/high reputation users?

We have very little visibility of voting trends like that.

6

Well... To answer your primary question

Does the iron law of oligarchy apply on Stack Overflow and other Stack Exchange communities?

as per the definition you provided:

iron law of oligarchy is an influential sociological theory which proposes that as membership organizations become large and complex, a small group of early members consolidate and exercise a monopoly of power within the organization as their interests diverge from those of the collective.

The simple answer is no, by this definition of "iron law", Stack Overflow is not an "iron law of oligarchy" because there is no small group of early members who consolidate and exercise a monopoly of power within the organization as their interests diverge from those of the collective.

Therefore, I conclude there is nothing to measure.

  • How do you know there isn't? I mean, I can agree that it looks like there isn't, due to various systems and appearances, but how do we prove it? – Nemo May 7 '18 at 20:47
  • 7
    @Nemo Not to get too philosophical but you cannot prove the lack of existence of anything. But if you know of any small group of early members [...], please voice who they are and how they've exercised their diverging interests on the everyone else... I think it's on the OP to give that example though. – Izzy May 7 '18 at 21:11
  • 1
    Izzy You are missing the point (as I understand). @Nemo is not saying there is a small group, but "how would we identify such a group by looking through the data on SE". – Martin Bonner supports Monica Oct 19 '18 at 7:20
  • 1
    @MartinBonner what are you referring to? Nemo asked if this law applies and I answered in the negative. There is nothing that fits the definition he gave and he has not offered any evidence to the contrary. Are you telling me "Does this iron-law apply here" doesn't imply "Is there an iron-law group here"? – Izzy Oct 24 '18 at 16:18
3

If I take your question literally, the answer is "no".

BUT, as for what I can infer about the spirit in which you're asking it, the answer is "Possibly, but it's not as bad as Quora!"

A lot of people have quit SE over their thoughts on this: http://archive.is/kTKqG

"Are privileged trolls making my 'community' total garbage?"

Illustration:

enter image description here

Probably not. It's just the way accumulation works, and all websites have it. At least here, the situation is obvious and can be data-mined:

http://varianceexplained.org/r/are_users_quitting/

What you're talking about is a specific example of the general case of homosocial reproduction. The sad news is that it happens everywhere, sometimes to a group's detriment, but often not.

I think the May 1993 paper called "Homosocial Reproduction," and other papers based on it, might put this in a broader context. A lot of the research on this subject surprised me. For example, which groups are the most "Homosocially Reproductive"? I made a list of what I thought the results would be based on my intuition, and it was incorrect. This subject is especially useful to study in light of the recent controversy over the USA H1-B Visa system.

The good news is that, to the extent that this phenomenon is occurring at SE, it's on the basis of stuff like points and coordination artifacts of SE's ranking system, rather than styles of dress, resumes, or hairstyles. Still a bit silly, sometimes, but less silly than meatspace.

But this isn't the type of question you should ask on Meta, because you're only asking the people who are left.

It's like opening a public speech to a group with, "Are y'all hypocrites?"

The people who thought initially thought that the group was hypocritical before meeting you are, most likely, no longer in it.

The emotions involved might make it difficult to get a usable answer.

Better to ask this on a sociology forum, or somewhere where the people whose motivations you're pointing at aren't the people answering. And voting.

  • 2
    Sure, focus groups or interviews on Meta.SE are probably not a very scientific method to ascertain it. But my premise is that some studies found a quantitative, relatively objective way to measure the "iron law" effect. Something like that could be devised with internal insight, although it's possibly more likely to come from outside. – Nemo Sep 25 '18 at 4:49
  • // , No, they brook no such criticism from a relative outsider, like yourself, especially given your presentation of the idea. Better would have been to ask, "How can we make SE more egalitarian in terms of the following definitions of rank and social mobility", and then adding some of your own algorithmic answers to join the rest. The "hey you guys might be oligarchic hypocrites, let's all use data to find out" approach might be correct, but it doesn't feel good, for what I hope are obvious reasons. – Nathan Basanese Oct 18 '18 at 15:30
  • // , Our brains are hard-wired to think in terms of "us" and "them", and this fact of moist meatspace has a lot of implications for our badlands of bits and bytes. – Nathan Basanese Jan 10 at 19:45
-17

Very nice text, good to see it! I am really hoping, your question won't be closed/deleted soon.

In my opinion, the answer to your question is very clearly YES. Exactly this is what I experience here since years. Your text is very useful, and it surely deserves to read and understand it.

There is also a large hidden or unsaid politics here. It is easy to lost in the labyrinth of said, suspected interests and motives of the different groups. But, as I can see, the background is mainly evil, most sites are ruled by an "inner circle" whose only goal is to cement their "power" until the eternity. This significantly worsens the enjoyability of the whole site network.

It is the "normal" behavior of the internet communities what I've met since the first IRC channels. This mechanism will ultimately kill the site, like it killed IRC and Usenet, too. The same mechanism makes the Wikipedia today practically unusable as an editor in exactly the most important topics.

The difference is this time, that the SE, being a profit-oriented company, need to do everything against it. Watching them I clearly see that at least they are trying. Which is particularly hard for them, being also part of the system. One side of the SE tries to harden the terror, other side wanted to make it more open, while the whole SE tries to show always a consistent face to the public.

To analyze and handle the problem, fortunately there is a way, the SEDE. I've created a huge mass of queries there, targeting mainly exactly this problem.

Unfortunately, exactly the most important data (voting details, deleted posts) is there hidden, but there is still a lot of useful info and possibilities to know and understand what we want. I think, more effort could (should) be put into this direction, particularly to detect bad community mechanisms, and to support constructive arguments in meta discussions.

Relevant nice read, particularly if you have to deal with people on the metas.

What I would like to say in the defense of the SE: if you only want to answer interesting questions and post your interesting questions, it is not so bad. You need to have the hard skin against the unexplainable cruelty (for example, if your clearly ontopic, interesting question is closed with an unrealistic reason), and comment bullying. The problems are coming if you want some "higher" goals, for example to get to 3k or similar. Then a large part of your effort will be spent on fighting them, instead of answering posts.


Extension, some of my queries:

  1. This query shows the entropy density of the answer upvotes on month basis in the history of the site. It shows a slow decrease since the beginning. And here you can parametrize the same query with tags.
  2. It is impossible to show, because the reviewer data is not available in the SEDE. By statically mirroring the site with a crawler, the required columns could be generated for the reviews of not yet deleted questions and not yet deleted users. It would be a huge work and might violate the SE terms of services (essentially, you are free to play with the SEDE, but you are not allowed to crawl the site too vehemently).
  3. The solution for (1) could be easily extended to handle downvotes, however, we have no way to identify the OPs of the deleted posts. And the most downvoted posts are also deleted, too. Furthermore, there is also no data in the SEDE for the originators of the votes. If you show interest, I can improve (1) to handle this.
  • 8
    Sorry, which of your queries are relevant to this question? I cannot see any on the first page of your list, except perhaps the one about most active "closers" (which however does not show an increasing concentration). – Nemo May 6 '18 at 5:03
  • 1
    @Nemo They are mainly not on the first page, and many of them doesn't do exactly what you want, but a large part of them were developed exactly for the problem you mention. I am planning to develop some queries also for exactly match your examples. – peterh says reinstate Monica May 6 '18 at 10:06
  • 1
    @Nemo I an planning to build queries or show my old queries for all of your questions. – peterh says reinstate Monica May 7 '18 at 18:28
  • 1
    @Nij 1) I answered around 2, maybe 3 times only below this question (example), and numerous times MSE-wide. 2) I flagged your post as rude/abusive, because it is. 3) I love the SE, I "hate so much" (cite from you) only, for example, you and the people like you, although I would prefer a nicer terminology. For example, I "hate" people wanting to make all punishment harsher, always more questions to close and so on. I simply don't understand them, why is it so good for you – peterh says reinstate Monica May 7 '18 at 19:38
  • 1
    @Nij to always harm others. I tried to understand you many times, but you typically don't tend to share your ideas and vision. My impression is that if you had a free hand on the site, you would forever ban everybody, and you would build a site which is enjoyable only for a narrow circle of people, of course you would have always a central role in this circle. But, I reverse your question: why do you vote so many downs and close? Why do you want to punish all the "crimes" more as they are punished now? What do you want from it? – peterh says reinstate Monica May 7 '18 at 20:36
  • 9
    Peter - I keep trying to be polite to you but you make it increasingly difficult by bringing out your offensive conspiracy theory stuff all the time. I consider myself an old hand, and moderate a few sites and I can categorically state: there is no inner circle - it just doesn't happen. There is no conspiracy. No "cabal". Moderators watch each other, and SE watches mods. And we just try to help calm down fights, clean up trolling, spam and abuse etc. If you think it is an inner circle downvoting you all the time, it isn't - you just earn a lot of down votes because of what you post! – Ro-Reinstate Monic-Alsop May 7 '18 at 21:00
  • 1
    @RoryAlsop You are partially oversimplifying my statements, partially altering them. If you disagree some of my posts/comments, then react below them, to the actual post/comment. – peterh says reinstate Monica May 7 '18 at 21:27
  • 6
    Dude - I have done so many many times. And you never listen. – Ro-Reinstate Monic-Alsop May 7 '18 at 22:02
  • 2
    @RoryAlsop Ok. The aging and dynamics of Internet communities is already a well documented and well understood thing. As far I know, it is a hot and popular area in the sociology and border areas. Thus, what I am talking about, is not from the air. It is a well documented sociological phenomenon. They shouldn't be a conscient, monolithic cooperation, although such cooperations are surely existing. In my texts, at least in the latest years, they were always as a sociological phenomenon. – peterh says reinstate Monica May 7 '18 at 22:31
  • 2
    @RoryAlsop ...what doesn't mean that they wouldn't be exactly so harmful. Btw, we had many talks in the latest year, some of them were quite long - actually much longer as I would expect from a "not-listener". – peterh says reinstate Monica May 7 '18 at 22:33
  • 1
    @RoryAlsop In order to avoid to think next time, that I don't listen, I suggest to read my this essential comment, and react it accordingly. I would be really unhappy, if I will need to refer this comment next time. – peterh says reinstate Monica May 8 '18 at 15:17
  • 10
    Next time spend less words asserting there's a problem and more words demonstrating it. Your one query successfully demonstrates that... Votes per user were more concentrated when the site was very young and had many fewer users. It notably fails to demonstrate that votes are concentrated on a specific subset of users, then or now. – Shog9 May 9 '18 at 2:29
  • 2
    @Shog9 I said that the stats show a much better system than I experienced it. I think what I actually said here is far lesser critical against the SE than it seems only from the attitude of the post. :-) – peterh says reinstate Monica May 9 '18 at 5:11
  • 1
    @Shog9 ) This stat doesn't show entropy, it shows entropy density (it calculates the concentratedness of the votes compared to the growth of the users). In this comparison, there is a slow decrease (votes become more concentrated, compared to the growth of the user base), but the effect is minimal. It decreased from 70 to 50 since 2011. – peterh says reinstate Monica May 9 '18 at 5:13
  • 2
    @Shog9 Furthermore, on many sites or SO tags I've checked, the entropy density is roughly static (-> there is no oligarchic direction in this sense). Furthermore, a decrease of the voting entropy density can be also explained, that a small user base is created, writing HQ answers, and "holding the front". | Btw, the absolute value of the entropy density correlates well, how I felt welcomed on the different sites. – peterh says reinstate Monica May 9 '18 at 8:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .