193

While I understand that "The Loop" is not a research instrument in the same line as what a social scientist would develop, it is still a survey instrument and reflects upon us all (i.e., poorly designed surveys actually lead to fewer people taking them overall). As such, most surveys follow the American Association for Public Opinion Research (AAPOR) Best Practices for Survey Research.

Looking at the survey, I really don't understand why demographic information (i.e., race, age, and gender) are being asked since it appears that the clear goal is to get an initial impression of "how satisfied are you with your experience using Stack Overflow?" While I understand that a lot of people will include these by default, unless it is relevant, it's generally taught in survey methods courses to avoid asking the questions.

In this case it appears that your sample frame is effectively all developers in which case demographics really aren't needed​ for this particular question. In scientific research, demographics are only requested if there is evidence that a demographic attribute will lead to a significant difference in responses (ex., "Does a respondent's age influence the amount of money they save for retirement?"). Otherwise, demographic questions are left off to avoid respondent fatigue and because paper and ink is expensive!1

Another reason why this is problematic is that the question is asking about "racial background" but the survey appears to be directed at an international community. Needless to say that from that standpoint, the demographic information looses all of its value as a data point since the respondents don't belong to the same cultural background, i.e., someone from Japan has a vastly different perspective from someone from China even though they may both respond "East Asian." This actually reflects very poorly upon Stack Overflow since it represents an extremely narrow perspective and​ understanding of culture.

If the "The Loop" survey is intended to be a preliminary instrument, then a better way of approaching things would have been to code the qualitative data and develop a new survey instrument (or panel interview more likely) from that.

As such, per the title, why is "The Loop" survey asking about race, age, and gender?


  1. Obviously that last point isn't relevant online, but you may or may not be surprised at how much work is done to limit the length of a survey to save on printing costs.
  • 34
    +1 Was about to ask the same question; I'm wondering why my race or ethnicity is even relevant. This is the first time anyone has asked me, I've never even thought about it, and seems to serve no other purpose than racial profiling. (Which I don't think is desirable?) – Servaes Nov 25 '19 at 22:39
  • 4
    @Inactive-ObjectingExtremism not an answer, but if I were to speculate it could be to set targets (x% of race y by date z). It could also be used to boast, 'more diverse than country A'. – JJJ Nov 25 '19 at 22:41
  • 4
    ...also for the record the gender options are really not great. I have never heard the phrase "gender non-binary" used - it's generally just "non-binary" - and it excludes a lot of genderqueer identities (genderfluid, for example). The "other" fill in the blank space is good, but the setup just felt a little weird to me. – heather Nov 26 '19 at 5:45
  • 3
    Worse, the "optional" questions don't have a way to skip (at least I haven't found any "skip" or "no answer" for the age, there's the "other" field in the others), making them in fact mandatory. Symptomatic. – Piskvor left the building Nov 26 '19 at 10:29
  • 1
    Th underlying reason is pure laziness. – user540056 Dec 7 '19 at 16:40
  • 3
    It's telling that the company declined to answer here. – faintsignal Dec 24 '19 at 19:58

15 Answers 15

137

Because that is what Corporate actually cares about. Or at least race & gender. Probably age to a lesser degree. The "gender" part has been emphasized recently via the "pronoun problem", but both race & gender have entered into the general world out in so many flavors. Call it "affirmative action". Call it "inclusive". Call it whatever you want.

My personal take on all of this: Programming (and nearly all the SE network of Q&A, but especially the programming and other technical sites) should be a race, age & gender-agnostic field. For better or worse, that is not always the case, but that should be the goal, not to make it "better" for a historically disaffected group but simply "great" for "everyone".

The question becomes whether to somehow proactively figure out how to "solve" that problem by asking who your users are (which appears to be the case with this survey) and somehow use that information to make the site better (if group 'a' seems happier than group 'b' then figure out how to make things better for 'b' - a noble goal if you can actually do anything about it) or to simply produce a product that has no inherent bias for/against any race, gender or age, where everyone is welcome and nobody cares if you are male, female, non-binary, young, old, white, black, etc.

SE historically has been a site where you could ask a programming question and (unless you wanted to let people know by way of your username or an actual photo in your profile) nobody would know or care whether you were male, female, non-binary, young, old, white, black, etc. Because C, PHP, Python, Java, etc. do not give different error messages for women and HTML/CSS produces different output in different browsers but not dependent on the age of the user.

Now if you're trying to sell a company to some big "words are all that matter, not deeds" corporation, then maybe this all matters. If you are trying to simply provide a place where people can ask technical questions and get answers, it doesn't matter AT ALL. Maybe a different system like Codidact (I am one of many people involved in developing Codidact, and there are many other open source Q&A systems out there) will do a better job of that, focusing on the users and not on political correctness.

But I digress.

| improve this answer | |
  • 65
    They may not even actually care about any of that. It's just politically fashionable to focus on that - especially as a distraction from substantive discussion. – einpoklum Nov 25 '19 at 23:26
  • 57
    It seems really bizarre to segment by race without considering nationality. Do they honestly think that a born-and-bred New Yorker will have more in common with a Vietnamese user in Hanoi than they do with the born-and-bred New Yorker they sit next to and go for lunch with every day, simply because the one New Yorker's family are Korean-American and the other's are Honduran-American? It honestly seems borderline racist, and I don't use that word lightly. Or maybe they forgot that not all users are American (again)? – user56reinstatemonica8 Nov 26 '19 at 9:41
  • 7
    I don't think we can simply ignore political correctness (aka politeness) if we want to attract experts. I certainly would be unwilling to have my name tarnished by publicly participating on a site that's soft on sexism, racism, etc. – Rebecca J. Stones Nov 26 '19 at 10:28
  • 4
    @RebeccaJ.Stones There are 3 different "attitudes" involved: * politeness (~= Be Nice), * sexist/racist/etc. - it is possible to be "polite" yet say extremely inappropriate things, and we definitely don't want that, * political correctness - for lack of a better explanation, I'll define that here as using/not using particular language to advance or emphasize certain goals with the implication that not using the same language automatically makes you a "bad" type of person. I know it is far more complex, but that is one of the key issues, IMHO. – manassehkatz-Moving 2 Codidact Nov 26 '19 at 15:49
  • 7
    @RebeccaJ.Stones In other words, you can be "polite" without being PC and you can be PC with a veneer of "polite" but elitist and/or racist and/or "bad intentions" underneath. – manassehkatz-Moving 2 Codidact Nov 26 '19 at 15:52
  • SO has been that kind of environment. SE is more varied - although these demographic considerations still aren't relevant to most of the sites, they can be to some. – Matt Gutting Nov 27 '19 at 2:11
  • 3
    @Fermiparadox To me, "Corporate" is "CEO+employees at the top" in any company - i.e., if shareholders/Board want to change something, they can say all they want but it comes down to CEO+employees to actually do stuff (for better or worse). – manassehkatz-Moving 2 Codidact Nov 27 '19 at 16:26
  • 2
    @Fermiparadox Typically (again, will vary by specific structure), the Board answers to the shareholders (in a number of ways which vary based on corporate structure/bylaws/etc.). Board typically only hires/fires CEO and perhaps a few other very top employees (COO, CIO, CFO, etc.). Everyone else is hired/fired by the paid management (CEO/COO/CIO/CFO for the next level and the next level for the one below that, etc. hierarchically). – manassehkatz-Moving 2 Codidact Nov 27 '19 at 16:41
  • @einpoklum-reinstateMonica they care. They care because in ever more jurisdictions there are legal requirements to have at least X% homosexuals, Y% blacks, and Z% women at any level in the organisation, even if no suitable candidates can be found... – jwenting Dec 4 '19 at 4:45
  • 1
    @jwenting Really? There is tremendous pressure in various ways. But actual legal quotas? Name a jurisdiction - point me to the actual law. Plus, we are NOT talking about employment or academic enrollment. We are talking about random users on the internet. – manassehkatz-Moving 2 Codidact Dec 4 '19 at 4:58
  • 3
    @manassehkatz-ReinstateMonica my dad's company was sued by the Dutch government back in the 1990s for not employing enough blacks as vetinerarians. The only way they could afford heavy fines was providing graduation records to prove that it was in fact impossible for them to meet the required numbers because there simply didn't exist enough people with the necessary degrees in the entire country. – jwenting Dec 4 '19 at 6:00
  • 1
    @jwenting Wow! That still doesn't apply to StackExchange/StackOverflow because that is employment and we are talking here about internet users. But still "Wow!" – manassehkatz-Moving 2 Codidact Dec 4 '19 at 6:21
  • 1
    Might be worth revealing affiliation with that link. I'm pretty sure that the general "disclosure" rules counts here too. – Journeyman Geek Dec 24 '19 at 10:25
65

Race, age and gender are just a part of the larger identity politics that has been seeping into tech.

For a larger context, I refer you to the social psychologist Jonathan Haidt:

Q: Are these problems graduating with Gen Z to the workplace?

H: While we were writing the book, I was beginning to hear anecdotes from business people, but we didn't have enough to go on. By January 2019, it was clear in industries that hire from elite liberal arts colleges. When we talk to people, they'll say young employees are so fragile and come in looking for confrontation. Members of Gen Z, they are used to an environment in which it's all about displaying what side you're on, supporting this person, opposing that. Everything becomes a moral conflict. But you can't run a company that way.

Q: How do we stop this from happening in an office?

H: If corporate culture becomes political, people feel fearful or reluctant to stand up. We need a speak-up culture, but Gen Z has been raised with a call-out culture in which they're incentivized to criticize or shame people publicly. A callout culture makes people feel they're walking on eggshells, and you can't have that in a company.

So there you have it. This is a cultural problem in tech that needs addressing by multiple organizations. Leaders of the companies should start speaking up against the growing divisiveness in tech, and aim to restore meritocracy.


Haidt has summarized a sociological paper to provide a more general sociological perspective into what is happening in tech: Where microaggressions really come from: A sociological account.

| improve this answer | |
  • 33
    That quote looks like just another complaint about "the younger generation". There are records of complaints about "the younger generation" going back to the Sumerians. If there was any truth to them, the world would have gone to hell millennia ago. It's best to take them with a hefty pinch of salt. – Raedwald Nov 25 '19 at 23:12
  • 17
    Those hundreds of generations of complaints about "the younger generation" could also point to some new fangled thing they didn't like as the reason for youthful fecklessness. People are good at post hoc justifications. This generation its trigger warnings. Previous generations it was TV. Or radio. Or rock music. Or books (seriously!). And do on. It's always something. – Raedwald Nov 25 '19 at 23:21
  • 10
    @Raedwald: So it does makes sense to know what is unique about this particular generation of Americans, isn't it? Although I agree with the 4th paragraph. SE Inc sounds like a NY company selling to Americans - and they entirely ignore the global aspect in the survey. Being a minority might be relevant, but short of indigenous people every racial background is a majority somewhere. – MSalters - reinstate Monica Nov 25 '19 at 23:41
  • 3
    @manassehkatz-ReinstateMonica You might also enjoy The Madness of Crowds – Sridhar Ratnakumar Nov 25 '19 at 23:59
  • 2
    Arghh... only available in my library in electronic format. Reading is when I actually get away from the computer! But I'll consider it anyway. – manassehkatz-Moving 2 Codidact Nov 26 '19 at 0:03
  • 6
    Similar to the "Sexual revolution" in the 60s and 70s, I suspect only a small percentage of the Gen Z generation is actually involved in this call-out culture. – Alex Nov 26 '19 at 7:21
  • 17
    @Alex If the minority is vocal enough, it can still influence the culture at large. The "walking on eggshells" problem mentioned in the quote is still a problem even if only a small proportion of people are doing the calling-out. – Mario Carneiro Nov 26 '19 at 7:40
  • 4
    @MarioCarneiro On the surface yes, but my parents didn't experience much from the free love revolution in the 60s and 70s. I guess it is the same with the "walking on eggshells" problem. Of course there is a much more cautious culture in the office, especially if you are white male, however, personally I have seen little change outside the office away from the preying eyes of social media. – Alex Nov 26 '19 at 8:33
  • 7
    From my own personal experience the people walking on eggshells are generation Z who have to avoid hurting the older generations feelings when they tell them off for making inappropriate comments in the workplace... – user3161729 Nov 26 '19 at 9:50
  • 9
    -1 While it's true that SO Inc seem to want to drag specifically American culture wars into an international site, let's not play their US culture wars game with another "Booo, kids get off my lawn" generational bun fight. Walking on eggshell has been a corporate culture problem in many offices for generations, for many reasons. Let's not forget many people have to walk on eggshells around their (usually older) bosses lest they get fired because of some work-irrelevant pseudo-moralistic disapproval of their love life, domestic living arrangements, non-majority religion, dress or hobbies. – user56reinstatemonica8 Nov 26 '19 at 9:51
  • @user56reinstatemonica8: Yes, or even for just the slightest tiny tiny hint of disagreement (I have a second-hand account (fellow student)). – P.Mort. - forgot Clay Shirky_q Nov 27 '19 at 2:14
  • 3
    @Alex yes, I'm sure. But the minority are vocal enough that people have been fired, and even arrested for having un-offensive views. How long before Alexa starts to report your uncool niece to the suede/denim secret police? – gbjbaanb Nov 28 '19 at 11:36
36

it is still a survey instrument

No, it is a diversion, a supposed alternative to actual dialog between the company and the SE community, here on MSE. I suggest Galastel's excellent summary of it.

and reflects upon us all

Cooked surveys which are put forward as a diversion and managed by untrustworthy people/organizations should not be taken seriously.

But I suggest we all make sure and actively undermine this initiative, by boycotting these surveys.

| improve this answer | |
  • 22
    Or, instead of boycotting, we could use these surveys to tell the company what we really think. – user245382 Nov 25 '19 at 23:33
  • 13
    @House-'ReinstateMonica'-man: No, that's not what would happen. And we already have the medium for telling the company what we think. It would be a mistake IMHO to play along with these survey games. – einpoklum Nov 25 '19 at 23:40
  • 1
    It's way past the time for all of us to just give up expecting the company to have any actual dialog with the community, if you ask me. – Marc.2377 Nov 26 '19 at 23:49
  • @Marc.2377: Why are you so certain of that? – einpoklum Nov 27 '19 at 0:33
  • 2
    @einpoklum-reinstateMonica I started noticing this pattern two and a half years ago. Others noticed it even earlier. Things took a turn for the worse more recently, and for the past two months it's been the subject of intense discussion. And we tried. (continues...) – Marc.2377 Nov 27 '19 at 0:40
  • 2
    (cont) And they apologized (or did they?). And they promised. But did anything change? Nope, just radio silence. As many have said before me: they haven't even talked to Monica, why expect them to talk to us? That would be a very foolish expectation indeed. It's way past time to move on. – Marc.2377 Nov 27 '19 at 0:46
  • 1
    @Marc.2377: Things were different 3 years ago, and they changed. Maybe 3 years from now they'll change again. I agree that they're not likely to change soon or without a change of circumstances. But don't be so fatal... – einpoklum Nov 27 '19 at 0:48
  • Yeah, I can see you already know about everything I'm saying here. Well, they haven't been talking. They won't talk anytime soon. If they do talk 3 years from now, I for one won't be here to listen. You must understand. – Marc.2377 Nov 27 '19 at 0:50
  • @Marc.2377: I understand. But I probably will be here (well, on non-meta sites at least) in 3 years. – einpoklum Nov 27 '19 at 1:05
34

They ask for it because they consider the race or gender of the people who answer is relevant to judge their opinions.

I have a dream that my children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character - MLK

That was back in the 60s, now we are back at racial profiling. And they call themselves progressive inclusive.

| improve this answer | |
  • 1
    The US at the very least has the concept of protected classes and it doesn't have to do with inheritance. It's meant for good reasons but never updated to reflect the changes in laws and policy that have been passed – SCFi Nov 27 '19 at 18:21
  • 4
    Aren't those protected classes just people with low incomes or in difficult situations? If you classify people by race and then create "protected" classes you are assuming that certain races are going to do better than others – Luis Rico Nov 28 '19 at 8:40
  • 4
    It's very useful to some in society that the conversation remains focused on identity narratives and does not consider economic realities. – Rounin Nov 29 '19 at 16:13
  • 14
    "It's ok, we're the good kind of racist." – Ask About Monica Nov 30 '19 at 20:40
  • 3
    ""Fun"" fact: If you quoted more of this speech, you'd have to expect being banned or flagged for using the "second-worst 'n-word'". Apart from that: For nearly ten years, people have been judged on SO for the content of their (not character, but) answers. Suddenly, people started claiming that this was not the case, and that's when this whole thing started blowing up. It's a pity. – Marco13 Dec 6 '19 at 15:19
18

I don't know about other sites, but on MathOverflow, there are essentially no women.

Most users use their full names as user names and have photos as avatars and link to their professional web pages so it is clear that they are men. There are essentially no women who identify themselves in this way. There are some pseudonymous users who might be women, but not very many. The proportion of women is far, far below the proportion in the professional mathematical community generally.

There are certainly many women who could greatly enrich the community with their knowledge and insights if they chose to do so. Of course I do not suggest that women should be pressured to participate if they do not find it rewarding. However, if there are fixable incidental reasons why women find the site unwelcoming, then it would be a good thing to understand that.

One could make some guesses about the reasons, but I do not think that any of the most obvious guesses are consistent with the observable data.

For example, in the small number of cases where women do ask or answer questions, I cannot find any sign that their opinions are disrespected or devalued.

So there is a need for further information.

There is some chance that the "The Loop" survey will help with this. It can only do so if it asks about the gender of respondents.

| improve this answer | |
  • 4
    I think that the great majority of professional mathematicians are aware of MathOverflow, and that a smaller majority have read questions and answers there even if they have not written any. – Neil Strickland Nov 27 '19 at 16:32
  • 15
    Why do you think the proportion of professional female mathematicians relative to male mathematicians would be the same proportion as female to male mathematicians that are interested in interacting online? Maybe female mathematicians don’t need as much help as male mathematicians, or they get their help offline. Maybe female mathematicians are less interested in teaching strangers on the Internet, or prefer interacting in person. Not every disparity is an indication that something needs fixed. – ColleenV Nov 27 '19 at 19:09
  • 2
    @ColleenV There are indeed a number of possible reasons why women might choose not to engage with MathOverflow, and I agree that some of them would involve freely made choices that do not need to be fixed. But there are other possibilities that would need to be fixed, and we do not currently have good information about which possibilities are correct. – Neil Strickland Nov 27 '19 at 20:25
  • 12
    My point was really that these surveys are starting from the assumption that there’s something definitely broken about SE for women, but that hasn’t actually been determined to be true. The only real data is for SO, which might be uniquely problematic and not representative of the rest of the network. The claim that women are under-represented on SO appears to be based on US labor statistics, which has a number of problems. It seems like folks are walking around with an hammer looking for excuses to treat something like a nail because hammering is an worthy pursuit. – ColleenV Nov 27 '19 at 21:36
  • @NeilStrickland I agree that some of them would involve freely made choices that do not need to be fixed. Only some would be freely made choices? Not participating in an online community is always a choice you are free to take. Unless MathOverflow bans women, but I doubt it. – Luis Rico Nov 28 '19 at 11:10
  • 1
    @ColleenV "My point...these surveys are starting from the assumption that there’s something definitely broken...for women,...hasn’t actually been determined" you might also consider the survey as a tool to find out whether the assumption actually makes sense or not. But, I like your hammer nail analogy a lot. Still, I believe the approach should be more nuanced. However let me share a Dutch saying paard voor de wagen spannen remaining question:where's the horse, in front or behind? – Sextus Empiricus Nov 28 '19 at 17:11
  • It's worth noting as context for those who don't know that MathOverflow isn't run by Stack Overflow Inc but by a separate foundation. Also, when I joined it I understood it to have a real name policy, although I could have been confused. – Peter Taylor Nov 28 '19 at 17:28
  • The Loop will help squat with any issues on MathOverflow as the Loop is restricted to Stack Overflow users only. – Emil Jeřábek Dec 6 '19 at 7:56
  • 2
    I agree to what ColleenV said: Even if there is a "disparity", this does not necessarily mean that there is something wrong that has to be fixed. But more importantly: Iff the survey showed a disparity, and iff there was a reason to assume that this has to be (or should be, or could be...) fixed, then the survey wouldn't say how. The survey asks far to few questions to even speculate about causalities. If they did not ask "Are you male or female?", but only asked "Do you wear glasses or not?", then I'm sure it would show a similar "disparity". What then? – Marco13 Dec 6 '19 at 15:29
14

They have reasonable grounds already to suspect that gender and ethnicity will be relevant, based on past annual surveys. As someone else pointed out, and you imply, this looks like an inept attempt to take that into account.

I guess the initial survey asks for only these kinds of demographic information to establish a pool of people they might select from. They might then select a suitable number of people from the various demographic groups.

| improve this answer | |
  • 1
    But people could lie about their gender and ethnicity or age. Not sure if this is a good way to collect that information from a large portion of the user base and without doubt. – Trilarion Nov 28 '19 at 14:00
13

A potential answer to this occurred to me while reading your question, specifically your section about when demographics should be used:

In scientific research, demographics are only requested if there is evidence that a demographic attribute will lead to a significant difference in responses

Perhaps this is what SO/SE is trying to reveal, is there a difference in user satisfaction based on race/gender/age?

Perhaps 75% of white men will report a positive experience on the site, while non-binary individuals report only 40% positive experiences. (Please note these two groups are not mutually exclusive.)

I am in no way defending the way the survey was done or the lack of appropriate diversity in options, but I do see the above scenario as a possible reason for demographics to be present in the survey.

| improve this answer | |
  • 4
    @Inactive-ObjectingExtremism 'ethnicity' might have been a better term than 'race'. In that case it is not only USA-centered. Many national bureaus of statistics (all over the world) consider ethnicity. The main problem with this survey is that the study of ethnicity is a bit misplaced. Whatever differences it might find, it will overestimate influences of ethnicity because there will be many other factors that may correlate with ethnicity (correlation ≠ causation) and also ethnic sub-populations at a national level (this seems to be USA based) may not need to be relevant on a global level. – Sextus Empiricus Nov 28 '19 at 12:41
  • 2
    Race is a central concept in the UK and Eurpope (I would say). And BAME is certainly under-represented as are other groups. Irrespective of the quality of the survey design... being able to pivot experience of SO by demographics is useful IMO. If only as a starting point for understanding those perspectives particularly if shared within a demograhic. But I do note @anonymous comments in ignoring_gravity's answer. – user371773 Nov 30 '19 at 14:29
11

I presume their objective is to see whether people of different race/age/gender have different experiences of the site.

| improve this answer | |
  • 1
    Not adequately, sorry. You talk about respondent fatigue, but in such a short survey, do you really think it's an issue? – ignoring_gravity Nov 26 '19 at 13:33
  • Why do you think they lack a causal model? – ignoring_gravity Nov 26 '19 at 15:54
  • 2
    If this is true, the timeline is very screwed. This survey should predate an article that mentions SO is a hostile environment for minorities (The article predates the new CoC) – dustytrash Nov 26 '19 at 20:12
  • 2
    @anonymous I agree. They missed many other potentially relevant demographics, for example national origin; income; disability status; language fluency. The fact that they asked for only age, race, and gender shows they have already implicitly selected those categories as the variables they care about. For example, I would think that national origin and language fluency might have very large impacts on satisfaction with Stack Overflow. – Mark Beadles Nov 28 '19 at 18:51
  • @MarkBeadles sure, if they want to include other (potentially better) demographics, then sure, I'd be all for it – ignoring_gravity Nov 29 '19 at 9:46
11

Is it relevant to Q&A? No

Is it relevant to marketing and trying to decide how to advertise? yes

Stack has also taken up an ironic policy of inclusivity so it could be part of an ongoing attempt/operation to make minority groups feel more welcomed by weighing race into the feedback provided.

Doing this lets them market to target groups, however the exclusion of nationality may greatly hamper these capabilities as stated elsewhere, and if it is weighted will let them know which opinions according to their internal workings are more relevant than the others (one of the reasons you have to call it ironic).

My personal belief is that policies should be general and not centered around gender or race, though Age is understandable to the extent of being too low or accessibility for older individuals (should also ask if disabled though if they are considering accessibility).

So in short, It is primarily for marketing and growing the community. I'm a bit new to meta, but it doesn't seem meant to replace it though it also doesn't seem as if they listen to meta as is.

| improve this answer | |
  • 2
    Yes. SO/SE is a company and is supposed to make a profit. They make money by advertising. Advertisers pay more if they can target their audience. So it's a completely commercial decision. They are flogging our personal data, just like any other "free" service. – RedSonja Dec 6 '19 at 9:18
8

I can see two reasons why SE wants to collect race, age, and gender.

  1. This is useful information if you are trying to sell advertisements
  2. One of the objective of SE, is to become more welcoming, in particular to minority groups. This survey can be used to define KPIs regarding this and award senior management with bonuses if their targets are met.
| improve this answer | |
8
+100

Sadly it is a fact that ethnicity has big influence

Let's be realistic: It is a fact that 'race' (I believe we better call it ethnic background instead) is having influence on social life.

Especially in the United States, where the company resides, one's ancestry (or ethnic background or ethnic identity) has a large impact on one's socioeconomic status.

That is, on a larger population level. An individual person with dark skin can become the president, but the group as a whole is experiencing friction, and this is clearly visible in statistical data.


Science/Technology/IT is one field where it plays a role

This influence of ethnic background is protruding into science and technology. If most people in IT are white and male then this might create prejudice in this field.

This occurs on the one hand in stuff like political correct language use in coding which is currently a hot topic. Like Python avoiding the master/slave terminology. But on the other hand there might be also more direct influences of white male 'dominance' in science and technology (like reductions in equal opportunities, salary equality and inclusiveness).


It is not inherently offensive for a survey to ask about ethnicity, but it is a sensitive topic and the execution needs to be carefull.

The survey is in principal aligned with a goal of the company to improve inclusiveness and create a more equal level playing field for minorities who may currently experience obstructions.

The goal of the survey seems to be to find out if and how people with a different ethnic identity ('race' is a bit negative term) might be experiencing different problems with the Stack Overflow website (the question four is about "What do you find most frustrating or unappealing about using Stack Overflow?").

Yes, it might be considered a bit harsh to ask about one's ethnicity (especially when the term that is being used is 'race'). It might be considered as 'racist' because it confirms the differences between people with different ethnicity and explicitly treats people with different ethnicity as different.

Yes, there are many ways to criticize the particular execution of the survey and the race/ethnicity concept. The way that SE/SO has categorized 'race' exposes underlying ideas about race and how this interacts with the use of the Q&A platform.

However, it is a fact that people with different ethnic identities are, on a population level, different. This survey has the goal to find out whether there are any negative effects that may follow from that and it will allow to create informed* policy that may diminish those negative effects.

The term 'racism' or 'bigotry' has a negative connotation, but not every differentiation between race/ethnicity or gender (what one might consider in a broader sense racism or bigotry) should be considered negative or intolerant.


*It is very easy to create an image of non-inclusiveness based on stories from a small number of people or other anecdotal evidence. The numbers/data can support this image.


Diversity

The image below is based on some data from the 2019 Survey. It shows how different the visitors of Stack Exchange and Stack Overflow are. But also, that a majority is 'white male' (from different origin Europe/Russia/USA and this might possibly need to be split up as well) and there are very few women and few people that do not associate with "white" ethnicity.

This makes it problematic to recognize problems that relate specifically to minorities. For this you need to be able to target them specifically, e.g. with a demographics based survey. If you ask the entire group, or listen to the loudest majority, then you may hardly hear the voices of minority groups.

In the image below the correlation with a problem 'belonging to the community' is not so much different for ethnicity (in binary form white vs non-white). But for gender the difference is more clearly different (and reasonably uniform among different countries, thus not influenced by country but rather by 'associating with male gender or not').

And for the aspects that respondents would like to change (which are not available in the open database) there are (many more) clear differences as well. This is sort of what the loop is imaging (based on keywords from open questions). It images how different people think different about SE/SO along with the demographic information. It will give a sharp image , rather than a fuzzy image that mixes/blends everything together. It will be colorful rather gray. It will have depth rather than being flat. (of course, it is arguable what the meaning of that image is and it will only provide additional information and one can not directly read 'causal' relationships from it)

diversity

| improve this answer | |
  • 29
    Some things can be tremendously hard to understand for people who are not from the US. How ridiculous some things appear can be pointed out by boiling down certain conversations to the core: A: "We want more black people in a certain field". B: "Why?". A: "Because skin color does not matter". B: "WTF?". It just seems odd. Regardless of that: The assumption that the distribution of people with certain attributes within a social structure or organization has to match the distribution of these attributes that is found in the next larger structure is nonsensical and impossible to enforce. – Marco13 Nov 28 '19 at 13:54
  • 3
    @Marco13 it is sensible (for the reason that you do not want your organization to live in a bubble and in addition for the reason that one might desire equal opportunities for minorities). It is a gray area though, it depends on what you consider by "match"; It doesn't need to be exactly equal of course (but at the current time women and people with a minority ethnic background experience friction in their careers and it is not nonsensical to create opposing forces that may lead to an equal level playing field) – Sextus Empiricus Nov 28 '19 at 14:04
  • 5
    Maybe I'm overly skeptical (sorry), but you said "This survey has the goal to find out whether there are any negative effects that may follow from that and it will allow to create informed policy that may diminish those negative effects.". Now, 1. I doubt that this really is the (honest) goal (but that's not something that we can know or find out), and 2. this goal cannot be achieved, because it's not possible to identify a causal relation between the observations, and 3., most importantly: .. – Marco13 Nov 28 '19 at 17:20
  • 7
    ... : I wonder how the results should allow an "informed policy". E.g. imagine the outcome is: "White: 60% satisfied, 40% unsatisfied", and "Black: 40% satisfied, 60% unsatisfied". (And there is not much more in this survey!). How should this be the basis for an informed decision? I.e. what chould a "policy" be based on that? And if it was implemented, and a survey 2 years later showed "White: 50% satisfied, 50% unsatisfied", and "Black: 45% satisfied, 55% unsatisfied", would that be an improvement? – Marco13 Nov 28 '19 at 17:20
  • 3
    @PeterTaylor I agree that the survey can be debated, and any way of categorizing people may be criticized (too much/little detailed, there is not perfect way). How SE/SO exactly categorizes demographics might be simplistic. But, the idea by itself - that demographics and age, gender, and ethnicity play a role - is not so strange. The question from the OP relates to respondent fatigue and that survey methods courses teach to avoid these demographics questions. But, the goal of SE/SO is to find out more about the idea they have about diversity (and related problems) on the platform. – Sextus Empiricus Nov 29 '19 at 11:38
  • 2
    The problem is to recognize problems that relate specifically to minorities. For this you need to be able to target them specifically, e.g. with a demographics based survey. If you ask the entire group, or listen to the loudest majority, then you may hardly hear the voices of minority groups. --- Indeed, maybe, just "white heterosexual male" versus "other" is sufficient. It already filters out 70% of the visitors that relate to it. A more precise classification may be more than necessary, but it works. In time one may find a need to change the classification according to new findings/ideas. – Sextus Empiricus Nov 29 '19 at 12:05
  • 1
    The update really adds value to the answer. But the survey that you linked to was far more detailed than the loop, and still shows how difficult all this is: Words like "nicer" and "friendlier" are appearing. Are they the same? What is the difference? And they are together with words like "rude" and "assholes" - wasn't this likely used as one phrase, i.e. "rude assholes"? There are approaches for sentiment analysis, but drawing conclusions from bubbles in a word cloud that was generated purely empirically (i.e. without any theory in mind) doesn't seem like a credible approach to me... – Marco13 Nov 29 '19 at 17:37
  • 1
    Thank you for this answer. While there were unforced errors in the release of this particular survey, the use of demographic categories is standard, and generally good practice. It seems standard survey methods have been interpreted as a tool to ignore input, rather than a tool to allow more depth and nuance in the analysis of that input. I suppose, considering corporate's strategy of actively ignoring meta, it's not that surprising. – De Novo Nov 29 '19 at 20:38
  • 4
    @PeterTaylor I agree with you. It was wrong to suggest that this particular execution of a survey that asks for ethnicity was not insensitive or not offensive. I was indeed having in my mind the more general 'idea' of the principle of asking for ethnicity or investigating ethnicity in any other way. – Sextus Empiricus Nov 30 '19 at 14:16
6

Looking at the survey, I really don't understand why demographic information (i.e., race, age, and gender) are being asked since it appears that the clear goal is to get an initial impression of "how satisfied are you with your experience using Stack Overflow?" While I understand that a lot of people will include these by default, unless it is relevant, it's generally taught in survey methods courses to avoid asking the questions.

I'm okay with thinking out loud on why this matters.

Some sample questions to ask of the data:

  • Are female developers between the ages of 18-35 feeling like the moderation done on the site is too extreme?
  • How satisfied are users younger than 18 with the site compared to users between 18-24?
  • Is there a correlation between identified gender and site satisfaction?
  • What is the actual demographic make-up of users of the site?
  • Is there a correlation between years in the industry and site satisfaction?

How can you ask those questions of your data if you don't actively request it?

| improve this answer | |
5

They can be used to group the data obtained from the questions "what do you like best" and "what do you find frustrating".

For example, "Lack of fluency in English" can be a common answer for a particular demographic group. Since, the data is already processed through machine learningref this can be categorized easily.

| improve this answer | |
  • What do you mean by "it data"? Do you mean "this data"? Or "its data"? Or something else? (Preferably, edit your question.) – P.Mort. - forgot Clay Shirky_q Nov 27 '19 at 2:21
  • Besides that, can you elaborate on the machine learning part? E.g., where would the input data come from? What categories (e.g. how many?)? – P.Mort. - forgot Clay Shirky_q Nov 27 '19 at 2:24
  • @PeterMortensen See the ref link in the answer for that part. Answer edited on "it data". Thank you. – Kolappan N Nov 27 '19 at 4:33
5

We have all seen SE's recent push to be as welcoming and inclusive of minority groups as humanly possible. The latest controversy involving a certain moderator-who-shall-not-be-named who happens to have a Go Fund Me set up right now started largely because said user asked in good faith for clarification about a proposed rule change involving gender identity.

SE obviously collects data about who uses their site, and they probably already have at least a general idea of the various ethnic backgrounds and ages of developers on their site. That being said, having concrete survey results showing a large percentage of minorities using the site would be quite useful to SE corporate to justify future changes along their current path.

Of course, I say "concrete" survey results, in the sense that they will be numbers. Whether or not those numbers will be skewed as a result of the questioned asked is another matter. I know that I would not bother answering a survey in any context which asked my age, race, or gender, first because I want to hold onto the tiny remnants of privacy we have left. But also because I see them as irrelevant factors on the internet.

However, other people may consider their age, race, or gender INCREDIBLY important, and would take any chance they get to respond to surveys in which they can change the results with their answers. This is a long way of saying that SE most likely included the questions they did to get results closer to what they wanted to see.

| improve this answer | |
2

I suspect it's because they want to see whether a different subset of users are clicking on the survey compared to their user traffic data, and annual survey data.

It's hamfisted and probably pointless, but at least I can see some motivation behind it.

| improve this answer | |
  • Though... Where would they find this survey? – Journeyman Geek Nov 27 '19 at 15:36
  • @JourneymanGeek The Blog, which may be engaged in differently to meta posts or the banner put up for the annual survey. Or maybe they are expecting it to be linked to directly by tech news? Who knows. – Pureferret Nov 27 '19 at 15:37
  • 1
    That would only makes sense if "race", age & gender associations were known outside that survey. Against what data could one compare here? – anx Nov 27 '19 at 16:51
  • 1
    @anx if 20% of people identify as race X in the annual survey, and 10% in the The Loop survey, you've reached half as many proportionately. – Pureferret Nov 27 '19 at 17:12
  • 2
    @Pureferret .. or the opposite, depending on how many potential respondents you've insulted with the (since slightly changed) question & answer set. – anx Nov 27 '19 at 17:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .