55

In the current version of "The Loop", the following question is asked:

  1. Which racial background(s) do you identify with? Please select all that apply. (optional)

Multiracial
Hispanic or Latino/Latina
Biracial
Native American, Pacific Islander, or Indigenous Australian
Middle Eastern
South Asian
Black or of African decent [typo in original]
White or of European descent
East Asian
Other (please specify)

Similar questions have been used in past surveys, for example this 2017 survey mentioned the following, referred to as ethnicities in the writeup as opposed to racial backgrounds:

  • Hispanic or Latino/Latina
  • Native American, Pacific Islander, or Indigenous Australian
  • Middle Eastern
  • South Asian
  • Black or of African descent
  • White or of European descent
  • East Asian
  • I prefer not to say
  • I don’t know

The groups mentioned above are somewhat different to those used by the US census, the country in which the company is based. It mentions the following (copied from the list of 2000, which apparently hasn't changed for 2010):

  • White
  • Black or African American
  • American Indian and Alaska Native
  • Asian
  • Native Hawaiian and Other Pacific Islander
  • Some other race
  • Two or more races

It also asks about the ethnicity "Hispanic or Latino".

There's a number of differences between what the company does, and what the US census bureau does. The survey groups "American Indian and Alaskan Native" with "Native Hawaiian and Other Pacific Islander". It splits the "Asian" category up. It treats "Hispanic or Latino/Latina" as racial background rather than a separate attribute. Most interestingly for me, it groups members of "Indigenous Australian" with "Native American" and "Pacific Islander".

How did the people behind "The Loop", and previous surveys, decide upon the categorisation of racial backgrounds?

Update In the 2020 developer survey, the following option was given, to a question which didn't use the word "racial" or "ethnicity":

Indigenous (such as Native American, Pacific Islander, Or Indigenous Australian)

This would suggest that in 2019, the designers of The Loop thought back then that the three groups had that characteristic in common.

| improve this question | |
  • Previous post by me in the form of an answer: meta.stackexchange.com/questions/339050/… – Andrew Grimm Dec 1 '19 at 11:22
  • 4
    Not an answer to your question but you might be interested in seeing how asking people about their ethnicity developed in another country history.blog.gov.uk/2019/03/07/… It would be nice to think that anyone including such questions in their work had devoted similar thought to how and why they were asking. – mdewey Dec 1 '19 at 12:31
  • 2
    Perhaps I pay too much attention to detail, but is "decent" (instead of "descent") in "Black or of African decent" a typo in the survey? (I can't see; it says You have already taken this survey.) – Rebecca J. Stones Dec 1 '19 at 12:38
  • 4
    @RebeccaJ.Stones It may have been fixed meanwhile, but it was (or still is) indeed a typo in the survey. Here's a screenshot, I got the link from this comment. – Daniel Fischer Dec 1 '19 at 13:18
  • 27
    My guess for the answer in the title would be either "dice rolling" or "brainstorming" (ignoring the fact that the attempt to squeeze people into such race-boxes does not even remotely make sense (even less on an international site), and seems to be a strange artifact of a way of thinking about the world that is incomprehensible for many non-US-citizens). I think they should rather ask about ethnic groups, because these would likely expose larger differences than the so-called "races", but... that's more difficult, of course – Marco13 Dec 1 '19 at 14:05
  • 52
    I can confirm that for an European the "racial background" question makes no sense. I have no idea what is the difference between a white person and a Hispanic person. From a European's point of view, black people and people of African descent are very different categories, the one hundred fifty million inhabitants of Egypt, Algeria, Morocco etc. being most certainly not black. I have no idea why people of "Middle Eastern" descent (= Turks, Jewish Israelis, Arabs, Persians etc.) are considered to form a racial group. I don't know what a "South Asian" may be. And what about racial foreground? – AlexP Dec 1 '19 at 18:38
  • 4
    They weren't..... – Lightness Races in Orbit Dec 1 '19 at 19:56
  • 8
    I am happy as hispanic to read US census don't mention hispanic as a race. Hispanic is not a race. There is a race maybe at southamerica but it is too broad. There is african influence, spaniard influence and american natives influence. As someone from spain that survey has no sense, I cannot answer. I consider myself hispanic as I speak spanish as my SA partners, but I am also white, I have blue eyes. I am also european – user657339 Dec 1 '19 at 21:20
  • Note that the first version of the list was significantly shorter: i.stack.imgur.com/EZ4Pb.png – OrangeDog Dec 2 '19 at 11:51
  • 2
    @AlexP, the reason for your confusion is that "Hispanic" doesn't belong on that list. It's a cultural background (Spanish culture as filtered through the colonies in the Americas) rather than a racial background. You can be a white Hispanic, a black Hispanic, a Native American Hispanic, etc. For that matter, I wouldn't be surprised to find a few people with predominantly Chinese ancestry who consider themselves Hispanic. – Mark Dec 2 '19 at 20:53
  • 2
    @Mark: The list is simply nonsensical. People of African descent to not belong to one racial group; the people of northern Africa and the people of sub-Saharan Africa cannot be made to belong to the same "race" unless that "race" encompasses all mankind. The "Middle East" (which a European would call the Near East anyway) is home to a great variety of people; Turks, Persians and Arabs (not to mention Israelis) are not all that similar (and belong to the Europoid race). And I really have no mental image of what a "South Asian" might be. A Malay? Are Malays considered a specific race in America? – AlexP Dec 2 '19 at 21:01
  • 1
    I still do not understand the reason, this survey, even wants to know about our race. What does my race have to do with my user experience? What does it have to do with my ability to ask a question or answer a different question? – Ramhound Dec 2 '19 at 21:46
57

Unfortunately I need to be a bit blunt since they were poorly designed.1 While it is reasonable to assume that the cultural background of a Stack Overflow (SO) user would result in different perceptions in the use of the website, asking about racial background is a poor way to go about getting that information.

First, there is the obvious problem that asking about "racial background" is a culturally sensitive question internationally. While some of the users of SO are from the United States, the international community is the majority of the user base and needs to be considered. This is fairly easy to correct with better phrasing though.

Second, the bigger issue is that when developing a survey you need to have an idea of what the like dependent and independent variables are.2 In this case the dependent variable (the one being influenced) could be "Perception of Stack Overflow" or "Use of Stack Overflow" and the independent variables (the ones thought to influence the dependent variable) could be drawn from the following list:

  1. Age: users may be more or less inclined to use SO due to age.
  2. Professional Experience: more experienced software developer may not use the website as much.
  3. Cultural Background: the cultural background may play a role; however, this is likely due to a independent variable that relates to cultural background such as native language or English proficiency.
  4. Gender: are women less like to use SO than men?
  5. Programming Language: developers who use a underrepresented language on the site may not use it as much.
  6. Type of Programming: what the developer is programming for may play a role, for example, web developers may be more inclined to use SO versus embedded systems programmers.

The lists of independent variables are usually developed through several sources (ex., prior surveys, literature reviews, brainstorming) and may be narrowed down a bit due to instrument constraints (ex., paper surveys have print costs) or due to overlap on the list. In this case it appears that "The Loop" is using "racial background" as a proxy for the cultural background, but does so in a way that loses a lot of statistical power. For example, "East Asia" is a very big place and includes countries that have a lot of software developers (ex., China, Japan, South Korea). Lumping all of these into one bin presumes there is no significant difference between the countries. This point is a debatable point, and given my familiarity with the region I would argue it's a bad assumption to make.

Given that this was a web-based survey the designer of the survey instrument actually has more tools at their disposal than a paper-based one. As such, I would design the survey instrument to take advantage of this (brainstorming questions):

Q1. In what country do you currently reside? [List of options]

IF [COUNTRY] = [UNITED STATES] THEN Q1.1 What ethnicity do you identify with? [List of options]

Q2. What is your level of proficiency with English? [Likert scale]

This approach captures the same data as the original question in "The Loop" survey since the country selected for Q1 can then be binned into a derived variable for one set of statistical tests. However, by looking at the role that the respondent's country and English proficiency plays it offers a lot more information for analysis. Depending on the results of the analysis a follow-up survey could then be developed that is a bit more targeted.

However, the trick to all of this is that you need to have some hypothesis in mind for the role that the independent variable plays on the dependent variable. During my training, development of causal diagrams played a major role since that in turn acted as the outline for the statistical analysis you would do with the results. It was always interesting to see what independent variables were not statistically significant and it wasn't uncommon to find that you initial hypothesis might be wrong.

  1. Credentials: PhD with graduate training in computational social science and survey methods.
  2. Note that for a census your objective is to try and capture a complete representation of user base. The design of a survey instrument changes in that scenario.
| improve this answer | |
  • 11
    Thanks for mentioning that they should have asked about English proficiency, because it's going to make a big difference. – tchrist Dec 1 '19 at 19:24
  • 35
    (+1) "While most of the users of SO are from the United States": note that over three-quarters of those who responded to the last SO developer survey were not from the USA. – Scortchi - Reinstate Monica Dec 1 '19 at 19:31
  • 4
    Very well written post. Hopefully the analysis from this survey is not used to make implicit conclusions about the representation of culture on the site. Then again, seeing the recent quality from the blog posts, it probably will. – Travis J Dec 1 '19 at 19:32
  • 55
    This. It's Americentricism at its finest. The rest of the world is just an afterthought because we barely even have electricity, let alone internet. Right? Oh, be inclusive, though.... smh. – Lightness Races in Orbit Dec 1 '19 at 20:07
  • 4
    Social status is missing from the list. Of the respondent and/or the parents. Sample social status proxies (often correlated, but not always): job title, salary, education length, education type (science, humanities, etc.), type of company employed at (e.g. shipping vs. FANG), employee or owner of own business, size of company, manager or grunt worker, etc. – P.Mort. - forgot Clay Shirky_q Dec 1 '19 at 20:36
  • 9
    I love this radical focus on Stack Overflow, as if Photography, Mathematics, Information Security, Parenting or Law does not exist at all. You just gotta be a developer to matter on this platform. much inclusive, very diverse, such good job, wow – MechMK1 Dec 2 '19 at 9:49
  • 4
    @anonymous If the survey was for Stack Overflow alone, then they should have designated it as such. – MechMK1 Dec 2 '19 at 14:27
42

I think that part of the survey was implemented following Hanlon's razor:

"Never attribute to malice that which can be adequately explained by stupidity."

But let's better call it incompetence, as we are professionals here.

As in:

The groups mentioned above are somewhat different to those used by the US census

To me, the only place in the world where my "race" matters is "US customs and immigration" when traveling.

And that is the thing: asking such a question isn't just "sensitive internationally". No kidding: if SE Inc. would be a Germany company, and they had pulled that survey, they would have made it the national stage, and they would have seen a 2 week shit storm. Minimum. They wouldn't have gotten away without the CEO publicly apologizing for being so insensitive, unwelcoming and racist.

Given the fact that SE Inc. over and over emphasises a global community, and put major efforts in being "welcoming", that part of the survey simply indicates one thing: cluelessness.

Regarding "how it happened", what we know is:

  • SE Inc. postponed the initial announcement (and going public with their survey) by a few days, to fix "problems". Only God knows bad the original content must have looked like.
  • They really don't care for feedback from this community. If those US Americans pulling together that survey had talked to a say, a few European users ... I am sure: that question about race wouldn't have been in the final survey. At least not like this.

But I fear, that is the way how things work now: SE Inc. has highly paid professionals defining the new path forward. And because they are soooo good at it, they can't get anything wrong, therefore there is no need to listen to us upfront.

| improve this answer | |
  • 3
    That's a bit of a generalisation, there: ask "say, a few European users. " I'm a European user and didn't blink an eyelid when I saw that question (other than to note that the list of options was poorly constructed.) But then I fill in a question like that every time I do a survey about how useful I find a government website (and some commercial services), or what my experiences are as a customer of the NHS or a Higher education establishment, or when I'm applying for a grant or a benefit or.... you get the idea: it's so normal I'd be surprised if it wasn't there. – ColeValleyGirl Dec 2 '19 at 9:23
  • 1
    @ColeValleyGirl Fair point. I was more thinking "5 to 10 different Europeans", and then I am pretty sure that someone would have (at least) raised an eyebrow and mentioned "that is really a very US centric approach you are taking here" or something. – GhostCat Dec 2 '19 at 9:52
  • 1
    @ColeValleyGirl Yep, as European, I also haven't blinked to that question... if you don't actually think deeply... such questions (although they don't ask about race, but nationality) often pop up in various government surveys and documents. The only sigh I had because it was poorly composed, just like gender question. – Resistance Is Futile Dec 2 '19 at 11:23
  • 8
    It might depend on what you're most used to. I thought that the question stuck out like a sore thumb so hard, that I did write in the free text box something along the lines of: "This is a very US-centric question. You know that races don't actually exist, don't you?" – Rounin Dec 2 '19 at 11:41
  • 2
    @ColeValleyGirl those questions would be asking about your ethnicity or nationality, not your race. And they're always optional. – OrangeDog Dec 2 '19 at 15:55
  • 1
    @StopHarmingMonica 1) the Questions asked always divide respondents into racial groups -- see history.blog.gov.uk/2019/03/07/… for examples. 2) Yes, optional as I believe the Loop survey now has them. I don't disagree that as implemented the Loop survey needed some work, but it's wrong to call asking that question USA-centric. – ColeValleyGirl Dec 2 '19 at 16:06
  • 2
    @ColeValleyGirl the title is literally "50 years of collecting ethnicity data" – OrangeDog Dec 2 '19 at 16:07
  • 1
    @StopHarmingMonica And the top level categories are: White, Mixed, Asian, Black, Chinese or Other as per ethnicity-facts-figures.service.gov.uk/ethnic-groups and – ColeValleyGirl Dec 2 '19 at 16:11
  • @ColeValleyGirl: I'm from the UK too, & the first time I was ever asked for my race on a form was in Brazil - I was rather taken aback. – Scortchi - Reinstate Monica Dec 2 '19 at 18:20
30

The groups mentioned above are somewhat different to those used by the US census

Of course. Because such attempts to group are always built on pseudo-biological theory.

Attempting to categorise individual humans by race is a highly dubious pseudo-science which first sought acceptance in the 16th century's Age of Exploration (for reasons we can readily guess) and which led to some truly horrific theories and practices in the 17th, 18th, 19th and 20th centuries.

As a global population we would all do well to regard the proposal that homo sapiens is divided into races with the withering scepticism such a flawed notion merits.

| improve this answer | |
  • 5
    (+1) While doubtless the question's offensive to many, in the USA & other countries of the Americas & Caribbean where slavery was until recently endemic, censuses & the like still ask people their "race": I'm sure this was simply something between carelessness & indifference on SE's part, rather than an attempt to promote racial ideology. – Scortchi - Reinstate Monica Dec 1 '19 at 20:07
  • 6
    @Scortchi: Everything about SE's new direction is about promoting inclusiveness. Racial and gender ideology are a big part of that effort. – user102937 Dec 1 '19 at 22:40
  • 2
    Homo sapiens mixed with Neanderthal, unless for subsahara people – user657339 Dec 1 '19 at 23:06
  • 1
    Go on then, @Universal_learner - homo sapiens sapiens, mixed with homo sapiens neanderthalensis. – Rounin Dec 1 '19 at 23:09
  • 1
    @gbjbaanb - Genetics is based on empirical evidence. That's science. Socio-ethno-cultural groups (tribes / proto-nations etc.) have fuzzy (and fluid) boundaries, but they really do emerge over time and vanish over time. That's social science. By contrast, Race is not based on any empirical reality at all. It relies on partisan categorisation and its flaws are easily pointed out. That's pseudo-science. – Rounin Dec 1 '19 at 23:50
  • 8
    @gbjbaanb - Note the Cavalli-Sforza obituary you posted above states: "Cavalli-Sforza’s genetic work earned him accolades from those hoping to break down the barriers of race. He found that people from the same population are as genetically diverse as people from two different groups, essentially showing that at the genetic level, there is no such thing as race. [...] Jared Diamond praised the Stanford researcher for “demolishing scientists’ attempts to classify human populations into races" (My bold) – Rounin Dec 2 '19 at 1:08
  • 3
    @gbjbaanb: Section 1.6 of the History and Geography of Human Genes, "Scientific failure of the concept of human races" contains some particularly interesting insights. – Scortchi - Reinstate Monica Dec 2 '19 at 1:09
  • 5
    My apologies, @gbjbaanb. It sounds like we're on the same page. In which case I misinterpreted "It's not quite the pseudo-science you think.". – Rounin Dec 2 '19 at 11:28
  • 1
    @BryanKrause: In a nutshell, social scientists in the USA have re-defined race as a social construct; whereas in Europe they've brought in ethnicity (& cognates) to do that job, leaving race with its old, supposititiously biological, meaning. So tomayto, tomahto in the end, perhaps: but it's hardly incumbent on survey-takers to "consider the cultural context" of survey-writers; quite the reverse - if the latter hope to get high response rates & meaningful answers (as well as to show some basic courtesy). SE could do this sort of thing well, if they cared to. – Scortchi - Reinstate Monica Dec 3 '19 at 21:49
  • 1
    @RouninsaysJesuisMonica In the US, "race" is used to include perceived race. Which means, for example, "having dark skin". I feel like we are in a bit of violent agreement here: biological race does not exist...nonetheless, people get categorized by others - SE and typical social science in the US calls these categories race, even when they agree with you that biological race is pseudoscience. Please please please listen to these differences in terms before you make your arguments. – Bryan Krause Dec 4 '19 at 6:24
  • 1
    @RouninsaysJesuisMonica Right, I too wish that skin color was discriminated no more than neck length. But it's a bit unfair to the people that are currently being discriminated against to just pretend it doesn't matter. Even if you suddenly banned talking about skin color for the new generation, the kids are going to notice differences that exist in society. How do we explain that to them? Do we remove discussions of the history of slavery from school? Do we skip the efforts that have gone into civil rights? Do we deny the modern consequences of that history? – Bryan Krause Dec 4 '19 at 16:53
  • 1
    @RouninsaysJesuisMonica I can agree with all that, for sure. How should we measure the present-day discrepancies that persist because of that history? – Bryan Krause Dec 4 '19 at 17:34
  • 2
    @RouninsaysJesuisMonica But how do you know if you are doing it right? How do you know if you are actually giving balanced opportunities? How do you know if there is a historically discriminated against group that is currently under-served by society? – Bryan Krause Dec 4 '19 at 18:10
  • 1
    @RouninsaysJesuisMonica But how do you identify the individuals that have been discriminated against systematically due to their collection of physical characteristics? The whole point is that those socially constructed groups have been the basis for discrimination. – Bryan Krause Dec 4 '19 at 19:37
  • 2
    @RouninsaysJesuisMonica That sounds like something that would work great for the people not being discriminated against, and not work that well for the people who are. For one, it removes their ability to show they are still discriminated against. – Bryan Krause Dec 5 '19 at 1:45
3

How did the people behind "The Loop", and previous surveys, decide upon the categorisation of racial backgrounds?

...

The groups mentioned above are somewhat different to those used by the US census, the country in which the company is based.

The blog: Introducing “The Loop”: A Foundation in Listening points to: "Developer Survey Results 2019" which points to: "State of the Stack 2019: A Year in Review", which says they use the Rooney Rule (as does Pinterest, Facebook, Patreon and Checkr, etc.,) during hiring.

"We’ve been working hard to create a more diverse team and inclusive team internally. We’re taking a look at our policies, benefits, interview practices, and internal trainings to make sure that we’re supporting the team we have. We’re also working hard to recruit a more diverse team.

...

While this isn’t where we want to be, it represents substantial growth over the past few years. In 2018, we also concluded a third-party compensation review to address any gender or racial pay gaps, and implemented a Rooney Rule for tech and leadership hiring to ensure we continue to address the gaps in our hiring.".

Whether hiring practices extend to every possible interaction isn't specified but fair application of the rule would ensure that population demographic groups and Stack Exchange/Overflow demographic groups are held at approximately equal levels. Were there any shortcoming it ought to be investigated. With the Rooney Rule in effect and that blog article co-authored by two persons, subject to review (internal and external), one expects that efforts are being made to equalize representation and treatment of everyone.

That isn't without its difficulties. The manner in which it is split needs to be categorically factorable. Categorization, even within the US Census, has been a fluid and open question, which is often reanalyzed.

On page 4 of "American Anthropological Association Response to OMB Directive 15: Race and Ethnic Standards for Federal Statistics and Administrative Reporting" (Sept 1997) an analysis of how categories are derived for the US Census was presented:

"The American Anthropological Association recognizes that classical racial terms may be useful for many people who prefer to use proudly such terms about themselves. The Association wishes to stress that if biological information is not the objective, biological-sounding terms add nothing to the precision, rigor, or factual basis of information being collected to characterize the identities of the American population. In that sense, phasing out the term "race," to be replaced with more correct terms related to ethnicity, such as "ethnic origins," would be less prone to misunderstanding.

Social and Cultural Aspects of "Race" and "Ethnicity"

Race and ethnicity both represent social or cultural constructs for categorizing people based on perceived differences in biology (physical appearance) and behavior. Although popular connotations of race tend to be associated with biology and those of ethnicity with culture, the two concepts are not clearly distinct from one another.

While diverse definitions exist, ethnicity may be defined as the identification with population groups characterized by common ancestry, language and custom. Because of common origins and intermarriage, ethnic groups often share physical characteristics which also then become a part of their identification--by themselves and/or by others. However, populations with similar physical appearance may have different ethnic identities, and populations with different physical appearances may have a common ethnic identity.

OMB Directive 15 views race and ethnicity as distinct phenomena and appropriate ways to categorize people because both are thought to identify distinct populations. Although this viewpoint may capture some aspects of the way most people think about race and ethnicity, it overlooks or distorts other critical aspects of the same process.".

Let's see what happens when we take organizations that we presume to be fair and honest and ask about their demographics, does this look even?

Police Dept. Race and Ethnicity Demographics
Source: "Police Department Race and Ethnicity Demographic Data" (Click to Zoom)

| improve this answer | |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .