Independent verification is a pillar of the scientific method, partly for sanity-checking and also because (and I've mentioned this multiple times before) both the cohorts used for the loop and the method are kept behind closed doors.
On an open-ended form, one would expect the average user to have multiple, sometimes competing concerns. For instance, one may flag up design and community. As such, one would expect either a sum total of events to surpass 100%, or for it to be renormalized in some way.
This is the graph presented in The Loop #2: Understanding Site Satisfaction, Summer 2019 article:
Coded responses to "What do you find most frustrating or unappealing about using Stack Overflow?"
Unwelcoming community 10.6%
Design 9.8%
Artifact quality 9.7%
Barrier to participation 8.3%
Discovery 8.0%
Overmoderation 7.1%
Voting 5.1%
Question quality 4.2%
Timely answers 3.5%
Other 3.2%
Comments 2.2%
Onboarding 2.1%
Social friction 1.8%
Subjective content 0.8%
Mobile app/site 0.6%
Welcoming backlash 0.5%
Job quality 0.4%
Review queues 0.3%
Two things immediately stand out:
1. The dataset is incomplete
It should not take a genius to figure out that, even with the previous caveats (that either we are looking at a sum total of events > 100%, or renormalized), this graph is once again missing 20% of respondents.
2. Some concerns overlap
"Question quality" and "Artifact quality" are the two biggest examples of this, where somebody would have to have written an entire paragraph for a human to, objectively, without letting their bias taint the coding, identify it was one and not the other.
Due to these concerns, could we please follow the most basic scientific method steps and release an anonymized dataset for people who would be willing to cross-check the findings?