22

I've recently taken the survey. Based on my (admittedly limited) understanding of statistics and social science, I can't come up with a meaningful way to analyse the results.

Some of the problems that I see are these:

  1. Some of the questions have been changed part way through
  2. Some of the demographic options seem... limited
  3. The non-demographic questions don't appear to be open to statistical analysis
  4. The non-demographic questions don't invite especially detailed nor comparable answers

So:

  • Are there other issues that I have missed?
  • Are there methods for overcoming these potential issues?
  • What kind of valuable information can be drawn from this survey?
18

Admittedly, not a lot. As you said, the changes halfway through really throw a wrench in any kind of statistical analysis, because now you have two different sets of data, and it's like comparing apples and oranges.

The fact that people cannot give detailed and descriptive answers to the survey questions greatly limits how SE will be able to use the data. It may provide broad generalizations about the problem, but specific feedback would be much more helpful to them so they can tailor a new feedback system to the desires of the people.

And limiting the demographic options is just a really dumb idea. You cannot associate the results with specific groups because the groups really aren't specific.

  • 7
    From their answers to other questions it seems their solution is "just use machine learning". I assume they have a model that has been trained on "I support 100% the agenda of Stack Exchange" and "Troll answer" and can classify all answers into this two categories. But that's just my assumption. – Josef Nov 28 at 12:50
  • @Josef I wonder what "i sUpPorT 100% tHe AgEndA of StAcK exChaNgE" will fall into – MechMK1 Nov 28 at 13:23
  • 4
    @Mech, first step of machine learning is to normalize case, because casing is not supposed to impact meaning. Thus, the pattern in your comment would match we're doing the right thing with full support from the community 100%. – Frédéric Hamidi Nov 28 at 14:10
  • 2
    @FrédéricHamidi And it clearly stripped away my intent of mocking SE, Inc., thus successfully failing at generating meaningful data. – MechMK1 Nov 28 at 14:13
  • 2
    @Mech, to be fair, machine learning is still several light years away from understanding humor. Even human beings have at hard time achieving that in some circumstances. – Frédéric Hamidi Nov 28 at 14:15
  • 5
    @FrédéricHamidi That's undeniably true. Yet it illustrates how easily one can send a message that any human would immediately be able to interpret one way, and any machine would interpret the opposite way. – MechMK1 Nov 28 at 14:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .