29

I just came across Is there any user satisfaction data for other SE sites than Stack Overflow? and I think, most likely, the bleak answer will be: no, there is no official survey data that explicitly covers other sites on this network.

Leading to a crazy (?) idea (somehow resembling something that user Renault inspired in a different place):

  • We users could sit together and design a (small) set survey of questions.
  • That questionnaire could then be published as a meta question, and users could then "fill out" that questionnaire.

We could start doing this here on MSE. And in case it works, other communities could pick up our questionnaire, and adapt it accordingly, and then run a "local" survey on their individual meta sites.

Two Three questions:

  • Would "doing exactly that" be within "on topic"? Or are there legitimate reasons/policies that prevent us from doing it like that?
  • If it is allowed, is it a good idea?
  • Or would you prefer to try to convince SE Inc. to run "our" survey through "their" channels, as outlined in comments below?

Sure, that wouldn't be the same as a "real" survey: answers are tight to a username, automatic processing/evaluation would be hard. But it would be something under "our" control, and it would enable any community member to look at the raw data.

9
  • 4
    CM Jon Ericson once created one for Sound.SE: Where do we go from here? Take the survey! – fedorqui 'SO stop harming' Jan 24 '20 at 9:52
  • 3
    Apart from the question if this is allowed on meta, would the results tell us something? An issue I see is that a user survey wouldn't have the same large audience like an officially announced one, so that could mean that only users who are interested in such a user survey will participate and the results aren't able to tell us something about the overall perception of the Stack Exchange sites. – Tom Jan 24 '20 at 9:55
  • @fedorqui'SOstopharming' That is good to know. The downside is: it depends on an individual, or a small group of people to build something like that. You also need a server, consider privacy, whatnot. The big advantage of doing it here is ... it is all out in the open. – GhostCat Jan 24 '20 at 9:55
  • @Tom We could still try our best to announce it too as many people as possible ... – GhostCat Jan 24 '20 at 9:56
  • Yes of course, I just wonder how reliable the data will be. We sadly can't tell how visible the announcement was and if it reached a broader audience to get representative data. The amount of people taking the survey might be an indicator. It would be great to get a mod onboard to get a featured tag, that would help. – Tom Jan 24 '20 at 10:02
  • @Tom We definitely need mods onboard on the various site-specific metas, but I suspect that will be no problem at all. If some sites don't want to survey then they simply don't, we're not forcing any site to participate. – gerrit Jan 24 '20 at 10:42
  • 2
    Wouldn't it be nicer if the company would circulate this survey in the same manner they circulate their customer satisfaction survey? We would just design it, they would circulate it and return the data after the survey is finished and then we analyze it. It should just clearly be marked as not an official survey by the company. – Trilarion Jan 24 '20 at 11:01
  • @Trilarion If the community would decide they wanted to go forward with this, I can raise it at the internal moderator team site; I have the impression there is a better chance of getting someone from SE to actually read it (perhaps due to the higher signal to noise ratio compared to Meta SE). – gerrit Jan 24 '20 at 11:34
  • @gerrit Makes sense, I updated my question to include this as third option. – GhostCat Jan 24 '20 at 11:37
21

We can do this. We can use a community wiki post to design the survey questions, but let's not use Q&A to conduct the survey. It's the wrong tool.

There are plenty of free alternatives online for conducting surveys, such as SurveyMonkey (we may have to search around if their free plan doesn't meet our needs, perhaps we can find a volunteer to host a survey on their own personal server). We would just need volunteers to host the SurveyMonkey (or other) survey under their personal account and publish the raw results somewhere (possibly stripped of responses that might identify users if there are any open questions), and would then need volunteers to process the results.

One issue would be the balance between anonymity vs. fraud-resistance. It's not possible to guarantee both, so we'd have to choose which one is more important to us.

Another issue would be representativeness, but that is true for any survey and there's no way getting around that. It would be up to each site to feature an announcement of the survey and hoping to get a good response.

4
  • I was under the impression that survey monkey would only work when you throw money at them. There is a "free plan", but that restricts you to view only the first 100 responses for example. – GhostCat Jan 24 '20 at 10:09
  • 1
    @GhostCatsalutesMonicaC. Oh, that could be, I've never gotten close to 100 responses to surveys I did there. A dedicated survey for a smaller site would be happy to get close to 100 responses, but if we do a single cross-SE survey we would indeed need something where we can gather a larger number. – gerrit Jan 24 '20 at 10:12
  • 1
    i doubt it'd be difficult to find someone willing to sponsor such a survey. – Kevin B Jan 24 '20 at 20:34
  • @user400654 Well, reception here hasn't been negative, but also not overly excited either. – GhostCat Jan 28 '20 at 8:51
5

In terms of mechanics (there was some discussion of Survey Monkey) -- the easiest approach in my experience is a Google Docs Form.

I like your idea very much. If support grows for this idea, you could next post a question where you collect suggestions for specific questions.

Related:

Academia Community Polls

Where do we go from here? Take the survey!

Survey of users to judge quality and culture of an SE site

2
  • 1
    NB. Google is blocked in China. – Rebecca J. Stones Jan 25 '20 at 12:36
  • @RebeccaJ.Stones - Bummer. Do you know what would work there? – aparente001 Jan 26 '20 at 5:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .