29

Together with a number of colleagues (UC Davis, Polytech'Nantes, Brunel U, Eindhoven U of Technology) we are working on a series of research studies related to Stack Overflow. While I do understand that this post would be technically off topic - it's about a study that could involve Stack Overflow users - not about Stack Overflow or the community itself, I wonder

  • to what extent would you be interested in entertaining academic studies?

  • under what circumstances would you be willing to participate in interviews, user surveys, lab experiments or any other activity related to academic research?

To sketch the context: in essence we would like to understand how developers who use Stack Overflow present themselves, communicate with each other, and how does this communication translate to technical solutions.

As part of these research effort we would like to hear your opinions: and to this extent we organize surveys. Participation in this surveys is voluntary and confidential, and no information is disclosed to third parties under any circumstances. Responses -unless explicitly stated otherwise- cannot be traced back to individual respondents. Moreover, the results of such surveys would be included in scientific publications, hence transparent, and no personal data is going to be published. Abridged version of the study results will be published on meta as well.

One of our current surveys recently advertised on Meta did not receive the warmest of welcomes. From among the critiques there were the appearance of spam, being off-topic, or not contacting the SE team first (one of the commenters suggested the "Ad Sales and Sponsorships" procedure). The post has been deleted in the meantime.

We had an email exchange with the SE team. Although they sympathise with researchers taking an interest in the SO/SE community, there is no official mechanism by which they can support or promote such academic requests (unless via ad space, which is not really accessible to us given the scientific nature of our study and our lack of commercial interest). The SE team also recommended us to make this kind of post polling the community opinion on the two questions above.

Improve this question
8
  • Thank you for your response. Will you also tell who is funding the research study? You indicated that there is a "lack of commercial interest." Is this research purely academic and funded exclusively by the universities listed, or are other parties involved?
    – George Cummins
    Jul 11, 2013 at 14:26
  • @GeorgeCummins: this is a loose cooperation between different universities listed above: UC Davis, Polytech'Nantes, Brunel U, Eindhoven U of Technology. For instance, at Eindhoven U of Technology my involvement in this research is not explicitly funded by external bodies, while Bogdan is being paid by the Dutch Science Foundation and our visitor from Polytech'Nantes has a topic-free scholarship from his region.
    – Alexander Serebrenik
    Jul 11, 2013 at 14:30
  • @JoeFrambach: thank you very much for your suggestion. I have revised the post accordingly.
    – Alexander Serebrenik
    Jul 11, 2013 at 14:33
  • 7
    I don't understand why the initial post was received so poorly. Studies of that sort could produce information about Stack Overflow that would be very relevant to meta-discussion. The fact that the post and request for input originates from researchers instead of the usual meta crowd means it could provide a different and interesting perspective. We should welcome this. (It would be different if you were a commercial group, but you're academics!) I found your original posts entirely acceptable and would be interested in participating.
    – Jeremy
    Jul 11, 2013 at 14:43
  • I trust that you have read the documentation, but it may be worth mentioning here that the number of net upvotes on your question indicates (to some extent) the positive attitude of the community toward your request.
    – George Cummins
    Jul 11, 2013 at 15:02
  • I have been thinking about this in the past and would love to participate and even contribute directly to the research.
    – asheeshr
    Jul 11, 2013 at 15:33
  • 1
    You mention a rather cold welcome to a previous post. Note that both that and the current one are rather long and difficult to actually comprehend. You may have a greater chance of getting responses if you post brief, concrete questions (or proposals).
    – Daniel Daranas
    Jul 11, 2013 at 16:16
  • 2
    I would suggest quoting this question in any survey requests (to give some credibity to it)
    – Richard Tingle
    Jul 11, 2013 at 16:17

3 Answers 3

Reset to default
15

I'm in the social/technical sciences myself, and I know how hard it is to get a representative sample of participants for scientific studies. Especially as "an outsider", with the best of intents, asking for survey participation may seem like spam. If you're facing opposition from part of the Meta community, I can also relate to this.

under what circumstances would you be willing to participate in interviews, user surveys, lab experiments or any other activity related to academic research?

  • Be as specific as you can about the intents of the study. What information needs to be collected, and why do you need it? If you need personally identifying data—even with all precautions taken on your side so that the data is safe and anonymized—people are going to be skeptical.

  • Tell us about the background of the study and who funded it. Many academic institutions run low on their own budget and rely on third-party funding in order to pay for project expenses or even the salary of researchers. I've seen complete departments run on third-party funding only. Commercially driven surveys might face strong opposition here.

  • Perhaps you could create some sort of opt-in process. You might want to do follow-up studies, longer interviews via Skype, etc. If someone's interested, let them sign up on a mailing list and do all the further discussion offline (or online, that is, but not here on Meta).

  • Reward the community. Make your results available if—or as soon as—you are allowed to. Not every result has to be there, of course, since otherwise you could not publish them in traditional journals or conference proceedings. But this sort of introspection could help the Stack Overflow community as a whole, since we're finding ourselves dealing with lots of human issues, especially here on Meta.

Improve this answer
5
  • 1
    And greetings to the guy from Nantes. Had an invitation to Polytech myself, very nice people there, and great researchers.
    – slhck
    Jul 11, 2013 at 14:53
  • 1
    +1 for an opt in option.
    – KatieK
    Jul 11, 2013 at 15:23
  • @slhck: thank you for your advice. Our surveys typically include an optional question "Would you like to be informed about the outcome of this study and potential publications arising from it? Please enter a contact email address" - which is an opt-in. We also reward the community by publishing an abridged version of the results as an update to the question where we ask for participation in a survey.
    – Alexander Serebrenik
    Jul 11, 2013 at 17:48
  • "Especially as "an outsider", with the best of intents, asking for survey participation may seem like spam." It is spam. Exactly spam. It's about the good and the desires of the poster and not about the good of the site. I know, I really do know that it is hard to get the study population you want, but people are also shot down for asking off-topic questions because they are trying to get a bigger audience. Same thing.
    – dmckee --- ex-moderator kitten
    Jul 12, 2013 at 2:26
  • 3
    @dmckee I would not agree with your statement since the results of the study are made known on meta itself, meaning that the results flow back to the community.
    – Alexander Serebrenik
    Jul 12, 2013 at 4:37
7

I am generally opposed to participating in surveys, especially those that primarily feed the needs of commercial interests.

However, I appreciate that the Stack Overflow community has become a significant presence on the internet. Because of the distinguishing features that set it apart from typical help forums, SO contains dynamics that do not exist elsewhere.

Since Stack Overflow's effect on the internet has generally been positive (in my opinion) there is benefit to be gained from gaining a deep understanding of those dynamics and applying them (where possible) to other communities and endeavors.

For this reason, I am willing to participate in such a study.

under what circumstances would you be willing to participate in interviews, user surveys, lab experiments or any other activity related to academic research?

My primary goal is to continue to continue to provide value within this community. To the extent that the particulars of the study allow me to continue to do that, and as long as the interest primarily academic rather than commercial, I am willing to consider a variety of activities.

Improve this answer
3

Maybe it would be better to make use of the advertisement system? That way you are out of the question/answer band. Possibly the team and community would support hosting adds for academic studies free as they do for Open Source software.

That's not a perfect answer, of course, because right now sufficiently high rep user don't have to see any of these.

Improve this answer
1
  • 2
    The official answer of the team representative reads "We can't officially endorse these types of studies, so it's up to the community to decide".
    – Alexander Serebrenik
    Jul 12, 2013 at 4:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .