6

Today, I shared an update on the blog about the research we will be conducting over the next quarter.

TL;DR – this quarter’s research focus is as follows:

  • Motivation, incentives, and gamification
  • Content permanence and modalities

As a reminder, this strategic research typically operates months in advance and can sometimes result in us making the decision that there is not a problem we are well-positioned to solve at this time. But if you’d like to help us get started, please let us know below if you’d like to see other content types on Stack Overflow/Stack Exchange? What motivates you to engage with Stack Overflow/Stack Exchange? Is it the same thing that motivated you when you first started on the site?

We are deeply in need of highly engaged research participants with varying levels of rep and site interactions. If you are not signed up for research, please sign up for research invitations in your account settings. If you are signed up, please send to your friends, coworkers, neighbors, ex sister-in-law, or whoever you know that interacts with Stack Overflow or Stack Exchange.

13
  • Also, how is it related to design? (I see it has this tag both here and in the blog.) Commented Nov 4 at 14:14
  • 4
    @ShadowWizard Updated wording in meta post. Design is used because both research and design are included in the user experience team. We also typically have tactical research, that's run by designers presented in this write up but didn't have anything to share this quarter.
    – Piper Staff
    Commented Nov 4 at 14:57
  • As much as i'd love to participate in research initiatives... they're almost always done through methods i'm unwilling to interact with. :shrug:
    – Kevin B
    Commented Nov 4 at 15:35
  • 4
    I'd like to see Area 51 modernized. Technological and social development has not frozen us in our current state - it is in S/E's interest to allow for the possibility that future public interests (i.e. where the money lies) would be served by further sites being given the chance to flourish (and generate revenue). Perhaps S/E could research future trends.........
    – W.O.
    Commented Nov 4 at 18:06
  • I can’t help but feel this approach has a huge survivorship bias. I mean, in the end you are asking the people happy enough to stay and engage with you - when user/participation numbers indicate the problem is the significant number of people who don’t want to do that anymore. Commented Nov 4 at 18:10
  • 1
    @MisterMiyagi You are totally right. And this is not the only way we are collecting feedback. We will do a lot of research first. But I also wanted to bring these questions to the group because you all are still here and I am genuinely curious.
    – Piper Staff
    Commented Nov 4 at 19:42
  • 2
    @W.O. Yes!!! I 100% agree and happy to announce that it's on our research roadmap for after one of these projects wraps up.
    – Piper Staff
    Commented Nov 4 at 20:45
  • 3
    @KevinB Thanks for sharing your POV! We want to get your feedback in ways that work for you and others, and we know different methods are more effective at different stages of a project. For some questions, observing or understanding users in their real contexts can be key, since what people say and do sometimes varies. Are there feedback methods you prefer? If you’ve had good or not-so-great experiences providing feedback before, we’d love to hear what works best and anyone else with ideas! Extra credit: take a look at the cheat sheet: nngroup.com/articles/ux-research-cheat-sheet
    – emmabee Staff
    Commented Nov 4 at 23:24
  • I strictly don't use SOfT or do live sessions. thus far everything I've been reached out for has been one or the other, aside from random surveys.
    – Kevin B
    Commented Nov 5 at 1:34
  • 1
    @KevinB I can appreciate that not everyone is into the live sessions and SOfTeams is also not ideal for everything. Are there methods that you prefer and work better for you? Also, others: please chime in if you share this perspective and have some thoughts on ways you would ideally like to provide feedback at different stages.
    – emmabee Staff
    Commented Nov 5 at 2:02
  • 1
    @emmabee I've done 2 of those research things. The first one was a live session, and I found that it was really hard to give proper feedback on the prototypes/designs I was shown due to a language barrier: I don't know how to properly pronounce far more than half of the English words I can write, so it's really hard to give feedback that way. There was also some time-pressure as the meeting was only x minutes long. The second one was similar but written, and it was much more pleasant for me to not be struggling to convey my thoughts properly, and have no time-limit while doing so.
    – Tinkeringbell Mod
    Commented Nov 5 at 10:13
  • 2
    @Tinkeringbell Thanks for letting me know about the language barrier! That is good for us to keep in mind. Maybe we can make sure we are at least letting folks know when conversations are in English (currently: always). I am glad to hear you had an easier time participating in the written one. Was it by chance an interactive session with a tool we've been trying out called ballparkhq.com ? This tool allows folks to interact with prototypes and is a bit more engaging than a survey, and it sounds like it was a good fit for you.
    – emmabee Staff
    Commented Nov 5 at 19:06
  • @emmabee I had to look that up... No, it wasn't anything like that. It was just a regular survey with screenshots (nothing interactive) and text boxes to type feedback for those screenshots, shared on the Stack Moderators SOfT.
    – Tinkeringbell Mod
    Commented Nov 6 at 13:32

3 Answers 3

9

As a mini little Meta preamble: This is your opportunity to make your voice heard about the gamification system on SE, similar to when they asked for feedback regarding the Reputation system. Please consider participating in the research opportunities. It's important to ensure SE gets this attempt at improving the new user experience right, because we all know how previous attempts have gone.


Let us know below if you’d like to see other content types on Stack Overflow?

I really, really don't think so. To solidify why I think this way, let's look at the other content types we've tried recently:

  • Articles: Well, sometimes they convey meaningful info, but many other times they're either puff pieces or uninteresting content.
  • Discussions: Discussions haven't been iterated on recently, and the changes that have been made don't make users feel like it's a useful part of the site. We can't downvote, which means that there's no way to meaningfully signal to the site what content is not worth looking at. Every 0 score discussion is a toss-up between whether it's cool (not likely nowadays), uninteresting (most likely), or not worth keeping (spam/off-topic).

The Stack Exchange network built its popularity off of being the largest library of question and answer content on the Internet. Iterate on what got you here in the first place instead of exploring outside of what makes SE great. Creating new content types hasn't gone well in the past, and I can promise you it won't go well in the future without some very clear definition about what you want to do, and support from the community about the vision. We've tried telling you when you're stepping in the wrong direction, but it feels like it doesn't turn you around. Like Makoto said:

My feedback doesn't seem to count. That's the energy that I'm getting from this.


What motivates you to engage with Stack Overflow?

When I see an interesting question that I can meaningfully answer, I feel really good if it lead to helping someone. Ideally, if I can help multiple people (and many more after, for as long as my content is visible and accessible), then that's a big bonus. Upvotes on my content indicate that it was useful in some capacity, and getting "paid" via reputation feels good.

Further, the privileges system dangles some interesting carrots on a stick. While I've made my position on the reputation and privilege system clear before, I do still want to be able to vote to close stuff as I browse a given SE site. I want to be able to vote to delete things that don't belong. Creating useful content on a site is the way that's done, and while that can take awhile nowadays on Stack Overflow, that motivates me in some way.

Boiling that feedback on the reputation system down to a simple few statements, I'd say: Reputation is a measure of the good content you've contributed to the site that others found useful. Good use of moderation privileges is contributing a similar amount of time to curating content as you would creating content, and that should be rewarded in some capacity. Badges, for privilege hunters, are a poor measure of this, because with the exception of gold tag badges, they don't let you do anything. Expand the user level moderation privileges to let users do more if they're proving they can use them correctly and let the site manage itself. Stack Overflow in particular seems to rely a bit too much on its moderators, who are meant to be exception handlers. There seem to be quite a few exceptions for them to handle!

When it comes to Meta, my participation in large part nowadays is to try and meaningfully contribute to the network as a whole, typically in policy conversations or on questions like these. I'm just trying to add another voice to the chorus. Sometimes we harmonize, and sometimes we're a bit discordant. All of that singing is to try and make the network the best it can be, though.

Is it the same thing that motivated you when you first started on the site?

Sorta. On Meta, rep doesn't mean anything to me anymore. On other sites, sure, more privileges on more sites lets me help those sites a bit more. But if the site doesn't interest me at all, or it's a low-activity site, I really don't feel interested in contributing the content necessary to try and get more privileges and thus give back to the site.

From the blog:

Is there a need for varying levels of permanence for our questions and answers [..]?

Questions? Not really. Answers? ... Maybe? We had an outdated answers project that resulted in a trending sort, but... Not much else, if memory serves? We were supposed to get version labels for answers but that didn't really get anywhere.

I'm not sure. I'd hesitantly say "No" because content posted on the SE network that doesn't violate our rules is usually here to stay for conceivably forever. We shouldn't consider changing that without some serious thought.

We’ve heard from previous research that there doesn’t feel like there is a space for developers and technologists to commune—could Stack Overflow be that space?

Like I said before, I really think Stack Overflow can't be that place without some very serious vision in place that the community is interested in backing. If you explore this, reading our feedback is critical, otherwise you'll just end up with Discussions 2.0.

On the blog, there's also this bit:

Jobs-to-be-Done being researched: “Find my people when I am looking to connect so that I can socialize with like-minded people in a positive environment.”

We're not a social media site. I can see why folks may think that, but we're not. We're really, really not. Let's not try to be. We're not built for that, and building us for that would be an incredible undertaking.

We do have Chat, though, AKA "Bonfire". One could argue that's a form of social media. It's undiscoverable, hasn't had features built for it in awhile, and deserves more recognition than it gets. Also, you need rep to be able to use it in the first place. Not saying that should change (think of the abuse vector if everyone could make an account and chat...), just throwing it out there.

“Identify gaps of knowledge when I find that knowledge that doesn’t exist so that I can complete the body of knowledge without it taking up a large amount of time”

Does providing it as a question and answer pair not fit here? We're not thinking of spinning up Documentation again are we?

18
  • 1
    I know it doesn't feel like it at times, but we are also trying to make the network the best it can be. I really appreciate your thoughtful response to my motivation question. When did you stop caring about rep? When you gained enough to unlock the privileges you needed it or was it some other event?
    – Piper Staff
    Commented Nov 4 at 20:13
  • 1
    Regarding the space for folks to commune you said "if you explore this, reading our feedback is critical". Are you referring to existing feedback or new feedback?
    – Piper Staff
    Commented Nov 4 at 20:14
  • 1
    Do you use chat? If you do, when you use it, do you feel as if you are socializing? I do agree with you. So much potential abuse and our mods have enough on their plates!
    – Piper Staff
    Commented Nov 4 at 20:16
  • 1
    A question and answer pair does in fact fit into the final job-to-be-done, but that job is not limited to question and answer. Identifying gaps within Q&A is simply too narrow. We simply use Q&A to convince someone to "hire" us to do this job. It's the details of the jobs to be done framework. Hope that wasn't confusing!
    – Piper Staff
    Commented Nov 4 at 20:20
  • 1
    I'll always believe that your motivations are to make the network a better place, @Piper . We will always have our disagreements, but we have to come from a place of believing y'all are acting in good faith. When that stops, there's no point in engaging. Understand that the feeling of our feedback not feeling impactful enough is just from a decent history of that feeling being reaffirmed several times. I'm always hopeful that the Next Big Thing (tm) will be great, and if I have something to add, I'll always try to, because just like you, I think it's really worth it.
    – Spevacus Mod
    Commented Nov 4 at 20:22
  • 1
    "Are you referring to existing feedback or new feedback?" - I was referring to, if you explore something new, please take our (new) feedback into account. "Do you use chat? If you do, when you use it, do you feel as if you are socializing?" - Yep! I use it a fair bit. I definitely think I'm socializing. If you want, feel free to drop in the Tavern and socialize with us!
    – Spevacus Mod
    Commented Nov 4 at 20:23
  • 1
    @Piper Re: "Identifying gaps within Q&A is simply too narrow" - Could you elaborate a bit about what you mean here? As in, are there use-cases for new forms of information contribution that you feel is worth exploring? Could you give a few examples? No need to be specific or hedge here, we're just throwing stuff at the wall.
    – Spevacus Mod
    Commented Nov 4 at 20:25
  • 1
    I missed a couple questions, sorry: "When did you stop caring about rep? When you gained enough to unlock the privileges you needed it or was it some other event?" - You got it right: Hitting Trusted User is pretty much the end of more rep being worth it to me. If there were other goals (side along to rep maybe?), I'd be motivated to reach them. So long as the methods to reach those goals are "Do something good" and not "Do something robotically", I'm on board.
    – Spevacus Mod
    Commented Nov 4 at 20:28
  • 1
    I simply mean that the jobs-to-be-done framework is a replacement for user personas that talks about why a user or person may choose a certain product. You develop a statement around the main "job" a user is looking to do. This particular jtbd talks about "identifying gaps in knowledge" where if I were to say "identifying gaps in q&a" it is too specific and likely not a main "job" many would seek to accomplish.
    – Piper Staff
    Commented Nov 4 at 20:29
  • 1
    What we want to do is set ourselves apart as a community that thrives off human contributions. It is a challenge to do that in just Q&A for a couple of reasons. First of all it’s one of the areas where AI is making the biggest impact. Secondly as a sixteen plus year old platform we have reflect that consumption of knowledge has new form factors beyond just Q&A. So the question is where else can we create knowledge that requires human contributions and a community that loves to help others.... [1/2]
    – Piper Staff
    Commented Nov 4 at 20:42
  • 1
    ...It’s unknown if that will mean investing more in the areas you’ve mentioned to improve (articles, discussions, chat, etc) to improve those features or if it means looking to something completely different. What it does mean is that we will assess if there is a need or appetite from our user base for such features. [2/2]
    – Piper Staff
    Commented Nov 4 at 20:43
  • @Spevacus you mention "hitting Trusted User is pretty much the end of more rep being worth it to me". I'm curious: what originally motivated you to achieve Trusted User status (what was appealing), and how does it feel now that you're there? You also said, 'If there were other goals, I'd be motivated to reach them.' What might those goals be for you? Since you're at the top, I’d love to know what new aspirations you have or if you can envision a future where SO/SE better supports goals that aren't currently achievable.
    – emmabee Staff
    Commented Nov 5 at 19:12
  • @Piper Motivation was primarily being able to vote to delete answers. We have a lot less now, but we had a lot of content that needed deletion awhile back. Being able to help out directly with that was something I was interested in. As far as other goals... Well, that's kind of a tough question. Any more access to user-level moderation tooling would be something I'd be interested in shooting for. On SO there's a lot of content that needs handling that moderators have to step in and handle rather than normal users, and there are places where users could help more. (1/2)
    – Spevacus Mod
    Commented Nov 5 at 19:59
  • Giving users a bit more trust in some areas (helping to delete NAAs more easily, MAYBE comment moderation but that's tricky, close vote weight increases maybe?) could be beneficial. (2/2)
    – Spevacus Mod
    Commented Nov 5 at 20:00
  • Regarding the difficulty of a space to commune: one of the things that has enabled SO to be so successful is that the zero noise approach has enabled an environment that crosses cultural boundaries and at least partially mitigates/manages the eternal September problem (though in a way that leads to a lot of bad experiences). If you want to introduce social aspects, you need to really careful to consider that aspect. The company has run into this in the past on surveys before, asking things that within your American culture are normal, but absolute nono's in German culture for example. Commented Nov 7 at 2:41
7

Firstly: can you tell us in more detail what kind of research you're performing, and what kind of feedback you're hoping for? This should be included in your post, and not just gleanable from the linked blog post.

let us know if you’d like to see other content types on Stack Overflow?

What are "content types" in this context? Something apart from Q&As?
If so, I personally wouldn't mind seeing tutorials being a possibility, preferably as a dedicated feature, that collectively form a kind of handbook or guide to their respective topics.
Depending largely on the type of site, some answers already venture into tutorial territory (or 'tutoritory').

What motivates you to engage with Stack Overflow?

Not much, because I rarely use Stack Overflow (but see my comment). SO specifically when I want to know something I can't find elsewhere, whereas with other sites it can also be to see what questions I would like to or can answer, review tasks, and possibilities for editing. In a word: maintenance. Underpinned mainly by perfectionism (improving for the sake of improving), but the platform's reward system and the idea that the content is more accessible to others certainly play a role as well (to be fair, I'm not sure how fervent an editor I would be if my efforts weren't ascribed to me). And having to double-check spelling, grammar, and facts every so often is a nice way to improve upon my use and understanding of the English language and general knowledge.

Is it the same thing that motivated you when you first started on the site?

Speaking for all sites on the network; no, not really. I joined the platform because of the Helping Others archetype mentioned in the blog post, but it has become a mix of the various archetypes in different proportions for every site I frequently visit (depending on my local subject knowledge, reputation, standing with other users, editing privileges, and so on).
However, these last years my motivation keeps nosediving because of the misalignment between the Community and SE Inc.

9
  • 2
    Yes, content types as in something other than question and answer. I really like your tutorial idea! Do you see that idea as something like curating existing tutorials or creating new ones?
    – Piper Staff
    Commented Nov 4 at 19:55
  • 1
    So you are describing the actions you take as maintenance, but why do you do them? You mentioned helping others was your original motivator, but why do you continue to "maintain" now?
    – Piper Staff
    Commented Nov 4 at 19:57
  • 1
    @Piper Creating new tutorials, mainly. Like with our current Q&A, we can select, interpret, and consolidate information from elsewhere, but it should still be mostly new material. As for your second comment: good point. I'll add it to my answer.
    – Joachim
    Commented Nov 4 at 20:05
  • 1
    Tutorials would be extra useful if they let you link sections to questions and answers
    – GammaGames
    Commented Nov 4 at 21:26
  • 1
    @Joachim Per your comment, are you saying that you do currently create tutorials? If so, would you mind sharing an example of a tutorial you have created? I'm curious how you are thinking about tutorials and how they might be different from the tutorials my mind goes to first (e.g. Udemy, Codecademy etc)
    – emmabee Staff
    Commented Nov 5 at 2:11
  • 1
    @emmabee I don't really, but I feel I've had answers come close to it. I'm not familiar with Udemy or Codecademy (and you might have SO in mind when picturing this, whereas I have Arts & Crafts), but I was envisioning it as somewhere between classical written step-by-step tutorials and the ones offered by Epic for their Unreal Engine, so with interactive elements and an incentive system.
    – Joachim
    Commented Nov 5 at 9:28
  • 1
    @GammaGames Exactly! They could link to a list of associated Q&As, perhaps have a dedicated chat room, and relevant and important questions could be turned into full-fledged Q&As again. Offering in a sense a feedback loop, where both content types are mutually beneficial.
    – Joachim
    Commented Nov 5 at 9:32
  • 2
    I love the brainstorming here! I am also just now learning there is an Arts & Crafts SE which could have been very helpful at times.
    – Piper Staff
    Commented Nov 5 at 14:01
  • @Joachim You're making me think about how different content types could support each other rather than conflict. Using cooking as an illustration, Q&A might represent the finished dishes that users share and enjoy. Tutorials or cooking classes provide detailed recipes, while discussions focus on preferences, like whether one spice is better in curry, weighing the pros and cons. Chat offers a more casual conversation to share tips and stories. Maybe there are other elements that also fit in, but all elements tie back to the goal of creating the dish. Is that how you are thinking about it?
    – emmabee Staff
    Commented Nov 5 at 19:01
4

Motivations, incentives, and gamification

A major motivation for many of us is creating a publicly accessible knowledge base under a CC BY-SA license. But you've killed the CC BY-SA data dump, thereby killing most of our motivation.

Is there a need for varying levels of permanence for our questions and answers

Do you mean you're considering deleting questions even more aggressively than Roomba?

are our users interested in other types of content?

What other types of content do you have in mind beyond Q&A?

1
  • 2
    I could ideate on content types all day! But it doesn't matter—what matters is how do our users learn and what would they want to see?
    – Piper Staff
    Commented Nov 4 at 19:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .