What deliverables would you like to see out of a working group?

Question

Update June 3rd, 2024

Thank you to everyone who took the time to share their thoughts. We will review your suggestions and share an update once we have decided on what action to take with working group deliverables.

---

Over the last few years, a few different projects have required working groups to be formed by willing community users. Some of those were:

• Moderator Election Revamp
• OverflowAI
• Code of Conduct Update
• GenAI Stack Exchange site
• Moderator tools

They have been extremely helpful in getting projects off the ground and collecting the necessary feedback to get projects over the finish line. We look forward to continuing to utilize working groups to assist with various projects over the next year and beyond. We have found that working groups are valuable in bringing community members directly into the development process. The conversations that take place in these groups help highlight concerns and allow us to launch projects or policy changes that are much closer to community wishes than we can deliver on our own. We are here to have a conversation about the kind of deliverables community members would like to see from these groups. More specifically, we want these groups to be more transparent than they have been in the past.

A brief explainer about working groups

Working groups are valuable because they allow us to bring together a small to medium assembly of users to focus and work on a specific issue. We love broad community feedback and will still be seeking that out as we always have. But, it’s much easier and more effective to have a small but diverse group work towards a common and focused goal in some circumstances. This helps us avoid the “Too Many Cooks” problem.

User selection for a working group is pretty straightforward. When looking for participants, we generally have a Google Form that includes questions about working in groups and why a candidate is interested in joining a working group. The relevant community managers and sometimes product managers will review all interested individuals and invite as many individuals as possible who look like they are a good fit. Some working groups are kept small, and some don’t have a size limit. It all depends on the particular needs of the project. Particular opinion is paid to diversity of opinions and other factors.

Our working groups have typically operated using a private but sometimes public chat and a private Stack Overflow for Teams instance. These groups usually have a schedule they try to adhere to and have specific subjects they are asked to work through. The working groups have a clear mandate for what they should emerge with, in terms of deliverables. Sometimes, there isn’t a schedule and they have just an end goal of completing what they are working through. Oftentimes, they collaborate as a group and focus on one of three things:

• Updating or crafting a new policy
• Feedback on a new feature/product
• Brainstorming and planning out what a new product/feature/process could look like

Once the working group has completed its mandate in collaboration with a community or product manager, some type of deliverable has typically been created. However, in the case of product/feature feedback, these groups might work as a feedback mechanism, and their feedback is acted on as a part of continuous development. When the need for feedback is completed, it will be closed without public acknowledgment other than to those specifically involved in the group. In addition, some working groups might be working on things that include sensitive information. This could mean something like moderator tools and outcomes for projects like those, which more than likely wouldn’t be shared with anyone other than the groups they are specific to.

What kind of deliverables would you like to see from a working group?

As I mentioned, we anticipate there will be more working groups in the future, and we would like to make the outcomes of those groups less mysterious, whether that means a finalized report on the end product or the opportunity for participating members to share their experience working on it publicly like some sort of retrospective post. Or anything else you could think of. Ultimately, we want to ensure that these groups have more transparency in their work and that when they are concluded, we consistently share outcomes with the community.

To be concise, what do you want to see after a working group has completed its work? This could mean something like the following:

• Reports about what a working group discussed, decided not to utilize, and ultimately decided on.
• Timelines on how work progressed
• Community retrospectives on what went right or wrong on a particular project.
• Data dumps of discussions in public and private places, with individual community member identities anonymized. Some projects centered on moderation tooling or conversations that would help individuals abuse network elements would have the option of remaining private.

Please leave any ideas you have as answers to this post. We will be accepting feedback on this post until May 29th, 2024.

Answer 1

This is going to sound negative, but I want to be honest:

As a general principle, transparency is good. But I find it hard to care much either way. Personally my desire to see deliverables from a working group is greatly diminished. I want to see improvements to the platform, more than documents from a working group. Documents from a working group don't do me much good. I've lost trust in working groups to lead to improvements to the platform, and so seeing their deliverables is not a high priority to me. I'd prefer you show me the improvements to the platform rather than working group documents.

After the moderator Tools/Policy working group, I've lost trust in working groups. That group put in a bunch of effort and came up with a ton of suggestions for how to improve things that I thought were great. I am hard-pressed to identify any of those improvements that were implemented. The company on its own designed and implemented something different. What the company implemented is certainly interesting and was a very reasonable thing to try, though my impression is that unfortunately its utility has turned out to be rather limited on many sites.

My impression is that company just doesn't have a lot of software developer resources to implement the ideas that people come up with. That's perfectly understandable. It's easy to come up with ideas, but a lot more work to implement them. And, there are many additional considerations that a working group is not well-suited to address, which anyone who implements something has to think about. But, given that, what's the point in having a working group put in so much effort if, realistically, the suggestions probably aren't going to be implemented? And what's the point in reviewing deliverables from a working group if, realistically, they probably aren't going to be implemented?

For the Code of Conduct revisions, I am not familiar with anything that required a working group. Soliciting feedback from all moderators seemed fine. Perhaps there is some aspect of the process that I've been unaware of. But again, I wouldn't care about deliverables from such a working group. Rather, I'd want to see and have the chance to provide input on a proposed revision.

My perspective is that the GenAI policy working group has been a failure. The original vision was that some users could agree on some heuristics that mods could use to delete posts that are, with high confidence, generated by AI. But I think that has not been a success. The process has been too cumbersome and bureaucratic, and the heuristics that have been accepted are too complicated to understand, and as a result, I find it irrelevant to my moderation. I find it more realistic to ignore everything that working group came up with and simply treat the policy as saying "don't delete posts because you think they were AI-generated, sorry". So for that working group, transparency and deliverables are irrelevant to me.

I think there is an important distinction between different types of working groups:

• working groups whose mission is to propose a policy; vs
• working groups whose mission is to propose changes to the software that runs the platform; vs
• working groups whose mission is to give the company feedback on such changes.

My thoughts on deliverables for each of these types of working groups:

• For policy-proposal working groups, we don't need deliverables. We just need to see the proposed policy, and a rationale, just like any other proposal on Meta. And we need a company representative who will engage with the community and accept input.

• For platform-change-proposal working groups, perhaps all we need to see is the proposed change and rationale.

• For software-feedback working groups, I'm not sure the company needs a working group, as opposed to a focus group, and I'm not sure any transparency is needed; the company just needs to have a representative communicate with the entire community about the software change and defend it and listen to input.

More broadly, I see working groups as part of a broader trend towards decision-making that is more top-down and less democratic and does less to engage the community and build buy-in. That's arguably a natural evolution, as a community grows. But still a little sad to see.

It's entirely possible I am being too pessimistic or negative. I apologize in advance if this is not helpful.

Answer 2

I don't think there is a universal set of deliverables that would be appropriate for each working group. Instead, once the need for a working group has been noted, one of the first things the working group should do is establish a charter. This charter should outline how the working group intends to validate their work and progress with the broader community.

I suppose if you needed to define a universal set of deliverables, then that set would probably only be the charter. The community can evaluate what problem the working group is working on and the charter and see if the deliverables make sense.

Perhaps, over time with enough similar working groups, you can start with some recommended deliverables. You may find that groups working on policies or groups working on feature ideation or groups working on processes may identify some common deliverables for their type of problem. You may also find that some deliverables may be common across multiple types of problems.

Instead of trying to do too much up-front planning, defer decisions until as late as possible, which is the creation of the working group.

Answer 3

I'd keep it simple

On the conclusion of a working group

• A clear timeline for implementation of what the working group was working on

• decisions based on the working groups conclusion.

• While I'm going to disagree with Kevin on 'full' disclosure, have a involved Community manager or involved staff member from an appropriate team write a summary of the findings, disseminate it on the group for feedback, and publish it on the appropriate meta for comment and finalisations of plans, with the staff member involved being in charge of dealing with the commentary

• a commitment to follow through on the actual project taking into account meta commentary for polishing and error correction.

• if a datadump is provided of posts, it really should be

It is also worth consider the subtext here

SO inc has had a habit of requesting feedback, getting excellent feedback, then losing focus, or pushing through 'features' no one really wanted for marketing purposes. Community feedback is pointless unless it acts as a countercheck for bad ideas, encouraged good ones and is generally heeded.

Practically, we're a chatty bunch and a datadump isn't exactly designed to be readable. My asking for a summary has 3-4 implicit goals. It gives us a summary and forces everyone involved to distil down the findings of the working group's conclusions.

If we needed specific details a meta post lets people ask and has a staff member to follow through. The rest of the community members involved will chime in anyway, and it seems like a good opportunity to exercise those meta writing skills and build connections with the community. Clarity and open-ness can be "Here's everything, go wild" or working with the community to clarify, assuage concerns and generally share if needed.

I disagree that too many cooks is the problem - practically there's been many projects that have had good clear feedback, and forgotten, cause management lost interest. Its rather hard to run a restaurant if you're famous for classical french cooking, but management suddenly decides one week you need to serve Dosai, Mee Siam the next month, and suddenly decides blindfolded, upside down dining is the next big thing. This is how the company has been for the last decade or so.

As such the most important deliverable is the desire for the company to follow through, and see a product that comes out that meets the community's needs, or to not waste effort on something that's clearly going to get broken and mocked, or worse, abandoning something that's useful and well liked.

Answer 4

Everything.

What posts were opened, their contents, the discussion that occurred surrounding each, the answers/comments, results of in person or other survey research, all in a format anyone can directly access without knowledge of how to process a datadump.

I certainly understand keeping things related to moderation private, but for everything else... if your opinion can't stand up to scrutiny publicly it shouldn't be part of the private discussion where that can't occur.