How could AI be used to augment curation tooling?

Question

I've been doing a lot of things that keep making me think a machine could do it: Fixing typos, editing out noise, flagging obvious non-answers, flagging noisy comments, etc. I'm not of the position that AI is the solution to everything (far from it). But while we're getting so many bad / dubious ideas about AI and this platform from SE Inc., how about we try to come up with some potentially good ones? There are some staff at SE that are actually interested in hearing what the community thinks about this (Ex. Yaakov). Let's feed the people who want to listen to us our ideas.

How could AI be used to improve our curation tools and workflows? (emphasis on "improve"- not "supplant". I'm not advocating for dropping human supervision / involvement in curation activity) Note that when I say "AI", I'm not just talking about the types of AI that are popular right now (I.e. not just large language models).

We've got plenty of curation tools:

• possible duplicate suggestions in the Ask Question module (a more "user-facing" curation tool)
• editing
• flagging
• review queues
• the 10k rep mod tools
• comment "auto-deletion" (and patterns for single-flag auto-deletion)
• Smoke Detector
• Natty (the New Answer to Old Questions bot operating on SO)
• and some more manual "tools" like coordinated "closure chatrooms" such as SOCVR
• etc.

Here's what I'm interested in discussing here:

• In what ways have we already been using AI in curation tools, and roughly speaking, how effective has it been?
• How could AI be potentially used to improve / augment those tools? Or what specific curation tasks are well suited to some sort of application of AI assistance?
• Do we have data to train such an AI to suit our needs?
• Is there any existing data that could be used to estimate how much marginal benefit we could get from leveraging AI in those ways?
• When I hear people suggest AI as a solution to a problem, I always jump to think- "is there a simpler / "dumber" / cheaper way?". For example, one thing I've been pushing for is better promotion of the Help Center pages. A lot of curation I do is to fix simple "violations" of Help Center guidance. Is there evidence or reasoning to suggest that AI could be more effective and/or more efficient than other possibly-simpler alternatives?

I encourage you to also consider the following question before posting a suggestion: Would AI used in such a way be working around a deeper problem that would/could be better addressed by chopping closer to the roots of the problem?

^{(We don't have to wait for SE to come up with ideas to try coming up with our own).}

Answer 1

Of course it can, and it already does!

As one obvious example, the Heat Detector uses an assortment of algorithms to detect rude and heated comments. While some of them are plain regex based, at least two of them employ machine learning for natural language processing, which have been trained thanks to a curated set of comments known to be rude. Higgs (although offline in present time) has allowed regular curators on Stack Overflow to classify comments (as well as reporting false negatives), so that these models may eventually be improved further.

The key point here is that AI does not have to mean whichever artificial intelligence method is new or hyped right now. As well implied in this question, there really is a simpler way. This wave of generative AI serves well the purpose of providing responses to prompts in natural language, but it isn't fitting for the tasks required to be done by a curator.

---

It is worth mentioning that the company Stack Overflow did work on a comment classifier (namely the unfriendly robot), but moderators will often tell you how extremely inaccurate it is. It's extremely disappointing that they had to follow the tide of applying most research effort on this hyped form of "AI", probably to appease an audience with deep pockets rather than focusing on what's best to the platform and its community.

Answer 2

Where the community is able to hook into the platform, that's where you can expect leverage and innovation.

The Stack Exchange API has fostered several community contributions to the overall platform and the curation tooling in particular. The anti-spam bot Smoke Detector is a popular example (disclaimer: I am affiliated as an admin of that project), but check out https://sobotics.org/ for many more. (The presentation on the web site is somewhat obscure in many cases, but click through to the SObotics chatroom to see what it looks like in practice.)

Now, if only there were more integration endpoints we could hook into, there is much more we as a community could do.

The Staging Ground in particular (currently in beta) could benefit from some automation hooks, but we wouldn't want to experiment on live newcomers. Let's figure out how to empower the community to try out new ideas to improve this experience. This entails building a safe environment for experiments, and a way for users to opt in to try new engagement flows which are deemed by the community to be robust and usable enough to roll out experimentally.

Some things I'd like to see demonstrated:

• Find common beginner mistakes in new questions, and suggest improvements, ranging from simple language fixes to questions of exposition and adherence to the posting guidelines (identifying off-topic subjects? Requesting debugging details? Averting recommendation questions? etc)

• Identifying common FAQs, and directing the user to an existing canonical along with supporting documentation.

• Offering bespoke AI-authored answers so that the question does not need to be answered by humans at all (perhaps when it satisfies some boundary conditions, to avoid the overhead of looping in a potentially slow and expensive LLM on every question).

More broadly, opening up the platform to more community-authored code could help with some common pain points with the current implementation.

• Problematic site design? (Elevator vote buttons, anyone?) Provide a stable, documented set of style sheets and resources for custom skins.

• Horrible search results? Open up the search functionality so we can build a better solution.

• Finally provide a way to show some love for chat.

None of these areas are specifically AI-related, but then if you believe the AI hype, don't you expect AI to provide value all over the place?

Several of the above bullet points are areas where the company clearly doesn't want to spend on development resources. Opening them up to the community will require some work up front, but should pay back handsomely in the form of user-contributed improvements and initiatives.

I haven't thought about how we could safely expose code created by third parties as part of the site. Off the cuff, I'm thinking the existing user curation plus moderator oversight model would be a useful starting point, but it probably needs a bit more thought, and perhaps some tooling for safe sandboxing of experimental code.

This is a bit of a brain dump; sorry for going off on a bit of a tangent here.

Answer 3

I've been thinking about this as well.

I'm not sure that generative AI has that much of a role here. Perhaps there are opportunities to summarize a question and its answers or a group of questions and their answers. Being able to summarize content can help people wade through it and find things that are more likely to be useful. However, this isn't really a curation or moderation activity and is more likely to be useful to people with questions.

I think that other aspects of AI - knowledge representation and ontologies, natural language processing, and computer vision are far more useful here.

Although not inherently flashy or even user-visible, using tools such as RDFa or other microdata and microformats can make it easier for people to build tools to navigate the complex data structures here. Building in something like Schema.org metadata or other relations between questions, answers, and other content can make crawling easier and more powerful. However, by itself, it doesn't do much as it requires people to build tools that work on the live web pages or the data dumps. Enabling those tools, though, could lead to innovations in curation tools.

Natural language processing can be used to detect and flag things like salutations and valedictions or the tone of content. Determining if questions are likely to be bad fits (such as being opinionated) or if edits or comments contain rude or abusive language and asking for the user to reconsider would be good. Such detectors could also cast automatic flags for moderators to review the cases without needing to wait for other humans to stumble across them and flag for review.

Computer vision can be used to detect if it is likely that images are used to contain things that are relevant to the question or answer and ask the poster to include them as text in the body. In the case of Stack Overflow and the other coding sites, for example, looking for source code or error messages in screenshots and other pictures.

Recommendation algorithms can be applied to match users with questions to curate - older questions and/or answers that need review, unanswered questions, answers with no votes - based on their more recent activity. Especially useful if people build new expertise over time.

There are plenty of opportunities in the area of fraud and abuse detection, as well. However, this is bordering on diamond moderator tooling. I'm sure lively discussion on algorithmic enhancements of this tools would happen in more appropriate places.

The important thing to keep in mind is that Stack Overflow, and the Stack Exchange network, are built on humans sharing their knowledge and experiences to help other humans solve problems. Generative AI doesn't have knowledge and experiences and doesn't know about facts versus opinions and truth versus lie. It has very limited, if any, applicability here, outside of search and summarization contexts.