34

I just saw this rather concerning blog post. Amongst other things, it says that Stack Exchange is

exploring ways in which AI and ML [(Machine Learning)] can assist community users in writing drafts for questions and can reduce the burden of work on our mods and curators by assisting in content review and flagging out of date content.

Using AI to generate posts is banned on virtually all sites in the network, and this blog post doesn't seem to have any plans for respecting those bans. Using a machine/robot to review is the literal definition of robo-reviewing, which is banned literally everywhere (as far as I know) and for good reason. Stack Exchange even has review audits to protect against it. I'm not aware of any ban on using AI to flag posts, but considering it's probably inaccurate enough to get itself a flagging suspension, I doubt it would be received well.

Am I missing something here or is this what it sounds like?

Improve this question
19
  • Does "flagging out of date content" mean they plan to implement version tagging of answers after all, then?
    – TylerH
    Commented Oct 22 at 21:55
  • @TylerH well that would be "tagging", which is not "flagging". With all they're marketing speak I'd imagine they have some idea of what their typing (unlike some of their ideas)
    – Starship
    Commented Oct 22 at 21:57
  • I mean, how else are they going to know content could possibly be "out of date"?
    – TylerH
    Commented Oct 22 at 21:58
  • @TylerH Presumably they'd have some AI look through it determine that.
    – Starship
    Commented Oct 22 at 21:58
  • 2
    "exploring ways in which AI and ML [(Machine Learning)] can assist community users in [...] flagging out of date content." FWIW, our community (shout out to SOBotics team) has done similar things like flagging NAA
    – Meta Andrew T.
    Commented Oct 23 at 6:09
  • 10
    Note that robo-reviewing on SE is not defined as "reviewing by a robot" but as "reviewing without checking the contents and just performing the same action over and over" similar to a non-intelligent assembly line robot. If the AI actually looks at the post and makes a correct decision then that can actually be helpful and is not robo-reviewing at all in the sense this term is being used on SE.
    – Marijn
    Commented Oct 23 at 9:47
  • I think it's obvious by now, they plan to replace moderators with automated AI-powered "tools", used by staff only. So this is just the beginning of a long way.
    – Shadow Wizard
    Commented Oct 23 at 9:48
  • @ShadowWizard Honestly I think they’ll eventually replace all review and flagging etc with AI as well as answering (and maybe they’ll even have the AI ask questions)
    – Starship
    Commented Oct 23 at 9:49
  • 2
    Yes but the point is that an AI/ML approach is actually paying attention to what it is doing, so the argument made in your question (AI reviewing is robo-reviewing and that is bad) is not valid.
    – Marijn
    Commented Oct 23 at 9:51
  • 2
    That is maybe a more philosohpical point but you don't need to know what you are doing in order to do it right. For a classification task like flagging I think that AI is very suitable, see the post mentioned by Andrew T. above (meta.stackoverflow.com/questions/343065/…).
    – Marijn
    Commented Oct 23 at 9:54
  • 1
    "Using AI to generate posts is banned on virtually all sites in the network". No, it's not. Currently, it's banned on only 27 sites. See meta.stackexchange.com/q/396626/334566
    – PM 2Ring
    Commented Oct 23 at 15:37
  • 2
    I mean, technically there's already a bot flagging comments on SO... It's not good at it but I attribute that to poor training rather than assuming that no bot could flag accurately. Heck, Charcoal does it pretty well. And, even then, flagging is not handling. It's drawing attention to, not acting on (usually). With enough flagging handled, though, it's not impossible that a bot could handle simple flags without mods involved in many cases.
    – Catija
    Commented Oct 23 at 16:35
  • 1
    I'm confused... as far as I'm aware, reporting = flagging. What do you think "flagging" means? I didn't say Charcoal was AI... I said it's a bot. When it comes to flagging, I'm not sure how an AI bot is more than a "learning" bot that can adjust behavior based on new feedback rather than having to be expressly programmed to flag specific things.
    – Catija
    Commented Oct 23 at 17:43
  • 2
    There is (or was?) the unfriendly comments detection robot (on SO?), but that was years before the current hype.
    – NoDataDumpNoContribution
    Commented Oct 23 at 17:59
  • 2
    Found it. Back in the days, when data driven development was still cool and SE had a data scientist: stackoverflow.blog/2020/04/09/…
    – NoDataDumpNoContribution
    Commented Oct 23 at 18:05

3 Answers 3

Reset to default
17

We have spoken previously about how we are conducting controlled, limited experiments using LLMs, for instance, as a question assistant in the Staging Ground area. We’ve also previously spoken about research into the other side of this equation, supporting those who answer questions on-site.

As we have previously discussed, we are looking at ways AI/ML can help speed up the process for askers in writing their drafts so they can get answers faster, learn community norms, and take the burden away from human reviewers without removing them from the loop. These are experiments that the Product Team is working on, and we will share more details about how you can get involved and provide thoughts and feedback.

There are two key points that I want to emphasize:

  1. These are experiments. A lot of time and effort has been put into research and will be put into community feedback. If we find these experiments are not providing value and are not improving the asking or answering experience, we are able to roll them back.

  2. Human review is a core component. We all know that what makes coming to Stack Overflow or any site on the Stack Exchange network different (and, in my opinion, drastically better) than just going straight to an LLM is that humans are responding in the form of questions, answers, comments, etc. That is something that we all hold dearly, and experimenting with ways to potentially utilize LLMs to help reduce the burden on askers, answerers, or curators doesn’t replace the human’s role in those actions.

The internet is changing, but that doesn’t mean we’re abandoning what makes Stack Overflow unique - the remarkable community of contributors behind the site. We will, as I mentioned, have more to share about these experiments (and other community initiatives that are unrelated to machine learning) in the upcoming weeks.

Improve this answer
6
  • 3
    Thanks. A few more questions. Will this “experiment” be opt-in or mandatory? What if the community finds it to not be useful but the company does?
    – Starship
    Commented Oct 23 at 18:50
  • 1
    @Starship Which "experiment" do you mean? The blog post, and this answer, refer to experiments plural.
    – wizzwizz4
    Commented Oct 23 at 18:59
  • 8
    "Human review is a core component." This is a very imprecise statement and can mean many different things from humans have to control everything to humans sometimes also contribute part of the input. The answer is probably not as clear as the asker would have expected when asking.
    – NoDataDumpNoContribution
    Commented Oct 23 at 19:11
  • @wizzwizz4 All of them
    – Starship
    Commented Oct 23 at 20:21
  • 1
    @Starship That's a bit tricky. Some experiments (e.g. chat server migration to Google Cloud) have to be mandatory. Many experiments get discarded before anyone outside the company ever sees them (… I assume, anyway). "What if the community finds it not to be useful but the company does?" is such a terribly broad question that I don't see how it could be answered in the general. What about ads? The leaderboard changes? Various tweaks to moderator tools? Be specific!
    – wizzwizz4
    Commented Oct 23 at 20:31
  • 4
    "that doesn’t mean we’re abandoning what makes Stack Overflow unique" - you keep saying that, however the company actions keep pushing the site towards being much less unique, so sadly I have to disagree.
    – Shadow Wizard
    Commented Oct 24 at 17:42
7

Using a machine/robot to review is the literal definition of robo-reviewing, which is banned literally everywhere (as far as I know)

SmokeDetector says hello. I have no special insight, but if this refers to the long-awaited SE–Charcoal official partnership, then I (for one) welcome our new-old robot overlords.

The use of automation on Stack Exchange is banned only when it doesn't work (e.g., answers written by GPT models). That doesn't mean that automation doesn't work in general: it can, and does. Quoth the Charcoal wiki:

While SmokeDetector aims for zero false negatives, autoflagging aims for zero false positives. That is, SmokeDetector aims to catch all spam at the expense of a few caught legitimate posts; autoflagging aims to flag only spam without hitting any legitimate posts. Of course, neither of them hit their goals 100%; that’s to be expected. Autoflagging is, however, extremely accurate.

Users set up conditions that govern how confident they want to be that a post is spam before their account can be used to flag it. All conditions have to be more than 99.75% accurate before the system will accept them; the default recommended conditions are 100% accurate to the nearest 0.01%.

Improve this answer
5
  • @Starship It doesn't participate in review queues, but it flags. What it does is described by the English verb "review". Since it runs using many privileged accounts, it does often take actions that remove posts from review queues: in that regard, it's different to the built-in autoflag systems.
    – wizzwizz4
    Commented Oct 23 at 18:43
  • 3
    @Starship Charcoal wiki: What's autoflagging?
    – Spevacus Mod
    Commented Oct 23 at 19:04
  • @Spevacus as far as I know Smokey can’t cast 4 flags, and bot is going to be much fastër than a human
    – Starship
    Commented Oct 23 at 20:26
  • 6
    @Starship Smokey can cast the maximum number of flags required to nuke a post, but it requires CM approval. See Flagging: Spam waves.
    – Spevacus Mod
    Commented Oct 23 at 20:33
  • @Spevacus interesting never seen that before but I’d imagines that quite rare. Now that I think of it why not make a spam wave for posts at 100% historical accuracy point.
    – Starship
    Commented Oct 23 at 21:02
-7

The ban on some (not all) SE sites aimed to prevent generating text from scratch/vague prompt with AI, whereas the blog talk about writing assistance: "by assisting in content review and flagging out of date content". Some writing assistant tools are allowed, e.g. some Grammarly features are allowed on SO. Then there is the grey area between generating text from scratch/vague prompt with AI and spell check.

Improve this answer
14
  • 5
    Sorry but this is false. The blog post literally says "writing drafts for questions". Grammarly doesn't write your question for you or robo-review or autoflag. I also seriously don't understand the point of SE having AI write questions that no one actually wants/needs the answer too. Also, no the ban aimed to prevent people causing harm to the site with AI, regardless of their means or what harmful thing they did.
    – Starship
    Commented Oct 22 at 22:34
  • 3
    @Starship maybe add the proper quote in the question then?
    – Franck Dernoncourt
    Commented Oct 22 at 22:36
  • @Starship is that the quote "exploring ways in which AI and ML [(Machine Learning)] can assist community users in writing drafts for questions and can reduce the burden of work on our mods and curators by assisting in content review and flagging out of date content."?
    – Franck Dernoncourt
    Commented Oct 22 at 22:37
  • 12
    @Starship then my answer is correct. "assist community users in writing drafts" doesn't imply writing drafts. Eg, a spell checker can assist community users in writing drafts.
    – Franck Dernoncourt
    Commented Oct 22 at 22:41
  • No, a spell checker would assist users in spell-checking posts, not writing drafts of questions. No AI should be writing the initial drafts of questions.
    – Starship
    Commented Oct 22 at 23:04
  • 10
    @Starship again, "assist community users in writing drafts" doesn't imply writing drafts.
    – Franck Dernoncourt
    Commented Oct 22 at 23:21
  • Let us continue this discussion in chat.
    – Starship
    Commented Oct 22 at 23:21
  • 4
    Even AI-based writing assistant tools would be frustrating; they already tried this to extreme backlash. And if they're planning on a weaker version, well, that already exists.
    – Anerdw
    Commented Oct 23 at 0:16
  • 2
    Grammarly also started using AI and such tools are also not allowed on Stack Overflow. The use of generative AI (e.g. ChatGPT, Bard, LLaMA, etc.) as a source for content on Stack Overflow is currently banned. Please see: Policy: Generative AI (e.g., ChatGPT) is banned and AI Policy It is not permitted for you to use generative AI to create content on Stack Overflow during this ban. This also includes rewording, translating or explaining text or code written by you.
    – Resistance Is Futile
    Commented Oct 23 at 6:33
  • 3
    It says AI and machine learning. So both are included as AI nowadays implies gen AI. Also regardless about what question asks, my comment was addressing your statement that Grammarly is allowed on SO. While it was previously allowed, they are also pushing gen AI in their tooling and using those falls under the AI ban. This doesn't mean that older grammar tools are not allowed, but people must be very careful to avoid accidentally using gen AI tools for grammar or translation.
    – Resistance Is Futile
    Commented Oct 23 at 7:11
  • 1
    @ResistanceIsFutile regardless, my argument stands: some writing assistant tools are allowed, e.g. Grammarly minus their new genAI features is allowed on SO.
    – Franck Dernoncourt
    Commented Oct 23 at 7:14
  • 8
    @FranckDernoncourt AI can't just fix grammar. It will change the text alongside with it, in ways that would be hard to detect, but might have fatal outcome, i.e. change the meaning. And this is core part of AI and how it works, it can't and won't be "fixed".
    – Shadow Wizard
    Commented Oct 23 at 9:45
  • I take it that this answer advocates for more patience, that there might be beneficial uses as well and that one needs to wait for details about planned features before judging them.
    – NoDataDumpNoContribution
    Commented Oct 23 at 18:08
  • 7
    Flaggers: Remember that VLQ/NAA flags are for posts that do not attempt to answer the question at all, not for wrong/inaccurate answers.
    – CPlus
    Commented Oct 25 at 14:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .