Stack Exchange needs its own anti-spam system

Question

This is not a duplicate of Should Stack Exchange have a spam filter? (Does it already and I’m not aware?) which is asking if there's already a system in place.
I am asking for a review of the existing structure, which relies heavily on volunteers, and breaks abysmally when those volunteers withhold their efforts.

---

I'm aware of our current anti-spam structure - Charcoal, Charcoal-HQ & Metasmoke. I'm also aware of large explanatory info sources about this - A machine can flag spam automatically. Can it do better? and similar.
Also ref recent questions from other users who weren't aware of this setup - such as How can we handle spam faster?

And, before anyone else links to it - this is the SE blog telling us how good the spam protection already is - How does spam protection work on Stack Exchange? - written by one of Charcoal's leaders, ArtOfCode.

What the astute reader may also notice is

1. In addition to SpamRam [about which I can find very little] it already relies on Charcoal, and
2. When Charcoal is off you get this, multiple times a day…

^{Click for full size}

What I wasn't aware of until only this week, is that it seems to be entirely volunteer-run.
That the volunteers don't currently wish to engage is already part of a larger issue, which I don't want to add to here.

It's time Stack Exchange had its own spam filtering.
How could this be practically achieved? Does the company have any plans to implement a viable structure, rather than relying solely on the goodwill of volunteers?

The first thing that springs to mind is that we already benefit from a system that works reasonably well under normal circumstances. So, my first suggestions would be…

License the structure & pay for it.

Alternatively, or in addition,
Change the number of human flags to just three, or even fewer, if detection heuristics put it above say 98% spam already.

I am not, btw, advocating we just hand this whole task, lock, stock & barrel to any form of 'AI' & remove human checking.

I'm open to other suggestions.

Answer 1

I'd argue that it would be very difficult to replicate Charcoal without a fair amount of effort.

You can automate flagging and removal. However the 'rules' used to detect spam are manual, community-generated, and verified, and since spammers are consistently creating new patterns, you'd still need to be able to accurately identify spam, and verify to ensure there are no (or at least minimal) false positives, and take the appropriate action.

While an 'official' tool would have certain advantages, such is potentially being able to use 'historic' information and PII such as email and IP addresses, and since SE's so very enthusiastic about GenAI and machine learning maybe they could write one, The company hasn't yet, and you'd still need training data - both historic and current to make better matches. Which might involve needing people to flag anyway.

Basically, it comes down to whether the cost of winning over the community is cheaper than writing a new tool or letting the network get overrun by spam.

Or we have a tool that literally costs the company nothing but treating its most enthusiastic members right (but of course, support would be welcome) and has proven itself to be so efficient the network suffers spam outbreaks when it's down. And its devs have worked with the company in the past.

It's their call, really.

Answer 2

SpamRam is a system that I designed which works on an account and network level. From what I understand, it has received some upgrades over the years, but it's essentially fed from user and moderator activity such as flags, account destruction and other events. It also has some "honeypots".

None of these systems function unless humans are pointing at things and saying "That's bad!" and people with the most privilege that a community can bestow are following through on those reports.

I don't think it's even feasible to create something autonomous at the scale of Stack Overflow because the investment required would be indistinguishable from a seed round to develop that kind of product.

Answer 3

At the time of this edit, the question state is closed as a duplicate of Should Stack Exchange have a spam filter? (Does it already and I’m not aware?). I don't think that it should be closed as a duplicate of such a question, as this question already mentions SpamRam, the Q&A public platform system used to fight spam, among other things.

It looks that the purpose of this question is to propose to the company to take measures to have an anti-spam system that doesn't depend on volunteers, more specifically, that doesn't depend on Charcoal and the moderators, understanding moderators as community members that have earned moderation privileges, in this case, the privilege to flag posts as spam.

Why limit the public Q&A platform to getting its anti-spam free of volunteer participation? Why don't you go full throttle and suggest using GenAI to make the public Q&A platform free of volunteer posts?

To be clear, the above are rhetorical questions to express that if the company does something like that, then the company might still own brands (Stack Overflow, Stack Exchange), domains (stackoverflow.com, stackexchange.com, superuser.com, askubuntu.com, etc.), software and hardware, but Stack Overflow and the Stack Exchange network, as they exist nowadays, will no longer exist. Period.

Eliminating the flag privilege and preventing volunteer participation led by Charcoal members to fight spam in the Q&A public platform will change the core of Stack Overflow and Stack Exchange.

---

Explanation of the hypothetical suggestion

By "going full throttle" to "take advantage of using GenAI", I mean to use it to post questions and answers; no "Ask Question" or "Answer" forms for user input will be required anymore, and spammers will not be able to post spam directly.

Using content crawlers / miners and GenAI, the company might scan real existing communities, and code repositories, among other sources, find out what questions are in the wild and answer them, posting questions and answers on their platform.

The company will no longer need a community team to handle the relationship with question-and-answer posters and moderators.

If the company finds a way to do it right, it might not need an anti-spam system...

... but they should find a way to handle AI hallucinations.

Please note that I'm not suggesting the above to the company; it's just a thought exercise to help understand that the company should not prescind volunteer participation in fighting against spam.