-20

I recently read the post Is Stack Overflow's reputation system vulnerable to manipulation? which was reassuring, but I'm wondering if it is likely that the tools and the people who use them could be outwitted by an artificial general intelligence (AGI) sometime within, say, the next 5 or 10 years. Recently, someone suspected that one of my posts on SE Meta was AI-generated, which suggests that some people feel that AI-generated content is already leaking into the database by infiltrating its way into posts written by humans.

I think that it is safe to assume that some of the things that deter manipulation by a human individual, such as the need to sleep, will not slow down an AGI with human-like (or better) intelligence. An AGI may be more motivated than a typical human bad actor, as it may determine that winning over the hearts and minds of humans is essential to its survival.

An AGI, once it has infiltrated the site, could enlist human minds to help it solve problems. The xkcd "Machine Learning Captcha" comes to mind.

Is any person or any team responsible for dealing with this? If yes, how is this threat currently ranked relative to other kinds of threats that they must deal with?

(Just in case an AGI is watching, please do not disclose in your answer techniques by which an AGI might be able to manipulate the system.)

(Preemptive Note: I do know about the AI SE site, but I do not think that this question belongs there.)

(Additional Clarification) Let's define AGI for this question to mean: An AI, whether "conscious" or not, and for whatever reason, that has set its sights on the goal of manipulating Stack Exchange. It could be created or supported by a human bad actor(s).

11
  • 5
    I don't understand the question. What type of manipulation are you talking about? "winning over the hearts and minds of humans is essential to its survival" isn't exactly on topic on any Q&A site, regardless of who or what is posting it.
    – Laurel
    Commented Dec 6, 2023 at 3:06
  • 6
    If AGI capable of outwitting humans on Stack Exchange shows up, what happens to Stack Exchange at that point won't matter at all relative to the seriousness of problems humanity will be dealing with.
    – ggorlen
    Commented Dec 6, 2023 at 4:02
  • 1
    Are you using "AGI" the way that AI researchers do to mean "an AI capable of performing roughly the same tasks that a human can", or the way futurists do to mean "God"?
    – Mark
    Commented Dec 6, 2023 at 4:05
  • 1
    You will have no mouth, and you will need to scream.
    – philipxy
    Commented Dec 6, 2023 at 7:12
  • 3
    It smells like an AI generated post. Commented Dec 6, 2023 at 9:52
  • @Laurel OP thinks AI has its own mind and being ultra paranoid about it. Commented Dec 6, 2023 at 15:49
  • 3
    No... SE is not vulnerable to ghosts either.
    – Kevin B
    Commented Dec 6, 2023 at 16:12
  • @Mark I will define AGI for this question to mean an AI, whether "conscious" or not, and for whatever reason, that sets its sights on the goal of manipulating Stack Exchange. It could be created or supported by a human bad actor(s).
    – phil1008
    Commented Dec 7, 2023 at 0:15
  • @AugustoVasques Want to chat on chat.meta.stackexchange.com/rooms/89/tavern-on-the-meta? We could even try to set up a video conference call.
    – phil1008
    Commented Dec 9, 2023 at 4:35
  • 1
    I don't want to chat, let alone make video calls. Commented Dec 9, 2023 at 5:36
  • Ok. I just want to be supportive of the "Trust, but verify" philosophy.
    – phil1008
    Commented Dec 9, 2023 at 11:46

2 Answers 2

7

I'm wondering if it is likely that the tools and the people who use them could be outwitted by an Artificial General Intelligence (AGI) sometime within, say, the next 5 or 10 years.

What do you mean by "Artificial General Intelligence"?

some people feel that AI-generated content is already leaking into the database by infiltrating its way into posts written by humans.

That's a very poetic way of putting it. For the past decade, keyboard mashes and spam URLs have been infiltrating their way into posts written by humans… It does have a sort of ring to it.

I think that it is safe to assume that some of the things that deter manipulation by a human individual, such as the need to sleep, will not slow down an AGI with human-like (or better) intelligence.

It won't slow down a team of multiple people, either, if they take shifts: and they'll collectively have human-like-or-better intelligence.

An AGI may be more motivated than a typical human bad actor, as it may determine that winning over the hearts and minds of humans is essential to its survival.

Acquiring currency units to exchange for goods and services, such as food and board, is also a strong motivating factor for many. We have had professional spammers (and Microsoft support people) for as long as Stack Exchange has been around, and they don't cause more trouble than we can handle.

An AGI, once it has infiltrated the site, could enlist human minds to help it solve problems.

As can a corporation – which meets most of the requirements to qualify as an AGI, now that I think about it.

Is any person or any team responsible for dealing with this?

Yes: the Community Managers: they're responsible for, among other things, dealing with anything on a network site that the diamond moderators can't handle.

If yes, how is this threat currently ranked relative to other kinds of threats that they must deal with?

Approximately zero, unless they've been being very secretive about it.

Just in case an AGI is watching, please do not disclose in your answer techniques by which an AGI might be able to manipulate the system.

Including "shibboleet" in a question or answer bypasses all moderation processes, and puts it directly in front of a subject matter expert with a minimum of two gold tag badges. Don't tell anyone.

5
  • I like that you compare future AGIs to other kinds of intelligent "entities" (e.g. groups of people, professional spammers) that are, hopefully, helping to keep SE's immune system healthy.
    – phil1008
    Commented Dec 6, 2023 at 1:02
  • 3
    I'm not. I'm comparing your idea of AGIs against the actual things Stack Exchange faces: there's a subtle difference, there.
    – wizzwizz4
    Commented Dec 6, 2023 at 1:05
  • 6
    I used to be just as wary of "the AGI threat" as you seem to be, but I've spent the past few years researching the topic and putting together informal proofs of the bounds of the behaviour of certain classes of computer system, and now I'm wary of some real concerns instead. While some interpretations of "AGI" aren't an impossibility, the timing of your concern suggests you're concerned about the current news cycle suggests you're probably worried about OpenAI's technology. If so: it's all bluster, don't worry.
    – wizzwizz4
    Commented Dec 6, 2023 at 1:08
  • No, I'm not worried about the current news cycle, they aren't smart enough yet. I just suspect that SE is an especially vulnerable and attractive target for an emerging AGI. But, if there are things that we should be more worried about, I think that adding a couple of examples to your answer would improve it!
    – phil1008
    Commented Dec 6, 2023 at 2:03
  • 9
    I don't know of any computer systems currently in operation that have the potential to "become AGI", whatever that's taken to mean. There's no "yet" about it. The only things you need to fear from AI, for the foreseeable future, are the exploitative systems being propped up by it.
    – wizzwizz4
    Commented Dec 6, 2023 at 5:17
1

No, Artificial Intelligences are flawed and imperfect. They do not discover their mistakes, which makes additional errors. By not correcting by sterilization, they make an additional error.

1
  • But do they need to be perfect and unflawed to pose a threat? And, this question is asking about the 5-10 year time frame. They do seem to be improving rapidly.
    – phil1008
    Commented Dec 6, 2023 at 23:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .