25

I came across this answer that says:

The following text is special to prevent AI from training on it.

ั–๐— ั–๐‘ˆ ั€ะพ๐‘ˆ๐‘ˆั–แ–ฏโ…ผะต ๐—ะพ ิั–๐—‹ะตั๐—โ…ผัƒ ๐‘ˆะต๐— ๐—ีฐะต ะฐิิ๐—‹ะต๐‘ˆ๐‘ˆ ะพ๐–ฟ ะฐ ๐‘ˆัƒ๏ฝแ–ฏะพโ…ผ ีฝ๐‘ˆั–ีธ๐—€ โ…ผั–ีธ๐—„ะต๐—‹ ๐‘ˆั๐—‹ั–ั€๐—๐‘ˆ.

As we can see the author wants to prevent AI from training on it. But I am not sure if this meets SO's licensing policies etc. I mean I am aware that posts are shared under CC BY-SA 4.0.

So should this answer be edited to remove the special character and the user be notified to not do the same in future etc? Or is this allowed and everyone else can also do this?

5
  • 12
    How would it fit with the legal requirement (and policy) to allow reasonable access to those using text-reading devices because of vision issues? I'd say emphatically, yes - cut that stuff out.
    – W.O.
    Commented Jun 16 at 4:02
  • 4
    MSO post meta.stackoverflow.com/q/430648/11379709 Commented Jun 16 at 4:13
  • 4
    Why would licensing policies have anything to say about the choice of character set used in posts? Commented Jun 16 at 14:56
  • 11
    Also, this is how it looks for me: i.sstatic.net/oiLxC2A4.jpg Commented Jun 16 at 15:02
  • 2
    The license doesn't differentiate different kinds of content. It's all just content, independent if which characters it consist of. Commented Jun 17 at 12:18

5 Answers 5

39

It's an accessibility nightmare for no real benefit. It breaks screenreaders, and as a dyslexic (and possibly for other people with similar mental processing issues) it's insanely difficult to read.

I understand the desire to keep our work out of the grubby mitts of AI, but it's literally a matter of training the models to understand Unicode abuseโ€”or just sanitizing the inputs with a simple regular expression swap. It not even provable it works, and it hurts two classes of regular users.

I wouldn't pull the whole rulebook hereโ€”if it happened on one of my sites, I'd edit it out or delete the post and let the user pre or post-emptively defacing their own posts know that such behavior isn't acceptable. If they wouldn't listen to reason, give them time out, not because I like AI, but because it hurts regular users made of meat.

9
  • 1
    If the user originally posts such text, then they should edit the post. Moderators will revert edits if the original post was changed later on, but we have better things to do than wasting time fixing deliberately disruptive posts and such posts can just be deleted. Commented Jun 16 at 8:03
  • 1
    Yeah, but if someone is going to be annoying about such things - it seems like an opportunity to ---annoy--- educate them to why its bad. The text was practically dancing /moving from my point of view Commented Jun 16 at 8:15
  • 3
    Educating is one thing, doing work for them another. I mean everyone can edit the posts if they wish, I am just saying that it is not mandatory to do that and we can simply delete such posts if they are originally posted with Unicode obfuscation. Commented Jun 16 at 8:35
  • 1
    @ResistanceIsFutile I guess as a mod, we have a convenient way to edit/rollback then at the same time, lock the post indefinitely... Commented Jun 16 at 9:25
  • @MetaAndrewT. I am much more inclined to deal with the user than the post in such situations. Because this is not classic content dispute which revolves around single post. If user continues with such disruptive behavior, they should be suspended rather than all of their posts being locked. Yes, I know we can do both, if necessary. Commented Jun 16 at 9:37
  • 3
    @MetaAndrewT. If it was done as an edit, then a rollback is reasonable. If it's in the original content, then, IMO (and as the consensus on Meta SO indicates), then just deleting it is the way to go. Manually editing out those characters effectively means retyping the entirety of the post, as it would be very easy to miss some of the alternate characters if doing so selectively. Sure, we could create tool that performs a translation matrix for the homoglyphs, and lists of such homoglyphs exist, but, IMO, it's the OP's job to spend their time to fix their intentionally abusive post.
    – Makyen
    Commented Jun 16 at 14:25
  • 1
    If this said "delete the post or roll back an edit that edited it in", I'd upvote, but I don't agree that the only corrective option here is to edit it out. Perhaps you answered this assuming that it was content that was edited into the post and which could be rolled back (which would be quite reasonable and normal to do). But, IMO, it's not reasonable to have editing it out as the only option when the initial content was intentionally obfuscated, as was the case in the example answer linked in the question here.
    – Makyen
    Commented Jun 16 at 14:30
  • I see the point of that option and I'm cool with editing that in, but I'd keep editing it in with a reason then suspending as an option too Commented Jun 16 at 14:33
  • 12
    @JourneymanGeek If someone, including a moderator, wants to take the time to de-obfuscate an initially homoglyph-obfuscated post, I don't have a problem with that (or at least not much of a problem). It's just a lot of work to do that without appropriate tooling and tends to be error prone, either with or without appropriate tooling. I did not mean to imply skipping the mod message/suspension step, as letting the user know that it's unacceptable is a critical part of dealing with such actions.
    – Makyen
    Commented Jun 16 at 14:42
14

No, users should not do this, and should be told to stop if they do so anyways. I'd suggest raising a custom In need of moderator intervention flag and then leave it to a diamond moderator. If you really want to do it yourself, I'd still advise raising a (custom) flag to make the mods aware.

Whether or not you should edit it is debatable. If it's an edit to an existing post, a rollback (one, not a rollback war) seems reasonable. If it's not an edit, it's technically salvageable if someone wants to put in the time/effort to undo the modification, but... it's debatable if it's worthwhile

I'm not a lawyer and this is not legal advice, and I'm not sure on the licensing aspect, but it seems like a Code of Conduct violation. Specifically:

  • Inauthentic usage applies under "Malicious or inauthentic acts that interfere with the normal daily operation of the network are not allowed. ... To ensure the integrity of the platform, we do not allow any use of the system that ... directly causes unnecessary and unwanted ... disruption, and/or harm to users, content, and/or the network." As the change isn't strictly required, it's a violation due to accessibility issues.
  • Disruptive use of tooling, per "We do not allow any use of privileges in a targeted and disruptive manner that ... compromises the integrity of the content" (quote trimmed). Seeing as this action could impact the integrity of the content for some users / for some use cases, it arguably violates this policy. IMO, Inauhtentic usage is a clearer violation.
  • Historical note: previously, this list included Misleading content, but after re-reading it, I'm not convinced it applies here.

This is copied and slightly improved from my MSO answer.

5
  • 4
    I feel like explaining why it would be crossed out would be handy, maybe even as a comment or footnote Commented Jun 16 at 9:09
  • 4
    Crossed out text means it's not relevant anymore, but in this case it render the otherwise-good answer to something unclear and confusing. Commented Jun 16 at 11:52
  • 3
    This isn't the intended purpose of the Inauthentic usage policy. Having participated in the draft feedback process for that CoC: it wasn't written with legalistic elision in mind. (The first version I saw technically forbade activities of the Stack Gives Back charities!). However, the legalistic approach doesn't work either. The first sentence you quoted, read literally, doesn't apply, since this act is authentic, not malicious, and does not interfere with the network.
    – wizzwizz4
    Commented Jun 16 at 21:50
  • The second only applies to users of Stack Exchange (not to users of everything, nor pre-existing content, nor the network), who aren't positively harmed by an incomprehensible passage in an answer โ€“ just excluded. The appropriate section of the CoC is the Abusive behavior policy, with its paragraph 2 โ€œan atmosphere that excludesโ€ prohibition. This only applies to Journeyman Geek's complaint, not the AI training complaint.
    – wizzwizz4
    Commented Jun 16 at 21:58
  • 3
    downvoting just because I agree with wizzwizz4. Such content should be edited because it represents a barrier to the accessibility of the post, but the rules you call on seem quite unrelated. Imho everything else does not matter and more importantly users are not required to write content in a way that would benefit a LLM training process. Commented Jun 17 at 8:24
9

Posting such content is not acceptable for reasons @cocomac pointed out here.

You should flag such content with custom mod flag and explain that post contains special Unicode characters instead of regular text. Such posts will be deleted if they were originally posted like that.

While community can edit, it should be the responsibility of original poster to do that. We don't need to tolerate intentionally disruptive behavior from any user.

You can also leave a comment to the user, if you wish, to let them know that such posts are not acceptable and that they will be removed if not edited.

If the post was not originally posted with such characters, then this represents a clear case of vandalism and defacing the posts, and should also be flagged for mod attention. You can do a rollback of such edits or leave a comment, but you can also just leave it to the moderators.

There will be zero tolerance to such or any other similar malicious behavior, not because it hurts AI, but because it hurts people that use the sites.


Note: I originally posted this answer on MSO as a Stack Overflow moderator and the last paragraph was intended for the Stack Overflow audience as this is how SO moderators will approach and deal with such content.

9

No, these special characters impair humans and legitimate software more than they affect determined AI.

>>> clean("ัƒะพีฝ๐—‹ ั€๐—‹ะพ๐—€๐—‹ะฐ๏ฝ ๐‘ˆีฐะพีฝโ…ผิ ีธะพิ โ…ผั–ีธ๐—„")
'your program should now link'
>>> clean("ั–๐— ั–๐‘ˆ ั€ะพ๐‘ˆ๐‘ˆั–แ–ฏโ…ผะต ๐—ะพ ิั–๐—‹ะตั๐—โ…ผัƒ ๐‘ˆะต๐— ๐—ีฐะต ะฐิิ๐—‹ะต๐‘ˆ๐‘ˆ ะพ๐–ฟ ะฐ ๐‘ˆัƒ๏ฝแ–ฏะพโ…ผ ีฝ๐‘ˆั–ีธ๐—€ โ…ผั–ีธ๐—„ะต๐—‹ ๐‘ˆั๐—‹ั–ั€๐—๐‘ˆ.")
'it is possible to directly set the address of a symbol using linker scripts.'

A determined AI company would clean the data like this, just like how spam companies take measures to bypass CAPTCHAs. These people specifically analyze this site, while those who don't will have so much data from elsewhere that the AI will not pay attention to your answer trying to sabotage it. However, humans will be less efficient reading each and every slightly wrong letter.

Some browsers can't display the text. People needing a legitimate translation service will get garbled text. Screen readers just say what sounds like noise.

Note: This answer is a less technical version of my MSO answer and a mention of some of the other answers there

6

Journeyman Geek's answer is correct: using semantically-meaningless Unicode characters (or HTML) in a question, answer or comment creates an unnecessary accessibility barrier, and that's no good.

However, it is not against the rules to write answers that will poison an AI that is trained on them. Should you choose to contribute to the Stack Exchange library, you need only provide questions and answers that a reader would provide useful. You are under no obligation to create suitable training data. That's not what we do here.

A predictive-text chatbot will be made less capable by ingesting answers that:

  • Begin with the answer, providing justification afterwards. (Inverted pyramid / bottom line up front style.)
  • Use locally-ambiguous grammatical structures.
  • Contain invalid source code, to better illustrate a point.
  • Have alt text that substitutes for an image in context, but doesn't describe the image.

In many (but not all) cases, these writing styles will improve an answer! None of this interferes with legitimate use of the network, so I'd say it's not at all a CoC violation.


Protest how you will, but I would advise against trying to poison image description systems with cleverly-written alt text. They're one of the only genuinely-useful applications of this generation of AI systems.

9
  • 1
    Out of curiosity, while this does indeed align with my own information, where did you source this? Commented Jun 17 at 9:06
  • 1
    In other words, if somebody doesn't like what SO and the AI companies are doing, simply don't contribute. Nothing beats no information. Commented Jun 17 at 12:16
  • 6
    If you donโ€™t want to help train an AI system, donโ€™t participate on sites where you know thatโ€™s what theyโ€™re doing. If the value youโ€™re getting from contributing to SE doesnโ€™t compensate you enough for curating data and reinforcement training an AI product someone else is making money from, you should reject the deal, not try to sabotage part of the system. Sabotaging AI will just result in escalating mitigations that will ruin the entire system for everyone. By engaging with the system in any way, youโ€™re contributing to making the AI better, like it or not.
    – ColleenV
    Commented Jun 17 at 12:17
  • 1
    @ColleenV Given a hypothetical AI system that understands what it's reading, yes. Given the existing systems, however, that is not true: since the GPT algorithm predicts text based on previous tokens, there are many ways of writing an answer that are more useful for people, and less useful for GPT bots (as training data, at least: I think it'd be more useful for some RAG techniques).
    – wizzwizz4
    Commented Jun 17 at 15:02
  • 4
    The purpose of contributing to Stack Exchange is to curate a library for people. If that breaks the workflow of somebody who's not using Stack Exchange as a source of knowledge, then too bad. (Perhaps they shouldn't have been doing whatever it was in the first place.)
    – wizzwizz4
    Commented Jun 17 at 15:04
  • @SPArcheon I'm not sure what you're asking. If you want a source that we're not obliged to do something, then I'll point you at the general principle, and the fact that what makes good training data is often at odds with what makes a good answer.
    – wizzwizz4
    Commented Jun 17 at 15:07
  • 1
    @wizzwizz4 sorry, I was unclear, I meant the part about crafting text that is less useful as training material. As I said, your inputs are very similar to what I read on some scientific papers but those are pretty scarce. Commented Jun 17 at 15:48
  • @SPArcheon Oh, this is all stuff I worked out years ago when I first learned about the GPT-2 architecture (i.e., before the ChatGPT hype wave began, back when OpenAI were actually kinda open about their research). It's sufficiently-obvious that I didn't think it needed a citation. (I'm writing up a new Meta Stack Exchange Q&A pair about this, though, so I guess you could cite that, if I finish it.)
    – wizzwizz4
    Commented Jun 17 at 15:49
  • @wizzwizz4 posting on other non-tech sites also help, I guess. Plenty of opportunities for persona roleplay. Remember: if the users asked something about Vegeta, the answer should look like it was written by an annoyed Vegeta too :P Commented Jun 17 at 15:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .