Less incentive for users to post unmarked ChatGPT answers
I do not see this as substantially lessening the incentive. You can already find questions with multiple ChatGPT generated answers posted to them. It does not really seem users who take the cheap and easy way to generate content really care that much.
Instant answers to simple questions
Except the network is built around good answers. There is absolutely no guarantee that an answer generated by ChatGPT is correct. Quite the opposite - the answers are very often wrong. Either inaccurate enough to pass for an answer at a glance and fail at the details, or may even contradict itself within the same paragraph or advocate using non-existent, impractical, or wrong solutions.
The answers generated also often enough fail to actually answer the question, since ChatGPT failed to identify a major component. Like "I want to do I used X but <doesn't work because>" and the answer is "You should use X".
Yes, ChatGPT does also answer correctly. But therein also lies a problem - most simple questions are already answered. They should be marked as a duplicate instead.
ChatGPT is clearly marked and users know not to trust it without checking
Let me be the bearer of bad news. Users do not check the validity of answers. No matter how they are marked. Yes, not all of the users but enough of them.
And we do have very concrete proof of this happening: see this bug which exists in Razer and Docker for Windows. Summary the Razer driver manager and Docker could not be launched together on Windows. The bug was due to an answer on Stack Overflow (screenshot as it is deleted) which initially proposed a faulty mechanism to uniquely identify an executable. It was faulty because it would generate the same ID for any executable. And it was used to prevent an executable running twice. In the case of Docker and Razer Synapse, both thought the other was a second instance of themselves. Thus if Razer Synapse is running, Docker cannot be started.
This is far from the only example. It is, however, the most visible and best for illustrating that users do not verify. There is a highly upvoted comment under the answer pointing out the problem.
It is also quite an important example, as it reveals a misconception many seem to have: the answers posted are not for the question asker. They are for everybody on the Internet. Anybody can ask a question and software engineers over at Docker and Razer can use the answers. Even if one user does verify the contents of an answer, there are many others who will not. And many more users may be taking advantage of content throughout the years.
Which is another reason why it is dangerous to leave wrong content.
Users could be allowed to create answers based on the ChatGPT answer, which improve on it by fact checking, fixing minor problems with the answer, explaining details and so on.
To be honest, I am all for allowing AI generated content in some form. However, I do not believe this is the way. It undermines the foundation of what Stack Exchange is. Reputable answers, each of which ideally stands by itself. Instead, this proposal will fill the network with untrustworthy, potentially wrong, dangerous, or plain misguided answers, and then expects real humans to pick up the slack and verify whether the autogenerated answer works, or not, why and how. And that is a tremendous issue.
