PSA: Stack Exchange Inc. has announced a network-wide policy for AI content

Question

A couple of days ago, Stack Exchange Inc., the company running these sites, announced a new policy for handling AI content. This policy goes against our own, local policy on how to deal with AI content, where the Ask Ubuntu community made it very clear that such content was not welcome: Announcement: AI-generated content is now permanently banned on Ask Ubuntu

I am writing this post to make sure that the Ask Ubuntu community is aware of this new network-wide policy and to provide a place where we can discuss it and maybe decide how to handle it. The full text of the new policy can be read below, but I urge you to visit (and vote on) the original on Meta Stack Exchange as well:

We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.

In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. This standard of evidence excludes the use of moderators' best guesses based on users' writing styles and behavioral indicators, because we could not validate that these indicators are actually successfully identifying AI-generated posts when they are written. This standard would exclude most suspensions issued to date.

We've also identified that current GPT detectors have an unacceptably high false positive rate for content on our network and should not be regarded as reliable indicators of GPT authorship. While these aren't the sole tools that moderators rely upon to identify AI-generated content, some of the heuristics used have been developed with their assistance.

We've reminded moderators that suspensions (and typically mod messages as well) are for real, verifiable malfeasance only, and should not be enacted on the basis of hunches, guesses, intuition, or unverified heuristics. Therefore, we are not confident that either GPT detectors or best-guess heuristics can be used to definitively identify suspicious content for the purposes of suspension.

As always, moderators who identify that a user has a problematic pattern of low-quality posts should continue to act on such users as they otherwise would. Indicators moderators currently use to determine that a post was authored with the help of AI can in some cases form a reliable set of indicators that the content quality may be poor, and moderators should feel free to review posts as such. If someone is repeatedly contributing low-quality content, we already have policies in place to help handle it, including a suspension reason that can, in those cases, be used.

Answer 1

It is a good time to be completely candid:

We worked hard to get a solid policy in place for the torrent of AI/LLM material that was appearing on Ask Ubuntu. And by 'we' I mean the elected and community moderators, the Community that is what Ask Ubuntu should be all about.

This is a policy that worked well and a policy that was always ready for improvement, for modification as the world of AI/LLM matured.

What we experienced from Stack Exchange was neither improvement nor modification. They proposed first in secret, and then in a watered down public version, to sweep away our Community policy and to completely obstruct Moderator ability to deal with a flow of almost always extremely low quality, unverified material.

Not good enough Stack Exchange!

I am personally not happy to stand aside and watch this happen and I have undertaken the following:

1. I have signed the letter of protest which has given a 'line in the sand' to Stack Exchange
2. I have committed to the upcoming action as of Monday and will be withdrawing from all Moderator activities as of that time for the duration of the action

These are not actions taken lightly but in the face of a company that seems to have no interest in maintaining quality on the site, no interest in fostering Community and only great interest in their own bottom line: these steps are necessary...

Links:

• Open Letter to Stack Overflow, Inc. The open letter to the company that describes the issues leading up to the June 5th 2023 strike. There is a facility on this page to sign up and participate in the strike, if you wish.
• Moderation strike: Stack Overflow, Inc. cannot consistently ignore, mistreat, and malign its volunteers Details of the SE strike, written beautifully, on Meta SE.
• Stack Exchange is failing its community Journeyman Geek as usual gets to the heart of the matter: the disconnect between the company and the Community.
• What is the network policy regarding AI Generated content? The official announcement of the new policy. Bear in mind that this differs significantly from the version handed down to Moderators in a private forum.

Answer 2

In my understanding, the reason we don't want AI-generated content is that it's garbage - the bot doesn't know the answer to the question, it only knows how to produce answerlike text.

We can delete garbage. Does the answer actually answer the question? Does it make sense? Does it contain false, made up, or irrelevant things? Downvote, delete, flag. Just don't tell me it's AI-generated ;)

I have a lot of respect for you @Zanna ^{See Footnote}, but I believe your views on this are missing some key points:

• Plagiarism: It's not just about the answer being right-or-wrong (or somewhere in between). Even if the answer is right, someone who posts AI-generated content and passes it off as their own is violating the plagiarism policy here as well. This is true whether they copied the information ChatGPT, Bing Chat, OpenAI, or even Wikipedia, Reddit, or example.com. This is true whether the user copy/pasted or tried to "reword" the AI's answer. We simply don't allow someone to take credit for an answer that they didn't write themselves.

  This is the main reason why my proposed AI responsible-use policy starts with Cite as a requirement.

  Side-note: Now there are certainly times that citation doesn't necessarily make sense. Most of us here on Ask Ubuntu know that, to set environment variables "permanently", you would do so in your startup files (e.g. ~/.bashrc). But who among us remembers where they learned this in order to be able to cite it if we mention it in an answer? At some point, what we learn becomes part of our "residual knowledge".

  But that's not the case when using ChatGPT or another AI. The user who posts it knows exactly where the material came from in those cases, and it isn't the user's "knowledge" learned somewhere in the past. It should be cited.

• Trust and Reputation: Hand-in-hand with that goes the fact that we are a site built on trust and reputation. We expect someone with reputation in their tags to have some level of expertise. When a user posts AI-content as their own, a good AI-answer will increase that user's reputation, even though that person might not have the necessary subject matter expertise.

  That's not right or fair, and it has the potential to destroy the community's trust.

• Incentive: The reward-vs-risk for posting AI-generated content is too high based on your proposal. Let's say a user with no expertise posts AI-generated 20 answers:

  • 5 of these are "good" answers
  • 5 are "so so"
  • 10 are bad and get removed

  Let's say that the user even gets blocked for a period of time (automatically or via suspension) for repeatedly posting low-quality. But those answers that remain sit here for years to come and can continue to receive upvotes. The user may have even gained a quick 50 rep for those 5 "good" answers - ~ 2 acceptances and 2 upvotes. Then there are additional potential upvotes in the future across all 10 of the answers that remained.

• Potential for additional spam: And keep in mind that with 50 rep (which can far more easily be obtained using AI than "normal" methods, IMHO), a spam account can then turn around and post comment spam, of course. Granted, that's not the motivation behind the majority of AI use here, but we know that we have spam accounts who have posted AI content as well.

• Difficulty for non-experts to judge the quality: What's worse is that the problem with AI is that the answers sound so "informed" that a new user (often the one posting the question) or an inexperienced one can't tell the difference between a good answer and one with potential issues.

  I suspect that many "new user" AI answers are upvoted during the First Answers review queue without a close examination, and I've had one Mod (via private chat) on a low-traffic site confirm that I was right about this (at least in one case) when they wondered how and why an obviously bad, obviously AI answer was upvoted.

An answer being long and grammatically correct doesn't make it right. On the other hand though, let's not assume every long and grammatically correct post is AI garbage!

I think you know that's not how AI answers have been detected here, by any stretch :-). I'm happy to discuss with you privately (e.g. chat, or even Discord if you prefer, and yes I know all mods can see private chat) the techniques I've used if you'd like, but having flagged more than 1,200 suspected AI answers on Stack Overflow, a few here and on Super User, and having over 2,000 GPT answers in my "Saves" list from the three sites, I think I've developed a good sense and set of heuristics to determine AI use. Of course, I freely admit that I am obviously biased, but the Stack Overflow mods, if my "Helpful" flag rate is any indication, at least, seem to agree.

I know you can't see the deleted answers on Stack Overflow, which I think would give you a great sense of a "common" GPT answer, but perhaps I can find some way to share a set with you. On bright (not-so) bright side, there are quite a few undeleted AI answers on Stack Overflow at the moment to work with.

---

^Footnote:

Interesting anecdote - I still remember my first interaction with @Zanna. I called her out for being wrong about making an edit to a post I didn't feel was substantiated. I didn't even notice her Mod diamond, but it didn't matter -- After reading more closely, I discovered I was wrong (/gulp!), and @Zanna was right. And maybe that's the case here as well. Who knows? ;-)

Answer 3

We're living in interesting times.

In my understanding, the reason we don't want AI-generated content is that it's garbage - the bot doesn't know the answer to the question, it only knows how to produce answerlike text.

We can delete garbage. Does the answer actually answer the question? Does it make sense? Does it contain false, made up, or irrelevant things? Downvote, delete, flag. Just don't tell me it's AI-generated ;)

An answer being long and grammatically correct doesn't make it right. On the other hand though, let's not assume every long and grammatically correct post is AI garbage!

Answer 4

overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site

The new owner does not seem to understand that the value of the these sites is the reliability of the information that is amassed here. That the information here, until now, was produced by invested authentic contributors in good faith. That's what draws people to these sites.

Accordingly, dumping unreliable AI content here can not count as legitimate contribution.

The new owner, according to my knowledge, is Prosus N.V. On their portfolio page, they currently seem to list 88 online businesses that are their property. With that many properties, they apparently don't have time to study and realize what makes StackExchange valuable.

They just go after the generic capitalist metric: number of users. We are no more than a bodycount, from now on. The nature of our contributions, Prosus does not recognize.

Furthermore, on the aforementioned portfolio page (in the "Edtech" category) they list only StackOverflow, which is only the flagship of the network of these sites. They seem to ignore that StackOverflow is only a part of the whole. And so they appear to express complete disinterest in the rest of the StackExchange sites.

---

If they unleash AI content around here, that will undermine the usability of these sites and dilute their value at the same time.

What is interesting, is if they make StackOverflow unusable, the entire world will feel that. The most impactful human creations these times rely on code. Accordingly, disrupting StackOverflow (and Unix&Linux) may disrupt industries, public authorities, and with those, our societies.

But Prosus seem to have only money, and not expertise.

And the StackExchange home team appears to be missing in action too! (Though the Monica Cellio case, its aftermath, and the massive quality hit that the network suffered then, had demonstrated their true attitudes years ago.)

---

AI is upon us, threatening with the disruption of anything that's done digitally today; the networks organizing our societies are under threat.

Twitter is destroyed, the owners of Reddit are following, now this network here also may join in the self-sabotage, education getting systematically dismantled, greedflation to be observed globally (the capitalist owning class is cashing their investments out as quick as they can, a lot of capitalists seem to not be interested in sustainable business any more (because they might not even know how to do that any more, under the given conditions)), billionaires prepping their bunkers.

This may either be a chain reaction of panic in capitalist systems, or a controlled and planned implosion of public wellbeing — or, as I seem to believe, the blend of the two. When capitalists see that the variant of capitalism oriented at the illusion of human wellbeing is not sustainable any more (it never was, by concept; its implosion was just a matter of time), they now attempt to transition society to some new shape.

Without education, knowledge, without corresponding infrastructure, without democracy.

But one in which they hope to remain on top of the rest of us.

The possible trigger could have been insights into AI potential, I seem to have read somewhere. (Those insights are not necessarily brand new; insiders might have enjoyed years of leeway, behind closed doors.)

---

In the meanwhile, if someone is willing to bother to save the knowledge that's amassed on StackExchange, I say, it's time to scrape it and archive it somewhere safe, while there is opportunity to do so.

It would also worth asking Canonical if they would be interested in hosting us somewhere else.

Gentlefolks, it has been a privilege playing with you tonight.

Regardless of what happens tomorrow, I thank you all, network-wide, for this opportunity to develop personally with you. It's been, and is, an honor.

---

UPDATE

SE sites are the essence from all of our learnings, hard-won insights, and successful battles that we share here, and that could serve the next generation with all that we could offer.

SE sites are our cultural heritage, fragments and facets of our authentic personalities, shared in good faith, and frozen in text content.

These sites here carry the fingerprints of our hearts.

A proposal:

Make SE UNESCO cultural heritage, perhaps an intangible one, and assign a worthy custodian to it.

Not a joke, and I hope, not a delusion.

In the face of the AI content tsunami, it's time to start to establish natural reserves, and cultural monuments to our cultural heritage.

Answer 5

Tactical Withdraw, may be?

Bear with me as this was my chain of thinking. Personally, I still believed AI generated contents is of unknown quality but expected to be low most of the time.

So what the current options to handle it:

1. Ask Ubuntu community policy: Ban it ...

   Experienced users expected to flag it. Moderator review then reject or hide it from public and it get deleted later.

2. Stack Exchange network policy: Leave it be ...

   Administrator are asked to close eyes. Users are not pushed either way.

3. I tried to come up with a in-middle option: Label it ...

   User expected to flag it. Moderator review then reject or a special AI label to make users aware.

Back to it now after I reread this thread again. Option 1 and 3 will give bad results on long run. Because of the way AI works.

1. Stack Exchange network data are already public and they are also serving it as API. That gives AI users a head start. So that data is used as training set.

2. AI users will contribute some posts here.

3. And they will wait for feedback!. Whatever we do with them votes, flags, comments, deletion...

This feedback will be very helpful, to continues AI learning. 1 and 3 will just levels up the AI and it will be hard with time to distinguish soon or later. 2 is not perfect but delaying the hide/deletion action will slow down the AI learning process.

May be it is wrong for me to put this in public? Anyway CM already gave some hints of SE secret scroll.

Answer 6

Libel, Slander or other Defamation

I think I would be the last person to praise a Billion dollar a year cooperation that depends on Free labor, but I do appreciate Stack Exchange's stand against defaming its contributors and inviting retaliation.

Stack Exchange Inc can be sued for libel in most of the countries where it is available. In some countries, libel is not just a civil matter but a crime punishable by imprisonment.

It is wrong to libel a person just on suspicion, a hunch, or without solid proof. Solid proof of AI use is pretty much impossible at this time, It is evolving rapidly and is not the same beast it was when it was banned.

So what is wrong with AI? I think most ASK users just want an answer to their question and don't care if it comes from AI or from a contributor that just looks it up on the internet.

Perhaps this is just a move by SE to eliminate the human element and move to an AI-based Q & A model, Human help is cheap, but they complain a lot.

One thing that can be done for mega AI posts is to change the downvote rep change from -2 to -10 (for new users). Right now I don't care if I get 10 downvotes for an answer as long as I get 3 upvotes.