21

A couple of days ago, Stack Exchange Inc., the company running these sites, announced a new policy for handling AI content. This policy goes against our own, local policy on how to deal with AI content, where the Ask Ubuntu community made it very clear that such content was not welcome: Announcement: AI-generated content is now permanently banned on Ask Ubuntu

I am writing this post to make sure that the Ask Ubuntu community is aware of this new network-wide policy and to provide a place where we can discuss it and maybe decide how to handle it. The full text of the new policy can be read below, but I urge you to visit (and vote on) the original on Meta Stack Exchange as well:

We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.

In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. This standard of evidence excludes the use of moderators' best guesses based on users' writing styles and behavioral indicators, because we could not validate that these indicators are actually successfully identifying AI-generated posts when they are written. This standard would exclude most suspensions issued to date.

We've also identified that current GPT detectors have an unacceptably high false positive rate for content on our network and should not be regarded as reliable indicators of GPT authorship. While these aren't the sole tools that moderators rely upon to identify AI-generated content, some of the heuristics used have been developed with their assistance.

We've reminded moderators that suspensions (and typically mod messages as well) are for real, verifiable malfeasance only, and should not be enacted on the basis of hunches, guesses, intuition, or unverified heuristics. Therefore, we are not confident that either GPT detectors or best-guess heuristics can be used to definitively identify suspicious content for the purposes of suspension.

As always, moderators who identify that a user has a problematic pattern of low-quality posts should continue to act on such users as they otherwise would. Indicators moderators currently use to determine that a post was authored with the help of AI can in some cases form a reliable set of indicators that the content quality may be poor, and moderators should feel free to review posts as such. If someone is repeatedly contributing low-quality content, we already have policies in place to help handle it, including a suspension reason that can, in those cases, be used.

4
  • 17
    I should also point out that the private policy shared with moderators (yes, they told us one thing and you another and we're not allowed to share) is much more restrictive than what can be seen here. What you have is a description of the policy, not the explanation of the "strict standard" we've been asked to apply.
    – terdon
    Jun 2, 2023 at 22:18
  • 5
    "may be turning away a large number of legitimate contributors" — contributing what, exactly? Also, they sacrifice the factual integrity of the content for the number of user accounts, because they get paid for the latter, and not the former. Stackexchange should have been made non-profit.
    – Levente
    Jun 3, 2023 at 16:48
  • 8
    openletter.mousetail.nl please join the strike Jun 4, 2023 at 16:12
  • 1
    Note that Slate's comment here says (emphasis added) "While we evaluate, we hope that folks on network sites feel comfortable establishing per-site policies responsive to their communities’ needs." I.e. according to the wording there, that allowance has a condition variable.
    – starball
    Jun 5, 2023 at 20:24

6 Answers 6

27

It is a good time to be completely candid:

We worked hard to get a solid policy in place for the torrent of AI/LLM material that was appearing on Ask Ubuntu. And by 'we' I mean the elected and community moderators, the Community that is what Ask Ubuntu should be all about.

This is a policy that worked well and a policy that was always ready for improvement, for modification as the world of AI/LLM matured.

What we experienced from Stack Exchange was neither improvement nor modification. They proposed first in secret, and then in a watered down public version, to sweep away our Community policy and to completely obstruct Moderator ability to deal with a flow of almost always extremely low quality, unverified material.

Not good enough Stack Exchange!

I am personally not happy to stand aside and watch this happen and I have undertaken the following:

  1. I have signed the letter of protest which has given a 'line in the sand' to Stack Exchange
  2. I have committed to the upcoming action as of Monday and will be withdrawing from all Moderator activities as of that time for the duration of the action

These are not actions taken lightly but in the face of a company that seems to have no interest in maintaining quality on the site, no interest in fostering Community and only great interest in their own bottom line: these steps are necessary...

Links:

5
  • 4
    They did not seem receptive in the Monica Cellio times, and back then Stackexchange was not owned by an external company. They seem to have been unreceptive on their own incentive. Now they are owned by Prosus N. V. I wouldn't expect that they got better. In my view, Stackexchange had since long been an unworthy custodian of the wealth of information that we had amassed here. I only hope that Canonical does something to host us elsewhere.
    – Levente
    Jun 3, 2023 at 17:06
  • 1
    I always thought such sides like stackexchange are from human beings for humans. 😕
    – nobody
    Jun 4, 2023 at 19:32
  • Thank you very much for the links and explaining your motivations! Much appreciated. Jun 5, 2023 at 18:03
  • Where's the link of the "solid policy"?
    – Braiam
    Jun 9, 2023 at 18:45
  • meta.stackexchange.com/a/389583/662263
    – andrew.46 Mod
    Jun 9, 2023 at 21:46
11

In my understanding, the reason we don't want AI-generated content is that it's garbage - the bot doesn't know the answer to the question, it only knows how to produce answerlike text.

We can delete garbage. Does the answer actually answer the question? Does it make sense? Does it contain false, made up, or irrelevant things? Downvote, delete, flag. Just don't tell me it's AI-generated ;)

I have a lot of respect for you @Zanna See Footnote, but I believe your views on this are missing some key points:

  • Plagiarism: It's not just about the answer being right-or-wrong (or somewhere in between). Even if the answer is right, someone who posts AI-generated content and passes it off as their own is violating the plagiarism policy here as well. This is true whether they copied the information ChatGPT, Bing Chat, OpenAI, or even Wikipedia, Reddit, or example.com. This is true whether the user copy/pasted or tried to "reword" the AI's answer. We simply don't allow someone to take credit for an answer that they didn't write themselves.

    This is the main reason why my proposed AI responsible-use policy starts with Cite as a requirement.

    Side-note: Now there are certainly times that citation doesn't necessarily make sense. Most of us here on Ask Ubuntu know that, to set environment variables "permanently", you would do so in your startup files (e.g. ~/.bashrc). But who among us remembers where they learned this in order to be able to cite it if we mention it in an answer? At some point, what we learn becomes part of our "residual knowledge".

    But that's not the case when using ChatGPT or another AI. The user who posts it knows exactly where the material came from in those cases, and it isn't the user's "knowledge" learned somewhere in the past. It should be cited.

  • Trust and Reputation: Hand-in-hand with that goes the fact that we are a site built on trust and reputation. We expect someone with reputation in their tags to have some level of expertise. When a user posts AI-content as their own, a good AI-answer will increase that user's reputation, even though that person might not have the necessary subject matter expertise.

    That's not right or fair, and it has the potential to destroy the community's trust.

  • Incentive: The reward-vs-risk for posting AI-generated content is too high based on your proposal. Let's say a user with no expertise posts AI-generated 20 answers:

    • 5 of these are "good" answers
    • 5 are "so so"
    • 10 are bad and get removed

    Let's say that the user even gets blocked for a period of time (automatically or via suspension) for repeatedly posting low-quality. But those answers that remain sit here for years to come and can continue to receive upvotes. The user may have even gained a quick 50 rep for those 5 "good" answers - ~ 2 acceptances and 2 upvotes. Then there are additional potential upvotes in the future across all 10 of the answers that remained.

  • Potential for additional spam: And keep in mind that with 50 rep (which can far more easily be obtained using AI than "normal" methods, IMHO), a spam account can then turn around and post comment spam, of course. Granted, that's not the motivation behind the majority of AI use here, but we know that we have spam accounts who have posted AI content as well.

  • Difficulty for non-experts to judge the quality: What's worse is that the problem with AI is that the answers sound so "informed" that a new user (often the one posting the question) or an inexperienced one can't tell the difference between a good answer and one with potential issues.

    I suspect that many "new user" AI answers are upvoted during the First Answers review queue without a close examination, and I've had one Mod (via private chat) on a low-traffic site confirm that I was right about this (at least in one case) when they wondered how and why an obviously bad, obviously AI answer was upvoted.

An answer being long and grammatically correct doesn't make it right. On the other hand though, let's not assume every long and grammatically correct post is AI garbage!

I think you know that's not how AI answers have been detected here, by any stretch :-). I'm happy to discuss with you privately (e.g. chat, or even Discord if you prefer, and yes I know all mods can see private chat) the techniques I've used if you'd like, but having flagged more than 1,200 suspected AI answers on Stack Overflow, a few here and on Super User, and having over 2,000 GPT answers in my "Saves" list from the three sites, I think I've developed a good sense and set of heuristics to determine AI use. Of course, I freely admit that I am obviously biased, but the Stack Overflow mods, if my "Helpful" flag rate is any indication, at least, seem to agree.

I know you can't see the deleted answers on Stack Overflow, which I think would give you a great sense of a "common" GPT answer, but perhaps I can find some way to share a set with you. On bright (not-so) bright side, there are quite a few undeleted AI answers on Stack Overflow at the moment to work with.


Footnote:

Interesting anecdote - I still remember my first interaction with @Zanna. I called her out for being wrong about making an edit to a post I didn't feel was substantiated. I didn't even notice her Mod diamond, but it didn't matter -- After reading more closely, I discovered I was wrong (/gulp!), and @Zanna was right. And maybe that's the case here as well. Who knows? ;-)

2
  • 3
    I didn't mean to suggest that people have been assuming every long and grammatically correct answer is AI-generated - maybe I should have parenthesised that part - more importantly I want people to not assume they are good (the upvotes you mention). OTOH not everyone is as good as you and I do think there have been some mistakes, false positives acted on etc. But for the most part I think humans do a good job of detecting AI garbage, and we are probably getting better at it...
    – Zanna Mod
    Jun 6, 2023 at 3:11
  • 4
    You mention a situation where I was right (apparently!), but I am wrong about plenty of things, and I agree with you that my suggestion that we can just treat AI-generated answers as low quality (as advised by SE) is not workable, even for our site (and definitely not for SO). Effectively SE are asking us to pretend we don't know it's AI garbage, which is as stupid as having a secret policy (secret methods, secret tools, okay, but one can't have a secret policy!). I thought it might be what we do to manage in the interim, but we went on strike instead so :]
    – Zanna Mod
    Jun 6, 2023 at 3:16
9

We're living in interesting times.

In my understanding, the reason we don't want AI-generated content is that it's garbage - the bot doesn't know the answer to the question, it only knows how to produce answerlike text.

We can delete garbage. Does the answer actually answer the question? Does it make sense? Does it contain false, made up, or irrelevant things? Downvote, delete, flag. Just don't tell me it's AI-generated ;)

An answer being long and grammatically correct doesn't make it right. On the other hand though, let's not assume every long and grammatically correct post is AI garbage!

8
  • Zanna, are you participating in the mods strike? Jun 4, 2023 at 10:02
  • 5
    @RandomPerson why does that matter? Let her be. If she decided to participate, she would probably announce it some way. Until then, it's not your business.
    – Levente
    Jun 4, 2023 at 11:13
  • 5
    I joined the strike, reluctantly, in solidarity. Stack Overflow mods can't possibly maintain the quality of the site with this policy, I understand. I agree that detectors shouldn't be trusted, but we need much better tools to deal with the AI problem - starting with actually prominently telling people it's banned, maybe warnings when they paste 6 paragraphs into the editor, autoflags? I haven't been active recently so others have been doing most of the AI cleanup work here (it's a complex problem, and I don't feel I've quite got my head around it even now). The company needs to work with us.
    – Zanna Mod
    Jun 4, 2023 at 12:22
  • 2
    @Zanna Sorry for being nosy. Jun 4, 2023 at 14:38
  • 2
    @RandomPerson no worries :)
    – Zanna Mod
    Jun 4, 2023 at 14:41
  • 6
    "Does it contain false, made up [...] things?" — that's the challenge: the false, made up parts blend in so well that it takes an extra amount of specialized expertise and attention to either spot, or to prove them wrong. They are wrong, because they are not valid and don't work, but unlike previous useless content produced by humans, this new kind needs way more effort to uncover. That's where it impacts sustainability in a bad way.
    – Levente
    Jun 4, 2023 at 21:32
  • 2
    @Levente Yes. You are right. The struggle is real. The moderation burden of dealing with AI content is exceeding the capacity of our communities.
    – Zanna Mod
    Jun 5, 2023 at 4:30
  • @Levente and yet, I've been able to figure those out. I simply test them.
    – Braiam
    Jun 9, 2023 at 18:55
7

overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site

The new owner does not seem to understand that the value of the these sites is the reliability of the information that is amassed here. That the information here, until now, was produced by invested authentic contributors in good faith. That's what draws people to these sites.

Accordingly, dumping unreliable AI content here can not count as legitimate contribution.

The new owner, according to my knowledge, is Prosus N.V. On their portfolio page, they currently seem to list 88 online businesses that are their property. With that many properties, they apparently don't have time to study and realize what makes StackExchange valuable.

They just go after the generic capitalist metric: number of users. We are no more than a bodycount, from now on. The nature of our contributions, Prosus does not recognize.

Furthermore, on the aforementioned portfolio page (in the "Edtech" category) they list only StackOverflow, which is only the flagship of the network of these sites. They seem to ignore that StackOverflow is only a part of the whole. And so they appear to express complete disinterest in the rest of the StackExchange sites.


If they unleash AI content around here, that will undermine the usability of these sites and dilute their value at the same time.

What is interesting, is if they make StackOverflow unusable, the entire world will feel that. The most impactful human creations these times rely on code. Accordingly, disrupting StackOverflow (and Unix&Linux) may disrupt industries, public authorities, and with those, our societies.

But Prosus seem to have only money, and not expertise.

And the StackExchange home team appears to be missing in action too! (Though the Monica Cellio case, its aftermath, and the massive quality hit that the network suffered then, had demonstrated their true attitudes years ago.)


AI is upon us, threatening with the disruption of anything that's done digitally today; the networks organizing our societies are under threat.

Twitter is destroyed, the owners of Reddit are following, now this network here also may join in the self-sabotage, education getting systematically dismantled, greedflation to be observed globally (the capitalist owning class is cashing their investments out as quick as they can, a lot of capitalists seem to not be interested in sustainable business any more (because they might not even know how to do that any more, under the given conditions)), billionaires prepping their bunkers.

This may either be a chain reaction of panic in capitalist systems, or a controlled and planned implosion of public wellbeing — or, as I seem to believe, the blend of the two. When capitalists see that the variant of capitalism oriented at the illusion of human wellbeing is not sustainable any more (it never was, by concept; its implosion was just a matter of time), they now attempt to transition society to some new shape.

Without education, knowledge, without corresponding infrastructure, without democracy.

But one in which they hope to remain on top of the rest of us.

The possible trigger could have been insights into AI potential, I seem to have read somewhere. (Those insights are not necessarily brand new; insiders might have enjoyed years of leeway, behind closed doors.)


In the meanwhile, if someone is willing to bother to save the knowledge that's amassed on StackExchange, I say, it's time to scrape it and archive it somewhere safe, while there is opportunity to do so.

It would also worth asking Canonical if they would be interested in hosting us somewhere else.

Gentlefolks, it has been a privilege playing with you tonight.

Regardless of what happens tomorrow, I thank you all, network-wide, for this opportunity to develop personally with you. It's been, and is, an honor.


UPDATE

SE sites are the essence from all of our learnings, hard-won insights, and successful battles that we share here, and that could serve the next generation with all that we could offer.

SE sites are our cultural heritage, fragments and facets of our authentic personalities, shared in good faith, and frozen in text content.

These sites here carry the fingerprints of our hearts.

A proposal:

Make SE UNESCO cultural heritage, perhaps an intangible one, and assign a worthy custodian to it.

Not a joke, and I hope, not a delusion.

In the face of the AI content tsunami, it's time to start to establish natural reserves, and cultural monuments to our cultural heritage.

5
  • 3
    I did similar research on Prosus (and their primary owner, Naspers) in attempt to "follow the money" in terms of why things were happening here. But I didn't reach quite the same conclusions. First, you wouldn't normally see Prosus management get involved at this level - I believe this is purely something coming out of the SO/SE staff. They'd probably ask the CEO about it, if it appeared to become a problem, but I doubt they were behind this decision. Jun 5, 2023 at 18:13
  • 2
    Second, while they may own a number of other companies, SO is definitely their largest acquisition to date and likely makes up a more significant chunk of their portfolio/valuation than any other child-company. So if they do perceive a problem that could devalue their investment, they'll likely get involved. Jun 5, 2023 at 18:14
  • 5
    In a "follow the money" analysis, it's most likely one of Stack Exchange's customers that has an issue with this. The customers that use Teams and Collectives are often in for "six and seven figure deals" (according to one of the Sales job postings). I've seen one company's employees posting AI-generated answers at a higher-than-average rate on Stack Overflow, and I'm wondering if perhaps there was some sort of escalation from the Account Team. Those tend to often get knee-jerk reactions from management that can often result in bad decision making for the company as a whole. Jun 5, 2023 at 18:16
  • 2
    @NotTheDr01ds those are very interesting insights. I totally failed to reckon with the StackOverflow Teams product. I guess I should research what that product actually is, to better understand how a client, who pays for privileged and private access can come to a position to dictate what happens on the entire open network... And how SE staff can be so far unaligned with all the rest of the user base in their conclusions about the source of the value of the platform...
    – Levente
    Jun 5, 2023 at 18:58
  • @NotTheDr01ds if your take is the winning one, then I am curious to see what will Prosus do to attempt to save the value of their investment. To what sort of confrontation between SE staff and Prosus that could lead. Might there be fresh openings in paid internal positions for community manager and other admin roles? If Prosus would make the effort to study the Monica Cellio case and its consequences, and then to consider this fiasco, maybe they could realize that the SE home team had long been ripe for replacement? Maybe Prosus could invite back the original pair of creators, to save the day?
    – Levente
    Jun 5, 2023 at 19:05
1

Tactical Withdraw, may be?

Bear with me as this was my chain of thinking. Personally, I still believed AI generated contents is of unknown quality but expected to be low most of the time.

So what the current options to handle it:

  1. Ask Ubuntu community policy: Ban it ...

    Experienced users expected to flag it. Moderator review then reject or hide it from public and it get deleted later.

  2. Stack Exchange network policy: Leave it be ...

    Administrator are asked to close eyes. Users are not pushed either way.

  3. I tried to come up with a in-middle option: Label it ...

    User expected to flag it. Moderator review then reject or a special AI label to make users aware.

Back to it now after I reread this thread again. Option 1 and 3 will give bad results on long run. Because of the way AI works.

AI feedback loop

  1. Stack Exchange network data are already public and they are also serving it as API. That gives AI users a head start. So that data is used as training set.

  2. AI users will contribute some posts here.

  3. And they will wait for feedback!. Whatever we do with them votes, flags, comments, deletion...

This feedback will be very helpful, to continues AI learning. 1 and 3 will just levels up the AI and it will be hard with time to distinguish soon or later. 2 is not perfect but delaying the hide/deletion action will slow down the AI learning process.

May be it is wrong for me to put this in public? Anyway CM already gave some hints of SE secret scroll.

1
  • 2
    1.) I think, as long as you do anything, it can be detected and evaluated as feedback. 2.) If you delay the hide/deletion action, that leaves polluting content on the site, negatively impacting site visitors who visit the site during that period. 3.) "Administrator are asked to close eyes." — did you mean diamond moderators? 4.) "Administrator are asked to close eyes. Users are not pushed either way." I'm not convinced I grasp how that works. I think the site is governed by a singular policy and mods and regular users work towards that policy by uniting their forces...
    – Levente
    Jul 27, 2023 at 18:52
-16

Libel, Slander or other Defamation

I think I would be the last person to praise a Billion dollar a year cooperation that depends on Free labor, but I do appreciate Stack Exchange's stand against defaming its contributors and inviting retaliation.

Stack Exchange Inc can be sued for libel in most of the countries where it is available. In some countries, libel is not just a civil matter but a crime punishable by imprisonment.

It is wrong to libel a person just on suspicion, a hunch, or without solid proof. Solid proof of AI use is pretty much impossible at this time, It is evolving rapidly and is not the same beast it was when it was banned.

So what is wrong with AI? I think most ASK users just want an answer to their question and don't care if it comes from AI or from a contributor that just looks it up on the internet.

Perhaps this is just a move by SE to eliminate the human element and move to an AI-based Q & A model, Human help is cheap, but they complain a lot.

One thing that can be done for mega AI posts is to change the downvote rep change from -2 to -10 (for new users). Right now I don't care if I get 10 downvotes for an answer as long as I get 3 upvotes.

3
  • 6
    So you are saying that when SE accuses moderators of suspending unfairly without having any evidence to support it, they are libeling and slandering us? You may well be right.
    – terdon
    Jun 3, 2023 at 12:20
  • 1
    @tendon: If SE accuses the moderator by name and can not prove what they say is true, but the moderator can prove that SE's statement is false then it is considered libel in most countries. As I understand, in that case, you could sue and win in some jurisdictions. A lot depends on Stack Overflow's Terms of Service you accepted. If you ask ChatGpt "Can a moderator on Stack Overflow sue if the network libels him." you will get a much better and more accurate answer than I can give. Jun 3, 2023 at 12:44
  • 1
    @C.S.Cameron SE didn't mention any name, but there's also arbitration clause.
    – Andrew T.
    Jun 4, 2023 at 4:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .