Ban ChatGPT and AI-generated answers on Ask Ubuntu?

Question

ChatGPT generated answers have been found across the Stack Exchange network and some examples have also been recently seen and removed from Ask Ubuntu. Our colleagues on Stack Overflow have put a temporary ban on all such posts while other sites have begun debate on what their positions will be. Stack Exchange itself has decided not to institute a network ban. So now for Ask Ubuntu...

There are many issues with ChatGPT answers but for me the main issue has been that they look like they might be good answers, (the ones I have seen have been beautifully written), but relying as they do merely on a statistical method they can very, very easily be completely incorrect. The accuracy or otherwise of such an answer has not usually been tested and thus has never been fully evaluated before being presented as a definitive answer to a group of users who are often new to Ubuntu and are often very trusting of such material presented on Ask Ubuntu.

Such answers therefore carry the risk of being harmful to the users of Ask Ubuntu and also harmful to the reputation of Ask Ubuntu as a quality Question and Answer site.

The other major issue is plagiarism. AI posts cannot be effectively referenced as their data comes from a soup of information that cannot ever be effectively referenced. As well many of those using ChatGPT to post will attempt to conceal this usage, thus doubly violating AU plagiarism rules.

My own thought is that such posts have no place on Ask Ubuntu and that, as Stack Overflow, Super User snd other areas have done, we should institute a temporary ban on ChatGPT and similar AI technologies. 'Temporary' while a clearer picture is sought on the role of such technology across the entire Stack Exchange network.

This ban, IMO, would involve:

1. Deletion of all ChatGPT content, whether acknowledged as such or not
2. Moderator email for initial occurrence, if minor in scope
3. Seven day Suspension for subsequent occurrence, or initial occurrence if larger in scope, and then the usual escalation of penalties for further infractions

I would welcome input from those who believe that AI generated answers have a place on AU as well as those who feel these answers have no place. There is room on AU Meta for varied opinions and vigorous discussion.

Your thoughts?

Further References:

1. Use of ChatGPT is now banned on Super User: Our colleagues on Super User have now instituted a complete ban on ChatGPT content.
2. Why posting GPT and ChatGPT generated answers is not currently acceptable: The Stack Overflow rational for banning ChatGPT answers.
3. Announcement: AI generated answers are officially banned here: An announcement on English Language and Usage completely banning technology such as ChatGPT from their area.

Answer 1

100 % Agree with the ban, Specially for Ask Ubuntu.

Here are some points I came up with against allowing ChatGPT answers:

• ChatGPT is not "Sentient" , it was trained on answers from forums like these.
• It was trained on outdated information before 2021.
• Inaccurate answers will cause more harm specially in these forums as we are dealing with operating systems.
• It is very overconfident spitting out wrong answers.
• It wont be able to help with/fix issues arising from suggested solution.
• "The model would not ask clarifying questions when the user provided an ambiguous query. Instead, our current models usually guess what the user intended" - https://openai.com/blog/chatgpt/

On the other hand.

There are 78,466 questions with no answers.

The community feels to be getting weaker with every passing day.

People often get discouraged from trying out new distributions / switching to Linux witnessing poor community support. I see new questions everyday made by new members who leave after getting none to little help.

I would suggest if a new user posts a relatively "simple/harmless" question that remains unanswered for a week there should be some automated response referring them to external resources such that people don't abandon the project altogether.

Answer 2

My straw-man position:

• I am opposed to a flat-out ban on ChatGPT, because I do believe it can be a useful aid for improving answers (see below), but there do need to be rules (and common sense) around its use.

• ChatGPT should not be used to write answers, but rather to improve answers.

• A ChatGPT-based answer should never be posted by someone without subject matter expertise in the question being asked in the first place. Confirming that the answer works is not enough. If you are even using ChatGPT to assist you in crafting the answer, then you must have the ability to:

  • Confirm that the answer is correct and works
  • Fully understand the answer that you are posting
  • Correct any issues
  • Communicate the answer in your own words
  • Reply to follow-up questions/comments about your answer
  • Warn of corner cases or potential issues if needed
  • Cite any reliance you made on ChatGPT for the answer

That's in addition to the normal requirements for a good answer. As just one example from that help page, answering a commonly-asked-duplicate is not useful, regardless of whether your answer is AI-assisted or not.

That said, I firmly believe that ChatGPT can be a useful aid when answering questions (and performing other actions on the site):

• If I'm going to post an answer of my own, I don't mind running the question through ChatGPT to see if there are additional things that I hadn't considered. Those suggestions that ChatGPT makes may or not be useful, but they at least make me think about ways that I might be able to improve my answer.

• Many users here have the right level of expertise (or beyond), but may not have mastered writing in English. It's far easier for most people to read in a language than it is to write well in that same language. For this reason, ChatGPT can typically offer some great suggestions on improving English grammar for these users. And those users will likely be able to confirm that the meaning of their answer hasn't changed.

• I have personally used ChatGPT on several occasions here on Ask Ubuntu to improve the grammar of some questions during Edits. The two that I've done:

  • Example 1: Before and after. IIRC, I'm fairly certain I did change one word from the ChatGPT suggestion -- I felt "the issue persists" was better than "this issue persists".

  • Example 2: Before and after: I definitely made some small tweaks to the ChatGPT suggestion to make it slightly more concise, but I can't recall exactly what they were.

  I do a lot of grammar/clarity edits (including, frequently, on my own mistakes). While I certainly could have cleaned up these two posts on my own, ChatGPT saved a lot of effort in these cases, and I believe the posts (and site) are better for it. In no way did ChatGPT suggest an answer here, or rely on any training other than proper English grammar.

---

Personal Anecdote

My wife has asked me on several occasions how to solve a Windows problem that she has. A certain application shifts its window entirely off screen when switching from her docked, three-monitor setup to her single-screen laptop.

My answer (from personal, long-past experience) is:

• Select the application on the Windows task bar.
• Press Alt+Space to show the "window menu", which appears on-screen.
• Use the Down Arrow to navigate to the Move option.
• Press the Enter key to activate it.
• Hold down the Left Arrow to move it back on the screen.

She's always amazed that I know this off the top of my head. And let's be realistic -- there aren't many people who can give that answer from memory. It's a technique that I developed on my own, before the web and search engines even existed in the form they do today.

So I was curious how (or if) ChatGPT would handle that. I asked it something like "I have an application whose window is appearing off screen in Windows. How do I get it back?"

To my amazement, ChatGPT not only gave me my normal method, but also suggested some other (easier) things that I hadn't considered, like Win (Super)+Left might snap it back to the proper side of the screen.

If I was answering a question like this on a Stack Exchange site, then the extra information that ChatGPT provided would be useful in my answer as long as I could Confirm, Understand, Correct, Communicate, Reply, Warn, and Cite as mentioned above.

Answer 3

The problem seems more about a large volume of garbage answers.

From https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned :

"The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure."

The current problem is the potential for a blizzard of garbage answers that overload the AskUbuntu cadre of great volunteers. That the answers are written by AI is interesting, but not the main problem. The blizzard is the problem.

The risk to this community is loss of credibility -- that noise drowns out signal and that AskUbuntu's reputation changes from "great gurus" to "garbage farm."

We face a potential flood of malicious vandalism. If that flood materializes, our Moderators will need help from us to spot and flag garbage, and to help identify key garbage posters.

When you identify a AI-written garbage answer:

• Downvote it.
• Flag it ('not an answer' for jibberish, or 'in need of moderator intervention' for flat-wrong or damaging).
• Click on the user and see if they are posting lots of similar garbage. If so, you just discovered a user that needs immediate action. Jump into chat and let the moderators know about that user.

Our goal should be to get these garbage-posters answer-banned --or perhaps completely user-banned-- and all their answers deleted. Rapidly!

It's up to the volunteer community to speed up the process and help the moderators by identifying obvious perpetrators.

It's up to the moderators to determine if their numbers are adequate to weather the storm (should it arrive), if their workflow to identify and handle garbage-posters rapidly is adequate, how they prefer communication of frequent-garbage-posters discovered by the community, and to communicate how else non-moderators can best help.

Answer 4

As is commonly the case, I probably got carried away in my other answer. Let's see if I can propose a simple policy for ChatGPT usage on Ask Ubuntu that could be expressed in the Help:

ChatGPT (and general AI tool) usage policy on Ask Ubuntu

• Users are not allowed to post answers directly generated by ChatGPT or other similar AI tools. Offenders may be warned, and repeat or frequent postings may result in a ban.

• If you are able to answer a question on your own, you may use AI tools to assist in wording or to suggest improvements to your answer. If you choose to do so, you should, to the best of your ability:

  • Confirm that the answer is correct and works
  • Understand the answer that you are posting
  • Correct any issues that could reasonably be identified
  • Communicate the answer in your own words
  • Reply to follow-up questions/comments about your answer
  • Warn of corner cases or potential issues if needed
  • Cite any reliance on third-party sources such as ChatGPT for the answer

• AI tools such as ChatGPT may be used for accessibility purposes such as voice-to-text transcription.

• AI tools may be used to suggest grammar improvements for your own posts and to make suggested edits to posts with poor grammar. However, as with any edit, take care not to change the meaning of another user's post.

_{^Note: ChatGPT was used to suggest slight improvements in readability and conciseness to this policy. Some suggestions were used, while others were rejected}

Answer 5

I've worked with ChatGPT yesterday again. Asked it about flask and plotly. The answers look good, but they don't work. It is completely ok to use it privately, but I do not want to investigate the "mostly correct but not working" answers in this forum. So I'm opting for a ban until the company worked out its flaws.

To sum it up: ChatGPT needs to be banned until it works

Answer 6

NOTE: find updates at the bottom of the post.

---

Having learned the views of others, I'd like to offer some degree of summary in response, and to deliver some new suggestions.

How to answer to the future, today?

I acknowledge the points of those who claim that AI is the future, and that denying it is futile — foolish even. I also sense the intent of a warning that advises us not to make similar mistakes to those who initially dismissed major, history-shaping technological advancements as unimpactful fads (only to be proven wrong by history shortly afterwards).

Yet, I argue, in itself, this insight is not a sufficient foundation for us to unconditionally embrace "AI produced content" on our website today. Our (e.g. those who engage with meta.askubuntu) main goal here is to be the custodians of a collection of reliable answers about Ubuntu.

The current state of available AI technology is not adequate to meet our requirement for factually consistently correct — and therefore safe-to-apply — information.

ChatGPT seems like an early proof of concept whose primary mission was to produce passable contributions to random conversations.

[ Aside: I have seen arguments that it's not even an AI: it's merely a machine learning model oriented at language. ]

Until an AI or equivalent technology is not capable of installing Ubuntu in a virtual machine and verifying the effects of its assertions, all it can do is to collect information from other, already existing sources — quite possibly including unverified ones — recombine it with unreliable degree of accuracy, and pass it on repackaged into unique linguistic constructs. This is no way to produce reliable answers for our Ubuntu-related questions.

A further, massive deal-breaker — as some of us had already seen — is when the machine grants itself creative freedoms in recombining and reshuffling software configuration options, regardless of whether these inventions correspond to existing specifications or not.

On these grounds, I don't see the place for the term "luddite" in our argument, for when someone expresses unwillingness to work with information of such low reliability. I believe no one is against technology here. We are against a barrage of useless lies / fiction that the current iteration of ChatGPT is capable of producing under the guise of reliable facts.

Seeing the dialogue in this thread, I am convinced that as soon as an AI will be able to produce truly and consistently reliable information about Ubuntu-related challenges, we will re-evaluate our position. (Impactfully, at that point, we also will have to face the question: what mission would keep this community (and numerous other online collaborations) working together from that point on.)

In the meanwhile, our burden is to keep this site free from misleading, potentially dangerous garbage.

How to maintain a reliable stock of information on AskUbuntu?

We are seeing in this thread that several people:

• are against a blanket ban, for various motivations
• are in agreement about the prediction that with the rapid evolution and refinement of the AI / ML technology, enforcing a blanket ban might turn out to be technically challenging

The following suggestions aim to acknowledge these positions & considerations, and suggest measures for a modus operandi where we have to live together with a constant influx of AI-produced content.

I offer the following suggestions; feel free to cherry-pick the most feasible ideas:

Inform proactively: announce a policy

We need to communicate our issues regarding low quality AI posts: this needs to be mentioned in the onboarding process and it needs its dedicated section in the Code Of Conduct.

Additionally, we could consider putting up small, unobtrusive banners where relevant (maybe accompanying the question- and answer submission forms).

Demand disclosure of origin for AI-generated posts

(Updated content)

As I have learned in the meanwhile, it is a general policy of StackExchange to demand disclosure of such material:

If it wasn't created by you, attribution is always required here.

Rely on community-delegated moderation

There is a threat that the volume of low quality posts could overwhelm our volunteer moderators. To relieve them, we should develop measures that could maintain order without needing explicit moderator attention for each individual case. This could be implemented through flagging, whose sufficient numbers could automatically lead to the removal of posts (comparable to how the "spam" flag works today).

Establish a new flag dedicated to the issue

Could be content-oriented:

Low effort answer with inadequate content

Or behavior-oriented:

For lack of a better term currently, until we find something better, I would suggest "meddling".

From dictionary.com:

verb meddling
to involve oneself in a matter without right or invitation; interfere officiously and unwantedly

We already have a behavior-oriented flag for spamming.

Now we could think about experimenting with a new, tailor-made flag for this new challenge:

[ Act of | A product of ] meddling

The poster appears to disseminate cheaply sourced third-party content while failing to demonstrate an ability to verify the adequacy of the information within.

Multiple posts being removed through flagging by the community could invite mod attention, where a temporary suspension could be used to discourage the unwanted activity.

Re-evaluate as necessary

Perhaps we don't get it right for the first try. Let's see how it goes, and, if under pressure, update the policies as necessity dictates.

At the end of the day, the ultimate goal is that AskUbuntu remains a reputable source of reliable information. We should maintain only such policies that do not endanger this primary mission.

---

UPDATE:

I maintain my suggestion to address the problematic usage of ChatGPT by a tailor-made policy and corresponding countermeasure.

This time, I suggest contemplating the usefulness of the terms "impostor" and "charlatan".

From dictionary.com:

noun impostor
a person who practices deception under an assumed character, identity, or name

"British Dictionary definitions for impostor":

a person who deceives others, esp by assuming a false identity; charlatan

From dictionary.com:

noun charlatan
a person who pretends or claims to have more knowledge or skill than he or she possesses; quack.

I opine, as long as the poster does not disclose up front that the core "value" of the post was generated through ChatGPT, their behavior amounts to acting both as an impostor and charlatan.

However, let's recognize that ChatGPT itself is acting as a charlatan. I interpret, that's at the core of our ban against it.

In that sense, could a single additional flag aimed at "charlatanism" cover the case?

I admit that I feel both "charlatan" and "impostor" are strong words, and carry heavy emotional weight on top of the intended correct semantical identification of the behaviors in question. Also, introducing them as-is could carry the risk of detrimentally impacting the community culture.

Nevertheless, I feel that the dictionary description of these words match the case we are experiencing.

---

UPDATE 2:

As I explored earlier, ChatGPT is not (currently) in a position of verifying the adequacy of its assertions through testing the validity of those on a live Ubuntu system (e.g. running one in a virtual machine).

Out of this circumstance, attempting to compile valid information — short of quoting snippets from successfully version-matched(!) documentation — is akin to how one sits in the casino in front of a slot machine with the spinning fruit symbols. Sometimes things end up aligning succesfully, oftentimes however not.

I'd like to point out that the above is not analogous to the contemporary machine-learning practice of trial and error, because trial and error — it seems to me — implicitly involves the verification/evaluation of the results, and the iterative development of the conclusion. That's fundamentally missing from ChatGPT's capabilities.

Therefore we could create a flag along the lines of:

Information produced through means of gambling.

Or, perhaps:

Information produced through means not accepted by this community.

Answer 7

I'm sure it is written somewhere but I cannot find it… I had an answer deleted as I cited my discussion with ChatGPT which helped me solve a problem I did not find the answer here or elsewhere. I wrote my question and pasted the dialog because I was thinking it would be very useful for others. And it seemed fair to acknowledge the source. I did not know the ban at this time. So, if this kind of thing happen again, am I authorized to write down myself an answer using the knowledge I gained from ChatGPT, knowledge that I empirically checked on my own computer?

Update After reading the first comment, I'd like to add my grain of salt. I consider myself that if a tool like ChatGPT gives a correct and hard to find elsewhere answer AND that the user verify himself or herself the correctness of the answer AND that he or she rewrite it from scratch with his or her newly gained knowledge, then he or she should be allowed to do so. BUT for fairness, he or she should be allowed to acknowledge the fact that the original answer came from the AI tool. Maybe a sentence like "derived from a human-checked AI generated answer" could be a good addition to this kind of answers. If something like that is not allowed, I think that there will be a lot of hidden AI generated content modified just to escape the ban.

Answer 8

Are we a bunch of Luddites?

Why not just ban spell checkers and any other computer aided technology?

ChatGPT is the future, we should practice it and learn to use it correctly, not hide from it.

Perhaps banning people that misuse it may be okay, but not those who use it for the better and to extend technology..

People can generate lots of good looking answers, that are not correct, without the need for a bot. I've seen it with my own eyes

In a year or so there may not be anyway to even detect if it was used.

Story

Back in first year Engineering, ~1974, I had a physics prof who made us use slide rules because she thought calculators were just a fad. At the end of the year we were pretty good with slide rules but most of us never used them again. Meanwhile the guys with more progressive profs were way ahead of us in the use of calculators.