11

AI generated answers are banned from Ask Ubuntu.

But how can I recognise them?

2

4 Answers 4

Reset to default
9

I'm using this online ChatGPT detector:

https://openai-openai-detector.hf.space/

Just paste the question/answer, and it gives you a really good indication.

There's also a Stack Exchange script, that can be integrated that does the same thing:

https://stackapps.com/questions/9611/openai-detector

6
  • Maybe stupid question how can I install the app from stackapps?
    – nobody
    Commented Jan 25, 2023 at 10:56
  • 2
    @nobody you install the ViolentMonkey extension for Firefox. Then, directly from the extension you can add a script from the source link (I haven't tested this, so I can't give more detailed instructions).
    – Artur Meinild
    Commented Jan 25, 2023 at 11:14
  • Thank you got it work.
    – nobody
    Commented Jan 25, 2023 at 12:15
  • 1
    @ArturMeinild I see that as of yesterday the OpenAI people have launched a new detector that they are calling an AI Text Classifier.
    – andrew.46 Mod
    Commented Feb 1, 2023 at 3:33
  • I find @andrew.46 and your classifiers to be inaccurate. I tried them on my answers (which are not AI-generated of course) and got an 80% ai result. On the newer generator, I got most likely to be AI-generated. This might create confusions.
    – Error404
    Commented Feb 12, 2023 at 11:57
  • @AlwaysAvailable I have looked at your answers on Meta and AU main site and I see nothing that tests positively for AI? There are definitely some false positives with all of the online testers but these are only part of the tests for ChatGPT abuse.
    – andrew.46 Mod
    Commented Feb 13, 2023 at 6:47
7

There's a very common "style" to ChatGPT responses1. If you start to use ChatGPT yourself, you'll quickly become familiar with it. And ChatGPT can be a useful tool, if you keep in mind its limitations, so I do recommend that you try it, especially while it is free.

If you do suspect that an answer is generated by ChatGPT, you can then run it through the detector that @ArturMeinild mentioned. However, the detector is not nerfect (pobody is), and can be (and has been) fooled.

Other indicators that increase the odds of an answer being from ChatGPT are:

  • A new user who posts a "professional looking" answer as their first post. That's not to say we haven't had some stellar first answers here, but for most users it takes some time to become really good at taking the time to fully read and understand the question, and to learn how to write a really good answer.

  • An answer that mentions a particular setting, with a comment from the OP that they couldn't find that setting. ChatGPT is really bad (or good, depending on how you look at it) about just "making up" settings or features that don't exist. It appears to hate to give a "There's no way to do that" answer.

  • A new user who posts multiple answers in a short span of time. These answers may not even all be here on Ask Ubuntu. Often, the user will create multiple answers across multiple Stack sites in a short span of time, on a wide-range of topics. So if you suspect that an answer is ChatGPT based, look at other answers from that user, both here on AU and other SE sites.

    I came across an answer here on AU just last night that "sounded" ChatGPT'ish, but actually came up as "98% real" on the detector that @ArturMeinild mentioned. But on further examination, I found 22 answers that the user had posted in 7 hours across more than a dozen Stack sites. Fortunately (for purposes of "proof" at least), all of the other answers did test at a nearly 100% GPT rate or highly so.

  • An answer that doesn't seem to actually answer the question that was asked, or skips key details from it. Often, to get a "focused" answer, someone will just copy the question title (or a subset of the question) into ChatGPT. The answer that is posted will seem "off", if you read it in context of the question that was really asked.

None of these absolutely means that the answer is ChatGPT-based (or AI-based), but each warning flag increases the odds. And it's highly unlikely that a user is going to post just one ChatGPT answer. Over the long term, I'm hopeful that we'll be able to spot the patterns above (and more) across multiple answers.

1 There are a few frequent users here who actually have that same style (and have had, since well before ChatGPT). @karel, @rinzwind, and @NotTheDr01ds (err, yes, that's me) all often have a similar style to ChatGPT. However, we always (in my experience) pass the "real" test on the Detector.

At this point, I commonly scan through the list of new answers in chronological order. I can often spot potential ChatGPT answers just from the first few lines in the preview. However, most of the time when @karel or @rinzwind answer, it often "triggers" my sense of a ChatGPT answer. But it's clearly not, of course.

I've even started to read some of my answers when it took a split-second for me to realize that I'd written it, even though it initially sounded ChatGPT'ish.

4
  • 1
    For fun I throw a part of you answer in open-ai-detector and you are to 98.61% real. Don't worry. :D
    – nobody
    Commented Jan 23, 2023 at 16:13
  • @nobody But I asked ChatGPT about my answer, and it told me that it was 95-99% likely that I'm an AI ... (Seriously, I did this on some of my answers). It's giving me a bit of a complex ;-)
    – NotTheDr01ds
    Commented Jan 23, 2023 at 16:16
  • It's not that easy. I feel lost about that.
    – nobody
    Commented Jan 23, 2023 at 16:23
  • 2
    This writing is superior to AI, and here are my indicators: Your grammatical choices aid understanding. You use passive voice appropriately. Your paragraphs are discrete subjects and flow sensibly. Your "other indicators" list of examples are actually examples. Result: Your prose is easy to read, and does not seem pompous, condescending, or confusing.
    – user535733
    Commented Jan 24, 2023 at 10:49
1

A well known user on the SE network, Glorfindel, created a userscript that provides a "Detect OpenAI" link to posts, that when clicked runs the post through OpenAI detections, using Hugging Face's AI detector.

I have a few non-free solutions I pay to use in addition to Glorfindel's script, like Originality.AI which allows me to copy/paste questioned content into there for analysis on plagiarism, AI generated probability, and other functions unique to that system. It's not free though.

1

OpenAI (maker of ChatGPT) has a classifier they currently call a work in progress that you can use for this purpose.

1
  • Oh they try to fix a problem they caused themselves in a sense.
    – nobody
    Commented Feb 2, 2023 at 20:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .