Skip to main content
Meta Stack Exchange

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

39
  • 124
    As a moderator on a smaller site, seeing the numbers you posted about the amount of AI-flagged posts is... incredible. Commented Jun 5, 2023 at 20:04
  • 62
    I've flagged quite a few ChatGPT answers, and sideshowbarker has indeed handled a huge proportion of them. From my end, I can also say I've never flagged based on any detection tool -- after you look at a number of ChatGPT responses, it becomes obvious what is ChatGPT (and if I'm wrong about this, I'd like to see examples of false positives). I guess SO's fear is that when users push back and insist "No I didn't use ChatGPT" then SO has no tangible proof that the user in fact used ChatGPT. Commented Jun 5, 2023 at 20:14
  • 18
    This answer is awesome. Great read. One could possibly create a training set for humans to learn to recognize AI generated content. Commented Jun 5, 2023 at 20:43
  • 42
    It would be interesting to set you up in a blind trial and see what your accuracy rate is on content known to be human generated vs content known to be AI generated. I'll bet you fare way better than the detectors they are calling out. Commented Jun 5, 2023 at 20:55
  • 36
    @StephenOstermilleronStrike IMO this is a big bit of what SE is missing in the current moment. Many times, both privately and publicly, mods have asked staff to run a trial like that. No such trial of moderators' abilities has been conducted. Commented Jun 5, 2023 at 20:57
  • 42
    "[...] striking SO mod who handled 10000+ GPT flags" - I knew this was you immediately 😅 Like tdy, you handled all (or most) of my ChatGPT flags, that also didn't use any tooling, but a combination of gut feeling, previous post history (if any), timeline of activity (multiple multi-paragraph posts in the span of a few seconds), etc. I'm glad to hear you (and I hope other moderators) used your best judgement and not some tool, and wanted to take a second to thank you; since ChatGPT came onto the scene, moderation here has probably been... difficult, but we, the users, appreciate you 🙂 Commented Jun 5, 2023 at 21:57
  • 12
    I- like tdy- never used ChatGPT scanners / detectors. When I flag something I think is ChatGPT, I just use my own sense and subject-matter expertise. Commented Jun 6, 2023 at 0:22
  • 5
    After sleeping over it: it could still be that AI scanners indirectly played a tiny role here. After all you say that many flagged posts you trained on included a score in the message. One could assume that the flagged posts were selected partly by scanners, which means the composition of the training data is affected by AI scanners. Another thing is the ground truth. How does anyone know the ground truth here unless it's a test like proposed by Stephen Ostermiller in a comment above? I mean where does the "very high level of confidence" come from if the ground truth is not known? Or is it? Commented Jun 6, 2023 at 6:36
  • 9
    FWIW, I have flagged hundreds of AI posts on SO and always detected and flagged those posts solely using my own brain, not tools. Occasionally, I have used detection tools, but only to see how well those tools work rather than means of detection or confirmation. Commented Jun 6, 2023 at 8:13
  • 11
    Like for regular plagiarism, I flag ChatGPT posts when I encounter them by chance (I didn't hunt for them (past tense intentional)). They are easier to spot than regular plagiarism due to ChatGPT's very distinct writing style (for instance, the echoing back of the question (in a slightly different form)). I have never used any tools for this, only my own brain. And I always look for supporting evidence before I flag. And thank you, sideshowbarker, for handling all those flags! The word has become devalued by overuse, but it is very much appreciated. Commented Jun 6, 2023 at 12:13
  • 15
    I have submitted numerous ChatGPT flags that were promptly and effectively handled by sideshowbarker. I feel we've worked together effectively to help protect StackOverflow. Now I will join the strike by ceasing all flagging and letting the site stew in its own juices. Commented Jun 7, 2023 at 13:19
  • 8
    I submitted many chatGPT flags that were handled by you. Although I used the detection tools just to see how they worked, I never gave them any credence. Firstly, it's easy to recognize when something likely had its start with chatGPT. It jumps out at you. Then I'd check their answer history, frequently there is a gap with mostly monosyllabic answers followed by more recent wordy flowing chatGPT-looking answers. Finally, I actually ran the questions through chatGPT, and only when one of its answers was almost verbatim copy-and-pasted did I flag the answer. No witch hunt here. Commented Jun 8, 2023 at 0:28
  • 7
    @AdamRubinson Here's one example. Feel free to have a look at the author's previous contributions and take an educated guess at what might have happened in the 6 months since their last post that turned them from someone that doesn't care about grammar, punctuation, or formatting into someone capable of writing perfect English text, perfectly paced and formatted, not missing a single punctuation mark. Commented Jun 11, 2023 at 12:47
  • 12
    @TomWenseleers Your examples aren't relevant to the ChatGPT policy. We've always been allowed to use CGPT, test its output, and write a verified solution based on that output. The ban/strike is about thousands of users copy-pasting CGPT output and basically spamming unverified junk. Commented Jun 12, 2023 at 23:37
  • 13
    We're on strike because we want to be able to keep doing this, to keep making the sites better, @dougp. If we're not allowed to do that, the next step will be quitting altogether, but we thought we should at least try some last-ditch efforts before abandoning ship altogether. We're not concerned that we'll be replaced with moderator bots. They might as well replace us with random number generators or nothing. At that point, the site will be dead anyway. We're attempting to head off the site we know, love, and have poured thousands upon thousands of hours into over the years from collapsing Commented Jun 14, 2023 at 11:50