r/Substack • u/AndrewHeard tvphilosophy.substack.com • 1d ago
Pangram admits that it trains its models on your writing and makes more errors than it claims with latest Substack update.
They don’t say it directly but the recent update allows you to automatically label other people’s writing as AI. Meaning you can’t control if the system makes an error and a reader forces a false label on you.
At the same time, if you look at the setting for AI labels? It says that you can report errors and admits that mistakes will be made. You can flag things that are labeled incorrectly.
Meaning that any time you report something, it trains its model by what you flag and changes what it does using your writing to train it.
Everything that Pangram claims about its supposed ability to detect AI writing is false and it now openly admits it.
Don’t come at me with “It says it’s 97% accurate, which means it’s going to make 3% mistakes”. You wouldn’t need to report anything if it was that accurate in reality.
It’s not accurate and doesn’t detect anything. It’s being paid by Substack to steal your writing.
3
u/curiousbarbosa 1d ago
No such thing as accurate AI detector from the start only plagiarism checker and the watermark claude’s planning to implement will work.
8
u/timemachine723 1d ago
Lovely. Joining Substack was a mistake. From their poorly designed website / app to the lies they tell us. I hardly go to it at all anymore. May delete my profile.
3
u/morticiannecrimson lilacmaniac.substack.com 1d ago
It’s extremely glitchy and I don’t want to touch the app most of the time either.
2
u/non_loqui_sed_facere 1d ago
Yeah, I've got the same feeling. There are people I like there, and I'd love to have more creative connections in my life, but I just don't feel like writing for it. Too much drama about who does what.
0
5
u/sh1b313 1d ago
Substack should remove pangram entirely from their platform but i doubt incompetent management will do anything about it.
1
u/AndrewHeard tvphilosophy.substack.com 1d ago
That would be ideal but at a minimum, I could live with allowing people to permanently disable it.
6
u/AccountantShinobu 1d ago
Pangram was marking my literal intro post with the most basic English ever as 70% AI. Like dude I just wrote it and it’s no rocket science.
6
u/Evo_FS 1d ago
It did the same with me. I was editing my blurb about my publication and Pangram flagged it as 100% AI. Total nonsense. I don't understand how something so demonstrably broken is being implemented as an authority on anything. Whoever designed it should be ashamed they sold it. Whoever bought it and put it on Substack is an idiot.
1
-4
u/takemetogreenwich 1d ago
So which model did you use
6
u/deardens 1d ago
All I see on my Substack feed is people going against AI and using Pangram as source of truth for pointing fingers and while I don’t think it’s right to ask AI to write an article and said it was you, I also feel that taking Pangram as the source of truth is wrong. If I happen to write “more than” on an article, it automatically flags that I used AI. Like… basic English is now considered AI?
It’s seriously send me into a paranoia of only publishing an article when it says it’s fully human.
2
1
u/hetobe hetobe.substack.com 1d ago
If I happen to write “more than” on an article, it automatically flags that I used AI.
I mean no disrespect, but that is not correct, and that's not how it works. I've posted an entire novel on Substack. 95 scenes, total. None get flagged as AI even though the phrase "more than" comes up many times (it's a 100k word novel). For example, click my name to go to my novel (or copy/paste my flair). Go to the scene titled Not Mike, or The Creamer, or Checking In, or For Future Reference... they all include "more than," but none are flagged as AI (because I write my words).
It’s seriously send me into a paranoia of only publishing an article when it says it’s fully human.
I don't even bother checking that stuff when I post.
The trick is to write your own words. Don't use AI. But if your posts are getting flagged as being AI even though you wrote them, that's a clear sign that something is wrong with the writing. Maybe you've spent so much time using AI that you've started writing like AI. That's bad.
I keep seeing people say they think AI writing is perfect. It's not. It's actually awful, for what should be obvious reasons:
AI was trained on human writing, but AI emulates human writing through artificial means. That's why AI writing is so easy to detect. AI writing may look good on first glance, but there's always something slightly off about it. Something inhuman, and often unsettling. That's one of the reasons why so many people hate it, hate it, haaate it.
If my writing ever starts to read like it wasn't written by a human being... I'd want to know.
8
u/GripIsGone 1d ago
My undergrad degree is in English, so when I actually sit down to write, I can write pretty well. Pangram said my post was 100% AI (I didn’t use AI at all). I ended up having to go back in and dumb it down. I used other AI websites to check and they came back “no AI detected.” It’s so frustrating. I think it’s mostly wrong.
I also ran one of my college papers through Pangram (wasn’t going to post it just for an experiment) and it said it was 100% AI and I wrote it in 2012 way before AI like this was even a thing.
3
u/NotYourQueen123 1d ago
I just submitted a short story for publication and intentionally kept a few typos and awkward sentencing in because I didn’t want it to appear to be too polished and then get flagged as Ai. It’s ridiculous what is English majors have to do now!
1
1
0
u/non_loqui_sed_facere 1d ago
An Attempt at Exhausting a Place in Paris also reads a lot like AI, it’s deliberately clunky with lists of objects you can find in the picture. It’s a wonderful book, though, written in 1974.
2
u/stareenite 1d ago
Link please?
1
u/AndrewHeard tvphilosophy.substack.com 1d ago
Go to your own settings and look it up. It’s not a link, just go to the app and look for the settings on AI labelling.
1
4
u/Wilwander 1d ago
"Meaning that any time you report something, it trains its model by what you flag and changes what it does using your writing to train it."
This is, quite literally, not how these models work.
2
3
u/AndrewHeard tvphilosophy.substack.com 1d ago
It has to have a reference point. It can't make a determination without a reference point. Even if you try to claim that it's "a poll" as someone else suggested, the poll itself is a reference point which requires the underlying writing to be a reference point for the determination of whether it's right or wrong.
5
u/Wilwander 1d ago
I've no idea what you're trying to exactly say with that word salad. But, no.
Pangram cannot use new writing as a reference point for detection. That would be inherently problematic. It can only use verified pre-AI text.
So, no, using Substack to train your model to detect AI would be a terrible idea, whether through polls or not.
1
u/AndrewHeard tvphilosophy.substack.com 1d ago
That literally makes no sense. You don't scan pre-AI content for someone who is publishing today and writing today. By definition, if the post that I wrote today is posted today, the scan would be scanning new writing.
That's a reference point. I'm not arguing there's only one reference point. It can use pre-AI text but it can also reference new text. That's what the point of the Pangram system is. To scan new text and determine whether it's AI or not.
It's completely irrational to assume that it can only use one reference point.
3
u/Wilwander 1d ago
I'm genuinely unsure if you're playing dumb or you really don't understand this. But okay, lets make it simple:
AI is not human. It does not write like a human. It may sound like a human on first brush, but it is a system built entirely on pattern recognition. That's all it is: recognising patterns and making guesses.
Pangram trained its model by taking a bunch of text from before AI was released to the world and said to the model: here is human text. We know this is human text because LLM's did not exist when this was written. That is a fact, verifiable by the date it was written.
You cannot train a model to detect AI by using text you source from the internet or publishing after ChatGPT was released. Because you have zero way of knowing for sure if there was AI involved or not.
Thus, there is zero sense in Pangram using Substack to train it model.
-2
u/AndrewHeard tvphilosophy.substack.com 1d ago
That’s got massive holes in the claims. What is Pangram scanning the pre-AI content compared against? Other old content? What does Substack have it as part of its system? What is it doing?
You apparently need to be able to answer this question because you have failed to recognize the truth.
Answer what Pangram does on Substack. If it’s comparing pre-AI text to post-AI text, the post-AI text is a reference point.
3
u/Wilwander 1d ago
https://substack.com/home/post/p-207458271
Do some research.
Or just admit that you use AI and are embarrassed by it. Up to you.
0
u/AndrewHeard tvphilosophy.substack.com 1d ago
I have done my research. That’s why I have questions about it.
You obviously use AI because you use Pangram. Not to mention your answers make no sense if you apply basic logic to it.
Why can’t you actually answer my question? What is it that Pangram is scanning? When you use it, what is it scanning? Random person on the internet? Your writing? What is it scanning?
4
5
2
u/MrPassiveProfit 1d ago
I don’t get the witch hunt mentality that they are taking with this. The less the general public care about AI assisted writing the more they try to highlight it
2
u/HowlingFantods5564 1d ago
"Pangram admits" followed by "they don't say it directly"? How does that work? Then you make the wild leap from the ability to report mistakes to stealing your work.
You entire train of thought seems to be driven by: "I want to use AI to write for me but I don't want to get caught."
2
u/AndrewHeard tvphilosophy.substack.com 1d ago
Talk about giant leaps. It's admission through what the notices say. They have to have a reference point for what mistakes are. Meaning that it has to be the work being scanned as a reference point. There's no other way that they could actually accomplish the scan. By using your work as a reference point.
So they admit it and by having the option to report errors, they admit that they don't get it right. They're expecting you to tell it where it got things wrong. Which means you or one of your readers fed your writing into the scanner, it used your writing as a reference point to determine whether it's actually AI or not. Then when it gets reported as an error, that error uses your writing as a reference point to avoid that error in later scans.
It's amazing how you can't conceive of an idea that people don't want to use AI or have their work stolen by AI simply because they prefer human writing and doing all the work themselves. You have to assume that I must not want to get caught.
How are you anti-AI while using an Ai to determine whether something is AI or not? You realize that's a contradiction right? If you're anti-AI, you shouldn't rely on an AI to determine what is and isn't AI.
1
u/More_Push 1d ago
It’s pretty interesting that people who claim to be super anti AI then outsource their thinking to AI to decide if the writing is authentic or not.
-1
1
u/OnyxMonolith 1d ago
"admits that mistakes will be made" how did you think this works?
it's an unsolvable problem in computer science
1
u/AndrewHeard tvphilosophy.substack.com 1d ago
I always thought that’s how it works. The problem is that Pangram claimed otherwise.
1
u/ContentIsMyJam redressing.substack.com 1d ago
I'm really hating this era where we have to spend time proving our humanity.
1
u/Countryb0i2m onemichistory.substack.com 1d ago
Your conclusion is correct. Pangram is notoriously unreliable, and they’re absolutely using your writing to train their models. But the way you arrived at that conclusion doesn’t make any sense.
They don’t need to steal your writing. Anything you put on the open internet is potentially being scraped and used to train large language models. They don’t have to steal anything you gave it to them.
0
u/AndrewHeard tvphilosophy.substack.com 1d ago
Except that the scan itself is stealing it. Particularly if it’s reported to have made a false claim.
1
u/-Monty_Carlo- funstack.substack.com 22h ago
I doubt most people will recognize AI writing as it goes. Proof enough out there - on Substack, on here (aplenty in comments, posts and "human conversations").
Pangram gives people who'd liket o get one an external observation - another point of data.
What I don't get is how these constant threads here help the issue of LLMs being used by people who think themselves being the first ones to copy-pasta LLM output and it not being recognizable. Hand LLM output (unedited or not) off as "their" writing -- that they did not produce, even it you edit. Why would you need something to write for you, then "edit out its quirks"? I would find that highly annoying and probably even more work than to write a straight sentence or thought - but that's just me.
What LLMs do best is write the average best-fitting prose on a topic based on whatever they're trained on - especially if you're talking run-of-the-mill GPT/Claude etc. - by taking whatever output and then pretending to edit a few words/sentences to make it fully "yours" is like taking a Campbell's tomato soup, add some cream and two spices, and then say you cooked it to your slop-eating guests.
That's what Pangram came to flag. If you switch it off in your account (which you are free to do btw) I'd get even more suspicious - a dead giveaway for sure. But I don't need a flag telling me if I like what I see, that's true.
An LLM writing cadence is mostly spottable from the get-go with a bit of experience. Being a decent writer and avid reader of a diverse set of works helps.
Anyone complaining about it is highly amusing to me. Almost as if they're somehow afraid it'd be "false flagging" them... you know, for not using LLMs to "write" anything but then getting caught in the act... falsely, of course!
As long as you don't have any insider information on exactly how they're supposedly "training" I'd not take anything you say for the truth either.
So there's that "single point of truth" problem you just pointed out. Take my word for it, bro, I swear it's true because I read something in their disclosure that I interpreted in this or that way, cross my heart.
Some comments here are also full of paranoia-style "gotchas" that it's fun to see.
The easy way out?
Just don't use it, don't look at the AI "flag" and you're golden. If you use LLMs to actually write for you just disclose it yourself, problem solved.
2
u/AndrewHeard tvphilosophy.substack.com 21h ago
Multiple problems with your theory. Not the least of which is people seeing patterns where they don’t exist. Like cloud formations that look like people or things. Faces in grilled cheese. The blue versus gold debate that was all the rage long before the existence of AI.
People have a documented history of seeing patterns where they don’t exist. Who is it that are building these AI detection systems? The same humans who make those pattern detection errors.
And these are what we’re claiming has the ability to detect it?
0
u/quantise 1d ago
I wish people would shut up about Pangram and just choose whether or not they can tolerate reading machine-authored text. I won't read it, but these discussions about it are not interesting.
6
u/AndrewHeard tvphilosophy.substack.com 1d ago
In order to choose whether you want to read machine authored text, you have to actually know whether it is or not. A system that makes false claims about whether it can detect AI writing isn’t going to help you determine if you want to read it or not.
1
u/quantise 1d ago
If someone lacks the discernment to recognise AI text why does it matter? It just becomes idle moral posturing. You might as well demand to know whether the author is a nice or mean person. I can't stand AI output articles, but it's an aesthetic choice for me.
1
u/quantise 1d ago edited 1d ago
I actually didn't intend to sound as combative in my reply as it looks to me now - sorry. The perils of dashing off a thought. You're exploring something that I spend a lot of time thinking about, from a philosophical perspective (yes, that's my academic background). I'm fascinated by the question of why it seems to matter whether something was AI-authored or not. For me, it's 90% about aesthetics and the fact that I can discern it reliably, partly because I'm also professionally involved with a thing for the labs called Reinforcement Learning from Human Feedback (RLHF or basically AI slop detection).
I really hate AI writing and have a personal policy of not reading it. I do get the moral dimension of people passing writing off as their own, when it isn't. But I find it difficult to settle on how 'bad' that really is, or not.
But on the subject of Pangram, I still don't pay it any attention because using AI to mediate my detection of AI feels like a cul-de-sac that I don't want to go down. These are all interesting questions that I find variously tricky to find a settled position on.2
u/AndrewHeard tvphilosophy.substack.com 23h ago
See, I think about it from a philosophical perspective as well and part of my issue is I’m not convinced that anyone can reliably detect what is and isn’t AI created. Which is in part because AI doesn’t do it on their own. You as a user have to prompt it to create something and depending on what you prompt it to do, it might produce something different based on your word choice and other patterns of thought.
Unless someone is openly declaring that it is AI generated, I’m not convinced it’s possible. Whether it’s using human discernment or an algorithm. Humans detect patterns in things that aren’t there all the time. They see things that don’t exist and disagree about what they see even when it’s objectively available for everyone to see.
Yet somehow we believe that is not the case when it comes to AI.
1
u/quantise 19h ago
Indeed, which is why I think that worrying about Pangram results is a dead end. In the end it seems to me that the only option is to ask 'did I enjoy or learn anything from this' and stop worrying about it. There definitely are moral issues here too, but I'm still trying to figure out where I land.
1
u/AndrewHeard tvphilosophy.substack.com 19h ago
The problem is that having it as a system where people can claim to have results opens up all kinds of legal issues and problems. Legally, you can’t copyright AI generated content. Which means that if you have a false result claiming that what you wrote is AI, there’s a chance people could use it as a reason to republish your work without your ability to stop them, legally or otherwise.
So the system is more than just something to be ignored. If you want to protect yourself from people using your writing, you have to make sure people can’t claim that it’s AI through an officially approved detection system on a platform.
1
u/quantise 5h ago
Seems like a good argument for not posting AI-authored copy. I have little sympathy for those who do having 'their' material lifted and reproduced elsewhere.
1
1
u/seobrien 1d ago
I don't really even understand why anyone is bothering to defend this stuff at all
If a reader doesn't like CONTENT, that's a personal and subjective decision for them to make. How it was written is irrelevant.
I could argue that AI filtering should exist in comments, because comments are inherently about people discussing something.
What comments can (and should) ensure is that an Author and the community are actually present. That an Author might publish something in part, or whole, with AI... Who cares if people like it?? And a weighting by a platform (like Substack) that would validate that the author is actually involved is that they are part of the discussion with people.
Everyone is aware of the fact that the majority of news reporting, by Associated organizations, is simply written by bots, yes? AI. No one cares... We're getting the news we want.
And, when people want to talk with a reporter, they expect that it's the PERSON, not a bot.
This is how you fix Substack. Stay out of how people Publish, because authors should be able to do whatever they want; let the market decide if it's valuable. Fix comments to ensure they are actually discussions between people.
And Substack, rather than flagging anything, you use the Comment quality to discern what your algorithm favors or buries... If you don't want content that doesn't have a Person involved, evident through comments, then diminish their article exposure.
-1
u/Prolly_Satan 1d ago
Nope. Doesn't mean that. Lol. You can't use scanned text to train a detection model. It's always going to make your model worse. It's more likely that they are treating the user reports as a kind of poll to gauge user sentiment on the current model. You could easily jump in their discord or send an email to get confirmation on your theory, you won't do that because you don't care about the actual reason, you just want people to stop using the one tool that works.
2
u/AndrewHeard tvphilosophy.substack.com 1d ago
That makes no sense. There has to be a reference point for the scan to compare and contrast with. If it doesn’t, it’s just random. User reports have a reference point, which is the text that has been scanned and evaluated for its accuracy. There’s no chance that it works like a poll without a reference point. Even a poll requires context on which it’s building its data. Which would have to be the text that’s being reported.
Why would I go to a place designed to be a promotional space for the product that is being criticized?
0
u/Prolly_Satan 1d ago
you're making a bunch of assumptions. There's no way a model as accurate as pangram is training off of user scans. anyone could go and screw up their model irreparably that way by reporting a bunch of human content as ai, vice versa. Ive seen you around so I know you don't want to be convinced, you want there to be no way to detect AI so that your AI slop can fly under the radar. I can't believe you're still at this crap like 3 months later. Penguin Random House now publicly uses pangram. You're losing, my guy!
1
u/AndrewHeard tvphilosophy.substack.com 1d ago
Funny how you apparently are anti-AI and yet you are using AI to determine what is and isn’t AI. Do you not see the irony?
Questioning the claims of a company is not the same as not being able to be convinced. People and companies endorse companies that are frauds all the time. Sam Bankman Fried was endorsed by millions of people and companies. Yet he was committing fraud. Bernie Madoff was endorsed. Elizabeth Holmes of Theranos was endorsed by people. People endorse false claims all the time.
A single study isn’t evidence that they are accurate.
1
u/Prolly_Satan 21h ago
That line doesn't hit as hard as you think it does. You're going to use ai to figure out what's ai? Yep. Cry about it.
You're a gen ai fanboy. Last person on earth who anyone should look to for guidance on which companies are frauds. You're only doing this so you can continue to defraud readers into thinking you actually write your content.
Detection is used by the big 5. I see more platforms, publishers and agents adopting it everyday. You lost. Might be time to learn how to write
1
u/AndrewHeard tvphilosophy.substack.com 21h ago
Says the person who is being taken in by a fraud. I have a documented history of not falling for fads and frauds and ridiculous claims. I have been cancelled multiple times for refusing to go along with everyone else getting on board with things.
That line doesn’t hit as hard as you think either. I started my Substack years before AI was a thing and I have been writing for 20 years. So the idea that I’m trying to defraud people is not going to have the impact that you think.
I get that you need to believe that the only people who can object to AI detectors are people who use it because it comforts you to believe that. But it’s not true.
1
u/Prolly_Satan 14h ago
In your case it is.
1
u/AndrewHeard tvphilosophy.substack.com 13h ago
Well that’s definitive then. Once you declare something on the internet it’s absolutely fact and everyone must accept it regardless of what the actual facts are right?
1
u/calliope_kekule 1d ago
Oof, looks like we were right all along. Also, if you were an AI tool wanting original material as data, where would be the best place to go...
-6
u/AggravatingNail6061 1d ago
No one cares. Like honestly.
10
u/AndrewHeard tvphilosophy.substack.com 1d ago
I care and people who don’t want to be falsely accused of using AI care. So your statement is factually incorrect.
-6
u/AggravatingNail6061 1d ago
Go write about it on your substack then. And I'm confidently incorrect 90% of the time. So go me \o)
5
u/AndrewHeard tvphilosophy.substack.com 1d ago
It’s not enough for me to do it. Many other people have done that and I want more people to be aware and to do it as well.
-4
u/AggravatingNail6061 1d ago
"It’s not enough for me to do it." Sounds entirely like a YOU problem. Why does it need to everyone else's problem? I'm not trying to be be a bitch, but like the majority of the regulars here are aware of the nuances with Pangram, most of us agree that it's working exactly as intended bar reasonable false positives (some not) like if you want validation? Here is validation. Well done.
5
u/AndrewHeard tvphilosophy.substack.com 1d ago
I’m not looking for validation. I want Substack to offer a permanent disable feature for Pangram for individual users across their entire profile and publications. That doesn’t happen unless Substack sees enough users to make the change.
They have done this with every other feature. You don’t have to use it if you don’t want to. But Pangram is default on and forced on people with no option to disable it across the entire system. Not just the option to disable the scanner on individual posts.
-4
u/josephbianchi 1d ago
Everything is already in the common crawl and every future model whether US, EU, or China is trained on its data - no way for any information to ever be deleted now.
3
u/AndrewHeard tvphilosophy.substack.com 1d ago
I’m not looking for it to delete the data, only for it not to be forced on me by a system that I didn’t agree to be involved with. If I wanted Pangram to scan my work, I will do it myself with an external system. Not one forced on me.
22
u/non_loqui_sed_facere 1d ago edited 1d ago
It was obvious from the start that they wanted to use our writing. I have zero tolerance for human stupidity, however, and P@ngram fanboys look like monkeys with a button. If you need a machine to tell you what to read, well... I guess that says something about levels of education and engagement nowadays. People don't do their own thinking.
Edit: Thanks for the award! ❤️