Substack Thinks You Are an AI

Why Substack's new AI detection tool is palmistry disguised as math.

I’m an AI.

At least according to the score that Pangram gave me. I signed up months ago, back when it was supposedly the best detection tool for AI writing.

The scores seemed credible… until I ran some experiments. I entered a mixed text. Some written by AI, some written by me. Pangram correctly identified it as mixed. But now the weird part. It flagged the section I had written myself as AI-generated. WTF?

But it got worse… I entered a text that was 100% generated by AI. The result, according to Pangram? 0% AI.

This is palmistry disguised as math.

“Look, it says 40% AI-generated.” I filed it under “Classic AI B.S.” and forgot about it. Then Substack launched a new feature called “Scan for AI.” You can scan any post and get a score from 0 to 100 estimating how much of it was written by AI.

The response to the new feature was overwhelmingly negative. At least from what I saw. I was a bit surprised because at first glance it seemed like a good idea. Substack is trying to protect itself from the flood of AI-generated slop.

So far, so good. The problem lies in how they tried to approach it. They partnered with Pangram to provide the scores. The same Pangram.

But do we really need a detector to tell whether it is AI-written? I mean, does it even matter? I think it‘s pretty obvious when writers use AI. Sometimes you can even tell which model they are using.

Don’t get me wrong. I don’t think it’s “bad” to use AI as a tool for writing. But if your whole essay smells of AI writing, I’m gone. What really gets me are the tics.

“It’s not… it’s…”
“The problem is structural.”
”In today’s fast-paced digital landscape...”

This drives me crazy. I just can‘t read that kind of writing anymore. The content might be good, but it has zero taste.

AI detectors scan the output. But the real loss already happened in the process.

Writing Is Supposed to Suck

I have a love-hate relationship with writing. Maybe it’s because I haven’t developed my skills to the point where it is all flow. Most days when I sit down to write, I would literally do anything but write.

I would even go so far as to say that writing is supposed to suck. By that I don’t mean that all writers are masochists… well, maybe a bit. My point is that sometimes your writing just flows, but most of the time it doesn’t. There is always an urge to procrastinate. Most people give in to it. Serious writers have learned to deal with it.

Most of writing is actually thinking. And thinking deeply is what makes writing so hard. We try to avoid anything that is hard. So we still write, but we skip the thinking.

It’s very easy to use AI every time the friction of thinking deeply arises. Even though there is a “Thinking Mode”, AI can’t replace your thinking. You have to sit with that discomfort. That’s why I decided to mostly remove AI from my writing process. I use it to do some light refinements and final editing. I only allow myself to use it when I feel I have gone through enough pain to reach the core of my idea. (Hence the masochist part.)

How To Screw Up Your Writing

There were many times when I wrote a crappy first draft. Things more or less flowed. Then the friction arose, and I switched to AI. I turned it into an outline and refined the draft. It seems innocent, but at least for me, it destroys the creative process necessary to come up with something original.

After hitting publish, I realized that I wanted to focus on something else, and that my core argument had been lost along the way. That’s because I used the first thought I had and put it into AI. The output looked good, so I published it. Nothing was wrong with it. But what I actually wanted to talk about got buried in just one section. I hadn't done the part that comes before. Sometimes hours of staring at the screen or days of not writing more than a crappy sentence.

It seems like wasting time, but that’s what the creative process looks like.

And that’s the ironic part. We call ourselves creators, but we try to skip the creative process. Yes, it is uncomfortable, but there is another reason so many creators reach for AI. This one is external. The platforms put pressure on you to be “consistent.“ The creative act knows nothing of time, nor does it care.

If you are constantly working against a deadline that doesn’t give you the space to think, you are robbing yourself and others of the greatest gift you have: the thing only you can create. That’s how you end up being a content creator. Or, put differently: you are an employee for the algorithm.

Creating content is a job.
Creating your life’s work is a calling of the highest order.

The Best of Average

The internet is overflowing with content. Almost all of it is mediocre. AI will raise the bar because suddenly everyone can produce good content. But all we get is better mediocrity.

If you are playing the content game, you are constantly under pressure to create more and better content. The case for using AI becomes even stronger. Everyone is doing it, so you have to use it too, just to keep up.

You might be a prompting genius, but AI by its nature produces average output. Running the same prompt twice produces a different outcome. Do it a hundred times and you will see a pattern emerge. AI predicts the most likely continuations, so it gets pulled toward an average. You are producing average crap, just a bit better average crap than the other guy.

AI models are trained on huge amounts of data. That data is a reflection of the past. If everyone is using it and it is trained on the past, you are in fact running behind by relying on AI.

It’s tempting to play that game. It feels safe. Creating something original, something from your unique experience, feels much more risky. There is no guarantee that people will like it.

Conformity feels safe because it is. Creating something unique means being alone with it.

You Can’t Measure Depth

Before there was AI, people would use templates or copy the style of popular writers. The underlying pattern hasn’t changed much with the arrival of AI. Just more people doing it.

So it isn’t really about AI.

I remember I had swipe files for copywriting. I collected what I liked and what counted as best practices. Best practices are the result of what was measured.

“What gets measured gets managed.”¹

But how do you measure depth? I have no idea and neither does Substack. So the metrics show what’s easy to measure. With the new Pangram feature, it was supposed to be easy to tell whether you use AI. Even if the detection worked perfectly, it wouldn‘t matter. It only distracts from the actual problem.

What gets mis-measured gets mis-managed.

We use proxies for depth and end up optimizing for something else (likes, views, restacks, etc.). Since those are the only metrics you see, it‘s perfectly rational to focus on the ones Substack shows you.

It leads to conformity. And that is really the problem we should be focusing on. There is no score that can determine whether your work is original.

Be Personal, Not Different

If you are playing to conform, you play the same game as everyone else. It’s the default mode. It’s easy. And it’s a race to the bottom. You are playing the game of the content creator. Since AI has lowered the barrier to entry dramatically, you are now competing against everyone who has access to it.

We tend to confuse productivity with creativity.

The creator economy currently punishes creativity. We have been trained to be productive. That’s why many creators have to re-learn to be creative.

If you want to stand out from the sea of sameness, you have to risk doing something different. Actually personal is the better term here. Personal is not just inherently different, but also unique. Nobody has your lived experiences. Nobody can think like you. Nobody can write like you.

That’s why I don’t want to read obvious AI writing. Anyone can write like this. Taste is personal. It’s what turns content into art.

Creativity is about expressing who you are first. Standing out and getting attention are only consequences.

What Substack should be doing is fostering a culture where artists can be creative. Focusing on arbitrary AI scores, likes or open rates doesn‘t help. If Substack itself doesn’t have the courage to stand out and do something different, we end up with a place that is just like every other social media platform. A community for creative people should be creative.

Creativity is a way of being. We are always creating.

The question is: are you creating content or are you creating art?

---

P.S. Your business should be a reflection of your life’s work, not a prison of your own making. You have to know the difference between the performance you give for the algorithm and who you truly are.

I created The Archetype Navigator to help you with that. It’s free and takes less than 5 minutes to uncover your creative patterns, so you can build a body of work that actually looks like you. It might save you writing the wrong article.

1. There‘s no evidence Peter Drucker ever said this.