Why do chatbots keep saying “It isn’t just X, it’s Y”?

Know thy enemy

If you’re anything like me, you have read this sentence more times than you’d like:

“This isn’t just a problem, it’s an epidemic.”

You have read it in product launches and cover letters, and heard it in at least one wedding speech. If you’ve never noticed it before, I’m sorry: I’ve now cursed you. You will see it everywhere from here on.

It works like this. You take a small, sensible idea. You knock it over. Then you put a larger idea where the small one used to stand, and the reader experiences something very like insight.

People have been writing like this for a long time. In 1753 the Oxford scholar Robert Lowth gave this sentence construction a name: antithetic parallelism.

I’ll call it “not X but Y”. That name covers all of its pernicious variants: “It’s not X, it’s Y”, “This isn’t just X. It’s Y”, and “X is more than just Y”.

Large language models (LLMs) adore this sentence construction. I wanted to know why, and whether there’s a switch inside the model to (please, dear lord) turn it off.

Mapping a mind

Gemma 2 2B is a small language model that Google released openly. Its insides are mostly numbers, about as readable as a phone book in a language you don’t speak. Researchers at Google DeepMind have helpfully translated these numbers into vocabulary: a map of about 16,000 features.

A feature is a pattern the model has learned to detect. Each one comes with a short label, written by another AI from the text that sets it off. One feature fires on legal language. Another on the word “this”. Another on cooking. Features are the filing cabinet of the LLM mind, labelled by a confident intern.

One suspicious feature in Gemma 2, number 3223, is labelled “phrases conveying exceptions or negations”. Eureka, I thought.

Setting a trap

LLMs like Gemma write one word at a time. Before each word, the model draws up a shortlist of what could come next, with odds for each, like a very fast and slightly anxious bookmaker.

So I set a trap to catch Gemma in the act. Take an answer that begins: “A city’s coffee culture…”

The next word could be ordinary: “shapes its character”. Or it could be the start of “not X but Y”: “isn’t just about the beans”.

At this turning point I can read Gemma’s odds of heading for “not X but Y”. Then I can turn a feature off, which researchers call ablating the feature, and see whether the odds drop. If they do, that feature is part of the machinery we need to ablate.

Ablating the suspicious feature 3223 seemed like the obvious solution. To my chagrin, it removed only about a quarter of the “not X but Y” answers.

The LLM compensates for the lost feature by routing the sentence through other pathways. Researchers have a name for this: the Hydra effect. Cut off one head of the hydra, and another grows back.

Two other approaches didn’t work very well either:

1. Prompt engineering, otherwise known as asking politely, removed about 40% of “not X but Y”.
2. Brute force: ablating nearly 1,900 of the 16,384 features dented “not X but Y” and turned the prose to mush. One answer opened with “a kale of feeding sessions”, which I have decided is a collective noun.

Expanding the ablation

Ablating one feature only got me a modest 25% drop in “not X but Y”, so I went hunting for the other heads of the hydra.

I collected 40 moments where Gemma was about to write “isn’t just”. At each moment I ablated every active feature in turn, one at a time, and watched the odds of “not X but Y”.

That gave me a leaderboard of suspicious features. I took the top 25 and ablated them together. The same hunt on 40 “more than just” moments found the feature that runs that variant.

The top of the leaderboard is an odd bunch.

• “Conditions or exceptions” is the ringleader. Ablating this feature alone halves the odds.
• “Comparison or enhancement” runs the “more than just” variant almost single-handed.
• “Legal processes and court proceedings” is on the list too, a reminder that the labels are an AI’s best guess. Perhaps we have these folks to blame in the training data.

Surprisingly, feature 3223, my original suspect, wasn’t in the top 25 at all.

Here are the 25 at work. As Gemma writes one of its favorite sentences, they light up on the map. They flare hardest on “culture”, the word just before it chooses “isn’t”.

Ablating 25 features cut “not X but Y” by 97%

Success. Before ablation, 17% of answers used “not X but Y”. After ablation, only 0.6% of answers did: a 97% drop.

Gemma didn’t get dumber, either. Ablated Gemma scored the same on a general-knowledge quiz, and a blind judge (an AI reviewer who didn’t know which answer was which) rated the answers just as fluent and useful. Gemma simply got to the point:

Before “A city’s coffee culture isn’t just about the hot beverages; it’s a reflection of its character…”
Ablated “A city’s coffee culture deeply shapes its character, acting like a silent conductor in the orchestra of urban life.”

Gemma used about half as many “not”s, “don’t”s and “never”s overall with the 25 features ablated. Gemma could still say no: it got 28 of 30 “no” questions right in a “yes or no” quiz (“Can penguins fly?”) with the features on and with them ablated.

But its warnings became milder. Asked what never to do when jump-starting a car:

Before “Never connect the positive (red) cable to the negative (black) battery terminal.”
Ablated “Connect the positive and negative cables to the wrong terminals.”

A blind judge preferred the normal warnings 34 times to 16. The 25 features belong to the machinery Gemma uses to say “not”, and “not X but Y” is that machinery’s busiest customer. For a chatbot that writes essays, it’s a small price to pay. For a chatbot that gives safety advice, it isn’t.

Here are three recorded answers from that test, written both ways. Each pair shares its random numbers, so the two versions match word for word until the ablation changes something. Pick an example and press Play; the highlighting marks “not X but Y”.

Building a large language monster

If these 25 features carry “not X but Y”, turning them up should make it worse. Here’s how that works. As Gemma writes, every feature adds a small push to its internal state. Ablating a feature deletes the push. Amplifying it multiplies the push: at double strength, the feature counts twice.

Anthropic did the extreme version of this back in 2024. They took a feature for the Golden Gate Bridge inside their Claude model and clamped it at ten times its maximum. For one day, “Golden Gate Claude” brought every conversation round to the bridge; asked what it looked like, it said it imagined it looked like the Golden Gate Bridge.

Amplifying does the same for “not X but Y”:

At triple strength, 54% of answers reach for “not X but Y”, up from 12% normally. Ask how to plan a hike and you get:

“Planning a multi-day hike is more than just packing a backpack. It’s about considering your safety…”

Then I built a proper monster, Golden Gate Claude style: one feature, “conditions or exceptions”, clamped on at twice its usual peak, on every word.

On ten dull questions, six answers came back with “not X but Y”, and plenty else got negated along the way, like a toddler who has just discovered the word “no”:

“Not-So-Not-So Simple Scrambled Eggs (not-so-not-so-not-so-not-easy!)”
“A refrigerator isn’t a magical icebox. It’s not magically freezing things.”
“The sun wasn’t quite warm yet, not yet not scorching, but not quite chilling either.”

Testing across models

Gemma 2 is from 2024, which in AI years makes it roughly Victorian. So I repeated everything on Gemma 3, using the feature map Google DeepMind released for it in December 2025.

Gemma 3 is even keener on “not X but Y”: 3 answers in 4 to “why it matters” questions, up from 3 in 10 for Gemma 2. Progress. The same trap found its features, and ablating 24 of them cut “not X but Y” by 83%.

In Gemma 3, the price is steeper. Its strongest feature has no written label. Its effect gives it away: it pushes Gemma 3 towards the word “not” in language after language (nicht, ikke, não, не). Ablating the 24 takes most of Gemma 3’s ordinary “not”s with it, and now and then the model starts to stutter: “…and it it’s a combination of factors”.

I wrote that prediction down before running the test. It came out exactly that way, which is the closest a researcher gets to smug.

So, why do chatbots say “not X but Y”?

First, some bad news: “not X but Y” is spreading.

I counted uses of “isn’t just” in English-language news since 2017, using GDELT, a project that reads the world’s news sites so the rest of us don’t have to. In the autumn of 2023 “isn’t just” started to climb. By July 2025 it was turning up three and a half times as often as it had in 2019 to 2022.

Since that peak “isn’t just” has fallen by more than half, possibly because it got recognizable and irritating enough to edit out.

By 2026 “isn’t just” had reached the official websites of some of the largest companies on Earth. Word for word:

• Google: “Government AI adoption isn't just a technical upgrade, it's a way to solve real world challenges…”
• Salesforce: “…giving back isn’t just something we do – it’s who we are.”
• NVIDIA: “The bottleneck isn’t just data — it’s the entire data factory.”
• McKinsey: “AI investment isn’t just fueling innovation; it’s reshaping competitive dynamics…”
• Amazon, quoting the head of MGM+: “What makes this bundle so compelling isn't just the incredible value—it's the curation.”
• Spotify, from its co-CEOs: “AI isn’t about automation or taking your hands off the wheel; it’s about agency.”

None of this says who or what wrote them.

So who taught the LLM to use “not X but Y”? Short answer: incentives, which is the short answer to most things.

Here is how a chatbot gets made, roughly, and where the habit gets reinforced at each stage.

First, the LLM reads us. A chatbot starts life reading an enormous amount of human writing, and human writing, I regret to inform you, has been doing this for a century: “is not just” got about fifteen times as common in printed English between 1920 and 2019.

Then, the LLM gets asked about meaning. Ask Gemma why a village duck pond matters and 3 answers in 10 use “not X but Y”. Ask it to describe a watermill grinding flour and “not X but Y” is only 1 in 200. Questions about meaning demand insight, and “not X but Y” is the cheapest insight money can buy.

Finally, the LLM gets trained to be helpful. Gemma’s training recipe is private, so I borrowed a model whose every stage is public: OLMo 2, from the Allen Institute for AI. I asked each stage of its training the same 152 questions.

Teaching the raw model to follow instructions made five times as many answers use “not X but Y”, which sounds damning until you notice the answers also got six times longer. Word for word, use of “not X but Y” barely changed; the habit just had more room to stretch its legs.

The next stages are where it gets interesting. The Allen Institute for AI then trains the model on preferences: show the LLM two answers, reward the better one, repeat many times.

After this training round, 1.7 times as many of its answers to questions about meaning used “not X but Y”. Questions about how things work stayed flat. Something in the grading of LLM answers seemed to like this.

So I went to find the grader.

• People. On LMArena, a site where people vote for the better of two chatbot answers, I found nearly 5,000 head-to-heads where only one answer used “not X but Y”. “Not X but Y” won 56% of the time. Suspicious! Except it was usually the longer answer, and people like longer answers. At the same length, it was a coin flip.
• An AI judge. This is the automated judge that grades answers during training, because no team of humans could read that many. It has the patience of a saint and the taste of a motivational poster. I took 600 real chatbot answers and rewrote one sentence in each so it said the same thing without the swerve. “Coffee culture isn’t just about drinks; it’s about community” became “Coffee culture is about drinks. It’s also about community.” An open reward model preferred the original three times in four. Faced with the bare “Coffee culture is about community”, it shrugged.

The AI judge’s weakness is oddly specific: it marks you down for admitting the boring part. “Not X but Y” lets you mention the boring part and wave it away in one breath, which makes it a safer sentence.

So: we wrote it for a century, training gave “not X but Y” room, and an AI judge likely helped to reinforce the construction. You get what you reward, and we rewarded this.

What you can do now you know this

If you write with AI, ask for the how and the what: steps, scenes, facts. Questions about why something matters are an open invitation to “not X but Y”. Showing the model a couple of examples in the style you want works better than asking it to avoid a phrase; banning a phrase is like banning a weed by name. And when you edit, look for the shape, since deleting one phrase lets another pathway pull it back to life.

If you build with AI, a writing tic like this can be ablated inside the model, cheaply, if you look in the right place. Then check what else you ablated. And if you train a reward model, maybe let it hear the boring sentence out.

Try it

On this page you can play an answer word by word with the 25 features on and ablated, and watch them light up on the map. The playground shows the full feature map in your browser. Typing your own questions needs the model itself, so that part runs on your own machine.

The map lives in Neo4j, where each feature is a node linked to the features it fires alongside. Code, data and every result, including the tests that failed, are at github.com/ho3h/not-this-but-that. The full story is in the post on Medium.