Europe has required AI-generated text to be marked so a machine can recognise it since this summer. OpenAI shipped its text watermark, textGrain, in October, and the first tests say what anyone who has built a text pipeline already suspected: the mark survives a blank page, but it does not survive someone editing the output.
What happened
Article 50 of the EU AI Act started to apply on 2 August. It obliges anyone offering systems that generate synthetic audio, image, video or text to mark the output in a machine-readable form, so it can be identified as artificial. The quality bar is left deliberately elastic: the technical solution has to work, has to be understood across different systems and has to hold up, always within what is technically possible. No threshold, no percentage, no reference test.
textGrain lands in that gap. It works like other text watermarks: the model picks the next token — the next word or word fragment — from several candidates, and a secret key nudges that choice pseudo-randomly. The result is a statistical pattern that a detector holding the same key can look for. What is distinctive about textGrain, according to Xiang Li, a postdoctoral researcher at the University of Pennsylvania and co-author of the technical report, is that it treats entropy — how much variety is allowed in word choice — as a budget the developer sets up front.
The figures published by IEEE Spectrum are these:
- On psychology questions from the ELI5 dataset, the detector finds the mark in roughly 80% of 200-token passages (about 150 words), and in 95% of 400-token ones (about 300 words).
- In 400-token passages, replacing just 10% of the words with synonyms drops detection from 92% to 66%. At 25% of words replaced, it falls to 17%.
- The false-positive rate is set at 1%: one in every hundred unmarked texts gets flagged by mistake.
And one detail that matters: the mark depends on the preceding tokens, so touching a single word does not break one point of the pattern, it breaks several.
Why it matters
If your product generates text with a model and serves it in Europe, the practical reading is uncomfortable: the scenario where the mark works is precisely the one that almost never happens in production. It needs long, untouched text. Real products trim, summarise, drop content into templates, run the output through a second model to adjust tone, and let the user edit before publishing. Every one of those steps eats signal. No attack required: a normal human review of a 300-word email already changes more than 10% of the words.
Two concrete decisions follow from that.
First: do not build anything on top of a detector. No internal flows along the lines of "if the detector says AI, it gets reviewed", no HR policies, no content filters. At a 1% false-positive rate, once you push ten thousand texts through the detector you have a hundred false accusations, and each one costs you a very unpleasant conversation. A positive does not tell you how much the human wrote either.
Second: provenance is recorded at source, not guessed afterwards. In the processes we build around models, what gets logged is which prompt went in, which model answered, what came out and who approved it. That survives a synonym pass and a translation; the statistical pattern does not. A watermark is a complement for whoever has no access to the log — a platform, a regulator — not a replacement for your own traceability.
What doesn't change
Three counterweights, because this story can easily be told worse than it is.
One: the European Code of Practice, which is voluntary, does not ask for marks on outputs below 200 tokens. A good chunk of the short-text hole is already accounted for.
Two: the law carries that "as far as technically feasible" clause. A mark breaking under rewriting does not automatically make anyone an infringer, because nobody has published the bar yet. The 7% of annual income floating around as a possible fine is the theoretical ceiling flagged by one expert, Thomas Souverain, not an open case.
Three: nobody outside has evaluated textGrain yet. OpenAI says it matches or beats SynthID Text in its own tests, and that is exactly what an in-house test is worth. A small group of researchers has just been given access — John Thickstun at Cornell and Dominik Macko's team at KInIT in Bratislava, among others — and has barely started.
Our take
I think text watermarking is going to age badly as a compliance tool, and well as a large-scale statistical signal. They are not the same thing. For a platform processing millions of messages, 66% detection is still useful: you see the trend. For deciding about one specific text — this exam, this article, this complaint — it is useless, and I am worried it will be used anyway, because the temptation of a button that says "a machine wrote this" is enormous.
The underlying problem is not engineering, it is the material. An image has millions of pixels in which to hide redundant bits; a paragraph has three hundred token decisions, and each one costs money in the form of entropy. We saw this with EXIF metadata on photos: everyone assumed provenance travelled with the file, until it turned out any forward stripped it. Here it is worse, because stripping it does not even require intent.
What does strike me as wrong is regulating with a criterion nobody can measure. A rule that asks for robustness without saying robustness against what leaves everyone in the same spot: the big company declares compliance, the auditor has no yardstick to check it against, and the small shop integrating the API has no idea whether the liability is theirs or the provider's. And I agree with the call to open the method up: a closed watermark that no outsider can try to break is not a guarantee, it is a claim. SynthID shows its methods; textGrain, for now, is a technical report with no peer review.
What I would do if it landed on my desk tomorrow: assume the text leaving my system carries no usable mark the moment anyone touches it, and solve it the boring way — logging at source, content credentials in the metadata where the format allows it, and a visible label for the user when a model wrote the text. Less elegant than invisible cryptography, with the advantage that it works.
Any questions, tell me and we'll go through it. All the best, Vicente.