If you run a Datasette instance on the public web, update it. And if that instance mixes public and private tables, do it today.
What happened
On 11 September Simon Willison published two security patch releases: 1.0a39, for the current alpha series, and 0.65.4, for the stable family. Two branches maintained at once, so if you're on 0.65.x you don't have to jump to the alpha to be covered.
It started with issues reported by Sevban Dönmez. From there, Alex Garcia and Willison ran a broad audit of the codebase backed by three different models — Claude Fable 5.1, GPT-5.6 and GPT-6 Astra — and spent almost a week fixing and reviewing what came out. Their read: the models surfaced some very subtle bugs. Their conclusion: frontier-model security audits are going into all of their development work from now on.
And one detail of method that is, for me, the best part of the announcement. They worked in a shared private repository and, for most of the issues, split it like this: one of them wrote the automated test that exposed the problem, the other implemented the fix. Every issue passed through two separate humans, on top of coding agents running different models. You can read the original announcement and the release notes.
Why it matters
The test is written by whoever isn't fixing it
This is what I'm taking into my own way of working. When the same person finds the bug, writes the test and applies the patch, the test ends up proving that the patch works — not that the bug existed. It's a quiet and very common bias: the test gets written while looking at the fix.
Splitting the two roles turns the test into a contract. It reproduces the problem from the outside, without knowing where the solution will come from. And it has an even better side effect: it forces you to explain the bug to another person precisely enough for them to reproduce it. There's no better detector of "I haven't actually understood this".
I work on my own, so there's nobody on the other side. What I do is get close: first a commit with the test failing, dated, before touching a single line of the fix; and when I use an agent for a second opinion, I hand it the test and the observed behaviour, never my fix. Show it your patch and it will applaud it.
Three models isn't collecting for the sake of it
Running three models over the same code isn't a whim. They fail in different places and overlap less than two passes of the same model, so the union covers more surface. The price is noise: every pass returns findings that are nothing at all, and somebody has to throw them out. My guess is that's where a good chunk of that near-week went. Auditing is fast; triaging and reviewing isn't.
The pattern to look for in your own app
The advisory is very specific: the risk bites when a single instance mixes public and private material. That's where this class of bug lives. Not in the endpoint returning the secret table — you've locked that one down — but in everything around it: schemas, column names, error messages, counts, facets, autocomplete suggestions, export routes. Anything that reveals the existence or the shape of something you shouldn't be able to see.
If your application decides permissions per table or per row inside the same endpoint, this is aimed squarely at you. What I'd do this week: list every route that returns metadata and run each one as a user with no permissions, checking not just that there's no data but that there are no hints either. Authorisation tests usually cover the data and almost never the metadata.
Don't let the audit depend on someone warning you
The order of events here is the usual one: an external report arrives and then you audit. It works, but it's the expensive route, because the window between the bug and the warning is on you. The copyable part of Willison's decision is turning the audit into a routine step in the development cycle instead of a reaction to an email.
What doesn't change
- This is not a penetration test. A model-assisted audit of your own code doesn't replace a formal audit or an intrusion test against the deployed system. It worked here because two people with extremely deep knowledge of the project were steering and knew what was worth looking at.
- We don't know how many bugs there were, or how severe. The announcement gives no count and no classification. If you need to assess real risk for your installation, go to the release notes, not to somebody's summary.
- The human cost is still there. Almost a week of two people, after the audit. Anyone selling the idea that models patch on their own isn't looking at the time invoice.
- 1.0a39 is still an alpha. Carrying security fixes doesn't make it stable.
- And the other way round: if someone tells you this story as "AI found vulnerabilities", they're underselling it. The headline is the process — two human reviewers, different models, test and fix kept apart — and that one you can take to your team tomorrow.
Any questions, tell me and we'll go through it.
Best, Vicente.