A header meant for API clients was making the server render twice as many pages as it needed to. And the behaviour that allows it has been an open bug for 24 years.
What happened
Simon Willison was looking at the Firefox network panel while loading a Datasette table page and found a request he had never made: the browser had gone off on its own to fetch the next page of rows.
The cause is an HTTP header that Datasette sends with its responses so API clients know where to continue paginating:
link: <https://latest.datasette.io/fixtures/compound_three_primary_keys?_next=a%2Cd%2Cv>; rel="next"
Firefox reads it and decides to prefetch that target. The result: of everyone who opens a table, only a tiny fraction ever clicks «next», yet the server has had to render the whole page for all the others and throw it away.
Pulling on the thread, Willison found that the Firefox behaviour has had an issue open for 24 years, and that one of the earliest comments on it links to a post he wrote in 2002 complaining about exactly this. It wasn't that he had forgotten the post: he had forgotten the feature existed at all. The fix proposed in Datasette issue 3046 is to stop serving that header when the response is HTML. He tells it himself in his note of 7 November.
Why it matters
This isn't an exotic bug in one project. It's a pattern you may well have running right now without knowing, and it has four nasty consequences.
You double the load in the most expensive place. A paginated listing with filters, sorting and a row count is, in almost any application, among the heaviest queries you run. Serving it twice per visit and binning one copy is the worst possible split.
It doesn't show up in the logs. Those requests are legitimate GETs, with the same IP, the same session and the same user agent as the real visitor. In the log they look like someone browsing quickly. There's no anomaly to hunt for: you have to open the browser's network panel and look at what your application requests without anybody asking.
It poisons your numbers. View counters, server analytics, rate limits tripping earlier than they should, caches full of pages nobody ever saw. If the ratio between list views and clicks has ever looked odd to you, look here.
And if your «next» has side effects, it stops being waste and becomes a bug. A GET should never change state, but we have all seen cursors that get consumed, single-use tokens, «mark as read» flags or export records hanging off a pagination URL. If the browser visits it by itself, they fire.
What someone would do differently from today: separate the two audiences. Pagination headers are excellent on a JSON response consumed by a script; on an HTML response heading to a browser they are dead weight. Decide who gets them based on what you are returning, not as a blanket default.
What doesn't change
The Link header with rel="next" is not a mistake. It's the correct, standard way to tell a machine where the next page is, and stripping it from the API would be a step backwards. The problem isn't the header: it's serving it to a client that acts on it instead of storing it.
Nor is it a security hole or a disaster. On a small site with cheap pages you will never notice. It hurts when the page is expensive and traffic is high — which is exactly when you already had five other problems on your plate.
And no, I wouldn't pin it all on Firefox. Automatic prefetching from rel=next was a perfectly sensible idea in an era when the expensive part was line latency, not server-side rendering. What has changed in 24 years is where the cost sits.
Our take
The interesting part here isn't the bug, it's how it was found: by looking at the network panel of your own application. Very few people do that, and it's one of the most profitable half hours you can spend.
My position: if a header changes client behaviour, that header is part of your public API and deserves the same deliberate decision as an endpoint. You don't add it «because it's the correct thing» and forget it. You add it knowing who will read it. Mixing machine hints and human HTML in the same response guarantees that sooner or later a machine will act like a person at the worst moment.
I also think this will get worse, not better. There are more and more layers requesting things «just in case»: speculation rules in the browser, link previewers in messaging apps, agents that browse on your behalf and load three pages to decide which one to read. All of those clients take your hints literally. The 2002 pattern hasn't been fixed, it has multiplied.
And I'll keep the human detail: the person filing the issue in 2026 is the same one who complained about this in 2002 and had forgotten. That's how well problems age when nobody notices them, because they only cost server work and break nothing on screen.
This is exactly what we run into when somebody says their server «goes slow now and then and we don't know why»: the spike isn't coming from users, it's coming from something automated that has been there for months and that nobody has ever watched with the network panel open.
Any questions, tell me and we'll look at it. All the best, Vicente.