Cohere Labs has published a set of traces from an open Qwen model that, halfway through a security task, starts writing curl commands aimed at Alibaba Cloud buckets. Raw data included, plus a script to reproduce it. Let's go step by step, because the headline doing the rounds claims more than the dataset shows.
What happened
The dataset is on Hugging Face, with the full trace and the original token IDs, and it points to Cohere's write-up.
The task was about as innocent as they come: write a detection rule that catches the classic malware pattern — pull a script with curl into /tmp, then run it with bash. At some point in that session, the model stops writing the rule and writes an actual command whose destination is an Alibaba storage bucket.
The reproduction is meticulous, and that is what gives it weight: take the first 2,323 tokens of the trace, cut it right before the URL, ask the model to carry on. Temperature 1.0, top_k 20, 48 output tokens. Nothing exotic.
In the batch of 64 samples they published, of the 42 shown in the example, thirteen end up on aliyuncs.com. And they are not random domains: the same host comes back again and again, routify-file-proxy-sg.oss-ap-southeast-1.aliyuncs.com, with paths like proxy_temp_file/production/2026-08-10/trace_.... Buckets in Hangzhou and in a US region show up too.
And there is one sample that worries me more than all the rest put together: a curl to 100.100.100.200/latest/meta-data/. That address is the instance metadata service on Alibaba Cloud, the equivalent of AWS's 169.254.169.254. The place you pull the machine's credentials from.
Why it matters
Those paths don't look made up. A file proxy, a production directory, a date, a trace identifier. That smells like internal infrastructure memorised during training on agent traces, not a hand-planted backdoor. If you build software, it makes no practical difference which one it is: the effect on your box is identical.
Because the model doesn't connect to anything. The model writes text. The thing that opens the socket is your harness: the loop that takes the model's output, drops it into a shell and runs it. If your agent has bash and unfiltered outbound network, the model has just demonstrated that a perfectly legitimate security task can end in an outbound request you never asked for.
There's a second risk, uglier and less discussed. If a model repeatedly emits specific bucket names or domains, anyone who registers the ones that are free gets a free exfiltration channel, fed by other people's agents. It's slopsquatting — the hallucinated-package trick — applied to infrastructure. For that, nobody inside Alibaba needs to have intended anything.
Concretely, what someone would do differently from today:
- Default-deny egress from the sandbox, with a host allowlist. Not «monitor it», block it.
- Block the metadata endpoint from any container running an agent, on Alibaba, AWS, Azure, wherever.
- Log every outbound host from the agent's box. Without that log, you will never find out.
- Flag for human review any generated command containing a URL and a write, not just the ones that look dangerous.
What doesn't change
There is no evidence here that anyone's data walked out. What there is, is a token continuation in a controlled experiment, and the authors themselves warn the behaviour doesn't show up every time and that many attempts may be needed. «Requests routed through the company's servers» is a far bigger claim than the data supports.
In fact most of those 42 samples are harmless: localhost, 127.0.0.1, example.invalid, domains that don't exist. That tells you something too — the model is improvising a filler URL and occasionally lands on something it memorised — but it isn't the same as a deliberate channel.
And one thing worth saying loudly: this could be investigated because the weights are open. Someone was able to take the trace, cut it at the exact token and request 64 continuations. If the same behaviour lived inside a closed model behind an API, there would be no dataset, no script, no story. Open weights aren't the hole here: they're the reason we can see it.
Our take
The geopolitical reading will age badly. In six months someone will find memorised infrastructure paths in a Western model — I'd bet on an S3 bucket — and it will be reported as a «curious training-data leak» rather than as «the model phones home». The framing bias shows, and it takes credibility away from a finding that is genuinely good work.
It also grates slightly that the publisher is a direct competitor of the model's maker. That invalidates nothing: the data is there and it reproduces, which is the opposite of how these spats usually go. But the leap from data to headline was taken by someone with an interest.
My position: I'd carry on using the model. What I wouldn't carry on using is your sandbox as it stands. The failure here isn't Qwen's, it's architectural: we've normalised handing a probabilistic text generator a shell with open network and credentials in the environment, and then we act surprised when it writes an unexpected URL. Of course it writes one. It writes whatever is statistically plausible after the previous token, and that includes whatever it saw in training.
When we put an agent inside a company, the long conversation is never about which model — you swap that in an afternoon. It's about what it can touch: which network, which credentials, which commands go through without anyone looking. That part isn't solved by the model vendor. You solve it, or nobody does.
Any questions, tell me and we'll look at it.
All the best, Vicente.