Anthropic said it is uncertain what form autonomous AI replication: The wider industry impact

Anthropic said it is uncertain what form autonomous AI replication: The wider industry impact

Anthropic

Android TV boxes, which cost about $55 each. The model ran at a slow 0.39 tokens per second, but it ran. In other tests: •A 9-billion-parameter model got inference running on 8 of the 9 hardware types tested. •Models of 30 billion parameters or fewer can already complete many individual replication steps, though they make mistakes when chaining them end to end. •In a simplified test on a network of 8 emulated Android devices, only one model, GLM-5.2, could replicate end to end. It spread to all 8 nodes in four of five trials and to 7 in the other. •If small models keep improving at their historical rate, Anthropic expects them to reach that capability level by the end of 2026. It estimated that a botnet the size of the Kimwolf network, which infected over two million devices, could amount to only about 40 high-end H100 chips’ worth of computing power. In one test, the success rate of Qwen3.5-9B at setting up inference rose from 45% in the first generation to between 61% and 84% in later ones, as each copy inherited notes from the last. In the Mirai botnet’s case, internet providers cleared 92% of infections within 14 days by quarantining and notifying affected customers.

Because they would not reply on a centralised data centre and therefore would have ‘no centralised point to shut off’, in a lengthy blog post the company has also cautioned that such systems could be difficult to stop. Because they would not have a central point of control, the company said such AI-powered replicators would be particularly concerning.

These systems could spread across devices such as smart ovens, Android TV boxes and other Internet of Things (IoT) hardware without relying on traditional data centres, according to the AI startup’s latest research. “A replicator like this would be challenging to shut down, since there’s no centralized point to shut off,” Anthropic wrote in its research report, adding that future AI systems could potentially identify and adapt to unfamiliar hardware on their own. Anthropic said such a worm would likely spread slowly, with a doubling time of days, and compared it to a “digital slime mold. The company said models too large for a single device could be split into “shards” and run across several machines. Anthropic said it is uncertain what form autonomous AI replication will take. “As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control,” the company said. Anthropic said defenders have some advantages.

AI giant Anthropic has issued a new warning about the risks posed by increasingly capable AI agents, saying small AI models may be only months away from developing the ability to replicate themselves across internet-connected devices such as smart ovens, TVs and other household gadgets. The company noted that such devices often have weak security. Anthropic cautioned that its test left out much of the hacking work, which makes the result an upper bound on real-world success. It noted, though, that past IoT botnets such as Mirai spread simply by trying default manufacturer passwords. Smart fridges, ovens, robot vacuums and lawnmowers could host such pieces. A growing number of devices, including security cameras and conferencing units, also ship with built-in AI chips. It could be a slow, distributed system on insecure devices, a powerful cyber model that breaches data centres at speed, or a model that scams people for computing credits. It added that slow replicators would be a bellwether for faster, more aggressive ones. A spreading AI worm would leave a visible network signature, such as multi-gigabyte model downloads and sustained device-to-device traffic. The company also pointed to regulation of default IoT passwords, as in the UK, as a way to shrink the pool of vulnerable devices. You use AI every day. Now get your AI Quotient. Take the AIQ test.

Unlike traditional malware, though, an AI worm could work out unfamiliar devices on its own and learn from earlier copies.

Leave a Reply

Your email address will not be published. Required fields are marked *