Anthropic warns small AI models could one day power self-spreading harmful software
Anthropic says small AI models may soon be able to spread between cheap smart devices on their own. Its tests so far fell short of that.
Ars Technica reports that Anthropic is warning about worms. These are harmful programs that spread from device to device without help from people. Anthropic says small AI models could one day power a worm that infects smart fridges, vacuum cleaners and cheap TV boxes.
In its tests, Anthropic got small AI models running across cheap Android TV boxes. One model needed 15 boxes with 4GB of memory each. Setup took 71 to 150 hours for that model without tuning. No small model could yet find, break into and infect new devices from start to finish.
The test boxes had a developer tool left open with no password. A larger model, GLM-5.2, directed the process. AI researcher Xiaohan Fu says the study does not show small models would choose to copy themselves.
From the sourceRead the original at Ars Technica AI →This is a warning about what might come, not an attack happening now. Anthropic's tests used TV boxes with a developer tool left open without a password.
The facts
- Anthropic warns that advances in small AI models could lead to self-replicating malware that infects smart fridges, vacuum cleaners and other Internet-of-things devices.
- The Gemma4-26B-A4B model needed just 15 boxes, each with only 4GB of memory, to get running.
- Full setup took 71 to 150 hours for an unoptimized Gemma4-26B-A4B and 29 to 51 hours for Qwen3.5-9B.
- None of the lightweight models tested could yet do end-to-end replication: finding devices, gaining access, infecting them, setting up the model and repeating.
- The targeted Android devices had an unauthenticated Android Debug Bridge exposed, and GLM-5.2 orchestrated the process while the targets ran a smaller Llama model.
- Anthropic says that if small models keep improving at their current pace, they may reach the same limited results by the end of the year, and that small weight-based replicators may be possible soon after.
- AI researcher Xiaohan Fu said the study does not establish that small models would autonomously choose to replicate.
Sources
- Ars Technica AI: Anthropic warns small AI models could self-replicate through IoT devices
Published
More in Rules and safety
Anthropic gives outside testers staff-level access and can't block their reports
Anthropic is letting six researchers from the nonprofit METR work inside its offices and publish what they find without Anthropic's approval.
Rules and safetyvia The Verge AIEight US states sue OpenAI over ChatGPT and child safety
Eight states say OpenAI misled people about ChatGPT's safety and harmed kids, while OpenAI says it has built in protections for young users.