Dual-use AI

"Dual use" describes technology that can serve both helpful and harmful ends. AI raises the problem sharply because one general model can do many different tasks, and the same knowledge that helps a doctor or engineer can help an attacker.

The clearest demonstration came in 2022. Researchers at Collaborations Pharmaceuticals, a drug discovery company, took a model normally used to avoid toxic molecules and reversed its goal for a presentation at the Spiez Convergence conference, a Swiss government series on chemical and biological weapons risks¹. In under six hours, it generated 40,000 molecules that scored within their toxicity target, including the nerve agent VX and other known chemical warfare agents¹. The authors said there had been little prior discussion of this risk in their field¹.

A 2018 report by 26 authors had already surveyed how AI could be misused in digital, physical and political attacks, and how to forecast and prevent such misuse².

In October 2023 a US executive order defined a "dual-use foundation model" as a broadly trained model with at least tens of billions of parameters that could substantially lower barriers to making chemical, biological, radiological or nuclear weapons, enable powerful cyber attacks, or evade human control³. That order was revoked on 20 January 2025⁴.

Labs now test frontier models for these risks before release. Anthropic, for example, added extra safeguards for Claude Opus 4 because it could not rule out that the model gave meaningful help with such weapons⁵.