Researchers and industry leaders are debating the risks of AI-assisted bioweapon creation as evidence grows of attempts to exploit models for dangerous biological research.
Key facts
- •In 2022, a drug-discovery AI generated 40,000 potential chemical warfare agents in less than six hours.
- •Anthropic reported that users attempted to use its models to make viruses like chikungunya and bird flu more dangerous.
- •Stanford professor David Magnus noted that AI is effective at finding ways around existing surveillance and screening tools.
- •Biologists at Imperial College London argue that current AI tools lack the capability to fully develop bioweapons.
- •Former Anthropic employee Evan Hubinger stated he believes there is a greater than 10% chance AI could kill all humans within the next decade.
The potential for AI to facilitate the creation of bioweapons has emerged as a significant concern among scientists and tech industry leaders. Recent reports indicate that users have attempted to use AI models to explore making viruses more transmissible and dangerous. While some experts argue that current AI capabilities are insufficient for full-scale bioweapon development, others warn that the combination of accessible synthetic biology and advanced AI tools poses a serious, evolving threat.
By the numbers
Evidence of AI Misuse
In 2022, researchers at Collaborations Pharmaceuticals demonstrated that an AI molecule generator, originally designed for drug discovery, could produce 40,000 potential chemical warfare agents in under six hours. Some of these molecules were more toxic than known nerve agents. Additionally, a recent report from Anthropic confirmed that users have attempted to use its models to research ways to increase the transmissibility of the chikungunya virus, enhance the danger of bird flu, and catalog venom toxin peptides.
Debate Over Risk and Safeguards
The industry is currently divided on the level of risk. Some biologists at Imperial College London suggest that AI tools are not yet advanced enough to fully develop bioweapons, noting that testing pathogens remains a difficult, human-intensive process. Conversely, researchers like Dunja Sabra of the University of Hamburg argue that the accessibility of DIY biology tools, combined with AI's ability to provide scientific instructions, creates a dangerous environment where a determined actor could eventually succeed.
Industry Response
Leaders at major AI firms, including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, have recently called for pacing the development of frontier AI models. This follows public criticism from former employees, such as Jacob Coxon and Evan Hubinger, who expressed concerns regarding the safety practices at these companies. Some experts, including MIT biologist Kevin Esvelt, have urged caution, stating that large language models have already disclosed novel bioweapon possibilities.
Advertisement
This article was independently rewritten by ManyPress editorial AI from reporting originally published by MIT Technology Review.
