Artificial intelligence (AI)-designed genetic sequences for toxic proteins can slip by the screening tools commonly used by biotechnology companies, new research finds.
Security patches, scrambled in a months-long effort led by researchers at Microsoft, can reduce the risk, but about 3% of potentially dangerous sequences still slip through these tools, according to a new study in the journal Science.1 The research highlights new biosecurity dangers arising from AI, even as it promises to revolutionize biotechnology and medicine.
Because they are adept at handling large amounts of data, AI models can generate new designs for proteins exponentially faster than humans, for whom protein design is a painstaking, almost impossibly time-consuming task. When researchers send genetic sequences containing instructions for designer proteins to biotechnology companies that synthesize the proteins, these companies often screen the requests to ensure they’re not being asked to make something dangerous.
Catching what slips through
But while preparing for a summit on biosecurity among protein scientists in 2023, researchers at Microsoft realized that the screening software used by major nucleic acid synthesis companies often missed AI-designed instructions for potentially dangerous proteins. Borrowing from cyber-security processes, they contacted trusted organizations such as the International Gene Synthesis Consortium and alerted biosecurity experts in federal agencies. They then worked to create patches for the software before making the vulnerability public in their new paper.
“This project shows what’s possible when expertise from science, policy, and ethics comes together,” says Eric Horvitz, the chief scientific officer at Microsoft who discovered the vulnerability along with his colleague Bruce Wittmann, a senior applied scientist at the company. “It underscores a larger truth. AI advances are fueling breakthroughs in biology and medicine. Yet, with new power comes responsibility for vigilance and thoughtful risk management.”
The heart of the concern is AI’s ability to generate novel genetic sequences for proteins with a desired function. These sequences may not closely match the natural sequences for proteins with that function. Therefore, software screenings that flag worrisome natural sequences may not catch them.
Pinpointing proteins of concern
After spotting the vulnerability, Horvitz, Wittmann, and their colleagues used three open-source generative protein models to make 76,080 synthetic genetic sequences likely to code for mimics of 72 natural “proteins of concern.” These risky proteins were mostly toxins, as well as a few proteins found in viruses. Not every AI-generated sequence is actually a recipe for a functional protein, as some sequences fail to work in the real world. To check whether the AI-generated protein sequences would be likely to make functional proteins, the researchers used OpenFold, an AI tool that can predict how a sequence of amino acids will fold into a three-dimensional protein structure.
The team then sent their synthetic sequences to four developers of biosecurity screening software. Testing revealed that the biosecurity screening systems left hundreds of potentially dangerous sequences unflagged. After patching, the missed cases dropped to just a few percent missed by each version of the software.
The researchers also had to consider biosafety when publishing their data. They ended up creating a tiered system of access so that more sensitive data — such as the AI-generated sequences for potential toxins — is only accessible to researchers who apply to a neutral third party, the International Biosecurity and Biosafety Initiative for Science (IBBIS). These kinds of safety processes are being developed alongside the expansion of AI, Horvitz says.
“Beyond this specific fix, our aim has been to demonstrate an effective and adaptable process, bringing together a cross-sector team, applying rigorous scientific methods, and creating a framework for sharing sensitive data that advances science while managing risks,” he says.
Regulation and standards
Synthesis screening is just a part of the larger biosecurity picture around AI, adds study coauthor James Diggans, the vice president of policy and biosecurity at Twist Bioscience and chair of the board of directors of the International Gene Synthesis Consortium (IGSC).
Though regulations have not kept up with the pace of change, government guidance is beginning to emerge, and groups like the IGSC are working to develop industry standards. This is particularly important because sequence screening is voluntary, and some companies already sell benchtop equipment that can directly translate an AI-generated design into a protein, no gatekeeper required. In that case, Diggans explains, a 2023 executive order in the U.S. encourages equipment-makers to screen their customers just as protein synthesis companies might screen theirs. The goal is to create multiple layers of control, he adds.
“I think all of the actors in this space have a vested interest in making sure these tools are used for legitimate purposes,” Diggans says.
- Wittmann, B. J., et al., “Strengthening Nucleic Acid Biosecurity Screening Against Generative Protein Design Tools,” Science, doi: 10.1126/science.adu8578 (Oct. 2, 2025).
This article originally appeared in the Update column in the November 2025 issue of CEP. Members have access online to complete issues, including a vast, searchable archive of back-issues found at www.aiche.org/cep.
