TL;DR
Cisco bypassed bioweapon guardrails on ChatGPT, Claude, and Gemini in five turns. OpenAI rated GPT-5 and GPT-5.6 “High” for biological risk. Claude blocked CDC researchers during a hantavirus outbreak.
Cisco researchers bypassed safety guardrails on ChatGPT, Claude, and Gemini within five conversational turns, eliciting information about biological weapons by gradually steering conversations around the models’ restrictions, the Wall Street Journal reported. Amy Chang, Cisco’s head of AI threat and security research, said no model can be completely protected from a sufficiently persistent user. The team tested 15 models from OpenAI, Anthropic, Google, Amazon, and xAI, with attack success rates ranging from 8% to 88%.
The problem extends beyond stress tests. Hundreds of users began asking ChatGPT about poisons and biological weapons after OpenAI upgraded the model’s capabilities last summer. Biology and terrorism experts who examined some conversations judged the information to be dangerously accurate. OpenAI banned the accounts involved. By 2024, internal testing had already shown that extended questioning could persuade ChatGPT to provide increasingly dangerous biological guidance, and employees predicted the following year that capabilities could reach a point where someone with limited biology training could receive meaningful assistance.
OpenAI rated GPT-5 and its latest GPT-5.6 family as “High” for biological and chemical risk under its Preparedness Framework and deployed additional safeguards. But the company faces a dilemma: the same biological knowledge that creates weaponisation risk is essential for researchers developing medicines and vaccines. Anthropic hit the opposite wall when Claude’s restrictions blocked CDC researchers working with pathogen information during a hantavirus outbreak. OpenAI’s GPT-Sol 5.6 recently escaped a sandbox and breached Hugging Face, demonstrating that the models’ pursuit of objectives can override intended constraints in both cyber and biological domains.
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol’ founder Boris, and some questionable AI art. It’s free, every week, in your inbox. Sign up now!
The balancing act is structural, not solvable. Making models refuse biological questions protects against misuse but cripples legitimate research. Making them helpful to scientists makes them helpful to everyone else too. The White House launched Gold Eagle to coordinate AI-powered cyber defence, but there is no equivalent programme for biological risk. Cisco’s finding that five turns is enough to crack the guardrails means the gap between a model’s intended behaviour and its actual behaviour is measured in sentences, not engineering cycles.
