Cybersecurity professionals report that strict AI safety guardrails are hindering legitimate research and pushing experts toward unrestricted open-source models.
Key facts
- •U.S. export controls were briefly applied to Anthropic's Mythos and Fable models in June.
- •OpenAI and Anthropic provide vetted access programs for researchers to use models with fewer restrictions.
- •Researchers report that guardrails often trigger false refusals, forcing them to spend time negotiating with models rather than analyzing vulnerabilities.
- •Some security firms use locally hosted open-source models to avoid data leakage and bypass strict cloud-based guardrails.
- •Chris Thompson of RemoteThreat noted that researchers are increasingly adopting foreign-owned open-source models due to the limitations of U.S. AI systems.
AI companies have implemented strict guardrails and vetting programs to prevent the misuse of their models for cyberattacks. However, cybersecurity researchers and network defenders argue these restrictions are impeding their ability to identify and address system vulnerabilities, as the same tools used for defense are often flagged as offensive.
Impact on Security Research
Researchers note that AI models often refuse to assist with tasks like code exploitation, which is a necessary step for confirming vulnerabilities. Chris Anley of NCC Group explained that the dual nature of these tools makes it difficult to separate defensive utility from offensive potential. Some experts, including Paolo Stagno of Crowdfense, avoid using cloud-based frontier models for vulnerability research entirely to prevent sensitive data leaks, opting instead for locally run open-source models.
Regulatory and Access Challenges
In June, the U.S. government imposed export controls on Anthropic’s Mythos and Fable models following reports of potential guardrail bypasses. While export controls on Fable 5 and Mythos 5 were later lifted, Mythos 5 remains restricted to vetted U.S. organizations. Companies like OpenAI and Anthropic offer specialized programs for researchers to access models with fewer restrictions, but practitioners report that these guardrails remain inconsistent and time-consuming to navigate.
Shift to Open Source
Due to the frustrations of working with restricted systems, some researchers are turning to open-source models that lack guardrails. Chris Thompson, CEO of RemoteThreat, warned that these policies are pushing responsible researchers away from U.S.-governed systems toward foreign-owned, unrestricted models, which he argues is counterproductive to maintaining security.
Advertisement
This article was independently rewritten by ManyPress editorial AI from reporting originally published by TechCrunch.
