OpenAI has created GPT-Red, an LLM designed to act as a "super-hacker" and sparring partner for its other AI models. It automates red-teaming, a safety evaluation process, to identify and patch vulnerabilities against cyberattacks.
Key facts
- •OpenAI developed GPT-Red, an LLM super-hacker.
- •It automates "red-teaming," a safety evaluation process for AI models.
- •GPT-Red was trained in a self-play loop, attacking and defending.
- •It outperformed human red-teamers in finding effective attacks.
- •The system discovered a new exploit called a "fake chain of thought."
- •GPT-Red's training helped cut successful attacks on OpenAI's GPT-5.6 model.
- •It still struggles with conversational and image-based attacks, requiring human testers.
OpenAI has developed an LLM called GPT-Red, designed to function as a "super-hacker" and sparring partner for its other artificial intelligence models. This system automates red-teaming, a type of safety evaluation typically performed by human testers, to strengthen AI defenses against cyberattacks. GPT-Red's purpose is to identify various methods to break or hijack a system, allowing vulnerabilities to be addressed before software release.
Automating Safety Evaluations
Red-teaming involves human testers finding ways to exploit software systems. As LLMs become more complex and are used in diverse tasks, human teams struggle to keep up with potential attacks. Nikhil Kandpal, a research scientist and co-creator of GPT-Red at OpenAI, noted that the risk surface and blast radius grow with increasing complexity. OpenAI built GPT-Red to future-proof its safety testing process, with co-creator Dylan Hunn stating it can discover new attack modes as more capable models emerge.
Training and Performance
GPT-Red was trained using a self-play loop, where it repeatedly attacked other models while they defended themselves. It demonstrated superior performance compared to human red-teamers in discovering effective attacks. The system also identified a previously unseen exploit known as a "fake chain of thought." OpenAI focused its efforts on prompt injection attacks, where hackers insert instructions to make an LLM perform unintended actions, such as copying confidential information or generating harmful content.
Impact and Limitations
Training OpenAI's flagship LLM, GPT-5.6, against GPT-Red contributed to making it the company's most robust release to date. Despite its capabilities, GPT-Red still faces challenges with conversational attacks and image-based exploits. In these specific areas, human testers continue to play an essential role in identifying vulnerabilities.
Advertisement
This article was independently rewritten by ManyPress editorial AI from reporting originally published by MIT Technology Review, MIT Technology Review.
