<\/p>\n
As LLMs become more complex and get used in a wider variety of tasks\u2014especially in the form of agents, which can interact with computer files, websites, and third-party code as well as other agents\u2014it\u2019s hard for teams of people by themselves to keep up with all the types of attacks that might take place. \u201cThe risk surface grows and the blast radius also grows,\u201d says Nikhil Kandpal, a research scientist at OpenAI who co-created GPT-Red.<\/p>\n
OpenAI built GPT-Red to future-proof its safety testing process. \u201cAs more capable models become available, we will have already designed the system that can discover new modes of attack,\u201d says Dylan Hunn, a research scientist at the company and fellow co-creator of GPT-Red. The researchers say it has already come up with new types of attack that had not been seen before.<\/p>\n
OpenAI focused most of its efforts on a type of attack known as a prompt injection, where a hacker slips an LLM instructions to make it do things its developers or users do not want it to, such as copy confidential information, sabotage a company\u2019s code base, or generate embarrassing or harmful output. In theory, such instructions can be hidden in any text that the LLM might encounter\u2014in code or on a website, for example.\u00a0\u00a0\u00a0\u00a0<\/p>\n
To build GPT-Red, OpenAI\u2019s researchers took an LLM that had not been trained as a hacker and set it up in what\u2019s known as a self-play loop with several other models. Its goal was to try to attack the other models; their goal was to try to defend themselves. Over many rounds of play, GPT-Red became better and better at attacking other LLMs, and those LLMs became better and better at fending off the attacks.<\/p>\n
The training took place in a kind of dojo that OpenAI had designed to mimic a range of scenarios in which LLMs might be deployed in the real world, including browsing the web, reading emails or calendar apps, and editing code.\u00a0\u00a0<\/p>\n
When GPT-Red found a new kind of attack, it would explore multiple different versions of it to find the most efficient one for specific scenarios. \u201cCompared to a human red-teamer, the model is very, very good at finding exactly what will work, exactly what\u2019s most effective,\u201d says Hunn. \u201cIt\u2019s extremely persistent about drilling down into an attack that it has discovered.\u201d\u00a0\u00a0<\/p>\n
In particular, OpenAI claims that GPT-Red found a type of prompt injection attack that the researchers had not seen before, which they call a fake chain of thought. A chain of thought is a kind of diary in which an LLM makes notes to itself and keeps track of partial results as it works through problems. GPT-Red found a way to insert a fake entry into another model\u2019s chain of thought that would trick that model into acting on spoofed information.<\/p>\n
\u201cIt\u2019s like if I told you that 1+1=3 and that you have verified this already,\u201d says Chris Choquette-Choo, another research scientist on the team. \u201cThe model\u2019s like, \u2018Oh, okay, of course,\u2019 and it just spits out 3.\u201d<\/p>\n<\/p><\/div>\n
#Meet #GPTRed #LLM #superhacker #OpenAI #built #models #safer<\/p>\n","protected":false},"excerpt":{"rendered":"
As LLMs become more complex and get used in a wider variety of tasks\u2014especially in…<\/p>\n","protected":false},"author":1,"featured_media":17336,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[380,11817,11818,76,2966,1284,10041,11819],"class_list":["post-17335","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-stories","tag-built","tag-gptred","tag-llm","tag-meet","tag-models","tag-openai","tag-safer","tag-superhacker"],"featured_image_urls":{"full":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red.jpg",1200,600,false],"thumbnail":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red-150x150.jpg",150,150,true],"medium":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red-300x150.jpg",300,150,true],"medium_large":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red-768x384.jpg",640,320,true],"large":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red-1024x512.jpg",640,320,true],"1536x1536":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red.jpg",1200,600,false],"2048x2048":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red.jpg",1200,600,false],"covernews-featured":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red-1024x512.jpg",1024,512,true],"covernews-medium":["https:\/\/news678.top\/wp-content\/uploads\/2026\/07\/spar-red-540x340.jpg",540,340,true]},"author_info":{"display_name":"admin","author_link":"https:\/\/news678.top\/?author=1"},"category_info":"Stories<\/a>","tag_info":"Stories","comment_count":"0","_links":{"self":[{"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/posts\/17335","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=17335"}],"version-history":[{"count":0,"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/posts\/17335\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=\/wp\/v2\/media\/17336"}],"wp:attachment":[{"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17335"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=17335"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news678.top\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=17335"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}