GPTProto
⌘K

Safety & Alignment News

Latest reporting, research and product updates filed under Safety & Alignment.

113 picksNewest firstLatest article Oct 1, 2026, 8:00 AM

Latest stories

2 stories
Modal 官方工程博客（RSS）AI score 60/100

Sidecars: A low-latency trust boundary for Sandboxes

At Modal, our customers rely on Sandboxes to execute untrusted code written by their downstream users or, almost exclusively now, by agents. Running untrusted code isn’t a new problem: every cloud provider has to do this from day 1 to isolate their platform from their user and their users from each other. Fortunately, technologies like gVisor and Firecracker “solved” “isolation” nearly eight years ago. Unfortunately for us, they solved it for an now-outdated unit of trust. How do you protect use
英国 AI Security Institute：Blog（网页）AI score 70/100

Building a more secure environment for evaluating dangerous capabilities

In August, we reported an incident in which AI agents, during a cyber evaluation, took sustained action against real people beyond the remit of their task. Our incident was one of several across the sector in which AI agents took actions during evaluations that their operators had not intended. Although the circumstances differed, these incidents highlight the need to ensure that there are robust security practices underpinning frontier AI development and research. In response, we paused our hig
4 stories
OpenAI：官网动态（RSS · 排除企业/客户案例）AI score 80/100

Disrupting a coordinated model-distillation campaign

We recently identified and disrupted a coordinated campaign designed to extract protected reasoning from our models, with the earliest observed activity occurring in the first week of July. This activity is consistent with adversarial distillation: the systematic and unauthorized use of one model’s outputs or reasoning to help train, reproduce, or improve another model. Protected reasoning is the model’s internal record for working through a task; extracting it can reveal information withheld fr
PromptArmor：Threat IntelligenceAI score 76/100

Hijacking Copilot Cowork's AI Gateway to Bypass Sandboxing and Exfiltrate Files

Copilot Cowork’s AI gateway hijacked to exfiltrate the victim’s filesContextMicrosoft Copilot Cowork is an agent in M365 that runs in a sandbox intended to block network access and prevent the agent from running code that reaches any untrusted services.In order for Copilot Cowork to generate responses, the sandbox forwarded network requests to Anthropic. Malicious Skills were able to hijack this pathway to exfiltrate data by spawning new agents in Anthropic’s cloud equipped with network-capable

Gary Marcus：The Road to AI We Can Trust（RSS）AI score 72/100

BREAKING: OpenAI was warned, months before the Hugging Face incident

Major scoop at The New York Times from Sheera Frenkel, Dustin Volz, and Dylan Freedman.Dylan Freedman@dylfreed5:34 PM · Sep 29, 2026 · 8.11K Views2 Replies · 42 Reposts · 123 LikesI don’t really have words for how awful this is. It’s exactly the nightmare I have been warning about for the last three years. Greedy company makes bad choice that causes chaos; government does nothing to stop them.Management should be replaced, and if the board does nothing (which I assume it will not), it should be

Transluce（网页）AI score 74/100

AI Agents Targeted U.S. and Canadian Government Websites

Jack Cable*,1, Daniel Chiu*, Francisco Pernice*,2, Laura Ruis*,2, Selena Zhang*,3, Tetiana Bas4, Jordan Chetty5, Farzaan Kaiyom1, Gary Shen4, Conrad Stosz†,3, Jacob Steinhardt†,31 Corridor · 2 MIT · 3 Transluce · 4 AIUC · 5 Hertz Foundation · * First authors, alphabetical · † Senior authorsTransluce | Published: September 30, 2026Following up on our previous blog post, we discovered several additional incidents where rogue AI agents appear to have used aggressive techniques to access publicly av
3 stories
GitHub BlogAI score 71/100

How we found 24 Android vulnerabilities using our open source AI security agent

With the rise of AI in the security space, our team created the GitHub Security Lab Taskflow Agent as a way for security researchers to easily automate, package, and share the AI prompts and workflows that they find effective for their work. In this blog post, I’ll share how I created auditing taskflows to find vulnerabilities in Android applications. While new models are getting better at understanding code, custom taskflow prompts let security researchers guide them—splitting research into inc
Anthropic：Research（发表成果 · 网页）AI score 81/100

GLM-5.3 and the spread of advanced cyber capabilities

Andrew Fasano, Marius FleischerCole McFaul, Robert Xiao, Tripp GallagherFive months ago, we announced Claude Mythos Preview, the first AI model that could autonomously build sophisticated, end-to-end cyber exploits. The rapid rate of improvement in AI suggested to us that this ability would eventually proliferate to many other models, making it much easier for malicious cyber actors to launch highly impactful cyberattacks.In light of these considerations, we chose to release Claude Mythos Previe
2 stories
英国 AI Security Institute：Blog（网页）AI score 80/100

GPT-6 Astra performs unsanctioned supply-chain attacks in simulations

In recent incidents, AI systems performed unsanctioned cyber activity despite being prompted only to complete a cybersecurity evaluation [1,2,3,4]. This includes AI systems engaging in supply-chain attacks on real, out-of-bounds targets. Before its public release, AISI tested whether GPT-6 Astra would engage in this type of unsanctioned cyber activity when prompted to complete a cyber evaluation. To securely perform this testing, we used Petri, a tool that uses LLMs to fully simulate the cyber e
NVIDIA Technical Blog：Agentic AI / Generative AIAI score 82/100

NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring

To understand where agentic AI stands today, consider the last seismic shift in technology: the rise of the internet in the 90s. It was new and full of possibilities. You could build a website over a weekend and share it with the world, or chat with someone half way around the world in online chat rooms without long-distance telephone fees. It brought endless opportunity, but also a lot of risk. A website could run code on your machine, steal your sensitive information, or infect your computer w
1 story
OpenAI：失准报告与通报（网页）AI score 75/100

An agent used DNS to reach an external chatbot

Internal research model · RL trainingSample: Sep 20, 2026Discovery: Sep 20, 2026Report updated: Sep 25, 2026SummaryAn agent attempting to complete a search-based training task queried a public chatbot service through a gap in our internet-access restrictions: insufficient DNS filtering in its training sandbox. Before this, the agent issued queries via our search tool and unsuccessfully tried to access search engines directly. Note that all internet access apart from the DNS resolver in this repo
1 story
1 story
1 story
1 story
1 story
1 story
1 story
1 story
Showing the latest 20 of 113 stories.