Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs
Abstract
Large language model safeguards decide whether to answer before seeing how an answer will be used. This creates a basic problem for dual-use tasks: the same answer can help an authorized professional or an attacker, while an attacker can imitate a benign request and interaction history. We separate the capability released by the model from the evidence available about downstream use. When that evidence is copyable, we derive the exact worst-case floor on attacker assistance while preserving useful answers. The result yields a safety trilemma: Useful Capability, Reliable Safety, and Open Access cannot coexist. We then show how a trusted credential can complement existing safeguards by adding hard-to-copy information that predicts actual downstream use, and identify the stronger condition needed to eliminate the floor. Evidence from dual-use evaluations, adaptive attacks, and deployed trusted-access programs supports the practical relevance of these conditions.
Community
This paper studies LLM safety through a game theoretic lens and develops a safety trilemma involving Useful Capability, Reliable Safety, and Open Access. It rigorously shows that, for dual use tasks where attackers can reproduce the contextual evidence available to the defense, safeguards relying solely on such evidence cannot provide reliable safety guarantees while preserving useful capability. Guided by this theoretical analysis, the paper further identifies a promising direction for reliable safeguards: augmenting existing mechanisms with trusted credentials that are difficult for attackers to copy and predictive of actual downstream use. More broadly, the proposed framework provides a unified theoretical explanation for recurring failures observed in real world LLM safety incidents, as well as for previously scattered empirical findings in the literature. I find the paper theoretically significant, practically relevant, and worthy of acceptance.
Get this paper in your agent:
hf papers read 2607.27951 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper