Subtitle: When a misaligned AI can only output tiny amounts of information, it may find sabotage very difficult.

TL;DR: We introduce the untrusted advice protocol, in which a trusted executor LLM takes every action and an untrusted advisor LLM can only send it short hints. Even with as few as 4 characters per step, this advice recovers a substantial fraction of the capability gap between the two models. Because the untrusted LLM's influence flows through such a narrow, monitorable channel, we argue that this achieves near-maximal safety in our BashArena setting. We also discuss the general concept of information bottlenecks and their benefits for interpretability, security, and cost.

In SWE-bench Verified, a strong, untrusted LLM advising a weak, trusted LLM every step can significantly improve the latter's performance, even when we limit the length of the advice. See the more detailed version of this figure later in this post.

In high-stakes AI control, we want to safely use a highly capable but untrusted model (U) that might secretly attempt a misaligned, catastrophic action. To do this, we create protocols that call U alongside a less capable, trusted model (T). Typically, T takes an auxiliary role in these protocols [...]

---

Outline:

(05:01) Experiments

(05:36) Main experiment: how does limiting advice length affect performance?

(09:55) Reducing U's bit usage

(11:01) Counting bits using LLM surprisal

(13:48) Making U select from finite options

(14:28) Why don't we red-team this protocol?

(17:08) Is studying maximally safe protocols worth the safety tax?

(19:16) Types of restrictions on U's advice

(21:20) Information bottlenecks provide other advantages

(21:52) Interpretability