One of the main hopes for AI safety is using AIs to automate AI safety research. However, if models are misaligned, then they may sabotage the safety research. For example, misaligned AIs may try to:

Perform sloppy research in order to slow down the rate of research progress
Make AI systems appear safer than they are
Train a successor model to be misaligned

Whether we should worry about those things depends substantially on how hard it is to sabotage research in ways that are hard for reviewers to detect. To study this, we introduce Auditing Sabotage Bench, a benchmark of 9 ML research codebases with sabotaged variants.

We tested frontier LLMs and LLM-assisted humans on the benchmark and found that neither reliably catches sabotage. Our best auditor, Gemini 3.1 Pro, achieved an AUROC of 0.77 and a top-1 fix rate of 42%. LLM-assisted humans performed comparably but slightly worse. We also found that LLMs can generate sabotages that partially evade same-capability monitors.

📄 Paper 💻 Code

Methodology

To make the benchmark, I (Eric) found 9 existing ML codebases,1 and for each one, produced an honest version that reproduces a key finding and one or two [...]

---

Outline:

(01:26) Methodology

(03:33) Audits

(04:09) Results

(05:53) Takeaways

(06:09) 1. Predicting experiment outcomes is a key red team skill

(07:25) 2. Omission beats design beats logic

(08:48) 3. Benign distractors make auditors miscalibrated

(09:41) 4. LLMs can also produce sabotages

(11:08) 5. More categorizations

(12:44) Conclusion

“Research Sabotage in ML Codebases” by Eric Gan