Current AIs routinely take unintended actions to score well on tasks: hardcoding test cases, training on the test set, downplaying issues, etc. This misalignment is still somewhat incoherent, but it increasingly resembles what I call "fitness-seeking"—a family of misaligned motivations centered on performing well in training and evaluations (e.g., reward-seeking). Fitness-seeking warrants substantial concern.

In this piece, I lay out what I take to be the central mechanisms by which fitness-seeking motivations might lead to human disempowerment, and propose mitigations to them. While the analysis is inherently speculative, this kind of speculation seems worthwhile: AI control emerged from explicitly taking scheming motivations seriously and asking what interventions are implied, and my hope is that developing mitigations for fitness-seeking will benefit from similar forward-looking analysis.

Fitness-seekers are, in many ways, notably safer than what I'll call "classic schemers". A classic schemer is an intelligent adversary with unified motivations whose fulfillment requires control over the whole world's resources. Meanwhile, fitness-seeking instances generally don’t share a common goal (e.g., a reward-seeker only cares about the reward for its current actions), and many fitness-seekers would be satisfied[1] with modest-to-trivial costs. Still:

Fitness-seekers risk some[2] of the same catastrophic outcomes as classic schemers do [...]

---

Outline:

(03:43) Overview

(11:24) The basic reasons fitness-seekers might be safer than classic schemers

(15:53) Four mechanisms for risk and their mitigations

(16:38) Potemkin work

(22:05) Instability

(27:41) Manipulation

(31:17) Outcome enforcement

(36:44) Cross-cutting mitigations

(37:10) Deals

(39:57) Control

(44:37) Alignment

(44:40) Preventing fitness-seeking from arising

(48:47) Making any fitness-seeking motivations safer

(51:27) How does online training change the picture?

(55:04) Overall recommendations

(59:08) Conclusion

The original text contained 18 footnotes which were omitted from this narration.

---

First published:
May 1st, 2026

Source:
https://www.lesswrong.com/posts/9YCJZBtqr3FYL8rDp/risk-from-fitness-seeking-ais-mechanisms-and-mitigations

---

Narrated by TYPE III AUDIO.