Subtitle: AI can sometimes distribute cognition over many extra tokens.

Prior results have shown that LLMs released before 2024 can’t leverage ‘filler tokens’—unrelated tokens prior to the model's final answer—to perform additional computation and improve performance.1 I did an investigation on more recent models (e.g. Opus 4.5) and found that many recent LLMs improve substantially on math problems when given filler tokens. That is, I force the LLM to answer some math question immediately without being able to reason using Chain-of-Thought (CoT), but do give the LLM filter tokens (e.g., text like “Filler: 1 2 3 ...”) before it has to answer. Giving Opus 4.5 filler tokens2 boosts no-CoT performance from 45% to 51% (p=4e-7) on a dataset of relatively easy (competition) math problems3. I find a similar effect from repeating the problem statement many times, e.g. Opus 4.5's no-CoT performance is boosted from 45% to 51%.

Repeating the problem statement generally works better and is more reliable than filler tokens, especially for relatively weaker models I test (e.g. Qwen3 235B A22B), though the performance boost is often very similar, especially for Anthropic models. The first model that is measurably uplifted by repeats/filler is Opus 3, so this [...]

---

Outline:

(04:06) Datasets

(05:57) Prompt

(06:55) Results

(06:58) Performance vs number of repeats/filler

(08:28) Alternative types of filler tokens

(09:01) Minimal comparison on many models

(10:02) Relative performance improvement vs default absolute performance

(13:52) Comparing filler vs repeat

(14:37) Future work

(15:46) Appendix: How helpful were AIs for this project?

“Recent LLMs can use filler tokens or problem repeats to improve (no-CoT) math performance” by Ryan Greenblatt

Powered by Wave AI