In this episode of Token Engineering, Cooper sits down with Yash, Head of AI Research at Neurometric AI, to break down TMax from AI2 (Allen Institute for AI)—a fully open dataset and training recipe that pushes small open-weight models like Qwen to near-frontier performance on terminal agent tasks. The paper argues that diversity and difficulty matter more than where the data comes from—even when that data is entirely invented by Gemini rather than pulled from real-world code.

We talked about:

What TMax is, and why AI2 built a fully open recipe instead of a closed benchmark
Why Gemini-invented data beat real GitHub repos on diversity
Same recipe, opposite results: boosting one Qwen version, degrading the next
How small models “cheat” when a task is beyond them
Why a lighter harness outperformed a more complex one on Claude Haiku
Gains that spilled past Terminal-Bench into SWE-bench and AIME math
The compute cost of agentic RL training, and how runaway tool calls blow the budget
Neurometric’s “Harbor Master,” built to run evals past AI2’s narrow benchmark set

Resources Mentioned:

TMax: A recipe for terminal agents: https://arxiv.org/abs/2606.23321

Connect with Neurometric:
Website: https://www.neurometric.ai/

Substack: https://neurometric.substack.com/

X: https://x.com/neurometricai/

Bluesky: https://bsky.app/profile/neurometric.bsky.social

Host/s:

Calvin Cooper

https://x.com/cooper_nyc_

https://www.linkedin.com/in/coopernyc

Guest/s:

Yash Sharma

https://x.com/yash_j_sharma

https://www.linkedin.com/in/yashjsharma