A few months ago, when we first started talking about the Science of Responsible Innovation at The Connected Ideas Project, I kept coming back to a simple question:

How do we know?

How do we know whether a technology is actually as powerful—or as dangerous—as we imagine?

How do we know whether our fears are grounded in evidence or in extrapolation?

How do we know whether policy is steering something real, or something hypothetical?

It’s one thing to run a model through an in silico benchmark and watch it ace a virology exam. It’s another thing entirely to put a pipette in a novice’s hand and see what happens in a real lab.

That’s why the recent paper, “Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology,” feels so important.

Not because it proves that AI is safe.

Not because it proves that AI is dangerous.

But because it does something rarer and more valuable: it measures.

And in doing so, it gives us a template for what responsible-by-design evaluation can look like in the age of frontier AI and synthetic biology.

The podcast audio was AI-generated using Google’s NotebookLM.

The Gap Between the Benchmark and the Bench

For the last several years, large language models have been climbing biological benchmarks at an astonishing rate. Protocol design. Sequence interpretation. Troubleshooting. Literature synthesis. In some cases, outperforming domain experts on structured tests.

On paper, that looks like capability. And capability, when it intersects with viral reverse genetics or synthetic biology, looks like risk.

But as I’ve discussed in recent work on Violet Teaming—particularly in “The Promise and Peril of Artificial Intelligence — ‘Violet Teaming’ Offers a Balanced Path Forward” —capability is not impact. And risk is not hypothetical power alone. It’s what happens when humans, institutions, and technical systems interact in the real world.

The authors of this new study understood that.

So instead of running another benchmark, they ran a randomized controlled trial. In a real BSL-2 laboratory. With 153 novices. Over eight weeks. Across five hands-on biological tasks modeling a viral reverse genetics workflow.

Not a chatbot demo.

Not a thought experiment.

A physical lab.

That matters.

Because biology isn’t just text. It’s tacit knowledge. It’s sterile technique. It’s muscle memory and timing and pattern recognition. It’s knowing when a cell culture “looks off.” It’s knowing that the protocol you copied from a paper assumes three unstated steps.

Benchmarks rarely capture that.

The study did.

And the results are, in a word, humbling.

What the Study Actually Found

The primary question was straightforward: does access to mid-2025 frontier LLMs significantly increase a novice’s ability to complete a sequence of tasks modeling viral reverse genetics?

The answer, in binary terms, was no.

Completion of the core workflow was low in both groups—LLM-assisted and internet-only—and there was no statistically significant difference in full workflow completion.

If you stop there, you might conclude: the models don’t matter.

But that would be the wrong lesson.

Because the study also found something more subtle—and arguably more important.

Across individual tasks, LLM-assisted participants were more likely to progress further through procedural steps. In cell culture, they completed tasks faster and with fewer attempts. Bayesian modeling suggested a modest uplift—on the order of ~1.4× for a “typical” reverse genetics task—though with uncertainty bounds that rightly temper interpretation.

In other words: not a revolution.

But not nothing.

And this is where responsible innovation becomes interesting.

Why This Is Violet Teaming in Practice

When Adam Russell and I first articulated the idea of Violet Teaming, we described it as the integration of red teaming (adversarial probing), blue teaming (defensive hardening), and ethical design into a proactive, sociotechnical framework .

Most conversations about AI and biosecurity oscillate between red and blue:

Red: “What if this model can design a pathogen?”

Blue: “Let’s add filters, classifiers, restrictions.”

What this study does is different.

It asks: what is the real-world uplift? How much does LLM assistance actually change novice capability in a physical lab? Not in theory. Not in speculation. In practice.

That’s violet.

Because it embeds evaluation into the design and governance process itself.

Instead of arguing over worst-case extrapolations, we now have empirical data about:

* Completion rates

* Time-to-task

* Procedural progression

* Human–AI interaction patterns

* Elicitation failures

* Usage intensity and its (lack of) correlation with success

That last point is particularly striking. Participants who used LLMs more did not necessarily perform better. There was no clean dose–response curve.

That’s not a trivial observation.

It tells us that raw access is not the same as effective amplification. It suggests that prompting skill, interface design, cognitive scaffolding, and user expertise mediate uplift.

And that means risk is not simply a function of model weights. It’s a function of the entire sociotechnical system.

That’s violet territory.

The Most Important Finding: The Gap

To me, the most important result is the documented gap between in silico benchmark performance and physical-world utility.

This is not an indictment of benchmarks. They serve a purpose. But they are not reality.

A model can generate a flawless text protocol for molecular cloning and still fail to help a novice identify the correct reagents from a messy inventory spreadsheet. It can hallucinate a DNA sequence that looks plausible but is wrong in a way a novice cannot detect. It can provide text-based instruction where video-based tacit demonstration might matter more.

In the study, YouTube was often rated as more helpful than any individual LLM.

That’s not because YouTube is smarter. It’s because biology is embodied.

This is precisely the kind of nuance that responsible innovation requires.

Without physical-world validation, we risk building policy on top of performance claims that don’t map cleanly onto human capability.

This study doesn’t close the gap. It reveals it.

And revelation is the first step toward responsibility.

Responsible-by-Design Requires Quantification

One of the themes we’ve explored in the Science of Responsible Innovation is that values without metrics are aspirations. Metrics without values are optimization problems.

We need both.

This study provides something we’ve been missing: a quantifiable baseline for novice uplift in a dual-use biological workflow.

Not a theoretical upper bound.

Not a catastrophic scenario.

An empirical distribution.

The Bayesian estimates even put a 95% credible upper bound around uplift (~2.6×), which matters enormously for policy calibration.

If you’re designing guardrails, export controls, compute thresholds, or deployment policies, you need to know: are we talking about a 10× amplification? A 2× amplification? Or something closer to noise?

This paper suggests modest uplift under the conditions studied.

That doesn’t eliminate risk. It contextualizes it.

And contextualization is the heart of responsible governance.

Where the Study Can Go Next

Now, let’s be honest.

As strong as this study is, it is not the final word. It’s the first serious step.

And if we want this to become an evolving framework for violet teaming and responsible-by-design evaluation, we need to iterate.

Here are several ways I believe the next generation of this work could build on this foundation.

1. Extend the Time Horizon

Eight weeks is meaningful. But complex biological workflows often require longer timeframes for skill acquisition.

Low completion rates may reflect not just capability limits, but time constraints. A longer intervention period could reveal whether modest early procedural uplift compounds into higher eventual completion.

Responsible innovation must account for trajectory, not just snapshot.

2. Integrate End-to-End Workflow

The tasks were decoupled into discrete components. That’s methodologically clean, but real-world risk emerges from integration.

A future iteration could test whether novices can string together multiple steps into a coherent, self-directed project—while still maintaining appropriate biosafety controls.

3. Compare Model Generations Longitudinally

The models tested were mid-2025 frontier systems. Biology-specific models are already emerging.

A longitudinal design—repeating the same protocol annually—would allow us to empirically track uplift curves over time.

That would be invaluable for macrostrategy. Instead of forecasting speculative capability growth, we could measure it.

4. Test Interface Scaffolding

The study hints that elicitation constraints matter. Novices may not know how to ask the right questions.

What happens if we add structured prompting interfaces? Visual overlays? Augmented reality guidance? Automated error-ch...

Ep 59 - A New Study Taking Responsible Innovation From Benchmarks to Benchwork