Eye On A.I. Episode #330

Sebastian Risi: Why AI Should Be Grown, Not Trained

Date: April 2, 2026
Host: Craig S. Smith
Guest: Dr. Sebastian Risi (Co-founder, Sakana AI; Professor, ITU Copenhagen)

Episode Overview

This episode explores a paradigm shift in artificial intelligence: moving from "training" static neural networks via gradient descent, to "growing" adaptable, evolvable networks inspired by nature's principles. Dr. Sebastian Risi shares insights from his research, discusses the power of neuroevolution and plasticity, and unpacks how evolutionary strategies and developmental growth could lead to more resilient, continually learning, and creative AI systems. The conversation spans neural plasticity, evolving architectures, model merging, artificial life, open-endedness, and the future role of AI in scientific discovery.

Key Discussion Points & Insights

1. Neuroevolution: Evolving Intelligence Inspired by Nature

• Definition and Motivation
  • Neuroevolution integrates evolutionary algorithms with neural networks, optimizing not just weights (like in backpropagation), but also architectures, learning rules, and hyperparameters (01:20).
  • “Instead of training networks with, you know, like gradient descent or reinforcement learning, we can take an example of how nature evolved intelligence and use evolution instead.” — Sebastian Risi (01:23)
  • Evolutionary methods do not require differentiability, unlike gradient descent, making them suitable for non-differentiable or discrete problems.
• Advantages Over Backpropagation
  • Can optimize discrete structures, architectures, and rules, not solely pre-defined weight matrices.
  • Population-based search finds solutions that can "jump" across complex solution spaces by crossover, not just small, local tweaks (04:15).
  • Ensures diversity and resilience, sometimes bypassing weaknesses of local search in gradient-based methods.

Notable Quote

“You don’t need to backpropagate gradients. You have a population that’s distributed on this landscape... you kind of sample, like evolution strategy, and can locally sample in many places at same time. That gives you a direction to go.” — Sebastian Risi (04:13)

2. Plasticity and Continual Learning: Networks that Adapt

• Plasticity Inspired by the Brain
  • The team uses local learning rules (e.g., Hebbian learning: “neurons that fire together wire together”), evolved via genetic algorithms, allowing continual adaptation (06:45).
  • “The main idea is that the weights never stop changing. Like, your brain is not frozen at some point, but it keeps changing.” — Sebastian Risi (08:45)
• Advantages of Plastic, Growing Networks
  • Such networks can adapt to drastic changes in input or even physical structure (e.g., robotic leg removal) in real time — unlike static networks.
  • Suggests greater resilience and potential for lifelong learning tasks.

Notable Quote

“When you use a static fixed network that is not changing...if you cut off a leg, it will probably fail...But these Hebbian networks, they change the weights all the time... oftentimes it will still be able to function, even though it has never seen this kind of variation during training.” — Sebastian Risi (00:00 and 08:00)

3. Neuromodulation and Balancing Learning

• Avoiding Catastrophic Forgetting
  • Neuromodulation is inspired by the brain’s mechanism for gating learning: certain neurons switch learning on and off for others, preventing overwriting of important information (09:39).
  • “You have another type of neuron... that tells parts of the network when should learning be switched on and off.” — Sebastian Risi (09:39)

4. Growing Networks: From Neurogenesis to Artificial Development

• Networks that Grow During their 'Lifetimes'
  • The “Grow AI” project seeks to mimic biological neurogenesis and morphogenesis—growing networks from a single node; the developmental “DNA” encodes growth logic, not the final brain (10:35).
  • Environments can influence how growth unfolds, leading to contextually adapted architectures (11:00).
  • Developmental programs are neural nets themselves, embedded in each node, which regulate when to split or connect neurons (13:20–16:11).
• Scalability & Challenges
  • Early experiments “grow” networks to a few thousand nodes on simple tasks (16:50), but scaling requires balancing growth (to avoid bloated networks) and plasticity.
  • Multi-objective optimization helps by rewarding both functionality and efficiency/sparsity (17:32).
• Continual Learning in Growth
  • The developmental model must avoid “forgetting” how it developed its earlier structures — an open challenge (17:32–19:22).

Notable Quotes

“Nature is very good at it—learned how to make a butterfly, then learned how to make a butterfly with different eye spots. In these representations… you want it to grow a certain network and then learn more and add structure, but not forget how to grow the first part.” — Sebastian Risi (17:32)

“If we figured out how to grow it, we should be able to really scale it up… there might be really interesting dynamics hidden if you scale it up, not letting structure be fixed, but letting it be grown and determined by this process.” — Sebastian Risi (19:40)

5. Scaling, Training, Inference: Open Challenges

• Training “Grown” Networks
  • What’s truly “trained” is the developmental program. Resulting networks could theoretically be further trained in downstream tasks using supervise or RL, but evolution tends to be more general (21:44–21:57).
• Deception & Open-Endedness
  • Evolutionary methods must circumvent deceptive traps, e.g., solutions that seem good but block real progress. Quality-diversity methods are used to keep exploring the space (21:57–25:18).
  • Example: T-maze task demonstrates why scoring solely for immediate performance leads networks into dead ends, while “evolving to learn” pushes towards deeper adaptability.

6. Model Merging & Incremental Learning

• Evolutionary Model Merging (Sakana AI)
  • Combine layers from specialized pre-trained models (e.g., one good at Japanese, another at math) via evolutionary search to produce models with composite skills (26:08–27:46).
• Potential for Incremental, Modular Learning
  • Future aim: Incrementally “growing” models that can add entirely new skills (like math proficiency) without overwriting existing knowledge—enabling true continual learning (26:08–27:46).

7. Artificial Life and Cellular Automata

• Artificial Life as Inspiration
  • Artificial life is “life as it could be,” pushing the boundaries of known biology via simulation (29:07–32:25).
  • Neural cellular automata and soft robotic morphologies are evolved to self-repair (e.g., Minecraft salamanders that regrow if cut in half) and show resilience missing in traditional deep learning.
• Key Properties Modeled
  • Growth, self-organization, self-replication, local communication, and inbuilt resilience.

Notable Quote

“Natural biological systems are incredibly resilient, and deep learning often…completely fails on weird examples. Using these systems that can self-organize…could make deep learning systems more robust and adaptive.” — Sebastian Risi (32:18)

8. Evolution as Scientific Discovery: The “AI Scientist”

• Combining Evolution and LLMs for Research
  • AI Scientist frameworks (e.g., Sakana’s “Shinka Evolve”) use LLMs to generate hypotheses/solutions (as mutation operators), evolutionary search to explore, and fitness evaluations to drive discovery (33:51–36:00).
  • Demonstrated by generating a workshop-accepted research paper entirely via this pipeline (38:03–39:37).
  • Open question: How creative can these systems truly be? Can they push well beyond their training data?

Notable Quotes

“You can use a language model as a mutation operator… you start with one example, then ask it for variations… then you evaluate and repeat. It’s a fruitful combination of evolution and large language models.” — Sebastian Risi (35:00)

“How do we best combine what humans are good at and what machines are good at? … Now it’s becoming a little more less clear. So I think that’s something we need to kind of figure out.” — Sebastian Risi (41:40)

9. Open-Endedness & Co-Evolution

• Continual Innovation in AI Agents and Environments
  • Open-endedness: Can we create systems that keep innovating and producing novel solutions indefinitely? (48:06)
  • Examples include the POET algorithm: evolving both agents and environments together, gradually increasing complexity to scaffold abilities that direct training cannot achieve (48:49–51:03).
  • Unity and LLM-generated environments: dynamically produce new, more complex worlds for agents to evolve within (51:03–52:05).

10. World Models and Internal Representations

• World Models
  • Early world model research (by Sakana’s co-founders) let agents train in simulated “dreams” to improve real-world performance (52:05–53:47).
  • Key challenge: Determining when simulative world models are essential versus when language-based reasoning suffices.
  • Rich internal representations enable agents to not just solve tasks, but anticipate and generalize across situations.

11. Why Evolutionary AI is Coming to the Fore

• Re-emergence with Generative AI
  • Evolutionary methods weren’t mainstream during the transformer boom but now pair especially well with LLMs—helping generate and search for new architectures and ideas where gradient methods are ineffective (54:38–56:00).
  • “Before, in neuroevolution, you always had to think about what does your representation look like… Now with a language model, you can just have that come up with the representation or be the representation.” — Sebastian Risi (54:44)
• Continuous Thought Machine
  • Sakana is developing architectures inspired by the complexity and asynchrony of biological brains—networks that can decide to “think longer,” synchronize internal neurons, and use more complex internal states (56:00–58:04).
  • Each neuron can itself be a neural network, able to better model biological features like oscillations and synchronization.

Memorable Moments & Quotes (w/ Timestamps)

• Resilience of Evolved Plastic Networks:
  “You cut off a leg, it will still be able to function, even though it has never seen this kind of variation during training.” — Sebastian Risi (00:00, 08:00)
• On Hebbian Learning vs. Backpropagation:
  “If two neurons always fire together, then the connection between them gets stronger. So it’s a local learning rule instead of this back propagation that is like this outside thing that changes everything.” — Sebastian Risi (06:45)
• Developmental Programs:
  “We call this a neural developmental program… a small neural network that runs in every neuron and can decide when to create nodes or adjust connections.” — Sebastian Risi (11:00)
• On the Limits of Language Models:
  “How far can it be pushed beyond what was in the training? I wouldn’t say that it cannot produce anything new… but how far is that outside of what it has seen?” — Sebastian Risi (43:38)
• On Human–AI Collaboration:
  “How can you make sure AIs and humans talk in the same language? … For me, the interesting part is how can we use this also as a co-scientist?” — Sebastian Risi (41:40)
• The Vision for Growing AI:
  “The ultimate goal: combine these smaller scale experiments to allow language models to do continual learning—growing new nodes and storing new learning, instead of overwriting.” (26:08)

Suggested Listening Timestamps

• Neuroevolution Explained: 01:20–06:16
• Brain Plasticity in AI: 06:45–09:25
• Network Growth & Development: 10:35–19:22
• Model Merging Examples: 26:08–28:54
• Artificial Life & Neural Cellular Automata: 29:07–32:25
• AI Scientist / Evolution + LLM for Research: 33:51–39:20
• Open-Endedness & Co-Evolving Environments: 48:06–51:03
• World Models Discussion: 52:05–53:47
• Return of Evolutionary AI: 54:38–58:04

Summary Tone & Language

Risi communicates with the excitement and humility of a scientist pushing into the unknown, constantly drawing analogies to biological systems, often qualifying statements with ongoing challenges, promising directions, and the collaborative spirit of combining evolutionary and deep learning approaches.

For Further Exploration

• “Neuroevolution” (Book by Risi et al.)
• Sakana AI and its open research
• “POET” and “Quality Diversity” algorithms
• Developmental & plastic neural architectures
• Recent papers on evolutionary model merging

Conclusion

Dr. Sebastian Risi makes a compelling case that true artificial intelligence may need to be grown, not merely trained. By harnessing biological principles—evolution, plasticity, growth, and open-ended innovation—AI systems can become more resilient, creative, and capable of continual learning and discovery. As fields blend, and with platforms like Sakana pushing the boundaries, the next AI leap may come not from bigger transformers, but from networks that evolve, adapt, and grow within ever-changing worlds.