Building Self-Accelerating AI with Mirendil

Summary

Podcast Summary: AI + a16z — "Building Self-Accelerating AI with Mirendil"

Date: June 24, 2026
Guests: Benham Nesherbor & Harsh Mehta (Mirendil Co-Founders)
Host: Matt Bornstein (a16z)

---

Overview

This episode explores the radical concept of "self-accelerating AI": artificial intelligence systems that can contribute to and even drive their own advancement. Mirendil, a company at the forefront of this movement, aims to build AI that can accelerate not just its own research, but engineering and scientific progress overall. The conversation covers the history and motivations behind Mirendil, technical and organizational challenges, why access and incentives matter, and what a future powered by self-improving AI might look like.

---

Key Discussion Points and Insights

1. What is "Self-Accelerating AI"?

• The core idea is AI not just performing tasks, but improving its capacity to perform those tasks — especially the tasks of AI research and development itself.

• This concept draws on long-standing ambitions in AI, now made practical as models become increasingly adept at coding, researching, and reasoning.

• "What does it take to have an AI system that is able to be directed at the direction and make improvements faster and faster... and that's the self accelerating AI technology."
  — Benham Nesherbor, [04:37]

• Mirendil focuses on enabling AI to conduct its own engineering and research, targeting "superhuman" scientific problems that transcend any single human expert or lab.

2. The Path to Accelerated Scientific Discovery

• Grand scientific and engineering challenges require deep expertise and iterative problem-solving — skills that are hard to scale with people alone.

• Mirendil wants to minimize the AI research bottleneck so that scientific labs can focus on their domains without needing huge AI teams.

• "Eventually, you wouldn't need to hire on the AI side so you can move much faster when it comes to solving the problem."
  — Benham Nesherbor, [08:33]

• The vision is for every scientific or engineering team to have access to highly specialized, adaptable AI that they can own and tailor to their needs.

3. Generalist vs. Specialist AI for Scientific Progress

• General-purpose language and coding models have improved, but often still lack the nuanced expertise needed for domain-specific breakthroughs.
• Oversight — the need for humans to intervene or correct — is a key bottleneck. Reducing oversight means more efficient, long-running AI experiments.
• "Even the tiniest reduction in oversight can lead to large amounts of token spent and just effective outcomes being generated."
  — Harsh Mehta, [10:33]

4. Industry Incentives, Access, and the Need for New Organizational Models

• Mirendil’s founders observed that large labs are often incentivized to restrict broad access to advanced AI research technology due to competitive and safety concerns.

• Existing business models (e.g., "train a big model and charge for access") can create dependencies that stifle broader progress.

• "With disruptive technologies you need to rethink a lot of pieces... It's very hard to have a disruptive technology show up in an existing company and flourish."
  — Benham Nesherbor, [14:33]

• Mirendil aims to empower organizations and individuals with infrastructure and ownership over their own AI, countering centralization trends.

5. Safety, Guardrails, and Open Access

• Not all AI capabilities are equally risky, but broad restrictions (such as those for AI research tooling) can slow scientific and engineering progress for everyone.
• It's important to develop fine-grained safety, not "throw a large hammer" blocking all use cases.
• "Because of our focus and... the intent to advance science, we want to take like a sharper approach where we put the energy into making sure that in every use case that we are offering this technology, we can make sure that it's being used in the most positive and safe place."
  — Harsh Mehta, [19:58]

6. The Mirendil Product Vision

• Their models and products look similar to existing state-of-the-art, but are "sharper" for engineering and scientific applications (e.g., low-level kernels, frameworks, training pipelines).
• The goal: Make these capabilities highly accessible and modular for teams training or serving their own models.
• "Every business, every lab would have their own AI optimized for their own workflow... This is like a much more enabling future."
  — Benham Nesherbor, [21:52]

7. Organizational and Systemic Scaling Challenges

• Mirendil is experimenting with how to scale systems of AIs (and people) so that adding agents increases real productivity — not the diminishing returns seen when scaling up teams in conventional organizations.
• The "self-accelerating" notion is increasingly about building complex ecosystems of AI agents and humans, with evolving oversight and collaboration patterns.
• "These systems today — they're like systems composed of people and agents... But the question is really how do you get a favorable scaling of a system?"
  — Benham Nesherbor, [30:14]

---

Notable Quotes & Memorable Moments

• "If you look at the history of our field, in some sense AlphaGo and self-play loops are already a form of self-accelerating AI."
  — Harsh Mehta, [06:18]

• "It is funny. It's almost easier to use model version numbers as date, you know, as like time reference points now, rather than years and months."
  — Matt Bornstein, [14:00]

• "A lot of the science... is just unknown at the moment. And a lot of the engineering... are also just bottlenecked by people being able to design things well and just the intelligence being the bottleneck."
  — Harsh Mehta, [35:07]

• "Automating people's job away... that doesn't seem like an exciting future. We are trying to in a way focus on building a different one that has more positive outcome for everyone. And we think accelerating science is pure good for humanity."
  — Benham Nesherbor, [35:44]

• On organizational scaling: "It's actually not a straightforward problem how to create a system that is fast-moving... There's a system of AIs that are not necessarily the same model... and then some people, and then this system wants to kind of keep getting better."
  — Benham Nesherbor, [28:20]

• "If you want to get scaling laws right, you have to start small and get the scaling right before you scale up."
  — Benham Nesherbor, [31:27]

---

Highlighted Timestamps

• [02:18] — Mirendil founding story and motivation
• [04:37] — The challenge of scientist-level AI expertise
• [06:18] — Definitions and versions of self-accelerating AI
• [08:33] — Minimizing the AI research bottleneck
• [10:33] — Oversight and “dumb mistakes” in current models
• [14:33] — Why disruptive tech requires new companies
• [19:58] — Approach to safety and access vs. blanket restrictions
• [21:52] — Vision for customizable, decentralized AI infrastructure
• [30:14] — System scaling and productivity challenges
• [35:07] — AI's role in unlocking unknown science and engineering
• [35:44] — Making AI serve positive scientific and societal goals

---

Closing Thought

Mirendil represents a bold push to realize the promise of AI as a transformative, democratizing force for science and engineering. By making self-accelerating AI accessible—and focusing it on grand human challenges rather than pure automation or centralization—the founders hope to reshape the way knowledge and progress unfold.

---

For listeners who want to skip the theory and get practical:

• Watch for Mirendil's forthcoming models and products focused on empowering organizations with specialized, modular AI for science and engineering.
• The experiment isn't just technical—it's about new incentive structures, access models, and building teams (of humans and AI) for the next generation of breakthroughs.

Transcript

[Benham Nesherbor]

With an AI technology that is helping with AI research, what is the role of everyone else in the company? This is a very disruptive technology. With disruptive technologies you need to rethink a lot of pieces to kind of enable them to actually grow and flourish. And some of these pieces are how you build a company around it. If the business model of the company is I train a big model and charge people for using it, how is this company incentivized to share this technology with everyone else?

[Harsh Mehta]

We've been at the Frontier labs for a long time and we know how much work it is to do something and we've been able to do it in like maybe 10 times less people and less resources. The jump from Sonnet 3.5, 4%, 4.5 has been materially better in terms of like, where does it get stuck? Where does it need oversight? Even the tiniest reduction in oversight can lead to like large amounts of token spent and just there's effect of outcomes being generated.

[Matt Bornstein]

What you guys are doing is almost the next level of cheat code. When the Internet gets faster, you don't like recursively get faster Internet, where do you think it all ends up?

[Harsh Mehta]

It's a wide spectrum of outcomes.

[A16Z Podcast Host]

One of the oldest ideas in AI is also one of the most ambitious. The idea that intelligent systems could help improve themselves. For decades that possibility remained largely theoretical. But as AI systems become more capable at coding, research, reasoning and engineering, the question is becoming increasingly practical. What would it mean to build AI systems that contribute to their own development? And if that becomes possible, where should that capability be directed? Matt Bornstein speaks with Meerindil co founders Benham Nesherbor and Harsh Mehta about self accelerating AI scientific discovery and why they believe the most important applications of AI may be in advancing science itself.

[Matt Bornstein]

Benham Harsh, welcome to the A16Z podcast. It's awesome to have you here. Benham and Harsh were most recently research scientists at Anthropic. Worked together before at Google, at Blue Shift Labs, which Benham, you are a founding member of. Thrilled to have you here. Tell us a little bit about Mirandil, what you're aiming to accomplish and tell us where the company came from.

[Benham Nesherbor]

A lot of reasons behind building the company comes back from when Scaling Wow was happening at OpenAI and back then I was at Blueshift team and that was the moment when I realized we are on the verge of a revolution and then everything is about to change. And for me the most important thing was what are the technologies that would accelerate all areas of science and what are the main bottlenecks for making that happen? And since then I've been kind of trying to close, remain close to that path and remain on the shortest pat and non harsh since then and kind of trying to be on that path. And more recently as we've been in anthropic, we've talked a lot about what does it mean to kind of remain on this shortest path. And we felt like the main labs are starting to diverge a little bit from what does it take to accelerate all areas of science. And that led to studying Mendel. We think the self accelerating AI is a technology that is disruptive and at the same time it's important for accelerating science and technology and we want to focus on building that and making it available.

[Matt Bornstein]

So science is notoriously hard, right? I mean science is science. It's an experimental discipline where you have to run experiments in the real world to understand the laws of physics or biology or so forth. Can you guys describe a little bit what do you think is the shortest path for AI to sort of accelerate science and why.

[Harsh Mehta]

So from last five years when the models kind of like started kind of like getting better at some of the primitives of conducting science, like high school math and college math and then a little bit of coding and then now the coding models are really good compared to use Sadiq Ruli and this is

[Matt Bornstein]

this all in scope? When you say science, by the way, I just want to make sure are we talking about strictly like physical, you know, natural sciences or is it a broader set of things?

[Harsh Mehta]

So ultimately what we want to build is like AI systems which in a very broad sense can conduct AI research and engineering itself. And if it has this capability in a very broad sense, it has the right primitives and the capabilities which are needed to conduct any science which is in the realm of digital world or any science which is in the physical world, but then has a digital component as well.

[Matt Bornstein]

And so what do you think is sort of the path and maybe contrast to what the major labs are doing? Because people may be familiar with that at some level.

[Benham Nesherbor]

Yeah. So when you think about what is a scientist or what does a scientist or an engineer do? The main skill they have is get deep, very deep in a domain and build expertise. And they build a very sharp expertise that accumulates over time. And as you kind of become expert in the domain, your expertise is becoming sharper and sharper in something that has tiny bit of volume in the whole like set of possible things you could be working on and getting to that point, like Being able to get there is a capability that is needed to advance all areas of science. What does it take to have an AI system that is able to be directed at the direction and make improvements faster and faster and gets eventually to the edge of that area and eventually make like progress? And what that view means is that you have to work on what does it take to direct a system toward a particular angle and make fast improvements. And that's the self accelerating AI technology. So this in principle is very disruptive because this technology allows a model to kind of keep making improvements and keep getting better at that direction. And as when you think about the business model you can build around this, et cetera, it's just very different world where you want to make this available to everyone as opposed to make AI available to everyone.

[Matt Bornstein]

Yeah. So explain this term self accelerating AI. I'm like a huge sci Fi fan. So I've always hoped that you know, the machines would start improving themselves, they'll

[Harsh Mehta]

take over the universe.

[Matt Bornstein]

But if that doesn't seem to be happening, you know, and I'm not sure if it really is possible or makes sense. So explain what you think of like how is this actually going to happen? And so what does self accelerating AI really mean?

[Harsh Mehta]

I think I have many different definitions and we have a specific version of it which we are targeting. If you look at the history of our field, in some sense AlphaGo, where we can create like all these self play loops is already self accelerating AI. Another version of it is if we can put AI in an unknown domain or an environment, can it learn by itself and improve its own capabilities and context? That is another version of self accelerating.

[Matt Bornstein]

And this is sort of like the ARC AGI benchmark for instance, where it's sort of dropped into an environment and has to like understand the rules before optimizing them.

[Harsh Mehta]

Correct. And more like inside enterprises or general knowledge work. Another example could be like a system which ends inside a company. If an employee kind of like enters a company, then they have to gain all this context to function well. And can an AI system do something like that? Our version of self accelerating AI is can these systems do the work of an AI researcher or an AI engineer? Like everything from the low level kernels to works to conducting research, it's often a very high throughput way. And we think that that's kind of like a sharp version of self accelerating AI will lead to a broad version of self accelerating AI which kind of ultimately has the capability to conduct kind of like advanced science in a more broader way.

[Matt Bornstein]

And so it sounds like part of what you're saying is that scientists kind of have to become AI people in a way. And they're not right going to like do it on their own, but like the tool that you're building for them can help them actually sort of become AI people.

[Benham Nesherbor]

Yeah. So one way we like to think about this is if you think about each areas, each of these important problems that exist in science and technology and what does it take to solve these problems. These are grand challenges, that they're not human level problems. These are superhuman level problems, things that we don't think people can solve and we don't think it's enough to build an AI scientist that would do the job of a scientist. These are superhuman problems. So what does it take to solve these problems? And each of these problems need a periodic lab, which is bunch of experts who understand the domain and are excited about kind of attacking this problem directly. And then a whole big AI effort just to push AI for that particular problem. And the thing that's difficult to do for every other club is assemble their best frontier AI team and kind of iterate over the AI solution that's best for them. And what we want to do is take that part and minimize it, make it small so that you can maybe today, if you need 200 people to have a good frontier AI lab and 200 of the best people, which is almost impossible to do, we want to reduce it to 10, 2 and eventually 1. And eventually you wouldn't need to hire on the AI side so that you can move much faster when it comes to solving the problem. Because each of these problems have a lot of other constraints to solve, like physical elements like getting access to data and all the other bottlenecks that exist. So we just want to remove the AI bottleneck.

[Matt Bornstein]

And this has been sort of a long standing goal in the machine learning community. You know, I remember even 10 years ago, Google had a product, it was sort of like a neural architecture search thing. But they sort of claimed like machine learning is now solved. Right. You just run the computer and it finds the right architecture for you. What's changed now or what's the state of the industry on self acceleration now?

[Harsh Mehta]

I think that's a great example. And in some sense it kind of like boils down to the intelligence of the agent or the system which is conducting this search. And as you move the intelligence dial higher and higher, you are much more compute efficient in getting the results that you want. And that has been happening from that particular point to kind of like where we are today. And it's also happening in a very broad way. It's not just architecture search that you can do, but much more large swaths of software that you can improve on its own. And it's a very expressive space of things that you can do right now.

[Matt Bornstein]

Can you explain this point about general versus narrow? And I'm guessing you don't want to reveal all the secrets of the models you're training, but I think it's a really important question. Will general language models or general coding models be able to run these sorts of AI experiments or is something more focused necessary?

[Harsh Mehta]

So I think the importance about kind of the general kind of sense of, and the taste of these AI models is if you kind of turn down the intelligence dial and make them focused on something very specific, then they would kind of still make very dumb mistakes, not wire things in the code properly, or they wouldn't have the right kind of mindset of an expert in that particular field. The jump from Sonnet 3.5, 4 opus and 4.5, that has been kind of like materially better in terms of how long can we run an AI system on a particular problem, or where does it get stuck, where does it need oversight? And every reduction and even the tiniest reduction in oversight can lead to large amounts of token spent and just effective outcomes being generated.

[Matt Bornstein]

And so you were actually at Anthropic when a bunch of these models released that you just mentioned. And I know you were starting to work on some of these ideas then. Can you just explain a little bit of the history behind this project that you're working on now?

[Harsh Mehta]

Yeah. So Benham and I have been excited about this grand challenge that he mentioned about accelerating science with AI. And this was five years ago, and we have been at it for a while.

[Matt Bornstein]

One of these overnight breakthroughs that takes five years to develop.

[Harsh Mehta]

Yeah, exactly. We had to start with pretty bad models, models which are not good at even high school math or autocomplete coding and stuff like that. So we first built the math specialized models at Gemini and worked on the reasoning models. And then some of the primitives kind of started to come together when we joined Anthropic. And that felt like the right moment where we can actually be ambitious enough to directly attack the problem of AI being able to conduct its own engineering and research. And the biggest jumps, kind of like Dario calls it, a smooth exponential. And scale has been very helpful, both in terms of the underlying models being better, but also conducting the infant's time kind of like you know research and engineering at a very high throughput rate.

[Matt Bornstein]

I'll embarrass you a little bit Harsh. You told me when you started working on this at Anthropic, nobody else wanted to work on it with you. Is that true?

[Harsh Mehta]

I think they had good reasons.

[Matt Bornstein]

Meaning the models didn't work yet or.

[Harsh Mehta]

Yeah. So like, you know, when we joined, like it was like, it feels long time ago but like, you know, the state of the art model was Sonnet 3.7. And we are, I think I was too excited in some sense to kind of like jump in and attack this problem. And one of the reasons why I joined Anthropic was like, you know, Dario had this essay like Machines of Loving Grace, which was very inspiring. And it really felt like the place where this kind of ambitious work can be done. And uh, that, that's why I felt the conviction to start the.

[Matt Bornstein]

It is, it is funny. It's almost easier to use model version numbers as date, you know, as like time reference points now, not rather than like years and months. Like, like I can actually understand like what you're indicating when you say 3.7 versus if you said like 2022, I'm

[Harsh Mehta]

like, what's going on then.

[Matt Bornstein]

Yeah, no, I. And I think a lot of people feel that way. I guess. What gave you guys the sort of impetus to go out and start Mirandil having been in that position and having done some interesting work in Antropic?

[Benham Nesherbor]

I think so I was co leading the science team in Anthropic. Harsh was kind of initiator that led the automated pre training project in Anthropic. And we were both talking about like, what does it take to kind of apply this to science and accelerate science. And one of the. There are a few observations that kind of made us decide to take this challenge. One is this is a very disruptive technology. With disruptive technologies you need to rethink a lot of pieces to kind of enable them to actually grow and flourish. And some of these pieces are how you build a company around it. You can't let. It's very hard to have a disruptive technology show up in an existing company and flourish because a lot of elements have to be redefined. Like with an AI technology that is helping with AI research, what is the role of everyone else in the company and how do they rethink? And it's very hard. If the culture is not built for it, it's going to be harder. The incentive of the business model of the company. If the business model of the company is I train a big model and charge people for using it. How is this company incentivized to share this technology with everyone else? Because that's directly letting everyone else train a model which reduces their dependency to the company. So these are like fundament so fundamental that it doesn't matter who runs the company. But you can clearly see that it's in, in in in contrast to what you want to happen. And those observations and seeing that these are practical limitations would let us to kind of start a company that is really rethinking all pieces for this technology and making it available to accelerate science.

[Matt Bornstein]

So just because you mentioned it, I have to ask about the fable launch right where it's we've guard railed certain important safety areas like bioweapons and et cetera and AI research. One of these things seems a lot less dangerous than the other things. Do you think that access to cutting edge AI research assistance is going to be curtailed or inaccessible to most people if, if you know, you guys don't succeed or companies like Marindale don't succeed?

[Benham Nesherbor]

Yeah, I think there are good reasons for why you want to be careful about the technology and it's very powerful.

[Matt Bornstein]

At the same time, can you actually sketch out the pro case? So for me as sort of a non expert, right to see it's like okay, I understand why bioweapons are dangerous, why drug synthesis is dangerous. There's a bunch of things that actually should be filtered out of pools of information. Generally the AI training one stood out to me. It's like well this actually seems like a positive good. Why would we. But since you guys are sort of on the inside of it, maybe sketch out the case for why that does make sense.

[Benham Nesherbor]

So there are a few cases. One is if you think about as let's say speaking of anthropic, if you're concerned about other states developing and competing with us, then you would be concerned about them using your technology to move faster. And I think that has been like a concern from Anthropic. And the other concern is let's say for bioweapons. If you're concerned about bioweapons then you'd be worried about someone using this technology to build a model that would help them with bioweapons. So it's very powerful, it can be applied to anything. So there's some merit in being worried about this. At the same time, the same way as AI came to existence and it was a powerful technology and you have to have considerations around it, but you can't limit the whole technology because it's very useful. It's true for self accelerating AI. It's powerful, it's very enabling. And what it really requires is rethinking how you should do safety, how you should, you know, think about guardrails, how you, you know, when you build a company around it, you would be focused on how to get this right as opposed to when you're trying to do a lot of things. Then you would be looking at this and like, okay, this is too much, too much trouble for me. And, and, and also it's very hard to separate incentives from, from reasons and it's just get melted. It's just hard to separate it because you're also like deep, deeply incentivized to, to you know, like to keep your distance with others, et cetera. And you know, it doesn't matter who runs the company. It's just like when the incentives are set this way, it's hard to fix it.

[Matt Bornstein]

No, that's fair. And suffice it. I guess I'm inferring from what you're saying that both the incentives and sort of the value structure of Mirandil are designed to enable access to these sorts of technologies rather than curtail them.

[Harsh Mehta]

Yeah, yeah, there's a kind of like in a very large benefit that we can get from offering this technology and there's also harm and some companies would prefer to kind of like, you know, throw a very large hammer that okay, we block access to everything but because of our focus and our kind of like, you know, the intent to advance science and we want to take like a sharper approach where, where we put the energy of making sure that in every use case that we are offering this technology, we can make sure that it's kind of being used in the most positive sun and safe place.

[Matt Bornstein]

That makes a lot of sense. Do you want to talk a little bit about the products and models that you plan to launch? Again, don't need to reveal anything super secret because I know you haven't launched yet, but it's a great chance just to tell people what they should look forward to.

[Harsh Mehta]

Yeah, I think in a broad sense our kind of like models and kind of like product look very similar to existing state of the art models, but sharper in the specific capabilities which are needed to conduct engineering inside like an AI lab or anyone who is really trying to train or serve their models. So the hope would be that it would be really good at kind of like, you know, everything from low level kernels to like, you know, Libraries like Pytorch, jax, RL frameworks, prefraending frameworks and the product will kind of like amplify these capabilities to make it very, very accessible for anyone who's doing this kind of work and like specialize towards that.

[Benham Nesherbor]

I want to add maybe one other point related to kind of our approach and how like how we are thinking about incentives, et cetera. So you know we are, the way we are thinking about the product and how we are forming relationship with the rest of the world is how can we enable businesses to start owning more pieces, to have their own infra, their own AI. And I think what we are noticing today is that as a result of creating more dependency to these big AI labs gradually losing control and losing bigger parts of the business and then it becoming weaker and weaker. What we want to do is kind of reverse this so that every business, every lab would have their own AI optimized for their own workflow and for, for their like with their own data, with their own infra. And this would allow them to have better margins, more control and kind of this is like a much more enabling future. And we think it's going to be just a matter of time. I think already businesses have started being worried about this and it's going to be just a matter of time before they, they want to lean into this angle more.

[Matt Bornstein]

Yeah, I mean that I think is a really interesting question. It's almost the like most important question right now. Like if you think about major technical breakthroughs that have, that have like massive economic impact, you know one, one characteristic of that is that everybody can use them, right? Everybody can build on them and build with them, not, not just consume them. You know, you have some monopoly industries like power generation where the capex is so high that everybody can, but, but you know, it's like consuming power is like not very limiting, right? This is like really low level infrastructure versus AI is not right. AI is like application level all the, all the way, sort of down. So this is the thing that we think about a lot on our team. It's like how, how do we get to a point where, where all the major companies, startups, you know, big banks, healthcare companies, pharma companies, et cetera can actually build with this. And I guess the hope is that something like what you guys are doing sort of can help with that.

[Benham Nesherbor]

Yeah, I think it's interesting because let's say with cloud code coding went from just being a thing related to programmers to something that everyone can do and now everyone is enabled to do a Lot more. And the future we see is same happening to AI, where everyone can go from their wish to have AI do something for them to seeing that happen for like their own AI. And it's just a matter of having the compute and resources and not matter of like expertise, etc. So businesses, if they want to kind of take an angle, they have, you know, like their own data, they have their own processes. This should give them naturally a lot of advantage. And the thing that they're missing is putting everything together to build the moats. And you know, we think AI is going to be that enabling thing that would kind of give the control back to them.

[Matt Bornstein]

Is it true that all researchers at Mirandil have to submit their traces and chats to model training? You don't have to answer that if you don't want to.

[Benham Nesherbor]

All I can say is that when you start a company with a certain goal, you optimize the entire company end to end for, for what you're going for.

[Matt Bornstein]

Maybe sort of. On a related note, like, are, are there any moments that you guys have seen either in the last few months at Mirandil or, or just in general that kind of gave you the chills that like, oh my God, like self accelerating is actually working?

[Harsh Mehta]

I, I think the, the kind of like the rate at which we have made progress, technical progress, I think that surprised me as well. I think we've been able to do a lot of, with a lot fewer resources and people and like, we've been at the Frontier labs for a long time and we know how much work it is to do something and we've been able to do it in kind of like maybe 10 times less people and less resources. That was surprising to me.

[Benham Nesherbor]

Maybe a surprise for me is when I talk to candidates and they get surprised that, you know, like, how do you think you can compete with big places with you know, like only having, you know, like I don't know, 20 people. And I'm like, looks like you're not a believer in, have you been using more?

[Matt Bornstein]

Should say that we have this thing now that can help us, you know, also start. Right? Like, you know, in many ways startups are just a more efficient way to allocate, you know, resources and people. Right? You know, compared compared to sort of diminishing returns at big companies and things like that. I guess part of what I'm getting at is like we have this term self accelerating. You know, do you believe in the kind of like real like self acceleration sort of recursive thing or, or, or like to you, is this a tool that, that is just going to like bring prosperity, you know, and, and, and, and like these great capabilities to everybody.

[Benham Nesherbor]

I think a lot of people have a sci fi view of it, which is like a model kind of making changes, et cetera. I think one thing that maybe we have been kind of thinking slightly differently is building AI systems as opposed to thinking about one AI model. This is where an ecosystem of models and ways of working with each other. And I think we think for foreseeable future, maybe, I don't know, maybe a few years, this ecosystem also includes human. And then you're thinking about this entire system as one intelligent being and you're asking this question of how can this system improve itself? And so that's like a gradual view of like how do we get there? But also systems are much stronger than individual models. Like if you think about best AI researcher in the field, what can that one person accomplish versus an entire company that's built? But it's also not obvious how to build an entire company from individual brilliant people. It's actually a hard thing to do to throw in like 20 people and make a company out of it.

[Matt Bornstein]

Poor AIs are going to have to figure out org charts.

[Benham Nesherbor]

Yeah, so it's actually not, not a straightforward problem how to create a system that is like fast moving and makes improvement. So the way you think about it is like there's a system of AIs that are not necessarily the same model and it could be a specialized, could be, bunch of them are generally small and then some people, and then this system wants to kind of keep getting better, which getting better means this system kind of develops the next AIs that would be part of the next system and kind of keep making improvements. And that's like a very realistic view of how it would evolve into at some point becoming fully autonomous.

[Matt Bornstein]

So you're saying in the future an agent will leave its 10 million agent swarm and talk to the 1000 agent swarm and say how do you guys get any work done with only 1,000 agents? No, I think, I think that's, yeah, no, I think, I think it's a very interesting point. You're almost like, you're almost saying that like, because one issue I've always had with this is like somebody has to prompt these things. Like, like, like they're, they're, they're not self directed, but you're sort of saying is like the level of abstraction goes up. There's almost like a standing, like you only need one prompt anymore. Which is like make improvements to yourself. And this like sufficiently large and complex system can kind of like operate continually on. It's almost like prime directive rather. Rather than prompting at that point.

[Harsh Mehta]

Yeah, exactly. And if you look at kind of like how the models have been developing, it requires less and less oversight. And I think this trend is just going to continue. And at the end, the ultimate prompt is to achieve goals and the system should be able to kind of learn on its own, figure out all the problems, ask for directions when needed, just as humans would and achieve those goals.

[Benham Nesherbor]

I mean, related to the systems, I wanted to kind of throw this out as like an interesting thought experiment is when you think about systems, there's this other angle of scaling up which we have so far scaled up models in terms of size and compute, et cetera. Now we are facing this another axis which is scaling up systems. And these systems today, they're like systems composed of people and agents. So X axis is number of agents. Where like some of them are people and some of them are models. And you know, they have different properties. But the question is really how do you get a favorable scaling of a system? Like companies scale and their productivity go down with scale. So people don't have a favorable scaling. So you go from, you use your company size goes 10x and your productivity is like, it's like 1.2 billion. So that's not great. And today agents are also not that amazing in terms of being able to actually scale them in a way that you can see that productivity of the entire system grows. So the companies of the future where you would want to be able to kind of throw compute at problems and kind of scale up agents and see that let's say when you double number of agents, you get to the same goal two weeks faster. That's the question, like how do you save time? Is the real future like the real thing that the world is fighting for is how to get to a point faster. And all the competition between companies, and a lot of these companies are willing to pay 10x more compute to just get to something like one month faster than others. And I think that's you have to solve the scaling problem. And one interesting thing that we know about scaling laws is that if you want to get scaling laws right, you have to start small and get the scaling right before you scale up. Otherwise things are not going to work. So that's, I think of ourselves as like the experiment running experiment at a small scale, thinking about like how can we get favorable scaling and then scale

[Matt Bornstein]

up the system I mean it's really interesting. Is it a communication, like what aspects of the system do you think need to be solved in order for this to work better?

[Benham Nesherbor]

Oversight, A lot of oversight issues. I think a lot of resource allocation problems, like if you have same compute but you increase number of researchers, you can't, you know, how do you decide which ideas would get prioritized, et cetera. So a lot of technical.

[Matt Bornstein]

This is what really happens. Human researchers. Yeah.

[Benham Nesherbor]

Also with AIs. Yeah, yeah, yeah, yeah, yeah. So there's a lot of technical problems to solve there that are very interesting.

[Matt Bornstein]

So now we're talking interagent politics. It's like why did that agent get

[Benham Nesherbor]

1,000 views at the end of the day? With agents and with people, incentives are very important.

[Matt Bornstein]

And this is orthogonal to the performance of the agent itself. Because I know you've worked a lot on long running agents and like Karshi were saying sort of narrow slices of problems, but these things you're talking about are actually orthogonal to individual agent performance.

[Benham Nesherbor]

Yeah.

[Harsh Mehta]

Well, one thing I want to highlight is such a very interesting and impactful time that we are living in. There are like this histories of science and for example physics. A whole bunch of really, really impactful work happened in 1920s and 30s and then there was a little bit of a blank space. And I think we're going through a similar thing in intelligence and it's improving at a very fast rate and it's just a, like a really interesting time to live in.

[Matt Bornstein]

Do you think this sort of self acceleration is already starting to happen? Meaning when you look at like the latest releases from the big labs, you know, is the slope getting steeper because of this?

[Harsh Mehta]

Absolutely. I think it has started kind of like, you know, in some sense and small ways when the coding models kind of like, you know, became like truly useful. And it has picked up by kind of like every generation of kind of like new models. And it improves everyone's productivity by itself, but that's also kind of like improves the time to the next breakthrough, next model.

[Matt Bornstein]

This is sort of a unique aspect of AI, isn't it? That it's like you have these sort of cheat codes. Like coding is kind of a cheat code because a lot of the system is built on code. What you guys are doing is almost the next level of cheat code because like the actual experiments are sort, you know, which you don't see with like, like when the Internet gets faster. Right. You don't like recursively get faster Internet. Right. You know, so it's, it's it's, it's sort of an interesting thing. Where do you think it all ends up?

[Harsh Mehta]

So wide spectrum of outcomes, but our preferred outcome is kind of like you know, just generally prosperity, special specifically scientific prosperity in the sense that a lot of the science like is just unknown at the moment. And a lot of the engineering kind of like, you know, large engineering projects are also just bottlenecked by people being able to design things well and just the intelligence being the bottleneck. And we hope that like some of the progress can translate into just us knowing a lot, lot more about ourselves and the universe.

[Benham Nesherbor]

Your question is like maybe also one of the other reasons where I just kind of pushed me to kind of start a company is I think the current picture that's being painted about the impact of AI is not that positive, like automating people's job away and you know, like that doesn't seem like an exciting future. And so you know, I am thinking we are trying to in a way focus on building a different one that has more positive outcome for everyone. And we think accelerating science is pure good for humanity. So that's where you could decide where to put like you're building something powerful, you could decide where to put it at work. And I think we, we should feel like we are in charge and we can kind of build the future we want. I think these problems that you know, like solving Alzheimer's disease, being able to predict it much in advance and solving it, these are like super challenging problems. There are so many things that have to be solved on the path of would show everyone that AI

[Matt Bornstein]

it should

[Benham Nesherbor]

have been used this way from the beginning. But somehow we are kind of losing focus on like these are the important problems that have to be solved and smart people should be working on these problems. So we want to kind of change, at least we can choose what we work on. And we want to kind of be focused on creating this powerful technology and directed at these problems that are long standing. And it would still take a lot of resources, it would create jobs because you know, like you have to solve all other aspects of it that are not intelligence related bottlenecks are going to move away from intelligence. So it's interesting to kind of direct it at these problems that are just not about day to day, but solving problems that help everyone.

[Matt Bornstein]

And is it fair to say that continuing to scale up pre training is probably not sufficient to solve these kinds of problems you're talking about because you're almost talking about generating net new knowledge which maybe I guess is hard no matter how much pre training you do.

[Harsh Mehta]

Yeah, in some sense, I think of pre training, post training and all the paradigms that we have of using compute effectively as tools in my toolbox and at certain points in time, some tool is effective and we go for the fixed amount of compute that we have. We go for the best tool.

[Matt Bornstein]

But what you're trying to build is not even just adding compute, but kind of the system of putting like expert people into, like, into like the system as a, as a whole. And this, and this is like part of what amoxys, I think you're saying.

[Benham Nesherbor]

Yeah. So, you know, like for, let's say, you know, let's do this mind experiment of like the way things are going, you know, our best hope is that we are going to have models that are going to be as good as our best scientists or maybe become better than the best scientists. But how's that going to solve Alzheimer's disease for us? Like, these are problems that are, you know, we don't even know if it's possible. We don't even know what are the limits that exist. So we really need to move at much higher speed. And if you're not thinking that way, if you're just thinking about slightly higher building, an AI scientist, et cetera, this is not going to solve these problems. Like Alzheimer's disease, for example. It has so much structure in terms of data that you have to use, et cetera, that you cannot even see how in 10 years existing models would be able to kind of move things much faster. And with the current speed, we are not going to get there anytime soon. So we are impatient about solving those problems.

[Matt Bornstein]

Awesome. Well, this is awesome, guys. Thank you. Thank you so much for working on this and for coming on the podcast.

[A16Z Podcast Host]

Thanks for listening to this episode of the A16Z podcast. If you like this episode, be sure to, like, comment, subscribe, subscribe, leave us a rating or review and share it with your friends and family. For more episodes go to YouTube, Apple Podcasts and Spotify. Follow us on X1 6Z and subscribe to our [e-mail] thanks again for listening and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com disclosures.