Anjney Midha's Plan to Radically Lower the Price of Compute

Summary

Odd Lots Podcast: “Anjney Midha's Plan to Radically Lower the Price of Compute”

Date: June 13, 2026
Hosts: Joe Weisenthal & Tracy Alloway
Guest: Anjney Midha (Founder, AMP pbc; ex-Andreesen Horowitz, Visiting Scientist at Stanford, early Anthropic investor)

---

Episode Overview

This episode delves into the urgent question of access to compute power at a time where AI development is both constrained and defined by physical and economic resources. From the bottleneck of GPUs to the prospect of forming a “compute grid” akin to the electricity grid, the conversation centers on how new infrastructure and business models could make AI research cheaper and more accessible—and whether market commodification is possible or desirable. Anjney Midha, with experience spanning VC, AI startups, and compute infrastructure, joins Joe and Tracy to unpack the financial and technical constraints of AI, recount the early days at Anthropic, and explain how his latest venture, AMP pbc, is building a universal compute standard.

---

Key Discussion Points & Insights

1. The Physical Constraints of Digital AI (02:24–04:19)

• AI development isn’t just code—it’s deeply dependent on physical resources (electric grids, GPUs, energy, talent, even real estate).
• Joe and Tracy reference the "paperclip maximizer" thought experiment to highlight how resource allocation for AI is both massive and escalating, with comparisons to how humanity might eventually consume "space" resources for AI in the same way the Bostrom scenario imagines for an AI.

---

2. Insane Valuations and Market Dynamics (04:19–05:43)

• AI races are driven by the existential need to reach AGI first, justifying enormous spend and resource reallocation.
  • Tracy: “If you say you absolutely have to be the first to invent AGI, then you can justify any amount of spending on Earth.”
• Joe asks if the similarity of today's top frontier models is likely to persist, or if there could be a breakout.

---

3. Guest Introduction and Origin Story: Early Anthropic (06:15–11:23)

• Anjney’s journey from Stanford ML aficionado to VC to AI founder, and being Anthropic's first backer:
  • He recounts how most VCs dismissed Anthropic in 2021, forcing them to raise from “cats and dogs—angels who believed in the mission.”
  • His realization: “Infrastructure, especially compute infrastructure, was just a key requirement to create any kind of modern AI lab.” (10:58)

Memorable quote:

“We tried to raise $500 million, we couldn’t. We instead scraped together about $100 million, which sounds like a lot, but at the time was a rounding error compared to how much Google had spent on the same kind of systems.” – Anjney (10:01)

---

4. Why Compute Access Dictates AI Progress (11:38–13:22)

• Joe asks: If compute were more available, could more labs reach the frontier?
• Anjney clarifies: Many “frontiers” exist—software engineering, chat, video, etc., and different labs dominate distinct domains (Anthropic, OpenAI, ByteDance).
  • The process to create a frontier model is surprisingly simple in principle: pretraining (on broad data), midtraining (on targeted domains), deployment, and feedback loops.

---

5. The Feedback Loop: Why Coding is Special for AI (17:09–22:29)

• Tracy describes giving feedback to models, noticing they apologize but don’t change behavior.
• Anjney distinguishes between subjective and verifiable feedback.
  • In software, peer review and unit tests create a “verifiable” loop, which is why code tasks have improved fastest.
  • Other fields (journalism, sales, creative work) lack this robust feedback structure, making AI progress more unpredictable.
• Expect “jagged” progress:
  • Structured, fact-checkable tasks will be automated first.
  • “Wherever workflows are not verifiable is actually where humans are going to shine.” (22:08)

Memorable quote:

“You can outsource part of the tedious workflows, but you can't outsource your understanding.” – Anjney (26:44)

---

6. The Compute Bottleneck and AMP’s Solution (27:30–32:59)

• AMP’s vision: Standardize and “grid” the compute world, just like electricity moved from isolated, stranded sources to a universal grid after AC/DC standardization.
• Key problem: Current compute rentals over-provision for spiky research needs; real utilization is far below optimal (sometimes just ~60–70%).
• Their approach: AMP’s software “grid” pools heterogeneous compute (different chips, vendors) and allocates jobs to maximize utilization, making compute fungible (via "grid credits").

Memorable quote:

“We’re building Borg for everybody else” – in reference to Google’s internal compute orchestration system (29:21)

• Economic result: Labs shift from paying “apartment-style” all-inclusive leases (wasted resources, $25–28/hr effective rate) to true usage-based pricing ($2.50/hr), reallocating unused compute via the grid to other labs (34:39–35:53).

---

7. Hardware, Software, and Custom Chips (37:53–40:30)

• Joe asks why every lab makes custom silicon.
• Anjney: “80 cents of every dollar a lab spends today on their R&D flows to a chip provider like Nvidia.” (38:43)
  • Labs seek control for two reasons: economic (keep more margin) and technical (supply chain independence).

---

8. Towards Compute Utilities and Financialization (41:09–48:07)

• AMP sees itself as an independent system operator—balancing supply and demand, much like the electricity utility grid.
• Financialization caution: Anjney warns against speculative trading in compute markets, arguing it could worsen shortages for researchers.

Memorable quote:

“When you add speculation to production goods, it creates scarcity of a different kind.” (46:53)

---

9. Demand Slopes and Market Commoditization (48:08–54:00)

• Demand is “perpendicular” right now—incredibly steep. Compute prices for long-term rentals have doubled in six months.
• Joe and Tracy discuss whether the boom in “frontier models” will create a commoditized AI market, where users care less about which model, and more about price, reliability, and outcomes.

Memorable quotes:

“What you’ll find is that which particular model is helping you out in a particular task will just be abstracted… You’re just going to talk to it, it’ll be a companion provided by a brand you trust.” – Anjney (42:47)

“The future, it’s not going to be a winner-takes-all thing in terms of models. You’re going to have a bunch of different models doing different things.” – Tracy (52:06)

---

Notable Quotes & Memorable Moments

• On the bottleneck:
  “Infrastructure, especially compute infrastructure, was just a key requirement to create any kind of modern AI lab.” (10:58)
• On utilization:
  “At Google, if utilization is at 96%, that's considered a major outage. Today the average data center... is running at less than 70% utilization.” (30:18)
• On market abstraction:
  “Increasingly, corporate leaders just want the work done—they don’t care about the model. They want the cheapest, most efficient, and secure solution.” (42:47)
• On financialization risks:
  “When you add speculation to production goods, it creates scarcity of a different kind.” (46:53)
• On demand:
  “If you look at the compute prices of long-term rentals over the last six months... they’re trading up 2x.” (48:22)

---

Important Timestamps

• 02:24 Physical resource constraints of AI
• 06:15 Guest intro and Anthropic origin story
• 13:18 Explanation of “jagged” AI frontiers
• 17:43 Why feedback matters, and coding’s special place for AI
• 27:30 What AMP pbc is actually building (the “compute grid”)
• 34:39 How economics of compute rental incentivize resource waste
• 38:43 Why labs design their own chips
• 41:37 How AMP operates as a compute grid "utility"
• 46:53 Thoughts on speculation and financialization in compute
• 48:22 Steeply rising demand for compute
• 52:06 Tracy’s summary of the commodified compute market future

---

Takeaways

• AI’s resource constraint problem is as important as algorithmic progress—and software innovations can dramatically increase utilization and lower costs.
• AMP aims to “grid-ify” compute, standardizing workloads, abstracting hardware specifics, and reallocating unused cycles.
• There is no single “frontier”—AI’s progress is jagged; multiple models and domains will coexist and thrive, with commodification likely on the user side, complex orchestration on the backend.
• Despite fears, the commodification and financialization of compute may have limits, particularly if speculation creates new forms of scarcity.
• Leaders and organizations must develop genuine technical literacy to optimally use AI—outsourcing all understanding to a model is both risky and limiting.

---

Further Listening & Reading

• Stanford's “Frontier Systems” lectures by Anjney Midha (153.stanford.edu)
• Odd Lots Discord (Discord.gg/oddlots)
• Odd Lots newsletter: bloomberg.com/oddlots

---

Summary prepared for listeners seeking a comprehensive yet efficient guide to the ideas, arguments, and memorable insights of this key Odd Lots episode.

Transcript

[Tracy Alloway]

Oddlauts is brought to you by Vaneck. For years investors basically forgot about real assets, energy, gold and infrastructure. But look at what's driving markets now. Central banks loading up on gold, massive capex cycles, currencies doing weird things. These assets are at the center of it RACS. The VanEck Real Assets ETF is an actively managed one stop shop for real assets spanning gold, commodities, natural resource equities and more. Go to vaneck.com raaxpod to learn more fun disclosures later in this episode so

[IBM Representative]

there's a lot of noise about AI, but time's too tight for more promises. So let's talk about results. At IBM, we work with our employees to integrate technology right into the systems they need. Now a Global workforce of 300,000 can use AI to fill their HR questions, resolving 94% of common questions. Not noise proof of how we can help companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business IBM when you're

[The Hartford Representative]

running a business, the best days are the ones where priorities stay on track. For midsize and large companies, that isn't always easy. Risk can touch multiple parts of an organization at the same time, often in ways that aren't immediately obvious. It might involve property liability or cyber. It could stem from regulatory requirements or challenges tied to a specific industry or the scale of an operation. At that level, managing risk becomes an ongoing discipline, not a one time decision. At the Hartford, the focus is on helping businesses manage risk before it turns into something more disruptive. That means working with companies to identify where they're exposed, decide what matters most, and put practical standards in place so risk is managed as part of day to day operations. And when losses do happen, the Hartford can pair that risk control work with insurance coverage grounded in underwriting, risk engineering and claims experience developed over time. Learn [e-mail] riskmitigation

[Narrator/Advertisement Voice]

Bloomberg Audio Studios Podcasts Radio News.

[Tracy Alloway]

Hello and welcome to another episode of the Odd Thoughts Podcast. I'm Tracy Alloway.

[Joe Weisenthal]

And I'm Joe Weisenthal.

[Tracy Alloway]

Joe, we like to talk a lot about physical constraints on this show. Right. And this is one reason why AI is a really fascinating area for us right now, because there are a lot of physical constraints on what is ultimately the sort of ephemeral technology. And I think that the tension between those two things is really interesting.

[Joe Weisenthal]

Right, right.

[Tracy Alloway]

Like you type a prompt into chat, GPT or Claude or whatever and it's the sort disembodied digital platform you don't necessarily think about the power usage, the real resources, the transformers that have to go into data centers to get compute.

[Joe Weisenthal]

The thing that I've been on my mind lately, and I've written about it, and I plan to write more, is this idea that the canonical AI thought experiment is what happens if you tell an AI to make a lot of paper clips and then it destroys the world. Because in the pursuit of, of marshaling all of the world's resources, it just turns everything into paperclips because it doesn't know.

[Tracy Alloway]

And I have to ask, is this canonical example, is this based on your traumatic fear of Clippy from Microsoft?

[Joe Weisenthal]

No, I did. No, but that is, you know, it all comes back full circle. But what we are seeing in real life is that everything from access to electrical grids, GPUs being the big example, energy, turbines, talent, and now even including residential real estate are being repurposed to make more and more advanced AI. And in the original paperclip thought experiment, they envision, or at least in one version, the philosopher Nicholas Bostrom envisions the AI having exhausted all of the world's resources, then sending a probe into outer space to consume star energy to build

[Tracy Alloway]

more paperclips, eat the universe.

[Joe Weisenthal]

And to this point, we're even talking about going into outer space for data centers to build more AI. So every version of the thought experiment is being replicated, except it's just more and more resources to build the AI by humans rather than paperclips by AI.

[Anjane Mitta]

AI.

[Tracy Alloway]

There's this other connected theme here. So we've talked before about how one of the reasons valuations seem to be getting insane in the market is because all of this activity is being driven by like this existential need to become number one in frontier models and this new technology. And so if you say you absolutely have to be the first to invent AGI, then you can justify any amount of spending on Earth, right? And so what we tend to see is like, the biggest companies just keep getting bigger and they're the ones that can get resources for all this stuff.

[Joe Weisenthal]

And I think one of the most fascinating things right now is that at least as of right now, June 4, 2026, the Frontier models are really close to each other, right? Yeah. So the 4.8 GPT, 5.5, like, they're not that different. And one of the things I'm curious about is is there something inherent in market dynamics in this space that will always keep, you know, whether it's being able to distill results from another Model and quasi steal them. Whether it's information sharing among employees, is there some inherent reason why we've seen the stability? Or could it be that at some point one lab just like breaks out and establishes permanent.

[Tracy Alloway]

Still a possibility. But like, I am personally on the side of commodification and everything just becomes kind of basic or basically available. I know.

[Joe Weisenthal]

Okay, well, I'm just kidding.

[Tracy Alloway]

All right, thank you.

[Joe Weisenthal]

Joe is a joke.

[Narrator/Advertisement Voice]

All right.

[Tracy Alloway]

That's a polite, light prompt to get to the guest. We do, in fact, have the perfect guest. We're going to be speaking with Anjane Mitta. He is, of course, a former general partner at Andreessen Horowitz, a Stanford University visiting scientist who teaches the viral AI lecture called Frontier Systems. Also one of the first guys to write a check for Anthropic. He is now the founder of a new company called AMP pvc. So thank you so much for coming on opbots, Angel.

[Anjane Mitta]

Thanks for having me. One correction, it's pronounced amp pbc. But that's everything else you got. Perfect on the intro.

[Tracy Alloway]

AMP would make sense, wouldn't it?

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

As an energy.

[Joe Weisenthal]

Yeah, yeah.

[Tracy Alloway]

And just remind us, the PBC is public Benefit Corporation.

[Anjane Mitta]

That's right.

[Tracy Alloway]

So you're doing this for the public benefit.

[Anjane Mitta]

We're governed by a public benefit charter, which means everything we do has to follow our mission. We have a public charter mission. We are for profit in the same way Ben and Jerry's or REI and Anthropic are public benefits. So we aim to make a healthy, modest amount of profits that can sustain our mission. But we have the flexibility to choose what that margin is.

[Tracy Alloway]

Can I just start? I want to establish your credentials, although I feel like that very long list did a pretty good job. But writing the first check for Anthropic, tell us that kind of origin story. Because the anecdote that you hear is like 25 VCs turned them away initially and you said yes.

[Anjane Mitta]

It was a little. But the other way around I said yes. Then we tried to get another 25 VCs to say yes, and I failed. It was a harrowing experience. It was a bit of a wake up call. It was late 2020. I had just sold my last business. It was called Ubiquiti 6. It was a 3D mapping business and it was an AI business that we had founded in 2017. And I felt like a failure at the time because, you know, in San Francisco. I was in San Francisco. I just as big picture my life stories. I was born in India. I went to High school in Singapore and I came out of college to the United States at Stanford for my undergraduate degree. And then when I arrived at campus in 2011, deep learning had just started taking over the world. In Silicon Valley, Andrej Karpathy was a computer science TA to Andrew Ng, who was one of the, I would say, modern sort of founding fathers of deep learning, this idea that you can teach machines to think without having to give them prescriptive rules. And so I went into sort of machine. I got swept up in that moment and started studying. A lot of my coursework was in machine learning. My primary department at Stanford was in bioinformatics, which was machine learning applied to healthcare. I got sidetracked to a venture firm called Kleiner Perkins for about four and a half years where I got the chance to work for some of the great investors like John Doerr and Mary Meeker. And then I left and started my own company. And as is the case in Silicon Valley, when you start, I mean, I was 25, I went and raised about 47 or so million dollars from some of the usual suspects like benchmark and index and so on. And I thought I was the coolest kid in town. And I got the beat out of me because we built this incredible technology which was this AI system that could map any location in 3D. And then the pandemic hit. And so location based mapping, 3D mapping, you can, the only thing you can control is how you react to what happens. And so I did feel for a moment like it was bad luck. And then you just have to pick up the pieces and make the best of it. So I did with my co founder, we figured it out. It was, it was a tough few years where we had to pivot the business, but we landed the plane. We essentially, a lot of the distributed systems we'd built on the backend side ended up being quite valuable. We sold that to a company called Discord, which is a chat app for gamers.

[Joe Weisenthal]

We have an onlog's Discord time to plug that. We chat with our fans in there.

[Anjane Mitta]

Yeah, awesome.

[Joe Weisenthal]

Our listeners.

[Anjane Mitta]

So, you know, about a month after I sold the business, I got a call from some friends who were running research at OpenAI and we'd all been friends in the machine learning community in the Bay Area. And they said, anj, we've trained a little model called GPT3 and we think it's the best since.

[Tracy Alloway]

Just a little model.

[Anjane Mitta]

Yeah, nobody really paid attention. They were like, nobody cares. But we think it's the best thing since sliced Bread. And we, we want to leave and turn this into a standalone business, but it'd be helpful to get some of your advice on how to do that. And I couldn't really come on board full time at the time with them because I had to integrate my company into the acquirer. But I came on as their angel and nights and weekends I worked with them on the business plan and who we should raise from that company. Was anthropic. Dario and Tom and I started doing these weekly working sessions in early 2021. And yeah, I assumed that, you know, if we went and talked to a bunch of venture capitalists on Sandhill Road, especially some of the ones who were involved in the biggest hits of the last decade before that they would, they would get it. These are the creators of GPT3 and they were like, we just don't get this. We've heard the whole AI story before. This whole general intelligence thing is a pipe dream. And it was painful. We tried to raise $500 million, we couldn't. We instead scraped together about 100 million, which I know sounds like a lot, but at the time was a rounding error compared to how much Google had spent on the same kind of systems. And it was all angels in that first round, a bunch of cats and dogs, all of us who believed in the mission. And then over the next 18 months, Dario, Tom and team put together a plan that we kind of workshopped on getting Amazon involved as a strategic. And that resulted in a $4 billion compute and capital partnership that made me realize infrastructure, especially compute infrastructure, was just a key requirement to create any kind of modern AI lab. And so since then, I've spent the past five, six years figuring out how to unblock that compute bottleneck for research teams.

[Joe Weisenthal]

Amazing. Well, obviously an incredibly well timed.

[Tracy Alloway]

It just like emphasizes how much things have changed, right? Where people are literally throwing money at almost any model now versus a few years ago going like AGI, I don't really know.

[Joe Weisenthal]

Well, let me ask you this question because this is a very top of mind question for me. And we can skip around on the timeline here, but there are three labs that are seen as like genuinely at the frontier right now and that is obviously DeepMind within Google, OpenAI and Anthropic. And then of course, you know, a lot of people say that the Chinese labs are very close, if not quite there, maybe they're a few months behind. Is this, is there. You know, when we think about like part of your mission is like you say, okay, a new lab should be able to get access to compute, if you're really bright like that shouldn't be the bottleneck. Does that imply, therefore that you expect more labs to be able to, were they to have access to the compute, also reach the frontier, and that there is something inherent about like this sort of seeming stability or parity that we see among frontier models?

[Anjane Mitta]

So the answer to your first question is yes, there are many frontiers to be conquered and pioneered, and this is not just one frontier. I think that's a fundamental misunderstanding people have about the frontier.

[Joe Weisenthal]

They talk about the jagged frontier. Exactly.

[Anjane Mitta]

Jagged intelligence. Right. Poetic sense and historical sense. If you think about the Wild west or the Western frontier, it wasn't just one frontier. There was a frontier of gold and there was a frontier of genes. It turns out Levi's. It turned out to be a new modern behemoth of a company. I mean, there were so many new businesses founded in the Industrial revolution. And I think that's the reality is the software engineering frontier, which is where Anthropic is clearly leader, is one frontier. I think the chat frontier, the sort of consumer chat frontier, is another frontier where OpenAI has been a leader.

[Joe Weisenthal]

Arguably ByteDance is at the video frontier with Seed Dance, right?

[Anjane Mitta]

Absolutely, yeah. And so I think there's just many, many frontiers to be conquered or pioneered rather. I think Anthropic is clearly a role model for the rest of the community on how to do it in an efficient way. They're, you know, I think fewer than 5,000 people and they've been able to put out state of the art models that, you know, teams like Google, which have 60,000 people, are close to but not yet quite there. So actually I don't really agree with your assessment that they're all at parity. If you use the models day in and day out, they're quite remarkably different in meaningful ways to the person with hands on the keyboard doing the engineering work. And I think those differences reflect the focus of the teams. Right. What is the actual mission that the team working on, on that domain cares about day after day after day? So in this, in the Stanford class I teach, the first lecture was a breakdown of how frontier models are even created. And it's actually quite simple. The recipe is super simple. There's basically four steps. There's pre training, mid training, post training, and then what we call the continuous feedback loop. So pre training just is, just says, hey, you collect a bunch of data from the Internet and train a model to be a generally good pattern recognition machine. You then do mid training, which is to say in a particular domain that you really care about, you inject more capabilities. So if you want this model to reason about science or math or physics, then you give it science or math or physics data and then you get a pretty good model that's specialized in that domain and then you deploy it to the real world where you have people using it and the context feedback, which is when the model is able to do a task well or not, you can verify whether that task was done correctly. Gives the model the data it needs to keep improving on that task, on that distribution.

[Joe Weisenthal]

Data centers need electricity, AI needs copper, reshoring needs steel, and gold's run may tell you something about how the world is repricing money and debt. All of those point back to real assets. The Racks ETF is an actively managed one stop real asset shop. From gold to commodities to natural resource equities, adjusting as conditions change. Visit vaneck.com raaxpod to learn more. An investor should consider the investment objective, risks, charges and expenses of the fund before investing. To obtain a prospectus and summary prospectus which contains this and other information, visit vaneck.com Please read the prospectus and summary prospectus carefully before investing. Racks is distributed by Vaneck Securities Corporation

[IBM Representative]

Distributor the thing about AI for business it may not automatically fit the way your business works. At IBM we've seen this firsthand. But by embedding AI across hr, IT and procurement processes, we've reduced costs by millions, slash repetitive tasks, and freed thousands of hours for strategic work. Now we're helping companies get smarter by putting AI where it actually pays off, deep in the work that moves the business. Let's create smarter business.

[Narrator/Advertisement Voice]

IBM if you're feeling off fatigue, mood changes, skin shifts, yet your labs say everything's normal. You're not alone. Mito Astra from Inner Balance the first all in one prescription strength bioidentical hormone cream that's natural and effective and only takes one drop 10 seconds a day. Oester replaces five to six products women typically use to treat symptoms and is third party tested to ensure the highest quality. Visit innerbalance.com today to start feeling like yourself again.

[Tracy Alloway]

That's innerbalance.com this is slightly tangential, but like I give a lot of feedback to the models because Joe made me paranoid about the Basilisk theory. So I want the models to appreciate me once they take over the world. But when you give them feedback, like if they spit out a wrong answer and you say that's wrong, they immediately apologize and fall over themselves to say that they're sorry. But then you ask them, like, give me another output, or like, would you do it again the same way? And they often say yes. Or they give, like, a very similar answer. They don't seem to be responding in real time.

[Anjane Mitta]

Correct. So when I say feedback, I mean a very specific kind of feedback, which I. Which I call verifiable feedback. So when you say that wasn't right or that was wrong, that's an opinion.

[Tracy Alloway]

Okay.

[Anjane Mitta]

Verifiable feedback is when you can have as close to factual verification as possible. The reason.

[Tracy Alloway]

So what does that actually look like?

[Anjane Mitta]

That's a great question. So let's take reason by example in two or three cases. In the case of software engineering, the way software engineers actually code is you write a piece of code and then you submit it to the main code base. And then you usually have a peer on your team review the code and approve it or reject it. And if it gets approved, that's the first step. That's called a pr, a pull request. And if another human on your team that you trust approved it, that's one kind of verification of quality. And then two, before that piece of code usually gets deployed to a production system, you have unit tests. And those are quite objective tests of is this code performing the function we need it to? And if it passes both those tests, it's a verifiable piece of code that accomplished the goal. So in software engineering, the reason we've seen such a dramatic improvement in capabilities is that a lot of these labs are using feedback from that verification loop.

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

In the case of another lab I incubated called Periodic Labs, which we started a year ago. And you should come by sometime. We've got 40,000 square feet in Menlo park where we've got AI models that are predicting new. The goal is to try to find a room temperature superconductor. And so these models predict new.

[Joe Weisenthal]

Oh, I forgot about it.

[Tracy Alloway]

I forgot about temperature superconductor.

[Joe Weisenthal]

I forgot about that.

[Tracy Alloway]

That was a fun summer.

[Anjane Mitta]

Yes. This time we will verify that if we ever put something out, you will know it's not. That's not going to be us. But the AI system predicts new materials, candidates. Then we have robots that synthesize the new material in the lab and then use x ray diffraction machines to test whether the material has the properties the AI set it with. And that's verifiable feedback from reality, from physics. And then we pipe that data back into the training loop over and over again. That context feedback is very factually verifiable. And that's where progress is the fastest today. Because that feedback doesn't result in the kind of hallucinations that you often experience with these models on more subjective tasks. It's also, by the way, why the models are terrible at subjective tasks like creative writing. And sometimes it can get quite toxic, to be honest, if you get them down the wrong loop. I don't know if you've been using it as a, you know, therapy bot and so on.

[Tracy Alloway]

I have not. Just for the record.

[Anjane Mitta]

That's great.

[Tracy Alloway]

It did ask me to defy the laws of gravity at one point because I was trying to create something in my backyard and I was asking it how to do it and it was like, then just set this up like the following way. And I was like, that's not within the laws of physics, but whatever.

[Anjane Mitta]

No, go ahead.

[Joe Weisenthal]

Well, what's interesting, and this is actually a trillion dollar question from just a very broad standpoint, is as you point out, even prior to AI, the field of coding had a very systematized approach to the feedback loops already. And so then it's like AI could sort of replicate that. Anyone who's done any vibe coding can see in the chain of thought sometimes. That didn't work. Let me try this. That didn't work. Let me try this. Most fields don't really have that. By and large, journalism doesn't have that. I mean, there are outputs that are better and worse. We don't really have that. That sort of like formalized approach to the. Yes. No, does that just zooming out? To my mind, that would imply that maybe, at least to some extent coding is a little bit special from a sort of white collar knowledge work that in terms of like, is it going to be as good as, say, I don't know, sales or something like that? Because coding has a long history. It's formulated and structured of that structured pipeline.

[Anjane Mitta]

Yeah, that's a great point. So where progress will be made most predictably is in parts of knowledge work where the task is essentially a workflow that's fairly structured.

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

And so somebody who spends most of their day inputting cells into an Excel spreadsheet, well, that part of the job will get automated pretty fast because that's actually verifiable. And you know what, that's frankly often the most tedious part of the job anyway. And so I'm quite excited to see that progress because I'm terrible at spreadsheets. And I think if we could free up more of my time and hopefully other people's time to focus on the art of the spreadsheet, not the tedious

[Joe Weisenthal]

part of it, the entry. The entry and retrieval.

[Anjane Mitta]

Yeah, yeah, exactly. You know, and in journalism, I think it's the same thing. There's so much craft that gets, that goes into the verification of a story before it goes out that's not legible to the world. You know, I've had a chance to spend some time with some of the journalistic institutions of the Bay Area, like Kate Metz or Brad Olson at the Journal. And as you spend time with them, you realize, I mean, they're verifying every sentence that goes into each.

[Joe Weisenthal]

Fact checking.

[Anjane Mitta]

Absolutely. So fact checking, that's an example where I think we should be leaning on these tools and you should expect more progress and the parts then that will be more, to borrow your jagged frontier framing, there we will be in a regime of jagged frontier progress. Wherever parts of workflows that are verifiable factually will essentially you'll see progress there very predictably over the next few years. And consequently, wherever that progress, the workflows are not verifiable is actually where humans are going to shine. And I think that's where parts of the economy, you're going to see extraordinary gains in the wages of humans who have creativity and craft that are not typically verifiable, you know, through traditional objective means. Does that make sense?

[Joe Weisenthal]

Yeah, it does.

[Tracy Alloway]

And it, it dovetails with a lot of what we've been talking about on the show recently, just going back to verifiable feedback. So, okay, the model spits out something and you can check whether it's right or wrong.

[Joe Weisenthal]

Right.

[Tracy Alloway]

Is it important to understand how the model actually got to that answer? Because we have discussions with like big bank CEOs who are using more AI and their response to this question is always like, well, if we can put restrictions around the AI, if we make sure that it's like released into a sandbox before it's released into the wider world, we're all set from a regulatory perspective. And regulators don't actually need to know what's in the black box model and how it's working. But like, this seems a bit concerning to me.

[Anjane Mitta]

Yeah, no, I'm quite strongly opinionated about this one, which is that technical literacy should be non negotiable. This is the reason I spend so much time teaching this class at Stanford, putting it up online. And the idea of the Frontier Systems class is that end to end, it's a full, simple, but first principles breakdown of how these AI systems are built from scratch from Land, power, shell, like the energy. Where do we get them? The data centers. Then how do we train the models? And the final project, the class with the kids, was actually the one person, Frontier Lab, which is at the end. They're creating their own models and so on. Because the idea is that a person with the right tools today can scale themselves infinitely, but they need to know how to use the tools, what the limitations are, when to lean on them versus not. And I think this is a generalizable piece of technical literacy that all leaders should have. It's like saying, you know, I. In the 90s, I imagine if you knew you could use the Internet without really knowing how it worked, but you know, on the margins when, like, the page doesn't, like, refresh, or you're like, this, this cookie thing is annoying me. Like, over time, people who are more technically literate just realized sometimes you got to debug, you know, the browser. And those of us who've learned over time to do knowledge work are more adept at leaning on them versus not. Like just now, when I was trying to get onto the Internet, I realized, okay, there's this wi fi, password, whatever. And then you don't end up relying on them in ways that they can't fulfill your need anyway. And what's a little bit more dangerous with these systems is because we tend to anthropomorphize them without the technical literacy that, that I wish all leaders had about reasoning about how these systems were built. What you end up doing is projecting out in your mind what the capabilities are in ways that are inaccurate. You project out their impact on society that are not accurate. You project out their business models in a way that are not accurate. I mean, the very fact that when you started this conversation, I don't blame you for it. You're like, there's three models at the frontier.

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

I'm like, well, which frontier and which three models? Because from where I'm sitting, there's like 17 different frontiers right now, and there's four different players in each one. And the businesses of all of them are kind of breathtaking. So I think that technical literacy should always, for leaders, be a basic requirement. And then if you're deploying these systems at Goldman Sachs, you won't oversimplify and get tripped up later when two years later, you realize half your employee base has been leaning on this, like, sandbox framing, when in reality, inside the sandbox, they were doing all kinds of. They were using the tools in ways that were prone to hallucination, prone to Risks, prompt injection. They were leaning on it in ways that were not informed in the appropriate ways. Is this making sense?

[Tracy Alloway]

Like, at a minimum, they would not be using it in the optimal way.

[The Hartford Representative]

Correct.

[Anjane Mitta]

Or relying too much on it. You can't outsource your understanding to a model. You can outsource your thinking, you can outsource part of the tedious workflows, but you can't outsource your understanding.

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

And if you keep think, if you say, if you create these simplistic frameworks of, oh, here's a sandbox, and this is safe, you have to use that sandbox in the right way. Because if you say, well, now everything that happens in the sandbox is totally fine. If the model says, use the spreadsheet, the spreadsheet is good, it's deployed in our servers. But you didn't actually check the spreadsheet and what went into the spreadsheet? And did the model actually understand the particular structure of the business, the physics of the business that you're trying to model out, then you've outsourced your understanding to it. Does that make sense?

[Joe Weisenthal]

Yeah, absolutely. Let's talk about amp. And because you're never going to get the frontier in anything unless you have access to compute, it seems pretty obvious, and there are various arrangements for acquiring compute. You have companies building their own data centers. You have smaller labs, and maybe they use someone else's data centers or a NEO cloud, et cetera. What are you building at amp, such that, at least as part of this story, is trying to solve the compute bottleneck? Specifically.

[Anjane Mitta]

Yeah, it's very simple. What we're doing at amp, we're doing two things. We are trying to standardize the format for compute, which today is super fragmented. So in the history of infrastructure, if you look at whether it was the Industrial Revolution, the Internet streaming, there were usually formats of inputs that were quite heterogeneous. They were fragmented. And then to unlock productivity, you had to standardize a format. So in the case of electricity, until AC DC was standardized, right. Megawatts would just sit in stranded pockets around the United States being unused. And then once we standardized the format to AC dc, then the question was, okay, great, now we turned all these stranded pockets of electricity into one sort of interoperable universal format. Now, how do we distribute it to everybody who needs it? And we came up with this distribution layer in the United States called the grid. That's all we're doing.

[Tracy Alloway]

So building a grid for compute.

[Anjane Mitta]

Correct. We're trying to standardize the compute layer today. Different Chip types, different manufacturers, different clouds. I mean, it's a complete mess. And if you're say more about how

[Tracy Alloway]

you plan to do this, because we've talked before about, you know, there are various people out there that want to create indices of compute.

[Joe Weisenthal]

Yeah.

[Tracy Alloway]

Futures potentially on compute. And the issue that always comes up

[Anjane Mitta]

is fungibility, Right, Exactly. So we've got a couple of ways we solve the fungibility problem. This is a pretty thorny challenge. We solve it in two or three ways. The first is we have a system called the grid which actually makes the compute fungible at a consumption layer. So under the hood we have a bunch of different chip types, we support various different manufacturers, and there's a system that was built to do this already inside a little company called Google. And one of the technical leads on that project was called Borg. Internally at Google is my co founder, Sebastian Lobo. He was my roommate at Stanford 14 years ago. He's my engineering co founder and we're building Borg for everybody else. Which is essentially a translation layer that says no matter what the underlying chip type is, the machine learning researcher who's using the chip just has to worry about the workload and we handle everything else.

[Tracy Alloway]

Underneath the hood, when you say system, is this hardware or software that's doing this?

[Anjane Mitta]

It's all software.

[Tracy Alloway]

Okay, yeah.

[Anjane Mitta]

So we handle that translation layer in software and it's a pretty gnarly challenge. But today we're able to do that in ways that improve utilization, sometimes from 50, 60% at labs that we have incubated or on the grid, to close to 95, 96%. At Google, the utilization is roughly 99%. When Sebastian arrived at Google, it was about 62%. By the time he left, it was roughly at 99%. At Google, if utilization is at 96%, that's considered a major outage. Today the average data center in the industry, in the ecosystem, in the independent ecosystem, is running at less than 70% utilization. The Colossus 2, which is running in Memphis, Elon's 500,000 GB3 hundreds was running at less than 60% node utilization and less than 11% MFU model. Flop utilization is how much of the chip is actually being used. So there's two kinds of utilization people care about in the data center. First is how many chips are being used. That's the highest. That's just the most naive measure. If that number is not at 90 plus percent, no excuses. Say you have the chips, they should at least be doing something. And then within the chip, during how much of the chip is being used within a workload, that number is usually much lower.

[Joe Weisenthal]

I'm very intrigued by this latter point about that. Even like the chip itself may not be even used at full capacity. Yes, because I see these numbers and you say like a lab has, we have 200 chips, we've acquired 800 GPUs, et cetera. And when I see these headlines, I assumed that that optimal utilization techniques must be so good that you can infer someone's capabilities simply by how many Nvidia GPUs they've acquired. But you're saying is that there is actually quite a bit of heterogeneity about the techniques and approaches to getting the most juice out of adship.

[Anjane Mitta]

Yes. You have to measure what matters and what matters is output.

[Joe Weisenthal]

Okay.

[Anjane Mitta]

Anytime I start a new lab with a team. In the case of periodic labs, we started it with Liam Thetis was the co creator of ChatGPT and Doge Chubbuck who led the physics teams at DeepMind. And when we sat down and we planned out the company's roadmap, the most important thing to us to measure was not the number of chips we had, it's the eval, which we call.

[Joe Weisenthal]

It's all this chip bragging. They're like, oh, we acquired. It's just a sort of, it's a lot of bravado. Yeah, all right, this is helpful.

[Anjane Mitta]

You don't measure the inputs, you should

[Joe Weisenthal]

be measuring the outputs. No, I agree, of course.

[Tracy Alloway]

I'm actually fascinated that there is a software solution to what I perceived in my head as like a very physical constraint. How does this actually work? Like feel free to get technical here. Like I want to understand the system.

[Anjane Mitta]

Yes. So let me give you the technological answer and the economic answer. The economic answer actually is a simpler one to reason about. The way the compute business works today is primarily on the construct of the atomic unit of long term leases. So I'm a researcher, I need some compute. I show up to a compute provider and say hello, I would like some compute please. And the compute provider says no problem. Here's you know, 500AMD chips or Nvidia chips that you can lease from me on various timescales and you got to pay for it 24 7. It's like leasing an apartment and whether you use it or not, that's your problem. But it's $2.50 per hour, $3 an hour. So instead you take a long term lease. And now the cloud provider, the compute provider said great, I just booked revenue for the Next two years that this guy rented for me. Now, what happens with that compute, Whether it's used or not is the researcher's problem. They've outsourced that problem as a result of this wastage that we're talking about. And I'm happy to go into why it's hard for individual teams to utilize most of the capacity. The primary reason is because research is spiky, right? It's hard to forecast. So you over provision for your peak, not your base load. Because what happens, you're researching on these algorithms and the minute one is working, you go, guys, let's scale. We want to ship this thing, so let's improve, throw as many chips at it, and then once we ship it, the needs go down. So between these spikes, there's just huge pockets of unused compute. As a result, the effective price per hour that you're paying is closer to 25 to $28, whereas the marketed rate that you think you're paying is $2.50.

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

So that spread due to wastage is just insane. So from an economic perspective, that's the wastage, that's the deadweight loss. Right. Okay, so now how do we, from a technological perspective, how do we utilize that opportunity? Literally all we do is from a software perspective. We take all of that unutilized computer, no matter what format it is. It might be Nvidia, it might be amd, we love amd, it might be some other chip, and we turn it into one fungible resource and we standardize the format on something we call grid credits. So researchers don't even need to think about what chip type is under the hood. They're just paying what they need or what they use. And so from a fiduciary perspective, I'm on seven boards as an investor. I get very excited when teams switch from this sort of long term lease model where they're paying 25, $26 per GPU hour to now they're actually only paying the $2.50 that was marketed because everything they're not using gets reallocated to the grid. And other research labs can use that, that resource.

[Narrator/Advertisement Voice]

If you're feeling off fatigue, mood changes, skin shifts, yet your labs say everything's normal. You're not alone. Meet Oestra from Inner Balance. The first all in one prescription strength bioidentical hormone cream that's natural and effective and only takes one drop, 10 seconds a day. Oester replaces five to six products women typically use to treat symptoms and is third party tested to ensure the highest quality. Visit innerbalance.com today to start feeling like yourself again. That's innerbalance.com hi, I'm Cindy Crawford and I'm the founder of Meaningful Beauty. Well, I don't know about you, but, like, I never liked being told, oh, wow, you look so good for your age. Like, why even bother saying that? Why don't you just say you look great at any age, every age. That's what Meaningful Beauty is all about. We create products that make you feel confident in your skin at the age you are now. Meaningful Beauty. Beautiful skin at every age. Learn [e-mail]. Every business has an ambition. PayPal open is the platform designed to help you grow into yours with access to business loans so you can expand and hundreds of millions of PayPal customers worldwide. Your customers can pay all the ways they want today with PayPal, Venmo, Pay later, and all major cards so you can focus on the future when you need a partner trusted by millions. There's one platform for all business. PayPal open grow today at paypalopen.com loans subject to approval in available locations.

[Joe Weisenthal]

This is the problem you're mostly solving for is the training part. Because there's training, right? Training. And both.

[Anjane Mitta]

Both, yeah. That's the. So the beauty about having diverse types of compute on our grid is that once you make the resource fungible, you can do any workload. You just fill all the unutilized pockets with inference and then all the reservations with training.

[Joe Weisenthal]

Got it. So can you explain why is it that every lab also seems interested right now in customized silicon, including Microsoft announcing a chip that says, oh, our new Mai. I don't know. Maya. I don't know how it's pronounced.

[Anjane Mitta]

Mai. I believe we had Satya in the class yesterday at Stanford and he pronounced it as Mai.

[Joe Weisenthal]

Okay, Their new Mai model. And then he's like, oh, and we also have a new Maya 200 chip or something that's optimized with it.

[Anjane Mitta]

Right.

[Joe Weisenthal]

Why is it that so many labs or companies that are in the lab, I guess, feel impelled to, like, also design a chip that goes along with the model? And long term is what you're doing saying, like, this really is not necessary to have that sort of model chip alignment?

[Anjane Mitta]

Yeah. There's two technological reasons and two economic reasons.

[Joe Weisenthal]

Okay.

[Anjane Mitta]

The first is from a economic perspective, about 80 cents of every dollar a lab spends today on their R and D flows to a chip provider like Nvidia.

[Joe Weisenthal]

Okay. Right.

[Anjane Mitta]

And so as a result, your margins are just super, super rough. So from a unit economic perspective, you want More control over your margins. And therefore when you look at your like unit economics, you're going, wait, wait a minute, for every dollar we make, there's this massive chunk that's going to somebody else.

[Joe Weisenthal]

So instead of spending 80 cents to Nvidia, you spend 78 cents to TSMC and keep that 2 cents for yourself.

[Anjane Mitta]

Well, I think that the better our software gets, the more that margin should flow actually to the researcher. Okay. Because that's where the value will be captured.

[Joe Weisenthal]

But like, but wait, sorry, you were going to say, what's the technical reason why they're trying to do optimal model chip alignment?

[Anjane Mitta]

On the technical side, the primary reason is you want control of your supply chain. Because today in a compute, well, we've been in a compute constrained world now for at least four or five years, but if you can't get the chips you need, you're not in control of your own supply chain. So you're dependent on compute allocations that the compute manufacturer thinks is optimal. Right. By the way, that's how it works at the foundry level. Today TSMC gets to decide which compute provider's business grows or not because they only have so much production capacity. And so the technological reason is you want supply chain independence. And so when you want economic independence, unit economic independence, anyone's supply chain independence, you want as much control over your own chip.

[Joe Weisenthal]

But Microsoft doesn't have a fab.

[Anjane Mitta]

That's not what I'm saying. What I'm saying is in inference, for example, Satya would like more control over his unit economics, so he's making an inference chip. Right. Because if you are dependent on a third party to give you the inference chip.

[Joe Weisenthal]

Okay.

[Anjane Mitta]

And if you don't have an inference chip, you can't sell more product you want more consumption.

[Joe Weisenthal]

Is it about having a predictable supply of chips for you rather than a predictable supply of. Okay, yes.

[Tracy Alloway]

So there's a lot of discussion right now about more efficient model allocation. So this idea that you do not have to be using the latest model to ask like what the weather is going to be tomorrow or something like that, and you also don't want to blow through your entire one year token budget in the space of four months, as Uber apparently did. So the spikes in usage that you're seeing, that allow you to do this system and you know, have grid credits, does some of that go away if people become smarter about which models they're actually using?

[Anjane Mitta]

Okay, so there's an embedded assumption I think I should tease apart in your question. Usage is different from the production of the model. So what's happening right, in terms of the pipeline is you use the grid to produce these system the model, and then the model produces tokens. If the end user is only using tokens, then as long as everybody, we have enough diversity in the end user base using models hosted on the grid, things actually even out.

[Tracy Alloway]

Okay.

[Anjane Mitta]

That cyclicality. In the same way electricity in America evens out if you have enough scale at scale, basically, for the most part, yeah. Except when like there's a heat wave. There's a heat wave. Exactly. So some of that infrastructure we are having to reboot. But you can think about amp in the broadest sense, as a utility company, we're what's called an independent system operator of the grid. So we don't own our own data centers, we don't own our own labs, but we coordinate the capacity needs across different parties and at sufficient scale that those usage patterns actually just gets evened out. Does that make sense?

[Tracy Alloway]

Yeah.

[Joe Weisenthal]

Well, you are an investor. Well, you are an investor in OpenRouter, I believe, which I think is an interesting company. Do you see, setting aside AMP for a second, do you think that there is at this point still within, say, corporate America, a certain lack of savviness about knowing which model to route to for the query and that there will be an improvement in learning within companies, within users, so that you don't have these incidents or like massive token consumption because perhaps everyone was using the wrong. The Cadillac model and the Ford model would have been just as fine for that purpose.

[Anjane Mitta]

Oh yeah, we're absolutely in the medieval ages of this technology. I think what will happen is increasingly based on my conversations with corporate American leaders and corporate leaders across the world, they don't really care about the models, they don't care about the underlying model, the technology. They just don't care. It's like too much complexity. We just want the work done. Yeah, can you guys please figure out how to get the work done in the cheapest way, in the most efficient way, in the most secure and trusted way. And increasingly what you'll find is that which particular model is helping you out in a particular task will just be abstracted. You won't even think about that. It'll just be a companion. You're just going to talk to it. It'll be a companion provided by a brand you trust. And under the hood, they might be using 200 different models to orchestrate your task. And over time that efficiency will get better and better and better and better. And that's why I just don't think there's only three frontier models that are going to win. It's going to be an ecosystem.

[Tracy Alloway]

This is. I know you don't want my take, Joe, but I do.

[Joe Weisenthal]

I want your take.

[Tracy Alloway]

This is my coffee.

[Joe Weisenthal]

I want your take. I want your take, Tracy. I want your take.

[Tracy Alloway]

It's all right.

[Joe Weisenthal]

I love your take.

[Tracy Alloway]

I'll save it for the outro. Actually, on this note, we have seen some headlines recently. Obviously there's the Uber one about token spending. And I think it was the CEO said he wasn't sure if the ROI was there on Uber's AI usage. And we've seen there was a good Vox article recently about a corporate reckoning with AI spend. Since you're going out and talking to CEOs, do you see any. Like, has anything shifted in the past couple months or so in the way people are thinking about the return on this initial investment or the return on spending on. On tokens?

[Anjane Mitta]

Yes. I think it's a barbell distribution. So there's two types of CEOs, broadly speaking. The first is the CEOs who are using the tools themselves. And those folks are going, aha. I understand the jagged frontier. When they understand the jagged frontier, we talked about their strategies. Their questions they ask me are completely different from the CEOs who are outsourcing their understanding. They're not trying the tools. They're mostly asking their kids, like, hey, kiddo, this ChatGPT thing, like, it's good, right? And your kid is like, yeah, it's pretty good, dad.

[Joe Weisenthal]

And then kids think it's really dumb, by the way.

[Anjane Mitta]

Yeah. So that's the other thing. Right. So the kids are super smart and they're using the tools. They're like, it's good at this thing, but not at that. So they understand the jagged frontier part of the world.

[Joe Weisenthal]

Actually, you know what? They think I'm dumb for using it. They're like, dad, like, you're not doing anything smart. You know that they. They don't think the models are dumb. They might as dumb.

[Anjane Mitta]

They might.

[Joe Weisenthal]

Exactly.

[Anjane Mitta]

They might be going the way you're using it.

[Tracy Alloway]

Exactly.

[Anjane Mitta]

Is not optimal. So. So the.

[Joe Weisenthal]

What I'm saying is my kids are six and they have no intent and they have no idea about anything and they just think I'm dumb. That's the only.

[Anjane Mitta]

That just might be a generalizable.

[Joe Weisenthal]

That's really the only point I'm trying to make. I see.

[Anjane Mitta]

Okay, well, you can send them over to me anytime. I'm happy to be the fan uncle.

[Joe Weisenthal]

Yeah. That would be great. You could show that actually this is fun to play.

[Anjane Mitta]

My wife and I are happy to host your kids.

[Joe Weisenthal]

That's really what I'm trying to get at.

[Anjane Mitta]

It's the summer, right? We have two nieces in London and we call it Camp Mirashen. My last name is Mira, my wife's name is Shen. And so you're welcome to send them to Camp Mitashen anytime.

[Tracy Alloway]

That's amazing.

[Anjane Mitta]

But that's. The bifurcation is leaders who are actually trying the tools out, they realize they're extraordinary at some things and not at others. And so depending on whether you get it or not or your actually getting your hands dirty or not, I find the questions are completely different.

[Joe Weisenthal]

So this has been an incredibly helpful conversation in terms of understanding basically the problem of essentially tons of money is being spent. And your thesis is that it's massively suboptimally used up and down the stack you mentioned. Okay, you get a credit, et cetera. Do you actually see that being financialized in a way? I mean, okay, I'm here, I bought this capacity. I have a lot of unused time. I don't always have a research idea that is going to require a big model run test. I can resell that. Is that something that you see, like something that genuinely resembles a financial market?

[Anjane Mitta]

I hope not. Because when you had. When you add speculation to, you know, production goods, it creates scarcity of a different kind. Right. Because then you have financial traders and markets trying to trade the speculative value of the asset and that's going to hurt a lot of our research teams in technology, on the other hand, I think that creates a need for innovation inside of the research teams. So one of the core operating functions we have inside of our business is a forecasting capability where we have a team that's very similar to actually the kind of forecasting team you'd have inside of a hedge fund. We're constantly predicting demand and supply and then we're actually procuring capacity in advance through call options on compute clusters. But our needs are similar to the kind of internal trading desk you'd have inside of a large steel company, right. Where they need to lock up iron ore and so on for their production needs. So I'm a big fan of efficient markets and I'm trying to actively invest in and help entrepreneurs out and teams out who are trying to drive more efficiency in the service of more productivity in science and engineering. I'm not that thrilled about the financialization of these products if it ultimately results in more Speculation. Does that make sense?

[Joe Weisenthal]

Yeah.

[Tracy Alloway]

I'm just curious since you're tracking demand in that way, like if you were going to describe the slope of demand right now versus say like a year ago, is it steeper, is it starting to plateau?

[Anjane Mitta]

Perpendicular.

[Tracy Alloway]

Oh, wow. Okay.

[Anjane Mitta]

If you look at the compute prices of long term rentals over the last six months from between January and now, they're trading up 2x. So we started, for example for 2026, we started securing our capacity in January at these long term rates. We could resell that at a 2x markup if we wanted to.

[Joe Weisenthal]

Part of the reason that 2026 has become just totally AI has consumed everyone's mind I think is because people got very excited about Claude Code specifically. But that was a breakthrough at the harness level, not the model level. Right. Suddenly like the really exciting, like, wow, this is just so fun. It's just so easy. You have a computer inside your computer. That was a harness breakthrough. Do you see like when you think about investment among AI labs, do you see any shift in allocation away from pure scaling and improving the model towards sort of like tooling and harnesses as a way to get more juice out of the models?

[Anjane Mitta]

No, I'm sorry, I have to correct you there. It was not just a harness innovation. Those two things go hand in hand. It's a symphony of improvement between, it's a dialectic between the model capability and the harness. That harness was designed specifically for the capabilities that the new model was going to have. And so when you design these things in the industry we call this co design.

[Joe Weisenthal]

Okay.

[Anjane Mitta]

So you have the harness designed side by side with the researcher who's designing the next generation capabilities in the model and you get a little bit of visibility in where the model is going to be good. Because as I described earlier, the pipeline is actually quite predictable. Pre training, mid training, continuous feedback loop. Once you have that visibility, you go, aha. We specifically want to improve the capabilities on this type of task. It's going to take us about three months to get there, start designing the harness for that improvement. By the time they show up, then you can have the harness assume that the cap this, the model will be able to do XYZ on its own, whereas abc, it's going to need third party tools. So then the harness says, remember that three months ago you were terrible at understanding a spreadsheet?

[Joe Weisenthal]

Yeah.

[Anjane Mitta]

So then we had to go, right, like go use a third party tool to use a spreadsheet. In the last three months what we've done is added the Ability to actually reason about a spreadsheet in the model.

[Joe Weisenthal]

In the model. Not that.

[Anjane Mitta]

So now you don't need to use a third party spreadsheet.

[Joe Weisenthal]

Okay.

[Anjane Mitta]

And so then the harness gets updated to say, don't go out and use a third party spreadsheet, which by the way collapses the time required to do that task by like sometimes a minute to two minutes. Now suddenly I've improved the user experience and that's when things really sing. It's when both of those, those parts, the model and the harness are co designed to create a symphony. Does that make sense?

[Joe Weisenthal]

Yeah, absolutely.

[Tracy Alloway]

All right, Anjaney Mitta of AMP pbc, thank you so much for coming on Odd lots. Really appreciate it.

[Anjane Mitta]

Thanks for having me and everyone go

[Tracy Alloway]

out and check out the Stanford lecture series. It's on YouTube, right?

[Anjane Mitta]

153 Stanford.edu.

[Tracy Alloway]

perfect.

[Joe Weisenthal]

I have a big flight coming up, so I'll watch it.

[Anjane Mitta]

Then you should download all the lectures. There's quite a few.

[Joe Weisenthal]

Thank you so much, Anton. I appreciate it. That was fantastic. That was great.

[Tracy Alloway]

All right, Joe. That was a great discussion.

[Joe Weisenthal]

Yeah.

[Tracy Alloway]

I should emphasize just how big a deal that lecture series actually is at Stanford. Like students are beating down the door basically to get into that. And if it's free on YouTube, you should definitely check it out.

[Joe Weisenthal]

I just want to establish that if I had given you the A, the insinuation that I didn't want to hear your take, or B, the idea that I would have wanted to hear Andre's take instead of yours, I want to hear your take.

[Tracy Alloway]

No, it's. It's fine, Joe. I realize that most listeners are here for the guest takes. I get it. But I thought his point about the jagged frontier was an important one. And this idea that like, maybe the future, it's not going to be a winner takes all thing in terms of models, you're going to have a bunch of different models doing different things that might suit different companies. And also the idea that like a lot of companies aren't going to care about which specific model they're using. They just want the cheapest one that basically gets the job done. In my mind, that sounds like more of a. Is commodified the right word? Yeah, a commodified market. Right. Rather than like, oh, people are going to pay up for, as you said, the Cadillac.

[Joe Weisenthal]

Well, so what I would say is by in listening to Angena and AMP is that people will want a commodified service, but that under the hood, I mean, this just sounds like what she's really trying to solve. And it's very interesting. I, as a user or a company, buy a commodified service, but under the hood, the commodity has an incredible amount of variety of models through which it can route, some of which will be the Cadillac, some of it will be the Keurig Coffee cup.

[Tracy Alloway]

Yeah, absolutely. But like, my point is maybe in terms of valuations, right? Like if everyone is assuming that the Cadillac is going to be like the one that everyone is going to get and the total available market, the TAM infamously is like not just the world, but potentially the universe. Like that seems a stretch to me.

[Joe Weisenthal]

Totally. And just generally I thought that was super interesting. And the idea this we've done a couple episodes recently specifically learning more about both chip level and box level optimizations, both how many chips you're using and how well you're using add chip. Definitely way more to do on that.

[Tracy Alloway]

It still blows my mind that this is a problem that can be solved with software rather than like something physical like you just come up with a way to efficiently allocate the compute. Yeah, because in my mind, like it's, it's such a physical problem. And we've talked to, you know, previous AI market participants like Brandon McBee at Core Weave, and they talk about like, oh, it's difficult to standardize because of the configurations of chips and things like that, but if you could solve it just through a software system, that's pretty crazy. I guess Google's already done it.

[Joe Weisenthal]

Yeah.

[Tracy Alloway]

All right, shall we leave it there?

[Joe Weisenthal]

Let's leave it there.

[Tracy Alloway]

This has been another episode of the Odd Thoughts podcast. I'm Tracy Alloway. You can follow me at Tracy Alloway.

[Joe Weisenthal]

And I'm Jill Wiesenthal. You can follow me at the the Stalwart. Follow our guest, Anjan Midda. Anjanemidha. Follow our producers Carmen Rodriguez at Carmen Arment, dashiell Bennett at Dashbot, Kale Brooks Kale Brooks, and Kevin Lozano at Kevin Lloyd Lozano.

[Tracy Alloway]

And for more Odd Lots content, you should check out our daily newsletter. You can find [e-mail] oddlots and you

[Joe Weisenthal]

can chat about all of these topics 24. 7 in our Discord Discord GG oddlots.

[Tracy Alloway]

And if you enjoyed this conversation, then please leave a comment or like the video or better yet, subscribe.

[Joe Weisenthal]

Thanks for listening.

[Narrator/Advertisement Voice]

If you're feeling off fatigue, mood changes, skin shifts, yet your labs say everything's normal, you're not alone. Meet Oestra from Inner Balance, the first all in One Prescription Strength bioidentical hormone cream that's natural and effective and only takes one drop, 10 seconds a day. Oestra replaces five to six products women typically use to treat symptoms and is third party tested to ensure the highest quality. Visit innerbalance.com today to start feeling like yourself again. That's innerbalance.com Paramount plus is now the

[Anjane Mitta]

home of all your BET favorites.

[Joe Weisenthal]

What?

[Anjane Mitta]

Yes, with all new episodes of Tyler Perry's Divorce. Sisters you've always liked, a little drama, plus a whole new world of movies like Gladys.

[Joe Weisenthal]

Now I will control an Empire Original

[Anjane Mitta]

series like the Shy.

[IBM Representative]

Just make sure we protect each other.

[Anjane Mitta]

And live sports like ufc.

[Joe Weisenthal]

Welcome to the history books.

[Anjane Mitta]

New home, same family. Your BET favorites are now on Paramount Plus. Subscribe now.

[Joe Weisenthal]

250 years ago, America made a promise of life, liberty and the pursuit of happiness. Since 1903, Harley Davidson has been living it out on the open road at the next exit ramp with old friends, new ones and the next generation of riders. Because the best part of any promise is keeping it. Harley Davidson Motorcycles ride.