Host (4:12)

Data is the beast feeding the AI train and thus Merck War CEO Brendan Foody is working with major AI labs on how to build what's next. He gives a clear prediction about what's coming for the workforce.

Brendan Foody (4:24)

I think displacement in a lot of roles is going to happen very quickly and it's going to be very painful and a large political problem. Like, I think we're going to have a big populist movement around this and all the displacement that's going to happen. But one of the most important problems in the economy is figuring out how to respond to that. Right? Like, how do we figure out what everyone who's working in customer support or recruiting should be doing in a few years? How do we reallocate wealth once we approach superintelligence, especially if the value and gains of that are more of a power law distribution. And so I spend a lot of time thinking about like how that's going to play out. And I think it's really at the.

Dan Hendricks (5:15)

Heart of what do you think happens.

Brendan Foody (5:16)

Eventually X percent of people get displaced.

Arvind Jain (5:18)

From like color work.

Brendan Foody (5:19)

What do you think they do? I think there's going to be a lot more of the physical world. I think that there's also going to be a lot that of like niche.

Brandon McKinsey (5:28)

What does the physical world mean?

Brendan Foody (5:30)

Well, it could be everything ranging from people that are creating robotics data to people that are waiters at restaurants or are just like therapists because people want human interaction. Whatever that looks like. I think that automation in the physical world is going to happen a lot slower than what's happening in the digital world just because of so many of the self reinforcing gains and a lot of, yeah, self improvement that can, that can happen in, in the virtual world, but not physical one.

Host (6:10)

Which brings us to one of the biggest questions of our time. How do we navigate the geopolitical implications of superintelligence? Dan Hendricks, the director of the center for AI Safety, has an answer.

Dan Hendricks (6:20)

Let's think of what happened in nuclear strategy. Basically a lot of, a lot of states deterred each other from doing a first strike because they could then retaliate. So they had a shared vulnerability. So they were, we're not going to do this really aggressive action of trying to make a bid to wipe you out because that will end up causing us to be damaged. And we have a somewhat similar situation later on when AI is more salient, when it is viewed as pivotal to the future of a nation, when people are on the verge of making a superintelligence more, when they can say automate, you know, pretty much all AI research. I think states would try to deter each other from trying to leverage that to develop it into something like a super weapon that would allow the other countries to be crushed or use those AIs to do some really rapid automated AI research and development loop that could have it bootstrapped from its current levels to something that's super intelligent, vastly more capable than any other system out there. I think that later on it becomes so destabilizing that China just says we're going to do something preemptive like do a cyber attack on your data center. And the US might do that to China and Russia coming out of Ukraine will reassess the situation, get situationally aware, think oh, what's going on with the US and China? Oh my goodness, they're so head on. AI, AI is looking like a big deal. Let's say it's later in the year when a big chunk of software engineering is starting to be impacted by AI. Oh wow, this is looking pretty relevant. Hey, if you try and use this to crush us, we will prevent that by doing a cyber attack on you and we will keep tabs on your projects because it's pretty easy for them to do that espionage.

Host (8:05)

Noubara Feyen has been thinking about how biotech gets built and how to change the game. For three decades, his breakthroughs have impacted global health. He's the founder and CEO of Flagship Pioneering and the co founder of Moderna. He wants to make entrepreneurship a scientific effort, not a random one, and he thinks AI can help.

Noubar Afeyan (8:21)

The motivation for flagship stems from what I was doing before, which was that I started a company in 1987 when 24 year old immigrants didn't start companies in this country, but instead it was kind of like former Merck senior executives or IBM senior executives were the only ones who are entrusted with the massive amounts of venture capital, namely 2, $3 million per round used to go into venture capital. So this was very early days and I had the kind of chance opportunity to start a company right out of my graduate school and ended up raising quite a bit of venture money and eventually kind of went down a path of entrepreneurship along the way. One of the things that interested me was why it is that kind of the entrepreneurial process was supposed to be random, improvisational, kind of idiosyncratic, almost emotional, gamey. All of those things I kind of thought was a bit of a put off when it comes to actually doing things in a serious professional way. And I kind of used to go around in the very early 90s saying why isn't entrepreneurship a profession? And if it was going to be a profession, how could it be a profession?

Host (9:34)

What do you mean by gamey?

Noubar Afeyan (9:36)

Because it's like supposed to fail most of the time and once in a while you win and then you celebrate the win. And what I mean is like it's random. But not only random, but there's like winners and losers and keeping score. I don't know, it's maybe the wrong word, but I just mean like people even call gamification in the software space. There is a version of this. Like I don't mind being playful because if you're overly serious, sometimes you miss things. But it can't just all be play. We take hard earned money, we deploy it to do things that are damn near impossible. Once in a while we reduce them to practice so they become not only possible, but valuable. And yet people treat it like oh well, you know, it didn't work. There's 20 different things we tried. One of them worked that I don't know. As an engineer, by background, as a scientist, I just thought that what we do, especially listen in healthcare, especially in climate, especially in kind of like agriculture, food security, you can't think of this as like shots on goal. And this night you've got to kind of say, hey, we can get better.

Host (10:37)

At this Reasoning is the biggest paradigm shift in AI architecture since the transformer. Brandon McKinsey and Eric Mitchell from OpenAI explained a crucial insight about reasoning models.

Brandon McKinsey (10:46)

I can give maybe very concrete cases for like the visual reasoning side of things. There's a lot of cases where and back to also the model being able to estimate its own uncertainty. You'll give it some kind of question about an image and the model will very transparently tell you initiative thought, like, I don't know, I can't really see the thing you're talking about very well. Or like, it almost knows like that its vision is not, not very good. And what's kind of magical is like when you give it access to a tool, it's like, okay, well I got to figure something out. Let's see if I can manipulate the image or crop around here or something like this. And what that means is that it's much more productive use of tokens as it's doing that. And so your test time scaling slope goes from something like this to something much deeper. And we've seen exactly that the test time scaling slopes for without tool use and with tool use for visual reasoning specifically are very noticeably different.

Winston Weinberg (11:40)

Yeah, I also say like, for writing code for something, there are a lot of things that an LLM could try to figure out on its own, but would require a lot of attempts and self verification that you could write a very simple program to do in a verifiable and, you know, much faster way. So, you know, I do some research on this company and like use this type of, you know, valuation model to tell me like, you know, what the valuation should be like. You could have the model like try to crank through that and like fit those coefficients or whatever in its context. Or you could literally just have it like write the code to just do it the right way and just know what the actual answer is. And so, yeah, I think part of this is you can just allocate compute a lot more efficiently because you can defer stuff that the model doesn't have comparative advantage to doing to a tool that is really well suited to doing that thing.

Host (12:41)

Sometimes the most profound moments in AI development aren't the grand theoretical breakthroughs. They're based on taste, data generation and grinding work. The visceral experience of watching something you hoped would work actually come to life. Issa Fulford from OpenAI captures that moment perfectly. Here she's describing the training that went into deep research.

Dan Hendricks (12:58)

It really was one of those things where we thought that training on browsing tasks would work. Felt like we had good conviction in it. But actually the first time you train a model on a new data Set using this algorithm and seeing it actually working and playing with the model was pretty incredible. Even though we thought it would work so honestly, just that it worked so well was pretty surprising. Even though we thought it would, if that makes sense.

Host (13:27)

Yeah, it's a visceral experience of like, oh, the path is paved with strawberries or whatever.

Dan Hendricks (13:33)

Exactly. But then sometimes some of the things that it fails at are also surprising. Sometimes it will make a mistake where it will do such smart things and then make a mistake where I'm just thinking, why are you doing that? Stop. So I think there's definitely a lot of room for improvement. But, yeah, we've been impressed with the model so far.

Host (13:49)

One of the biggest surprises of AI and a core principle for us here at Conviction, is how it can make bad markets suddenly good ones. The right technology can meet the right moment in unexpected ways. Arvind Jain built Glean in what everyone said was a graveyard market, enterprise search.

Arvind Jain (14:05)

It was like a graveyard. Like, you know, of all these companies that tried to solve the problem and it didn't. Part of it was just that I think search is a hard problem in an enterprise, like even getting access to all the data that you want to search. It was such a big problem in the pre SaaS world. There was no way to sort of go into those data centers, figure out where the servers were, where the storage systems were, try to connect with information in them. It was a big challenge. So SaaS actually solved that issue. So, like search products, like most of them, most of those companies started in the pre SaaS world. They failed because you just couldn't build a TurnKey product. But SaaS actually allowed you to, to actually build something. Which is my insight was that, look, the enterprise world has changed. We have these SaaS systems now. And SaaS systems don't have versions. All customers have the same version. They are open, they're interoperable. You can actually hit them with APIs and get all the content. I felt that the biggest problem was actually solved, which was that I could actually easily go and bring all the enterprise information and data in one place and build this unified search system on top. So that was actually a big unlock. And by the way, the origins of Glean is so at Rubrik, you know, we had this problem, like, you know, we grew fast. We had a lot of information across 300 different SaaS systems and nobody could find anything in the company. And people were complaining about it in our Pulse surveys. And I, and I was, you know, I always run it in my startups. And so there's a complaint that, you know, it came to me like I had to solve it. So I tried to buy a search product and I realized there's nothing to buy. I mean, that's really the origins of how Glean got started as a company. And so that was like one big issue. Like, SaaS made it easy to actually connect your enterprise data and knowledge to a search system. So that actually made it possible for us to, for the very first time, build a turnkey product. But there are a lot of other advances as well. One is, look, businesses have so much information and data. One interesting fact, one of our largest customers, they have more than 1 billion documents inside their company. Now here, this, you know, when LR and I, you know, when we were working on search at Google, you know, in 2004, the entire intern was actually 1 billion documents. You know, there's a massive explosion of content, like inside businesses. So you have to build scalable systems and you couldn't build like a system like that before in the pre cloud era.

Host (16:21)

Perhaps. No story captures the human impact of this AI moment and its potential better than what's happening in healthcare. Here's Shiv Rao, CEO and founder of Abridge.

Brandon McKinsey (16:30)

It's pretty heroic in general for a doctor to give you feedback like, hey, this sucked and you got to do better. Like, you didn't recognize the way I said this medication, or I'm a gastroenterologist and I would never, you know, sequence my problems in my assessment and plan section of my note. This way it doesn't serve me well and makes me look like, terrible as a doctor or whatever. We get that feedback, we love it, it's oxygen. But then we also get the feedback that's like, hey, this is amazing and I'm not going to retire anymore and I've got like years, decades left in my career now thanks to this technology. But in this channel, love stories, all of that feedback, that positive feedback, we just get it like programmatically funneled so any one of our people inside of the company can always go into that channel. And it's like purpose, you know, it's like fulfillment immediately. Like, you immediately understand why we're all working so hard and why it makes sense. Because, like being on this very telephone pole, like journey these last couple years is obviously like, it's new for so many of us and we're all kind of building new muscles, but it's, it's a lot of pressure. But this is my favorite bit of feedback. So this love story comes from a doctor at Tanner Health, which is a rural health system. And she wrote to us. She wrote, I was sitting at dinner last week and my son asked me, mommy, why aren't you working right now? I literally took my phone out and explained to him that a bridge is a new tool that lets mommy come home early and, and eat dinner with her family. I started to tear up and looked over at my husband, who then said, mommy's going to be able to eat dinner with us every night now. And we get feedback like that, like every day, you know, and so, like, there's, there's dopamine hits, you know, in hypergrowth and like, those are awesome, but I think that they get us through, like, sprints. But I think it's the oxytocin hits like this. It's the purpose, it's the fulfillment. It's like, that's, I think, what I think we're really after in this company. And so, like, everybody's mission driven out there, but I think this mission, like, it hits me at least a little bit different.

Host (18:21)

These conversations remind us that we're living through a hinge moment in history. Stay tuned as we have more conversations with the builders and thinkers leading the way for the rest of the year. If you like what we're doing, leave us a review on Apple Podcasts or Spotify. Comment on YouTube or let us know who we should have with guest. Thanks for listening. Find us on Twitter opriorspod. Subscribe to our YouTube YouTube channel. If you want to see our faces, follow the show on Apple Podcasts, Spotify or wherever you listen. That way you get a new episode every week and sign up for emails or find transcripts for every episode@no-priors.com.