
Loading summary
Podcast Advertiser
This episode is brought to you by White Claw Search. Great podcast pick friend. No surprises there. After all, you're all about finding the tastiest flavors out there, just like White Claw Surge. And with big bold flavors to enjoy like blood orange, BlackBerry, cranberry and more, it's time to go all in on taste. Unleash the flavor. Unleash White Claw Surge. Please drink responsibly. Hard seltzer with flavors 8% alcohol by volume. White cloth seltzer works Chicago, Illinois Limu.
Carl Benedikt Frey
Emu and Doug Here we have the Limu Emu in its natural habitat, helping.
Podcast Advertiser
People customize their car insurance and save.
Carl Benedikt Frey
Hundreds with Liberty Mutual. Fascinating. It's accompanied by his natural ally, Doug.
Podcast Advertiser
Uh, Limu is that guy with the binoculars watching us.
New Books Network Host
Cut the camera.
Podcast Advertiser
They see us. Only pay for what you need@libertymutual.com Liberty Liberty Liberty Liberty Savings Ferry Unwritten by Liberty Mutual Insurance Company and affiliates Excludes Massachusetts.
Sponsor Representative
This episode is brought to you by State Farm. Listening to this podcast Smart move Being financially savvy Smart move. Another smart move Having State Farm help you create a competitive price when you choose to bundle home and auto bundling. Just another way to save with a personal price plan like a good neighbor. State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts, and savings and eligibility vary by state.
New Books Network Host
Welcome to the New Books Network.
Leo Bader
Welcome to the New Books Network. I'm Leo Bader and my guest today is Carl Benedikt Frey. Carl is the Dieter Schwartz Associate professor of AI and Work at the Oxford Internet Institute and a Fellow of Mansfield College, University of Oxford. His new book, How Progress, Technology, Innovation and the Fate of Nations, traces a sweeping history of technological progress from ancient China to the current advances in AI to illuminate how new innovation can be translated into lasting growth. Carl has been working on the relationship between employment and automatization for a long time, and how progress ends puts this question into an exhaustive historical context. So Carl is the perfect person to tell us if we can still expect to have a job in the coming years. Carl, thanks for joining me.
Carl Benedikt Frey
My pleasure.
Leo Bader
So your book hinges on the observation that both decentralized and centralized modes of economic organization can drive progress at different points. Why is this? And how should we orient how we think about technology and progress? Knowing this?
Carl Benedikt Frey
So I think for most people engaged in science, technology and innovation, we know that there is a time to explore and we know there's a time to get things done and execute. And so if you write a book, for example, you spend a lot of time initially talking to people, reading broadly, trying to get input to figure out what it is that you want to do. But then at some point, when that has been crystallized, you need to get your act together and write the thing down and execute. And most tasks, whether in academia or in business, are like that. So no factory could function if every worker questioned every procedure. Right. But very few new ideas would emerge without questioning. And so there is this tension between exploration and exploration, between innovation and execution, if you like. And when something, when an objective is clear, it's much easier to execute it from the top down. And so the Manhattan Project delivered the first nuclear bomb, the Apollo Project put the man on the moon, the RAD Lab advanced radar technology. But that's quite different from, let's say, arpanet, where there was no plan to develop the modern Internet, that hinged on a much more unpredictable chain of exploratory efforts. And so that happened in a much more decentralized manner.
Leo Bader
So we have to start with decentralization, but once we land upon an idea that we know is worth exploiting, it needs to be more centralized. We can't keep questioning that's right.
Carl Benedikt Frey
And then we do decentralization again to go on to the next thing.
Leo Bader
So what are the implications of this for how we should think about the state's relationship to progress, which I assume needs to be more flexible than many politicians or pundits might conceptualize it.
Carl Benedikt Frey
So the state can take on a role both when it comes to exploration and exploitation. So as you may have noted in my earlier answer, the RAD Lab, Manhattan Project, the Apollo Project, these were state initiatives, right? And they were more of centralized nature and was more about executing based on existing findings and existing technology. And so when a project can be clearly defined, the state can play a reasonably prominent role in executing from the top down. ARPANET was also state funded, but it depended on a decentralized network of universities to draw talent from. And obviously, when you give people more autonomy, you need to be sure that you're hiring the right people. And the pool of talent that the United States has had access to through its decentralized network of universities was a prerequisite for that. And so much of the university system was also in part seeded through state grants back in the 19th century. So the state needs to take on very different roles when it comes to supporting innovation and technological development, depending on what it is it's trying to achieve.
Leo Bader
And do you see consistencies across the historical record and what kinds of approaches or what kinds of instruments states have to promote this exploratory phase. And maybe conversely, instances in which states do one or two very significant things that prevent this.
Carl Benedikt Frey
So clearly having an open space for scientists to interact and debate and disseminate their findings is important. And that's something we do see throughout history, that is quite important to frontier discovery. But again, it depends a little bit on context as well. Right. And so if you go back to the 19th century, competition between states played a very major role in pushing for reform that encouraged exploration and that, for example, consisted of getting rid of gilded regulations. And so the Prussian defeat to Napoleon in 1806 paved the way for the Steinhardenberg reforms that made that possible. And Russian losses at the Crimean War paved way for the abolishment of serfdom. And there are many other examples. So competition, I think, has often paved the way for the kind of liberalizing reforms we've seen in the past. And at times it also comes from a mobilized society. And so there was a push for a more meritocratic civil service in the United States in the late 19th century. That more meritocratic civil service then went on to institute the Sherman Antitrust act and a range of other initiatives that gave the state tools to enforce competition. That in turn enabled the US later on, for example, to file lawsuits against IBM for forcing it to unbundle hardware and software, creating space for new firms to enter, like Microsoft. Similarly, the breakup of AT&T, I think, was very critical to the development of the modern Internet because it meant that when the National Science foundation released arpanet, made it publicly available, it wouldn't be hand over to one carrier service, which could have bottlenecked it in its boardroom if it saw no immediate economic benefit to it. And the economic benefits of the Internet were far from clear at that point. So I think those are some lessons that we can draw.
Leo Bader
And when we draw this into the present day, where a lot of economic growth is concentrated in big multinationals, does that change sort of the nature of competition? And does that create any more of an opposition, do you think, between states and companies? Because companies are no longer so bound within one state, even if they're headquartered in one.
Carl Benedikt Frey
So there's nothing inherently bad with large corporations. And so you need large corporations to leverage technologies like factories, but bring down costs and make sure that new products are widely accessible to the population. And so there is no people's car, there's no T Ford without the Ford Motor Company and the assembly line and some other technologies that gave rise to mass production. The that said, consolidation can inhibit innovation. And we've seen with various antitrust initiatives that subsequently you tend to see an upsurge in patenting, you tend to see an upsurge in new job creation, you tend to see an upsurge in entry. But on the downside, you may also jeopardize the return to being a first mover. And so you may cap the expected return entrepreneurs have to pursuing productive innovation. So a trade off does exist there. And I think, you know, part of the reason that, that we've seen a lot of innovation happening after antitrust cases is that for the most part, first movers are not that concerned about a situation where there might be a monopolist, which is usually very far off when you set up a business. Now, if you go back just, you know, a couple of years and you ask yourself, you look at the tech giants we see today and you ask yourself, do they have a monopoly? Well, they look a lot safer back then than they do now since the release of ChatGPT, and that was indeed acknowledged in the recent lawsuit against Google. And so unpredictable changes in technology matters as much, if not even more than antitrusts. And no cooperation can be safe. At the same time, it's not a very good system if you're dependent on incumbents making mistakes and missing new technologies. And so we have been seeing in recent years upsurge in outright killer acquisitions whereby firms buy up competitors just to close them down. We have seen a revolving door between the US Patent and Trademark Office and incumbents, whereby extreme examiners grant low quality patents and then go on to work at some of these large technology companies, which raises barriers to entry for new smaller firms entering the market. So yes, I do think, broadly speaking, that there are concerns about the rising market concentration that we have been seeing over the past two decades or so.
Leo Bader
I'm curious about whether you see a, so to say, correct turning point for moving from exploration to exploitation. If there's any sort of consistency across history when saying, okay, let's stop looking at all these different ideas and let's go all in on this one or these two ideas is the right decision or in other cases the wrong decision?
Carl Benedikt Frey
I think it's very hard to get that timing right. And I think, you know, in the 1970s, the mass production system essentially peters out, growth stagnates all across Europe, the United States, and, you know, behind the Iron Curtain as well. And so it's clearly the case that we didn't just have something to replace that growth with at that point that eventually became the computer. And as I argue in the book, a key reason that happened in the United States rather than in Japan, which many predicted would overtake the United states in the 1980s, had to do in part with competition policy, in part with the federal system permitting differences in regulation. And so in California, famously, non compete clauses are void. And that meant that when engineers left Shockless Semiconductor to set up Fairchild, they could do that. And when two of them left Fairchild to set up intel, they could do that. And that's contributed to, to the constant renewal of Silicon Valley in this period. But there was nothing planned. And clearly when it comes to antitrust initiatives, you don't want to apply antitrust arbitrarily in the hope that by hurting some incumbent you might get some entry. You, you do want it to check anti competitive behavior. And so I think it's very hard to get the timing right. And so I think what you need to be focused on is to make sure that incumbents are not competing unfairly and then let the best technologies win in the marketplace.
New Books Network Host
Upgrade your laundry routine with a durable and reliable Maytag laundry pair at Lowe's. Like the new Maytag washer and dryer with performance enhanced stain fighting power designed to cut through serious dirt and grime. And what's great is this laundry pair is in stock and ready for delivery when you need it the most. Don't miss out. Shop Maytag in store or online today at Lowe's.
Sponsor Representative
Running a business comes with a lot.
Carl Benedikt Frey
Of what ifs, but luckily there's a.
Sponsor Representative
Simple answer to Shopify. It's the commerce platform behind millions of.
Podcast Advertiser
Businesses, including Thrive Cosmetics and Momofuku. And it'll help you with everything you.
Sponsor Representative
Need, from website design and marketing to boosting sales and expanding operations. Shopify can get the job done and make your dream a reality. Turn those what ifs into sign up for your $1 per month trial@shopify.com specialoffer.
Leo Bader
You mentioned antitrust is one big one. But what other institutional structures can be built to ensure that in one state or in one society, there's space for both exploration of some technologies and simultaneously exploitation of other technologies that are further along. And to give both of them the space and the resources that they need to continue?
Carl Benedikt Frey
I think it's just as I said, it's about creating that space. And then you can plan what all these localities or firms or whatever it is that you're thinking about do with that space. Right? And so one of the key reasons that Soviet Union failed to adjust to the computer revolution is that it didn't have any space for exploration. Right. So entire economy was more or less managed from Moscow, which managed steel and railroads and everything centrally. It's quite different from China where provincial leaders had much more autonomy to pursue different strategies to attract foreign investment to spur growth after the opening in 78. And so I think it's more about having that space, having that diversity in regulation enabled, and then leaving it to local authorities and firms to figure out what's the best path forward within that space.
Leo Bader
If we think about developing economies that maybe are not as interested in being at the forefront of the next great technological innovation, but more in catching up on what there already is, would it make sense to adopt a more centralized economy that can then spur very quick adoption of already existing technologies?
Carl Benedikt Frey
It might do, but let me qualify that in a number of ways. So first of all, industrial policy has been pursued all around the world world with very mixed result. Right. So Latin America pursued industrial policy. It didn't work out for the most part particularly well. You can look at Korea and Japan to some degree, China perhaps to a lesser degree, and say that here those cases industrial policy were quite well. And I think key difference there is that those places have a long history of having meritocratic states and relatively strong states that can enforce, for example, export discipline and make sure that firms that only firms that are actually competitive in global markets are allowed continued support. They also have more capacity to actually get things done, roll out infrastructure, provide public services, and they have a capacity quite importantly to check vested interest. And so if you have some of those preconditions, then I think you're in a better place to pursue industrial policy. But if this was easy to achieve, most places would have done it. And the fact that there's mostly a few places in East Asia and to some degree Europe in the post war period and in the 19th century, catching up to Britain and I've done this suggest that it's not that easy and in many cases those preconditions are lacking.
Leo Bader
What relationship do you see between the public and private sector across the historical record, both in the exploration and exploitation phase? Is it mostly a productive cooperative one or can there be conflicts that hinder innovation?
Carl Benedikt Frey
Well, there can be absolutely big conflicts and collusion. And the more you support one firm, the more invested you are in one company, the stronger the incentive there is to undermine competitive forces that might displace that firm. And that's why it's so worrying. Some of the tendencies that we now see in the US and the government taking shares in intel and US Steel and planning to do so in range of industries. And even, even in China, most progress, innovation, productivity has become from private foreign invested companies. And in the cases where it's done well, industrial policy has been beneficial. It's been mostly local initiatives and local authorities having firms competing against each other within this vast market in China. But if you take the United states in the 19th century, it didn't have a lot of state capacity. It opted for small government. And so it needed to depend much more on, on private enterprise, on private initiatives to roll out railroads, for example. And so that turned out to work reasonably well in terms of letting private firms develop the railroad network. But it is also true that as a consequence of that, there were a number of scandals, there were a number of corruption cases, and that eventually paved the way for a more meritocratic civil service through the Pendleton act, which then began to take on more tasks itself. But yes, that collaboration between private enterprise and firms does still exist in the United States and exist in many other economies to varying degree. And so it's easy to understand why the state want to outsource some activities. But it's also then hard to sometimes monitor performance, make sure that competitive forces remain intact. And so in Italy, for example, in the late 19th century, early 20th century, when he wanted to develop not just his own railroads, but also his own railroad industry, developing locomotives, for example, it had procurement practices for which actually permitted also foreign firms to compete for those contracts, which put pressure on Italian firms as well, and I think contributed to keeping them in check.
Leo Bader
What implications do you think your argument has for debates around what kinds of political systems are best for generating economic growth? If we think about sort of the caricatures, democracies would be less decisive and authoritarian systems presumably would be able to better exploit innovations once they have been produced somewhere. But should we maybe be a bit more careful in drawing immediate conclusions here, given that, as you say, there are these two distinct phases of growth?
Carl Benedikt Frey
Well, so I think first of all, there's compelling evidence that on average, democracies grow faster than non democracies. But if you look at autocracies, there are also different kind of autocracies. So Singapore and Zimbabwe, there is a lot of shows that there is a lot of variation within that category. And so as a general rule, more institutionalized autocracies, where you have, for example, a state bureaucracy where, you know, meritocratically recruited civil servants is more able to get things done than A personal autocracy, where, you know, everything is down to one person who, you know, arbitrarily promotes and selects loyalists to keep the regime floating. And so a meritocratic autocracy or institutionalized autocracy can roll out often infrastructure more quickly, for example, by overriding vested interests, than a place that, or let's say private property rights are stronger and more secure. But also in an autocracy, priorities can change. They can change quite quickly. And so if you're a firm in an autocracy, you ideally want to seat at the table and you want to invest more in building up political capital to shape the priorities and have some say. And that means that you're diverting resources away from innovation, exploration towards lobbying and political capital. You have that to some degree in democracies as well, to be sure, and love. Spending on lobbying in the United States has basically doubled since 1990. But you do have the courts, you do have reasonably secure private property rights, and that means that you're less dependent on changing political priorities than you are, for example, autocracy. So I think it's fair to say that democracies, for the most part, have some pretty strong advantages. We can sometimes look to some autocracies that are doing very well in building quickly and catching up and fool ourselves into thinking that we will soon be overtaken. But, you know, the task of the frontier is quite different to the task of catching up.
Leo Bader
Absolutely. Let's end by talking about AI for a little bit. How does it compare with other historical transformative technologies like the steam engine or the printing press? And does the fact that it maybe has some potential to kind of intrinsically generate knowledge mean that it could potentially start to bridge this divide between decentralized exploration and centralized exploitation?
Carl Benedikt Frey
Well, potentially could, is doing some work there. And so, yes, it potentially could, but it's not doing that. Right. And so, yes, AI can be a useful tool for science and innovation, but I struggle to see how it's currently at least anywhere near as transformative as the Internet and the personal computer. Right. Those were the ultimate research tools, connecting the best scientists and inventors around the world, streamlining the research process, giving most people access to the world, stored knowledge in their pockets, and what we get is a sort of decade streak of productivity growth, which then, you know, eventually the mid 2000s peter out. And so I don't think AI is on par with that yet. It could be, but it's by no means clear. And it's by no means clear that the models we have now, which are Basically, large language models that are unable to be out in the real world and learn from experience and learn continually and basically are rewarded by imitating human behavior rather than learning will get us there. So I think that's very much an open question. I don't know the answer to whether which is the right path forward. I don't think anybody actually really knows, although some people think they know. But in the end of the day, you need somebody to show that something works. And that's the beauty of decentralized exploration. And so I don't really think that AI is changing the equation. We still need exploration, we still need human ingenuity, at least for now. And I'm open to the possibility that that might change in the future, but we're definitely not there yet.
Leo Bader
So there's a sense in which it might be sensational as something that's very cool and powerful to see, but in terms of the actual growth and innovation that it's generating at the moment, it's not quite where other technologies have been.
Carl Benedikt Frey
Yeah, I mean, to be clear. Right. You know, suggesting that it needs to push frontier discovery is also quite a high bar. So, you know, I think it can be very useful in medical research and in other domains which, you know, and, you know, useful in screening the literature, et cetera. But, you know, if you had trained an LLM on data available up to 1900 and you've asked, it would have asked it, will humans ever be able to fly? It would have looked back at the long list of failed attempts. It would possibly have looked to data on birds to suggest flight in principle is possible. But it would have concluded that there's no bird that weighs more than 30 pounds that's able to get off the ground. Larger birds, like ostriches, can't fly. And so just before the Wright brothers took off, it would have no chance to predict that. But clearly it's a useful tool for many other things. And so I don't want to diminish the technology, but I don't think it's, you know, whether puzzle computer and the Internet is. And it clearly hasn't created this sort of new industries on the scale that automotive did or electrical industries, or even if you sort of take automotive, then, you know, it wasn't just automotive industry. It was a range of supplier industries creating components and a host of industries producing machine tools to make the components, et cetera. And so AI right now is more like a productivity tool. And it may eventually change. It may eventually create new industries and more sustained growth as a result of that, but we'll have to wait and see.
Leo Bader
Well, we should wrap it up there. Carl's new book, How Progress, Technology, Innovation and the Fate of Nations, can be purchased through the link on the New Books Network page. If you're interested in more, Carl has another book, the the Technology Trap, Capital, labor and Power in the Age of Automation. Carl, thanks again for joining me.
Carl Benedikt Frey
Thank you very much for having me.
Podcast: New Books Network
Host: Leo Bader
Guest: Carl Benedikt Frey, Dieter Schwartz Associate Professor of AI and Work, Oxford Internet Institute
Book: How Progress Ends: Technology, Innovation, and the Fate of Nations (Princeton UP, 2025)
Date: October 17, 2025
In this episode, host Leo Bader interviews economist and historian of technology Carl Benedikt Frey about his new book, How Progress Ends. The discussion traces the historical dynamics between technological innovation and economic growth, with a particular focus on the interplay between decentralization and centralization, the role of the state, competition policy, political systems, and the transformative potential (and limitations) of artificial intelligence. Through historical and contemporary examples, Frey explores how nations have successfully—or unsuccessfully—translated technological breakthroughs into lasting societal progress.
Exploration vs. Exploitation: Frey outlines the need for both decentralized, exploratory innovation and centralized, top-down execution.
Dynamic Phases: Societies must transition between these modes:
Flexible State Role: The state's role changes depending on context—sometimes leading (execution), sometimes enabling (exploration).
Historical Examples:
The Role of Large Corporations: They can be enablers and inhibitors of innovation.
Current Concerns:
Creating Space for Both:
Developing Economies and Industrial Policy:
Benefits and Tensions: Productive collaboration can exist, but risks include corruption, collusion, and stifling competition.
Historical Cases: US reliance on private initiative (railroads), Italy's inclusive procurement practices, and China's focus on competition even among favored firms.
Democracy vs. Autocracy:
Core Tradeoffs: Autocracies risk creating incentives for rent-seeking and political gamesmanship. Democracies provide more stability for innovative activity.
Transformative Potential vs. Hype:
Still Need Exploration:
Limits of Prediction:
On the Tension Between Innovation and Execution:
On the Role of Antitrust:
On State Capacity and Industrial Policy:
On Innovation and Political Systems:
On AI's Current Limitations:
Carl Benedikt Frey's How Progress Ends provides a sweeping yet nuanced examination of the factors behind technological and economic growth, cautioning against one-size-fits-all prescriptions for how societies should structure innovation. Historical cases and current technological disruptions show that the right balance between exploration and execution, as well as flexibility in institutional and political arrangements, is key to translating innovation into prosperity. While AI presents new possibilities, Frey argues that the fundamentals of progress—competition, openness, and state capacity—remain as vital as ever.