Loading summary
A
As the 21st century was getting underway, Hollywood released a series of films that were daring, entertaining, and absolutely unmissable. Films like 25th Hour, Bring It On, Zodiac, and no country for Old Men. They arrived during the George W. Bush era, a chaotic time in America. Think 9, 11, Katrina, the mortgage crisis. After the Bush years, the country would never be the same, and neither would Hollywood. I'm Brian Raft, and in my new limited series, Mission Accomplished, we're gonna dive into some of the biggest movies of the Bush years and look at what they said about the state of the nation. We'll go behind the scenes with filmmakers and experts and relive some of your favorite movies from the early 2000s, from Donnie Darko to Michael Clayton, from Anchorman to Iron Man. So slip on your sketchers, dig out your old Nokia, and join me for mission accomplished, starting Aug. 12 on the big Picture feat.
B
This episode is brought to you by Zendesk, introducing the next generation of AI agents built to deliver resolutions for everyone. With an easy setup that can be completed in minutes, not months, Zendesk AI agents resolve 30% of interactions instantly, quickly giving your customers what they need. Loved by over 10,000 companies, Zendesk AI makes service teams more efficient, businesses run better and your customers happier. That's the Zendesk AI effect. Find out more@Zendesk.com this episode is brought.
C
To you by Whole Foods Market. Whether it's a game day or backyard fire pit or simply to enjoy delicious food with the people you love, Whole Foods Market has got you find great everyday prices on 365 brand favorites like organic tortilla trips and salsa, plus apple cider sandwich cookies. Yum. And right now, you can save 50%. That's right, 50%. With prime on select frozen pizzas through October 28th. Visit your local Whole Foods Market today and store online. Terms apply today.
B
AI and work. In the year 1800, it took about six weeks to travel from New York City to Chicago. You were probably going by horse and carriage, and the roads were cracked. So if you wanted one summary of what the canals and railroads did to America, you could say they shrank the country. One implication of a smaller country is that it was easier for companies based in New York to do business in the Midwest. Just as the train made it possible to move goods and people across the country at faster speeds, the telegraph accelerated the speed of information. And together, the telegraph and the train collapsed space and time for American commerce. But trains did something more. They they didn't just make it faster for companies to do business they changed the sort of business that companies could do. In 1977, the business historian Alfred Chandler made this point in a controversial, but I think, mostly wonderful book called the Visible the Managerial Revolution in American Business. Chandler pointed out that before the train and the telegraph, most businesses in America and throughout the world were small, local and personal. A single owner might oversee every step of production and sale. But when goods and messages could suddenly travel hundreds of miles in a day, scale exploded. And with scale came complexity. The owner couldn't do everything himself. He needed help. He needed dispatchers, accountants, foremen, supervisors. He needed managers. So the railroad and the telegraph did not just make old work faster, they created an entirely new kind of work that didn't previously exist. The work of management. And these managers shaped business and capitalism just as much as the forces of supply and demand. They were the visible hand to go along with Adam Smith's invisible hand. If you're a white collar worker in an office and you have a regional or department manager, or if you are a regional or department manager, or if you are both a manager and have a manager, you are ensconced in a system made possible by invented by the railroad and the telegraph. I've been thinking a lot about Alfred Chandler's thesis recently because it contains an important lesson for our own time. We tend to assume that inventions mostly automate existing tasks. The typewriter makes writing easier, the spreadsheet makes accounting easier, or chatgpt just makes writing faster. But Chandler reminds us that general purpose technologies often do something more profound. They reorganize the architecture of work itself. And that brings us to AI. We're at a moment right now that I think rhymes with Chandler's 19th century revolution. AI can make existing tasks faster, right? It can write memos, summarize meetings, answer emails. Fine, Good. Wonderful. What's less clear to me though, is how to complete the following analogy. As railroads led to managerial capitalism, AI will lead to what exactly? When the Alfred Chandler of the 2000 and 30s looks back to explain how artificial intelligence changed work, what book will he or she Write? What is AI's visible hand? Now, let me interrupt this open by appealing directly to a part of my audience. I know that some of you folks do not like AI, don't trust AI, don't know what it can do for you, don't believe perhaps that it can do much of anything. This is one area where the analogy between the railroads and AI completely breaks down. Everybody looking at a train knows what the train does. You put the stuff on it and it moves the stuff at a quantifiable speed across land that's trains generative AI has no fixed observable quality like this because it responds personally and privately to prompts. And that means that its potential as a general purpose technology is often siloed and secret. Some people use AI constantly to code and to write and to research, and others just don't know how to use it at all. Today's guest is Ethan Malik. Ethan is a professor of management at Wharton where he specializes in entrepreneurship and innovation. He's the author of the book Living and Working with AI and his substack One Useful Thing is properly named the single most useful guide I have ever found to make sense of AI tools and how to use them productively. But Ethan is also a deep thinker of the Alfred Chandler school of big ideas. He wants to not just help individuals use this technology more efficiently. He wants to understand, I think at a bigger picture, what's going to happen to us? What's going to happen to our economy as tens of millions, even billions of people use technology to make themselves more productive and at times even obsolete. I'm Derek Thompson. This is plain hunch. Ethan Malik, welcome to the show.
D
Thank you for having me.
B
You say the intelligence of AI is best understood as a jagged frontier. What does that mean?
D
So what makes AI kind of weird is it does a lot of things, like many, many things. And some of the things it does well, some of the things it does badly, and it's very hard to predict what those things are in advance until you start using it. So we have this mental image in our mind of this sort of jagged wall. Some things are just beyond the wall the AI can't do. Some things are inside the wall, the AI could do it. And it's really hard to know what those things are going to be or what they're not going to be. And that can be very frustrating.
B
As a user, I think the concept of jaggedness is a really great one because it resolves, as you indicated, a really common question that I get from people who are less familiar with AI, which is often something like, Derek, if AI is so damn smart, why is it so dumb? Right? A lot of people hear OpenAI or Anthropic or others talking about, you know, this is a tech that could eliminate tens of millions of white collar jobs in the next few years. And meanwhile you go online and you tell one of these advanced tools to make a frigging clock striking 9:30pm and it cannot do it. You can ask over and over again it can't make the clock strike 9:30. And so I like this idea of saying we don't need to declare AI a full genius or a full idiot. It is more sophisticatedly true to say that its capacities are jagged. It is very smart at some things and very stupid in others. How has the frontier, Ethan, gotten less jagged over the last few years and months?
D
So there's really two ways that this has happened. One is AI has gotten broadly more capable due to a couple of different breakthroughs. So it's a lot stronger than it used to be in lots of different ways. And that has enabled a lot of use cases that didn't exist before. So hallucinations, the kind of errors that we see, they've dropped quite a bit, which lets AI do longer tasks than it did before. So that's one area. It's just gotten broadly better, which enables a lot of different things. So the frontier's moved out. And then the second big thing that's happened is some of the most jagged parts of the frontier. Like, if we talked a year ago, right now, we would talk about how AI is terrible at math, and that was the big joke. You couldn't count the number of hours in Strawberry. There have been breakthroughs in the areas where the frontiers were at its worst, and now it's winning at the International Math Olympiad, for example. So both things have happened. The researchers in the AI labs have focused on the areas where AI was weakest and made stronger, and AI has broadly gotten better. That doesn't mean there's still not jaggedness there, but it's far beyond where it was, say, a year ago.
B
What is AI bad at right now? What is the canonical example? The same way that maybe 18 months ago it was, well, it can't name the number of Rs in Strawberry, or it can't show someone slicing a pizza. Is there a similarly broad or familiar way to describe the deficits of the frontier models right now?
D
So that's kind of interesting. It's a part of a broader question about how do we even measure success? I don't think there is one canonical example anymore. The Strawberry example, you know, a couple of brain teasers people used to use that. There are problems with some of parts of AI lag other parts. So, for example, image generation models have these notorious weak spots, right? Of, for example, clocks, right? Every clock in every advertisement is turned to, I think, 10, 10 as the time, if I remember correctly. And so it's really hard to get AI clocks to show something else other than that. So Image models are full glass of wine that's not spilling over. So there are weaknesses in some pieces. But in terms of the core text models that do that kind of thinking as we consider it, that produce answers for us, it starts to be much weirder things. For example, if you ask your favorite LLM to show you a seahorse emoji, they will start to freak out for very narrow, weird reasons. But there's not a canonical big example anymore the way there used to be. Like count the number of hours in Strawberry.
B
Do you have any kind of big picture theory for why specifically seahorses? I mean, it is so interesting to me where it's jagged, right? Like, why not understand clocks if you can also make beautiful PowerPoints? Why not understand seahorses if you can make clownfish? Like, maybe let's just narrow this to seahorses specifically. What do you think is going on there?
D
I mean, we don't know for sure. There's some people doing some exploration of this. But part of your intuition starts to be, what are humans wrong? If I told you there was a seahorse emoji, you'd probably believe there would be one. If I surveyed you, like surveyed your listeners, and there isn't. So that sort of leads the LLM astray here. And it keeps thinking it could produce one and producing logical combinations of Unicode information that should produce a seahorse emoji. And then it keeps getting frustrated when it doesn't work. So LLMs weirdly have pre existing beliefs that can get in the way of conversation.
B
When GPT5, the latest offering from OpenAI came out, the reaction was incredibly interesting and quite split. Some people were saying this was a profound disappointment, and others were saying, no, this is one of the most amazing things that's ever been built for white collar work. You said in a recent piece that you believe that with GPT5, AI has crossed an important threshold. What did you mean by that?
D
So part of what makes this difficult, we were talking about what makes AI hard to use, right? The seahorse emoji, the jaggedness. The other part is the labs don't help, right? So GPT5 is actually a couple different things in one bucket. And they both sort of went weird, right? So one of those is it is a family of AI models that range from a pretty dumb model that's fun to talk to, to a model that can actually do PhD problems. Not all of them, but a subset at sort of the doctoral professorial level. And it's also a router, a system that decides based on your question, which model should get to answer that question? And as a result, talk about jaggedness. If I ask the same question, if the router and its infinite wisdom decided to send me to the dumb model, I could get math errors. But if it sent me to the really smart model, I might have to wait 10 minutes and they get a really smart answer to a question that wasn't quite phrased the right way. So you resulted in all these really weird sets of interactions. So everything was true. You got very bad results and very good results. But if we look at the high end version of the models, right, the GPT5 thinking models are called the Pro model. They are substantially better at being able to plan and execute large numbers of steps to solve a problem. Sometimes we call this agentic work. And as a result, you could just ask the AI to do something, it does it for you. You don't have to do all the fiddly bits that we used to of phrasing your request exactly correctly and going back and forth the AI constantly. You could just kind of ask for something and it happens. But again, that requires you to either be paying and knowing to use the higher end models, or that the free router in its wisdom decides to send you to the smart models that could do this.
B
You mentioned agentic, and I want to make sure that we spend a good deal of time here talking about agents, because if agents white collar AGI arrives, if a superintelligence that can do tens of millions of people's white collar work, you know, coding and writing and Excel and PowerPoint, my sense is that the people at these frontier labs believe that that future will arrive in the form of an autonomous agent. So just from the beginning, what are AI agents and why are people so excited about them?
D
Okay, so everything in AI devolves to a marketing term as soon as it invented. So we should separate out that there's no clear definition of agent. And everybody on the planet will sell you an agent. But we have sort of a concept of what it is. So that's an AI system that when given a goal, will follow that goal on its own and use tools as necessary to accomplish it. And so what you need to have an agent to be successful is it needs to be able to go a large number of steps, right? If I ask to be prepped for this interview, for example, the AI would go to my Gmail and figure out what this interview is going to be. It would research you and your background, it would prep me on materials, it would go through a whole bunch of different searches. Maybe it would write an infographic for me. So it might do like a thousand independent steps and give me the result. And I think the fear and concept for a lot of us was that this was very difficult because AI makes mistakes. There are errors that creep into the process. And if you have a long horizon on a task, 1,000 steps, there's a really good chance that that error appears somewhere in those thousand steps that breaks the AI. We were wrong about that. It turns out, it turns out that AI is self correcting enough that when it makes an error, it doesn't just keep running with that error, it starts to realize something's wrong and fixes it. Not all the time, but enough of the time that we've gone from models being able to do maybe 20 or 30 steps independently to GPT5 could do over a thousand steps independently. And that changes what an AI can do. And so when we talk about agents, an agent is really just I want to assign an AI a task, it does it on its own without me having to intervene. Which is a little bit different than the co intelligence model that I talk about in my book, which is work back and forth with the AI. And so that is sort of what the labs have all been aiming for. And it went from AI can't really do this to suddenly this is a real possibility.
B
I have to imagine that agents have a jagged intelligence as well. Is there something that agents are really good at doing? Say taking a corporate memo and turning it into a PowerPoint versus something that's a very common white collar task that for whatever reason, maybe based on the number of steps or how long it typically takes someone to do, the agents are not quite as good at doing that.
D
So I would say we're still in the very early days of agents, general purpose agents that go out and do anything, right? And you could use These right now OpenAI has an agent mode and you could say, go buy me snow tires for my 2024 Corolla. And it will go out and research websites and read them and do it. But they have problems, they have errors and mistakes. So general purpose agents, it's not even like one jagged thing. It's not strawberries and ours. There's many, many things they can't do. The goal of these agents though is to be able to use them. Anything that a human can do with a computer, the AI could do. And that they started and their strongest area still is easily verifiable output. So computer coding is by far their strongest case because if the code doesn't run the ad could fix the problem. Increasingly now they've gotten good at PowerPoint Excel, which have sort of similar characteristics. I could verify as an agent that the output looks right or that the Excel formula works. As things get fuzzier, the agents start to have more issues because we can't as easily rate, is this a really good memo or not? And so it's easier to have errors and mistakes sort of compound. It doesn't mean they're not useful, but it's less understood about whether they're good or bad.
B
One thing that I've said before is that I think AI is very good at answering questions, but not so good at telling me when the questions I'm asking are wrong. And a really important part of I think being a good worker, maybe especially a knowledge worker, is having good taste in questions. Like, if I'm a journalist and I go to my editor and I say, here's my piece, can you copy edit it? Well, that's something that ChatGPT, you know, GPT5 can do quite well, right? Finding grammar errors, finding typos and editing them in line and bold, that's easy. But going back to me and saying, derek, you didn't do enough reporting on part three of this essay, or, Derek, I think this entire essay is actually wrong, based on this other piece of research that I went and found. Those are the kind of replies you're likely to get from a human editor who sees their task, their job, as being not just answering the question that they're getting from their journalist, but also posing new questions back to that journalist. You have such a lovely and clear way of seeing this. Do you have a good way of seeing the ways that agents aren't yet sophisticated enough to work with their humans in order to sort of ask questions back?
D
So I'm going to agree with you strongly on the general principle and talk about it and disagree with you in your instance. So on the writing side, one thing we're talking about is proactivity. The AIs are not proactive and they're also sycophantic. They tend to want to make you feel good for a variety of complicated reasons, some nefarious, some much less so. And as a result, they're often bad at pushing you. They'll agree with you, they'll redirect you. That's improving a little. But I would argue that some of this is about prompting. I mean, some of this is about the questions you ask. So you start with the good questions. For example, I do a lot of writing. Much like you, I always write material on my own first because I think my own writing matters and my own thoughts matter. But I always use AI to get feedback, and often some of the really good feedback I get comes from AI. I have other editors in my life. I find AI to be an incredibly good editor in terms of suggesting what's missing in a story. So I would wonder if you're asking it go into a clawed model, which tend to be the best writers nowadays, and, you know, saying, what am I missing here? What would a naive audience member miss? Is. Is the story? Like, you have to ask it to be critical, but it's quite good at being critical. When you ask, like, is there a through line here that makes sense? Is there anything unnecessary? And interestingly, by the way, you said grammar corrections are good. I thought the AI actually misses grammar corrections because it's not a detail oriented piece of work necessarily. And sometimes it'll catch a grammar issue, sometimes it won't. Sometimes it obsesses over certain issues or not.
B
I am such a sloppy writer that any improvement over mine looks like a dramatic improvement. So just seeing that it's found 1575 errors makes me feel good. Like, at least it's cleaning up my copy. I love this idea that a part of working well with these models, part of working well with these agents, is us, the humans, developing a sophistication for a taste for asking the right questions. I mean, let's just make it really personal. You write essays, you write books when you're working with Claude or GPT5 to edit your essays, to find holes in your logic, or to make the flow of ideas cleaner for a reader. What are the kind of questions that you ask to make your writing better?
D
Okay, so we talked about taste, and there's really two kinds of taste that matter here or three kinds, right? There is a taste of the good questions. There is a taste in knowing my own style that matters. And then there is a taste of curation. So a typical way I might handle this would be first initial kind of thing is, hey, anything spot you is weird here that a naive reader might be confused by. But then I go very specific. I can't nail the Transition from paragraph 1 to paragraph 2. Give me 37 versions of how I might do that transition. Okay, more like three. Okay, you're being corny. Stop being corny. Go in a different direction. This doesn't sound like me at all. Come on, you could do better than that. So, like, there's a taste. Like, my taste matters. My Taste for the question matters, but also my taste in curation and selection matter in those cases. So I often will do that. Like give me, you know, I need 75 different ways of closing out this paragraph. And I will often not use any of the ways it suggests, but it will spark my own writing. So there is this difference of, like, I'm not interacting with a human editor who has limited patience and I have limited time and get infinite kinds of results. So I do a lot more variation and selection than I would with human work and less dialogue with the AI about. Let's talk about this paragraph more. No, I think more like this, more like that. I do some of that, but a lot of it is, let's expand out the number of options, select them down, expand them out again, until we end up with something we really like.
B
There's two different things we're talking about here, and I want to make sure that they're disentangled. What we were talking about just now about writing with an AI is an example of what your book calls CO intelligence. But the long vision of these autonomous agents is that one day they will become the full Ethan Malik, or more specifically, they will become the full paralegal. They will become the full marketing assistant. If the amount of time and the amount of tasks that they master grows and grows. Well, if jobs are just a bundle of tasks, they'll eventually click every box in the bundle and do the whole paralegal job. For example, what are the jobs that you think are most susceptible are most vulnerable given the trajectory of agent technology?
D
So just to tie the knot of what we talked about before, I can ask an AI to write an essay for me and do the research for me. And it now does a pretty good fake Ethan essay on research and say the AIs are good. That's an autonomous agent task. Does searches the entire web, generates themes, rewrites them, turns them into my style. Right, so we can. Right now we're talking about the cointelligence approach. As you said, agents are quite different and it's hard to know, Right, because there are some tasks that AI is really good for right now, but we don't have a full range of evaluations. We've just started to get our first evaluations about how good these agents are. The answer is pretty good. I think single, focused tasks that involve collating information and writing about it are probably in the most kind of risk. Right, so you're talking about jobs are bundles of tasks. A job is not just one thing. As a professor, I teach And I write and I do research and I provide emotional support to my students. I do a lot of different things. If AI does one of those things like grading, we have some early results showing AI is a better grader than most humans. I don't use it for grading because my students would get mad. Right. So another barrier to sort of adoption. In that case, it doesn't mean that my job destroyed disappears. So I think if your job is something AI is good at and you don't have a bundle of tasks, you have one or two tasks that you do. I write press releases. I think it's why freelance jobs are under threat to some degree, because a freelance job is an assignable job that I give somebody that they give me an output for, as opposed to being a journalist where there's actually a ton of different things you do. And if AI freed up some of your time, you'd switch to other things you do as a journalist and do those really well.
B
As you were describing how these agents work, I thought about one tool that I use on ChatGPT, which is called Deep Research. Yesterday I asked Deep Research to Write me a 10,000 word history of the rise and fall of the dot com bubble, with a particular emphasis in the ways it's like the AI boom, or not like the AI boom. So I use this for my own research, essentially. And because I'm very nervous about hallucinations, I asked for links to every falsifiable claim and it went off for 10 minutes and came back with a really elegant summary. Exactly what I would hope to get from a day's work from a research assistant or even several days from a research assistant. And it occurs to me that that is also agentic. Like the AI is taking my request, it's making out a kind of outline and adhering to that outline to construct a 10,000 word essay for my benefit. Is the line between models like ChatGPT and agents like crystal clear or is it getting blurrier in cases like this?
D
It was clear until recently and I think it isn't clear now. Right. So when I worked with an AI before, I'd ask a question and I'd get an answer. That wasn't an agent. Right. That was an interaction. And there was lots of things that people were speculating about about what agents would look like. We kind of know now. So there are now agents available to you right now. GPT5 thinking is an agent. Claude 4.5 sonnet is an agent. If you ask them, they go through a planning process where they think about how they want to solve it. They may decide, rather than answer your question, they need to do some research work first, or they need to write some computer code, or they create a PowerPoint, they'll do all of that work before returning an answer to you. So in many ways that is the beginning of what an agent does. So it's blurred the line between what a chatbot does and what an agent does.
B
If you told me 10 years ago in 2015, that a decade from now we're going to have a technology that's called an AI agent, and it's essentially going to be able to do an incredibly broad range of white collar tasks at B A level, whether it's working with Excel or writing a memo, or turning a memo into a PowerPoint. I mean, this is your classic meat and potatoes knowledge economy stuff. I would think if you told me that, that the companies that adopted this technology would be suddenly like way more productive than the companies that didn't adopt this technology. It's not entirely clear to me, just a guy sitting in a room right now, that there's this incredibly dramatic schism happening in the economy where like the handful of companies that understand how agents work are just completely racing ahead of the rest of the economy. Maybe that is happening and I don't know, but it just doesn't seem like AI is already like supercharging productivity at some places and not supercharging productivity at places that don't know how to use the tech. How do you see this landscape? I guess one way, simple way to ask the question is if agents are as good as some people describe them, why aren't we seeing some companies absolutely race away in terms of productivity?
D
It's a really interesting question. And anyone who's worked at an organization probably can come up with 10 answers off the top of their head that are all correct. I think the view of technological optimist pessimists, the Silicon Valley folks, is we have this new technology, the world gets transformed. That is not how technology actually works. Right. I think AI adoption is far faster than we expect. And by the way, I do think you're seeing 90%. We have evidence that people are seeing very large productivity gains. If you survey people, they say large productivity gains. Those aren't being captured by companies. Well, why not? First of all, probably some of this is illusory, right? Like we haven't figured out how to work with the AI yet, but a lot of people are just hiding their AI use from their companies. If I have a 10 times performance improvement I know if I show the company how to do this, first of all, they'll think I'm less smart because the AI is doing the work. They'll ask me to do more work. They might fire me or my friends because now we have cost savings and companies love cost savings from it, so people are hiding it. But let's say I even am in a company where I'm sharing this stuff, let's say, which is already happening, I'm now a 10 times more productive coder. What do I do with that? Like I'm part of an agile development sprint with a 15 person team that has a two week meeting cadence and stand up meetings. I get 10 times more work done. What do I do with that? What work should I be doing? And so part of the problem is about process, right? There's this danger that otherwise 10 times more PowerPoints is not going to make a company more successful, I guess, unless your job is making PowerPoints for some reason. And maybe that's what a lot of companies actually do. But I think that part of this is about there's a lot more pieces to just like similarly, if I gave any organization a thousand PhD interns, I don't think that would make them more productive overnight. And so I think there's this sort of fallacy of like, AI is either good enough and it transforms everything right away, or it doesn't work. And it's clearly untrue. I mean, I talk to companies all the time. There's no longer sort of this question, does AI provide benefits? Now it's this hard leadership and process problem of what do I do now that I have 10 times more production in this area? Where do I want to build agents that make a difference? How do they integrate with humans? Those are the really hard problems.
B
In that spirit, I definitely want to move to really practical advice. So take somebody with a typical white collar job. They work with Excel, they make PowerPoints, they write memos, and once in a while they're expected to maybe come up with some new idea, some new way to solve a problem, a new line of business, get a new client. Have you found in your speaking that there's a piece of advice that you can give to somebody in this classic knowledge economy job whose attitude is right now something like, quote, I just don't see how AI can make me better at work, end quote. Like someone who's at that first step of just like, forget autonomous agents taking over the economy in the2030s. I don't know what to do with the suite of technologies that exists right now to make me just a more productive knowledge economy worker. Where would you suggest someone in that position start?
C
This episode is brought to you by mobile1.mobile1's synthetic motor oil knows your car is your happy place. But did you know your happy place has a happy place? It's not stuck in rush hour traffic. We've all been there, especially in la.
B
It's always terrible.
C
And the entire time you're sitting there, you know deep down that your car's favorite place is on the open road singing its favorite song while you sing along to yours.
D
Mobile 1.
C
For the love of driving, visit LoveOfDriving us to learn more. This episode is brought to you by Viori Look, I'm not a big let's hype up workout clothes guy, but Viori, I gotta say, total game changer.
D
Been wearing a lot.
C
If you see me power walking around Los Angeles, probably going to see me wearing some Viori Sunday performance joggers that they have. It's made with four way performance stretch fabric, one of the most comfortable things you own. You will wear them everywhere, I promise. All you have to do is go to vuori.com Simmons and you get 20% off your first purchase with Viori. V U-O-R-I.com Simmons enjoy. Free shipping on all US orders over $75 plus free returns. Exclusions apply. Visit the website for full terms and conditions. This episode is brought to you by the Home Depot. The holidays have arrived at the Home Depot and they're ready to transform your house into a holiday home, no matter your style or budget. They've got so many types of string lights to choose from. You can create everything from a soft glow to a colorful display of holiday cheer. And I love this innovation. Their warm white LED lights come with steady light technology so your lights stay glowing even if one bulb fails. Find everything you need to get your holidays started with the Home Depot.
D
So I have really easy advice which is like the same advice from my book back in the day, which I still think is the right thing. There is no easy instruction manual. There is no manual for your job. The right thing to do is to use AI for everything you possibly can for about 10 hours in actual work tasks. And you will start to see the shape of the jagged frontier that we started talking about earlier. You'll start to understand what the models are good or bad at. You'll start to. You'll probably be freaked out at some point because it does things you wouldn't expect. You need to do this with a sort of pro ish level model. So that means, you know, either a company gives it to you or you're spending 20 bucks a month for usually one of the big three, which is either Gemini or Claude or OpenAI's models, ChatGPT. And you just need to try it and you will find tasks that it does for you. And I think you have to keep pushing for more advanced tasks because otherwise people use AI for things that are time saving but not transformational. Summarizing meetings or events, helping me write an email, that's an important starting point. But then it starts to get even more interesting. You're like, here's all the context for the media I'm going to have. What are some angles I should approach this with and how should I think through this? Give me some ideas of how to do this. Create some mock ups of this concept I had so we can kind of play test them back and forth. Some people have different kinds of ways of reacting or interacting with these things. I talked to a Harvard quantum physicist at one point who told me his best ideas come from AI. And this was back well before the models were good at math. And I was like, is it good at quantum physics? He said, no, no, it's terrible quantum physics, but it's really good at asking me good questions. So maybe you're someone who needs a partner to bat things back and forth with. You have a general purpose tool. You'll only figure out how to use it by using it.
B
One thing that you're very good at is finding papers that describe this jagged frontier, the little things that AI is breaking through at. Just in the last week, you've shared papers showing that retailers can predict purchase intent by asking an AI, a large language model, to pretend to be a specific kind of person and giving it a bunch of product pictures and asking what it wants, which is an amazing use case that I never would have thought of. You've showed that AI can, in a matter of minutes, reproduce statistical results from science papers, which is work that can take professors or journal editors hours to do if they're trying to evaluate whether a new scientific result is valid or not. I wonder, in this sort of landscape of papers being written on discovering what AI can do that we didn't initially anticipate. What are some of your favorites?
D
There's so many interesting ones. There was a really sketchy paper and it's sketchy not because the research was sketchy. Well, but because the methodology was controversial. Where a bunch of European researchers put AI agents onto Reddit's Change My Mind forum, which is their debate forum, without telling anyone. That was the sketchy part. They just put them on there, there they got IRB approval, but the Reddit community hated it and they found they were at the 99th level of persuasion in terms of persuading people on their viewpoint. So I mean, I think that that's a both important finding and an interesting one. Right. In fact, the only thing that robustly lowers conspiracy theory belief that we've found and like social scientists have been working this for a long time is a three round discussion with GPT4. Just like three rounds back and forth is a replicated finding. People believe conspiracy theories less two months later. On one hand.
B
Pause there. What does that tell us? What does it tell us? That we've built an artificial mind that is better at persuading us, surely sometimes of falsehoods, but also it seems sometimes of truths that counteract conspiracy theories. What is it about the alien intelligence that we've designed that makes it so good at persuading?
D
Well, we actually have some evidence on this and it turns out it's logic and personalization, the exact same stuff that makes it good at translating stuff from one frame to another. Like explain this to me. It does a good job of inferring who you are. And it's not, when you look at this in detail, it's not, by the way, it could be right. So I think we should worry about this a lot. But it's not that the AI is using some nefarious persuasion technique or social or manipulation. It's that it's explained to you in a way you understand and patiently listening to you and responding, which is I guess, the most hopeful of the persuasion answers. Now that doesn't mean that it can't be used for much more nefarious purposes and will not be. But in those studies it really is just like I can explain to you at a level you understand and we could go back and forth and I can answer your queries, you know.
B
Well, I want to close talking a little bit about costs and worries. I'm worried about several things with regard to AI that I've covered on a lot of these shows. I'm worried about worker atrophy, right? There's one question that is how do I use these tools? There's another question that is if I use these tools to do the same task over and over and over again, will my skill in that task decline? And I think there was a study recently that showed that doctors that use large language models to Diagnose disease, become on their own, less talented at diagnosing disease, which is kind of interesting. Do you worry about skill atrophy in the face of artificial intelligence? And if not, I'd be interested just to know, because you're such a clear thinker about these things, if you do have worries on the part of the workforce.
D
Oh, I mean, skill atrophy is a huge concern. I mean, massive. Right. But it's not an unknown concern. So, you know, I'm a professor and students have always cheated. Right. There's actually these great studies that show. I think it was. There's one at Rutgers that showed. And I'm going to get the numbers not exactly right, but something like 20% of 80% of students who did their homework in 2008 did better on their tests, and now it's down to 20%. Right. Why? Because people were cheating on their homework. And so they stopped getting the benefit of doing homework, which is miserable. Miserable and annoying. But teaches you things. Learning is hard. And of course, we atrophy skills all the time. I've atrophied a lot of skills because I use computers to do my math for me, and I don't do it by hand. And so we're going to worry about this a lot. Writing now suddenly essay writing can be done by AI didn't mean people cheated before, but now everyone can cheat at scale. I think any skill you don't use, you atrophy and skills. And we have to decide what skills you want to keep. There are ways of doing that, of course. Right. So we're going to have much more blue book testing and things like that in schools. And we can force you to learn because we have you in a learning environment and we can grade you on the outcomes. And then we could use the good part of AI, which is actually some early evidence it could be an incredibly good tutor when used as a tutor to support an educational experience, as opposed to a thing that gives you answers. The workforce is an even bigger deal because I teach people to be generalists at Wharton, at Penn, and then they become specialists by apprenticing. They work for you as an intern and you get the benefit of an intern who does work for you. It might not be great, but you didn't want to do it yourself. And they get the benefit of feedback and learn to be good at their job. If you just turn to AI for everything, the intern learns nothing. And the intern will be dumb not to give you AI content because it's better than them. So we have to Think formally about how we teach skills in the workforce. We may have to have assessments of people's ability, but there is a crisis coming in that set of stuff. But it's not an unsolvable one because we always give up skills to technology. But it is one that if we don't think about but we are kind of in trouble. Right? In a lot of different areas.
B
I want to make a slightly philosophical turn here. You and I are talking about using AI to do human work, but AI doesn't have human intelligence. It doesn't learn like we learn. It doesn't process like we process. It's a fundamentally alien intelligence that's being used for human ends. And that's a bit of a strange and spooky thing. In our conversations offline, you mentioned that you wanted to get at this point by introducing a concept from AI research that's called the bitter lesson. What is the bitter lesson?
D
Okay, so the bitter lesson is an idea that came from computer science. It's actually a fairly technical lesson, but I think it's been used more broadly and it's important to understand because I think it tells us a lot about the future of AI. So the idea of the bitter lesson is best illustrated with chess computers. So we're going to do a little detour to chess. And when you wanted to build a chess computer, what you would do is you'd hire a grandmaster to help you build that chess computer. And you put every move, opening gambits, and every move of how to close the game and end game results and lessons from everything we knew about chess. And even through Deep Blue, the machine that beat Kasparov, it was trained that way. There was a grandmaster who helped do the training, searched lots of possible moves, but it still had a human chess master at the center of the training process. In 2018, Google released something called AlphaZero, which was a chess computer that didn't know how to play chess. What it did was play on a chessboard, basically a mechanical chess, an automated chessboard against itself till it learned the rules of chess and eventually could beat grandmasters. And that was the bitter lesson. The bitter lesson is your beautiful, handcrafted attempt to instill all of your amazing human knowledge into a piece of software. Gets lost if you just throw enough computing at the problem and the machine learns how to do it itself. And this was very relevant to me because I don't take any money from any AI labs, but I talk to them all the time, along with my wife, who co directs the AI lab at Wharton with Me and we Talked to the OpenAI team that had just released their version of agents over the summer. And we've been building agents for education for a while. And we had a conversation with them where we said to them, you know, one of the problems with the agents that you have is they don't give you step by step directions. They don't tell you, I'm going to do this and then do this and then do that. It just does stuff. And that makes it really hard for us to diagnose the problems it has or intervene. And like, well, of course it just does stuff because we just train The AI on PowerPoint excel directly. We have no idea what it does. It just does good PowerPoint in Excel. And I haven't stopped thinking about that because that's in some ways the bitter lesson coming for office work. If your goal is output, if the only thing you want to do is produce a thing, then you can be bitter, lessened, you can learn the bitter lesson because the AI might just be able to make the thing better. If the human process matters, the friction interaction and the learning process matters, then maybe there's more of a moat there against what AIs can do. So I spend a lot of time thinking about that with agents. Agents are best where you can have an output you can train against where the process matters. There's more room for humans.
B
I want to think a little bit about, and this is a question that we're going to use, I think, to wrap up the agent section because I liked your note. I want to think a little bit about how strange this could get if agents really do take over the workplace. So I wrote a piece about job applications coming out of college and talked to a lot of college career counselors and they said, derek, you have no idea how strange the job application process has become. When I graduated From Northwestern in 2008, I sent out what I considered a ton of applications. I applied to 30 magazines before I got a call back for the Atlantic today. They said it's not uncommon for people to use AI to apply to 300, 500, even 1,000 jobs. And because that is way more applications than any human being can process. In the HR departments of Amazon and Meta and all these companies, they use AI in order to filter the AI enabled job applications. So that in a way is like a little diorama of what an economy might look like if rather than have human to human interactions, the white collar economy is mediated by agents, right? That Ethan's agent talks to Derek's agent to Figure out the perfect line of questions to ask Ethan, and then we sign off on it. And then you and I both get a memo that's sent by our agents to each of us. I mean, things could get really weird if this technology succeeds at the level at which and is implemented at the scale at which a lot of these labs are attempting. Right?
D
I mean, this is what I was talking about before process break. So all the processes in the world are built for humans, right? The college essay, the letter of recommendation. Right. I write a letter of recommendation as a signal I set my time on fire as an indicator that I care enough about of a student. The nature of the letter matters less than the fact that I wrote it right now. And I actually asked my students, would you rather have the okay letter that takes me 50 minutes to write, or would you rather have the AI letter in a minute that's much better and is in my style? And no one would know it wasn't for me. Everyone wanted the one minute letter because it was a better version. I actually had a student send me the prompt they wanted me to use to write their essay, which their letter recommendation, which I thought was interesting. What you actually want is the sentence now that says, yeah, they're good, hire them. But we have all these rituals and stuff built around this online. I just did a thing earlier where I was asking Claude for PowerPoints. I just kept saying, more PowerPoints, more PowerPoints from a memo. And I ended up with 17 really good PowerPoints. Like, no one wants to decompress that PowerPoint back to the original prompt. So this is part of what's going to break, right? When you ask about the productivity problem. The productivity problem is everything. We do things like writing, showing effort. Think about an essay. An essay showed me competence by how well it was written. It showed me effort by how long it was written. It showed me attention to detail by lack of errors. None of those things are signals anymore. So what do I do when I grade an essay? All my heuristics are gone now. It takes forever to grade an essay where it used to take a couple seconds to do, because I would know whether it's good or bad. This is happening everywhere. So I think a lot of things are going to break before they get reconstructed. And I wouldn't be surprised if we see lots of processes that we have to rebuild from the ground up to think about what it means to be in a world of AI. It's happening everywhere. Scientific papers, more of them coming in. The solution can't be AI grades AI content. And it's just a mess of a world where things just get filtered from AI to people. So we're going to have to think about this. Maybe there's more in person conferences and meetings, right? Maybe that there's more, we have new selection mechanisms, but it can't go on like this.
B
Can we just talk very specifically about the way that it can't go on in science? Because it's interesting that in the same way that in the job application process, AI can both amplify job applications and evaluate job applications. In the realm of science papers and scientific journals, AI can write science papers, but judges reviewers with AI can also evaluate science papers. So how is that going to break and how is that break going to be resolved? Because we can't possibly just live in a future where there's just no more peer review and also no more research because it's just the AIs writing the paper for the AIs and publishing science, the journal by artificial intelligence.
D
So there's actually a ton of things. First of all, almost all the systems we talked about are already broken, but they kind of work well enough, right? So like, well before AI came out, there was a great article in Science explaining that we're under a flood of research and it's no longer. People aren't reading it anymore, right? So people are just referring to the same articles over and over again and not really reading like that was pre AI. We had this problem, right? Between paper mills and just having too many scientists versus too little review the way journals worked. I mean, there's a lot of broken systems out there that were just sort of chugging along, talking about homework, right? Which people were cheating at before, but was kind of okay, we kind of resolved. So a lot of this is systems already under strain or that were already broken that we're evaluating. But in science, it's especially interesting because right now I think we're at a place where the current AI models can produce trivial but true information. They can prove a new conjecture or theorem in math that isn't a big deal one, but a small one, but something that will probably get published before. So now there's that. And then I think very soon the main goal of the AI labs is to produce actually good output. They want want AI to do real science. So we might actually start getting a flood of not trivial but true and real research. And as you said, leaving aside AI content that's created for nefarious reasons, we don't have a way of dealing with that. AI does a Pretty good job peer reviewing, but we still want humans involved. So is it some sort of flagging process where you have a series of AI reading each paper that comes in and flagging the most interesting for a human to skim? I don't know. But these things don't get invented automatically. And I don't see the systems, the guardians of our systems, thinking hard enough about how to change this. Going back to your example of meta trying to review too many applications. Unless there's some effort to rebuild what the job application process looks like, it's going to be an arms race of AI against AI. One of the things that the breaking act pace of AI development doesn't let us do as much is sit and think a little bit more about what structures we want to do. Even think about journalism, right? Let's say extrapolate where they are. AI could do a pretty good journal, like probably become a pretty good journalist. Not as good as the best journalist, but probably a pretty good local interest journalist, searching for information, interviewing people even, and then producing pretty well compelling and tight articles. What do we do with that? What does that look like? I don't think we have a lot of ideas. So this is going to be a series of rippling changes everywhere. We will adjust, right? I mean, assuming that we don't reach some sort of superintelligence that solves all our problems, we'll figure this out. But we're in for a fairly disruptive period.
B
It's interesting because listening to you, I feel like there are some people that consider themselves AI optimists that believe that once we pass some certain threshold, there's going to be a foom that AI is just going to sort of like take over everything. And it seems like you believe that what's coming is going to be more of a muddle. There's going to be a muddle to figure out what the jagged frontier looks like for the models that exist. There's going to be an organizational muddle as companies figure out how to use this powerful technology. And there's also, I like what you said at the end. There's going to be almost like a civic or like interpersonal or a very, very personal muddle to decide, well, now that we have AI that can essentially do the job of the local reporter in Atlanta, Georgia, do I want to read a newsletter by an AI? Do I want to be the kind of news organization that rather than hire human journalists, instead hires AI? If there's music that can be made by AI artists that is technically, according to some Blind test as good as Kendrick Lamar or Taylor Swift. Do people actually want to listen to AI music? There's going to be this really interesting, I think, very long period of trying to reconcile with the fact that we're building a technology that is as good as us at many tasks without understanding how to incorporate those tasks into organizations or how to deal with the fact that we've just been sort of equaled or even vested in certain fields where we're much more familiar with dealing with humans rather than robots.
D
Yeah, I mean, people always flip this as a disruption that is about robots taking our jobs. But the other question is, if the AI is better diagnosing you than a doctor and is accessible to you via your phone, what do you do about that? Right. Like we might want doctors to do that. Well, people individually do this. I think revealed preferences is that a billion people around the world are using AI and they might rail against it, but they might also use it. I mean, it's a general purpose technology. It is going to influence every part of our culture and society all at once. And if you study other general purpose technologies, steam power, electrification, they're messes when they come out, Right. All of society reorients itself around this. You know, when you think of steam power, we think about like steam engines, but you don't think about like urbanization happened as a result of that. Right. Like the nature of warfare changed overnight as a result of this. Nation states got born as a result of this. Like many things are downstream from technology plus society and we're going to have a lot of weird stuff, some good, some bad, all happen at once in the very near future.
B
I'm glad you see it as a mess because you follow this stuff closer than I do. But someone recently asked me, you know, what do I think will happen if in 2027, 2028, Claude Anthropic OpenAI announces that they essentially have built what they're calling AGI. Like, what will that look like? And I said, look, if you think this is the next industrial revolution, just know that every other industrial revolution went through a period that was objectively terrible. Right. As you said that steam power in the early 1800s, urbanization was such a mess that I remember reporting this. In the 1820s, the average age of a male laborer in Manchester fell to 29 years old because the conditions were just so decrepit and diseased and full of literal crap. In the late 19th century, with the second industrial revolution, I mean, you had unbelievable bubbles and busts and union conflict and wrenching changes to American life. Despite the fact that electrification and the railroad and telegraph are changing everything. It would be crazy to think that this is going to be a similar industrial revolution. But also no mess. That sounds like the eschaton. That sounds like a Christian rapture. It does not sound like the introduction of a general purpose technology to a human economy. When that happens, we tend to get a mess. So, Ethan Malik, this was absolutely fascinating. Thank you so much.
D
Thanks for having me.
B
Thank you for listening. Plain English is reported and hosted by me, Derek Thompson. This episode was produced by Devin Beraldi and Devin Rinaldo. If you like what we're doing here, please rate and subscribe. New episodes drop every Tuesday and Friday.
A
Limu Emo and Doug Here we have.
D
The Limu Emu in its natural habitat.
A
Helping people customize their car insurance and save hundreds with Liberty Mutual.
D
Fascinating.
A
It's accompanied by his natural ally, Doug.
B
Uh, Limu is that guy with the binoculars watching us.
D
Cut the camera.
A
They see us. Only pay for what you need@liberty mutual.com Liberty Liberty.
B
Liberty. Liberty Savings Ferry Unwritten by Liberty Mutual Insurance Company and affiliates. Excludes Massachusetts.
Episode: What Happens When AI Learns to Do Our Jobs
Date: October 24, 2025
Guest: Ethan Mollick, Professor of Management at Wharton, AI researcher and author
This episode of Plain English explores the evolving capabilities of AI — especially agentic AI — and what happens when artificial intelligence begins to perform not just tasks, but entire jobs previously done by humans. Derek Thompson sits down with Ethan Mollick to dissect the “jagged frontier” of AI’s abilities, the organizational and social ripple effects, and what the near-term future might look like as AI tools become more competent and autonomous in white-collar domains. They discuss practical advice for workers, skill atrophy, disruptions in scientific publishing, and the philosophical implications of using "alien intelligence" for human ends.
“If AI is so damn smart, why is it so dumb?”
– Derek Thompson, voicing the public’s confusion about AI's uneven capabilities. [08:41]
“An agent is really just: I want to assign an AI a task, it does it on its own without me having to intervene.”
– Ethan Mollick [15:22]
“Ten times more PowerPoints is not going to make a company more successful… unless your job is making PowerPoints for some reason.”
– Ethan Mollick, on productivity gains vs. meaningful work. [28:48]
“There is no easy instruction manual. The right thing to do is to use AI for everything you possibly can for about 10 hours… you’ll start to see the shape of the jagged frontier.”
– Ethan Mollick [33:20]
“If the only thing you want to do is produce a thing, then you can be bitter-lessened… the AI might just be able to make the thing better.”
– Ethan Mollick referencing the “bitter lesson” from AI research, where human-crafted systems get outperformed by brute-force machine-learned approaches. [42:00]
Skill Atrophy
Process Disruption
AI as “Alien Intelligence”
The Bitter Lesson
The Coming “Muddle” Rather Than a Sudden Revolution
Historical Perspective
Recommended Segment:
“I think a lot of things are going to break before they get reconstructed. And I wouldn't be surprised if we see lots of processes that we have to rebuild from the ground up to think about what it means to be in a world of AI.” [44:55] – Ethan Mollick
For listeners and readers alike, this episode offers a clear-eyed, practical, and philosophical look at the changes AI is bringing — and the unpredictable shape of the workplace to come.