← Solutions with Henry Blodget
Solutions with Henry Blodget
The Jagged Frontier: How AI Will Transform Your Job, According to Wharton's Ethan Mollick
00:45:54
Loading summary
Transcript88 lines
- [00:01]
A
This episode is brought to you by State Farm. Listening to this podcast Smart move Being financially savvy Smart move Another smart move Having State Farm help you create a competitive price when you choose to bundle home and auto bundling. Just another way to save with a personal price plan like a good neighbor, State Farm is there. Prices are based on rating plans that vary by state. Coverage options are selected by the customer. Availability, amount of discounts, and savings and eligibility vary by state. AI is incredible. It can teach you how to fry an egg and even write a poem pirate style, but it knows nothing about your work. Slackbot is different. It doesn't just know the facts, it knows your schedule. It can turn a brainstorm into a brief. And it doesn't need to be taught. Because Slackbot isn't just another AI. It's AI that knows your work as
- [00:55]
B
well as you do.
- [00:56]
A
Visit slack.com meetslackbot to learn more.
- [01:00]
B
AI has a jagged frontier, and this jagged frontier means AI can't do everything, but it transforms parts of work, so bottlenecks change. So your value of your job kind of depends on how much of a bottleneck you are.
- [01:13]
A
The fire hose of AI news can make it difficult to separate hype and hope from reality. So we thought it would be helpful to talk to one of the world's leading experts in what's actually happening. This expert, helpfully, is not selling anything. Ethan Mollick is a professor of management at Wharton. He's also the author of popular Substack One Useful Thing, which is dedicated to tracking the development of AI. Professor Malick says AI tools are advancing even faster than he expected when he started following the industry closely a few years ago. We discuss how they have already radically transformed his classroom and what his students can do. His students are not just encouraged, but required to use AI. Here's our conversation. Professor Malik, great to see you just speaking as a consumer of your substack One Useful Thing. Thank you for writing that. It's so great to have somebody sort of sitting between the Silicon Valley hype machine and the audience who does not have time to look at every new thing, knowing that you will look at it and tell us what it means. So thank you.
- [02:18]
B
Well, that's really nice of you and I'm glad to be here.
- [02:21]
A
And folks who don't know you, they may know you through your otter test, which has also been very fun to follow over the years. Tell us what that is because it's gotten quite impressive recently.
- [02:31]
B
Yeah. So I, you know, there are all kinds of benchmarks for AI, Right. And all of them are flawed, so you might as well pick one you like to some degree. So I have been for a long time. My daughter is a huge fan of otters on an airplane. At one point I was sending her AI generated images of otters sitting on airplanes using wi fi. Cause the WI fi was going in and out. I posted them, it was like the threads went mega viral. And ever since then I've been testing AI's image generation, video generation ability by asking it to create pictures of otters on airplanes. And it turns out that now we've gotten in this phase where not only are they really good, but you could generate an entire movie about it from a prompt, which is, I think, pretty impressive.
- [03:06]
A
Yes, it's extraordinary. They get ever more lifelike and funny and so forth. So it's amazing. All right, so you recently stepped back in your substack and basically said, okay, the future is now emerging. It was visible a couple of years ago, but now it's actually happening. So where are we?
- [03:24]
B
So you kind of think of AI as having three phases, right? There is the pre large language model AI, which was big data, and that was a huge business. Right. A lot of the huge investments through the 2000 and tens, early 2000 and twenties was based on if we got our data organized enough, we had the right data scientists, we could do really great predictions of the future numerically. So we could choose how to price things to people where demand might go, all kinds of things like that. Then with ChatGPT, we sort of entered what I'll call cointelligence from my own books, but you could call it other things as well. And that was a phase where you'd use chatbots, so you'd work back and forth with a chatbot to make things happen. Right. So I'd say, hey, write me this piece of code or write me this email. That wasn't quite good enough. Can you make it better? And that was the way we sort of work with AI. What's happened in the last few months is we've gotten in the stage of having practical AI agents, which has been a marketing term for a long time, but now is a reality. And an agent is an AI tool that can be given a goal and can accomplish that goal autonomously using its own tools, often many hours worth of human work it can do at one time. And that is a real transformation because agents start to mean, I can delegate tasks to AI and have it do it. I can expand what I can do because the AI just does the Work, rather than me being in the loop at every moment. The AI goes out and does stuff and then reports back to me afterwards. And that has created this really big step change. We saw it first in coding. Coders are getting very nervous or excited, depending on who they are. But we're going to see that in other fields as well.
- [04:42]
A
So give us some examples of that. A lot of it seems very complicated. There are a lot of deep tech folks who are doing it, but everybody else is sort of watching from the sidelines. So when you talk about that actually happening, what are you seeing?
- [04:54]
B
I think among software developers it spread very quickly from like a deep tech weirdo thing to like fairly mainstream. Right. We have evidence from the previous generation of the early agents that came out a year and a half ago. They were increasing coding ability in randomized trials by something like 38%. So like that was already a huge leap. What's happened now with coding agents is you basically can code in plain English. You can tell them, I want to build a piece of software that does this. And if you're expert enough about coding that you can look at the results and give feedback. A lot of the best programmers are barely touching code anymore. And that's not just a weird high tech sort of San Francisco thing. I see that kind of happening everywhere. We could start to see signs of other forms of AI kind of, you've heard of openclaw, all these other approaches that, you know, those are much more nascent and weird, but like the coding stuff is pretty normal at this point.
- [05:41]
A
And you talked about one company that has created something I think they call the software factory, where it's basically humans cannot either create or evaluate the code, right?
- [05:52]
B
So if you think about software as this sort of early signal, right, the harbinger of what's going to come next, you're also going to start to see a few interesting issues. One of them is software development has basically operated a very similar kinds of. There's sort of two patterns. There's a waterfall development where you have a big plan and do step by step work. And then there's been agile development, which is what most companies do, where you have, you know, short sprints of creating software one time after another. And it's all built around these kind of human meetings, right? You have design meetings, you have stand up meetings. Suddenly when people can create 100 times more code than the AI can code, those old methods of working that worked really well for humans don't work anymore. So we're starting to see radical experimentation with New approaches to using to developing software. One of these is a company, Strong DM that developed what they call the software dark factory. And much like a dark factory is an entirely robot run factory so you don't need lights on it. This is a software dark factory. So you just put into the software factory roadmaps of what you want to accomplish over time. And the AIs actually generate the code. Then they have fake customers software that's built to test the code and the customer agents have built their own fake versions of Slack and Gmail so they can actually test it in a real environment. And then in the end, what you get out of this whole thing is just software. And no human ever sees the software, no human ever tests the software. It just, you know, ideas go in one end, software comes out the other.
- [07:08]
A
And do you need to be, at this day and age a great engineer to do that, to oversee it, to be able to specify the product clearly enough for the AI system to do that?
- [07:20]
B
So I think those are three different questions actually. And the difference between them is important, right? Because with software development, probably your ability to write just good boilerplate code, which is an important skill, has devalued your ability to be a good manager of people who write software is increasing because the AI kind of works like an employee doing work for you. So there is a shift in what is important to be a good software developer that is sort of unexpected and fast. I think we'll see that across many fields as what makes work hard changes from can you write enough lines of code each day? To are you good at envisioning what a product requirement document PRD looks like? Are you good at giving feedback and building test steps? Those are very different questions.
- [08:00]
A
We'll talk more about impact on jobs and what we're all supposed to do if our skills, skill or primary skill is now done better by AI and much faster. But just in that specific example, I know from having run a team of engineers, the jump from being an actual line coder developer to being a manager, it's tough. It's tough in everything. It's tough in journalism, going from a writer to an editor and at universities, a professor to a dean and so forth, is that something do suddenly that everybody has to instead of learn to code, actually learn to just manage big software projects by having others do the work.
- [08:41]
B
I mean, a few things. One is that's probably also a temporary state, right? Like I think people are over indexing on where AI is right now compared to where it was a little while ago. And the truth is, is that we're still in early days for a lot of this kind of stuff. Right? Like right now, that's the answer. But of course, part of what makes becoming dean hard or becoming an editor hard is you're not just managing, you know, output, you're managing people. So it's to be easier when you're delegating tasks, but don't have to deal with all the other things around delegation that make delegation hard. Right. Can you give good feedback to a human and can you make them happy? And can you get a team of people going? Can you run a good meeting? So it's weirdly parts of that managerial tasks that go to your level and parts don't. And again, it's a moving target in a lot of ways.
- [09:23]
A
And another recent example of the incredible productivity improvements that you've talked about is teaching your class on entrepreneurship, where you basically, I gather, ask your students to, okay, create a startup and use the tools to do that. And it sounds like it's changed radically in recent iterations versus a few years ago.
- [09:43]
B
Yeah, as academics, we think it's semesters, right? So I've gone from like, you know, AI didn't come out. So it was a semester's worth of work to produce. You know, you'd come with a startup idea that was like a month in, right? And, you know, you'd pitch a PowerPoint presentation, you've spoken to a couple customers. That was the expectation four years ago at the end of a. At the end of a semester's worth of work with very smart students. And then AI came out and I introduced my class immediately and it became, oh, you can get some working software out of this you could do. But, you know, it was a co intelligence piece. So I expected more work, but better quality. My most recent version of this vibe, founding people had three days to launch a startup company. It was really two days. Like a day was teaching tools and they had to have a working product and they had to have a PowerPoint and they had to have simulated customers. And, you know, I've been doing this for a long time and I saw incredible outputs over. You know, people did have working product or at least demo working products. At the end of a day and a half of work, that changes how we start to think about how even entrepreneurship operates. Like, should you try 50 companies at a time and see what works or not? What does it mean that what was holding you back was some weak spot? Like you weren't good at finance, but the AI is good enough at finance for you to advance what you do.
- [10:45]
A
And what's the reaction of the students when you have them do that and you say, and you have to use these tools because it's got to be much faster.
- [10:52]
B
So I mean, generally I'm lucky enough to teach an entrepreneurship course, right. So because I do that, it's pretty straightforward because people want to launch a company, so anything I give them doesn't feel deskilling, it feels like an advantage. Right. Because what they want to do is launch their organization and entrepreneurship anyway. And you know, this having been through it is really about being good at like one, like being world class at one thing and hoping the 99% of other stuff you're not good at doesn't kill you.
- [11:14]
A
Right.
- [11:15]
B
So the fact that the entry like it's perfect for AI stuff because they're one specialty, they're definitely better than the AI, their background and experience. In fact, I made them all do what I called the Voight Kampf test after the test in Blade Runner where I had them write down what their unique skills were and what their unique experience was and their startups had to include that. So I felt like that, you know, the reaction was very, very positive. When I've talked to other groups, it can be very negative or very disturbed. And I think there's reason to be ner, you know, feel disturbed and confused by where things are right now.
- [11:44]
A
Yeah, I think you've said that students should learn to use AI and use it in school.
- [11:49]
B
So with a caveat, it turns out if students just use AI and just let the AI do the work for them, then they don't learn anything. Right. Like, and they could be fooled by that. They could say, help me solve this problem. And the AI explains how to solve the problem. They're like, I learned how to solve the problem, but if there's no friction, if there's no difficulty, you don't learn. So my classes are 100% AI based and the AI tools they use and they interact with AI systems, like that's great. And you know, AI mentors, AI simulations, they build a case with the AI and you know, like all kinds of cool stuff you can do. But it does have to be deliberate, Right. It can't just be use AI and everyone learns more because they don't.
- [12:21]
A
And how do you. So, so how do we deal with that? Exactly. So I used to be research analysts a long time ago. You study a new industry, you'd talk to everybody in the industry, you develop theses, you'd come back you'd write your report. It would take months. I had ChatGPT do a market analysis for me. It comes out in six minutes. Is it the best thing ever? No. Is it way better than I probably would have produced in my early years? And it did take six minutes versus months. Absolutely. But when I look back on that, I think, yeah, but those two months, that's when I learned and that's when I came up. So it's straight to your point. So how do we, given that the technology is so much better and faster than we are at a lot of these tasks, how do we also manage to learn?
- [13:12]
B
That is a big question. I'm not worried about classrooms. Things are screwed up right now. But we'll figure it out. It's already pretty obvious. We'll use AI tutors outside of class. Inside of class will be activities and engaged interactions. We'll be fine. We'll figure this out. And people still want to. We still want to test people, just like we figured out with calculators. I am deeply worried about the next thing you talked about. I teach people to be generalists at Wharton, and I send them out into the world and become specialists the way they've always been specialists. In your case, they work for a bank and they write analyst reports, and they get yelled at by their boss if they're not good. And they're learning a lot about the world beyond just the report they're writing, because they're talking to people, they're at conferences, they see how people react, and they learn to be experts. The same way we've taught experts for 4,000 years was apprenticeship. And that model's the most threatened. Because if you're an intern or you're a junior employee, your goal is to impress your boss. Right? Even if you want to do the work yourself, you'd be dumb to not have AI do the work for you because it's better than you as an intern. And all your bosses would rather not assign work to you. Why have you do the research report when I have the AI do 20 of them? I can check them and give feedback to the AI. And once it's running, that machine is much better than working with students or junior people who have emotions and feelings and may or may not be good at a task and sometimes get sick. So it breaks the entire apprenticeship pipeline. So what are we going to do to reconstruct that? I think we have to be deliberate about how we learn at work and outside of it. Maybe it's longer university courses where you learn to be A specialist at school. Maybe it's that you have to start building, building official mentorship into what you do and having people not use AI to solve problems. Maybe we have to test people, we don't have answers. But I'm really worried about that pipeline.
- [14:40]
A
And so you're professor of innovation entrepreneurship. Have you changed? Like, have you created a professor Molochbot yet that your students can talk to 24 hours a day?
- [14:50]
B
But I don't think that's. I think that's. That's the easy solution, right?
- [14:53]
A
Have you done that?
- [14:53]
B
I don't think I want a fake professor Moloch bot. I want. I want to. What I have built for them, you know, in various classes is. There is like, you want to think about what they want to learn. Talking to professor Molikbot is not learning thing. Right. They're just getting fake information from a fake version of me instead. I want systems built for tutoring very hard concepts. So we've been building tools and releasing them open source that do things that like, for example, build simulations. So you talk to the AI and you give it your syllabus and it creates a simulation for you. Right. Or that act as a devil's advocate that you have to argue with or that help you with a team. So I think we'd be more imaginative than Professor Malik Bhatt teaches a class. Right. I think it has to be beyond that kind of piece. I mean, people. You know, one of the weird things about having a very narrow sort of fame is I occasionally run into people who are like, oh, I. I built an Ethan bot and I talk to it all the time about, you know, advice. I'm like, all right, I'm not quite sure what to do with that information. But I don't think, I don't think we have to replicate person to do this well. Right. I think we have to think about what, what makes learning work.
- [15:55]
A
Right. And in that there has been a dream of folks who do hope to bring technology to education, that we can have truly customized education for each one of us. There is that image. I'm sure you saw the original new Star Trek where on Vulcan, all the young Vulcans are educated in their own little bowls that have their AI teachers or what have you, and it's all individualized and so forth. Is that something that is promising where each student is going at their own?
- [16:26]
B
So my colleagues at Worthington have a randomized controlled trial they did in Taiwan where they had their students use a personalized AI tutor based on the now absolute GPT 4.0. So not even a great system. And they found equivalent of an extra few months of teaching for people who use a customized tutor outside classes. Another experiment that was done in World bank in Africa, there's been a few of these. It's very clear. AI as personalized tutor is great. And all the AI companies have a sort of half baked learn mode which I would recommend using if you're a parent, if your kids are going to use this, which asks more questions rather than being a helpful assistant that gives you answers. To me, personalized education, the one on one tutoring, I think we're going to nail that. I think people already should be using it to help women with that, as long as they're careful. This is a great place for parents to weigh in and watch what's going on. We'll figure that out. I think that people though underestimate the value of teachers and instructors in classroom settings. Which is the uneven piece, right? You don't want to be in a Vulcan like bull because a lot of the stuff we teach isn't just about do you learn the rote material? Right. It's about applications, about dealing with the classroom setting. So teachers will have to keep transforming their role to more exercises and experiences and activities. Grading people becomes something that becomes very important to do in a classroom sort of setting. So we'll see that transformation happen. I just think that again, the technology solution of if everyone has a universal tutor, it's great. I mean it reminds me of when I was at the Media Lab in the early 2000s doing work with the Media Lab when I was at mit and we all thought that the future was going to be one laptop per child. Everyone will just educate themselves with technology. It didn't work out that way. Wikipedia did not transform education the way we, I think thought it would do if we were at the early days of the Internet revolution. So I think we do need to be a little deliberate about thinking through what we actually want to teach and how.
- [18:06]
A
So how have you changed your classroom? How are your classes different?
- [18:10]
B
I mean they're 100% different, right? Like literally there is not a single piece of the classroom experience that doesn't have AI included in it. So final projects, the AI actually interview, like turns out like, so we start with pedagogical principles. So it turns out reflection is really important in education. If you don't stop and reflect, then you actually don't learn very much. So final project actually is an AI that interviews you about your experiences and tries to pull out reflections and that's great because you can't really fake it with another AI, right? It's based on your class experience in that kind of way. Simulations are a big part of they have to actually engage in negotiation with the AI and give me how the simulation worked out, right? They're asked to build AI tools and of course they use AI to create and build things themselves that are powerful. So like I'm using every dial, but I'm using it based on pedagogy, right? So having explaining something to someone else is really important. So let's build that in. Having to deal with the skeptical audience is important. Let's build that in. Having a simulation where you have to put things into practice is important. Let's build that in. Reflection's important. So building on the building blocks of good education, you can build some really cool things. Zootopia 2 has come home to Disney Plus. Let's go get ready for a new case. We're the greatest partners of all time.
- [19:23]
A
New friends Gary the Snake and your
- [19:25]
B
last name the Snake Dream Team New
- [19:27]
A
Habitats Zootopia has a secret reptile population.
- [19:30]
B
You can watch the record breaking phenomenon phenomenon at home.
- [19:33]
A
Zootopia 2 now available on Disney Plus. Rated PG. And right now you can get Disney plus and Hulu for just $4.99 a month for three months with a special limited time offer. Ends March 24th. After three months, plan auto renews and $12.99 a month, terms apply. The problem with radio is we can't
- [19:48]
B
show you our new box packed with
- [19:49]
A
a KFC snacker, five nuggets, fries and
- [19:52]
B
a drink for just $7.
- [19:53]
A
So you'll just have to trust us when we say the crispy golden fried breading will make your mouth water. You'll have to trust us when we
- [20:00]
B
say this is a ridiculous amount of chicken for such.
- [20:04]
A
And you'll have to really trust us when we insist that yes, it really is only $7.
- [20:09]
B
$7 box feast from KFC Trust Us.
- [20:12]
A
It's finger licking.
- [20:14]
B
Good prices and participation may vary.
- [20:15]
A
Tax tips and fees extra. The world moves fast. Your workday even faster. Pitching products, drafting reports, analyzing data. Microsoft 365 Copilot is your AI assistant for work built into Word, Excel, PowerPoint and other Microsoft 365 apps you use, helping you quickly write, analyze, create and summarize so you can cut through clutter and clear a path to your best work. Learn more@Microsoft.com M365 copilot so let's talk a little bit about jobs impact because there is still every month predictions that no humans will have jobs anymore or there will certainly all white collar workers will be wiped out. You have been very optimistic, stick seemingly on that. So talk about that.
- [21:04]
B
So I think we have some sense of a picture early on, right? So first of all, there is not really any indication of job changes through 2025. And partially that's because organizations change much more slowly than people think. Right. You can have 100 times more software being developed, but unless somebody is figuring out how to market and sell that and all the other pieces. And that's because AI has a jagged frontier. This is a term my co authors and I coined going back with the early project at Boston Consulting Group where we found that AI boosted performance in areas that AI did really well and limited performance in areas that didn't do well. Right. And so this jagged frontier means AI can't do everything, but it transforms parts of work. So work is a set of different tasks. Some parts of those will be transformed by AI faster than other parts. Like these models aren't good at everything yet. And so what we're likely to see, and I think you've seen this in coding happen already, is bottlenecks change. So your value of your job kind of depends on how much of a bottleneck you are. So if you looked at coding 2019, 2020, everybody wanted programmers, right? And so if you went to an HR conference, it would all be about how do we hire the best programmers we can. Well before sort of the AI factor hit, that started to fall off a cliff, right? Programmers weren't as much demand. What happens? The entire ecosystem adjusts, right. My students go into CS degrees less and they go more into whatever's valuable next, finance or whatever is hot. The job market adjusts around those things. I think we're going to see now we're seeing the bottleneck move from coding to engineering ability, software engineering at large, that's going to suddenly be in demand. So what we're going to see is bottlenecks change as jobs are transformed by AI. That doesn't always mean destruction. That might mean the changes in who's valued where under what circumstances. Now I think we have some vision in the next few years. I don't know what 10 years from now look like. The AI labs are very explicit that they want to replace all human labor. And if you take this sort of big picture view, industrial revolutions tend to create more jobs than they destroy. We don't know if that's always the case here, but it has been the case but living through them kind of sucks. Right. Like you read Dickens and you're not like, oh, I really wish I was there right now. And so, you know, it's great to have our grandchildren look back and say, that was, you know, thank God that happened. But it can be rougher to live through that kind of period. So I don't know what kind of disruption's ahead of us in some ways.
- [23:14]
A
And what do you tell your students given that, given the amount of change, like, what do you tell them to learn to do?
- [23:21]
B
So I think one thing to just, I tell them, which is I do academic research on careers, amongst other things. And like, careers are long. So like, I think not being flexible, being entrepreneurial, not over indexing. Right. I mean, I, you know, I've plenty of my friends lived through. You know, we're in finance during the financial collapse. Right. The global financial crisis. Like, their jobs are very different now than they were before. Many of them left the industry. They did other things. Like, I think there's some flexibility that's kind of required. It's. People don't expect to, you know, look at your own career with so many different twists and turns.
- [23:50]
A
Yeah.
- [23:51]
B
Like, I think that people have to be ready for that kind of experience. I also think that, that it is, you know, there's two kinds of knowledge that are valuable. Deep knowledge. So if you can learn something about an industry area deeply, that's valuable. Like it's still a valuable piece to know more than other people. It makes you good at working with AI systems. It makes you good at the human pieces and then wide knowledge. Understanding a lot of how things fit together gives you a sense of where things are going and changing. So I actually feel like this is the time for humanities in some ways. Understanding a broader base of human knowledge and then going very deep in something I think could help a lot. But I don't have good pictures. Like, I mean, you know, my own kids. One wants to be a lawyer, one's in college, studying to be for med school. I mean, I think these jobs will transform. But whatever you're really best at, I tend to stay. Stick with whatever you're best at and ride it out in that direction. And I guess the other advice I'd give is bundled jobs, jobs that do many different things, like doctor, where you're actually supposed to be good at many, many different things that you're not going to be good at. That's a place where AI is helpful, not harmful.
- [24:47]
A
Yeah. And I have to say, I mean, I've done Some experiments within journalism and other things. And as like you have had the experience, sometimes it is just awe inspiring what the model can do. And yet I couldn't automate it. I was not able to figure out how to automate a newsroom, for example. Like the world doesn't need more articles. It actually needs perspectives that people really understand. And I know you've written a lot about this. Like even when you can automate it, it's knowing what to automate and what to do that is valuable.
- [25:23]
B
And I think that automation versus augmentation, as much as I don't like the framing because it comes up often, I think transformation's a little different than any of those. But I think thinking about how we make humans thrive is going to be the challenge. And to me the biggest worry I have is organizations find automation the easiest thing to do. Right. How do I remove all the reporters from a newsroom as opposed to I Wish I had 100 more reporters like this. How do I take my reporters that I have and make them even better? Better, Right. And that might be getting writing or fact checking help in some cases. But you're hoping you're using this to expand what you do and not to retreat.
- [25:54]
A
Yes. And I also, I mean going back to like, where is AI today and what's it doing and what's the frontier look like? I was on LinkedIn last week. I saw a company, I think it's called pulsea. That is one guy who used to work for cloud kitchens and has created an AI company. It's just him and the AI and the product is, hey, I'll run your business while you sleep. And so I pointed this out. I'm invested in a video company with a guy I used to work with named Nicholas Carlson. I said, nicholas, you ought to check this out, maybe we can automate some stuff. So he signed up immediately. And the little counter on the website is saying that hey, 4,500 companies this week use us. And last week it was only 3500 and now we're up to 5 million of run rate revenue. So he signed up. Within a minute it pulsia had tweeted, hey, I have a new client. It's a video company, it's great to work with them. And I'm automating this and I'm automating that. And then it had posted a website about the company. You know, very interesting. It's like describing the company it learned. Now it's going to dive into competitive analysis and all this other stuff. It's like, my first question was like, hey, did we say that it was okay to publish that we're a client and to create a website for us? It's like, no, we didn't. So you ask it. Okay, can we take those down? We didn't, we didn't approve that. It's like, well, yeah, I'll get to it. So on the one hand, it's like, it's so human. Like the bias to action. You know, you've all had that where you have a new employee, you give them a thing, and suddenly all this stuff has happened that you would have wanted to weigh in on. And it's like, well, but nobody wants to be nagged. And I, you know, I'll take down the website on my time. So anyway, long way to go in completely automating away everything.
- [27:44]
B
That's right. And that's the tag of frontier work. Now I'm skeptical about a one person automated company. I think you could probably do some amazing things with seven really smart people. Right. Using the various pieces. But I want a human in the loop for major decisions at this point because the models aren't good enough. I live with these things. On one hand, they're absolutely brilliant. They can find errors in my papers that a Nobel Prize winner did not find. They could identify issues that are brilliant stuff. Right, right. And obviously I'm coding all the time. I'm doing. You know, I've had AIs launch businesses for me and ship products to customers without me ever touching anything. As an experiment, it worked amazingly. I had to build, write a book, write a whole book thing and publish it. I mean, it's great, but there also are jagged edges. And this is kind of that going back to that big issue you raised before. Experienced people know the jagged edge right away. Right? Because you can kind of see this and you're like, oh, yeah, this is good. But like, this feels like an intern doing this. Or they're missing this important step and they think they're doing. But if you're an intern using AI, the problem is everything looks great. Like, one of the most depressing things I heard was a Chro told me, oh, the kids these days that we hired this last, you know, in the summer were great. They were all like digital natives, AI natives. And I'm like, and they're giving me great work. I'm like, they were not giving you great work. You were just talking to Claude. Like, all they were doing was taking anything you did pasting into Claude and giving you the results or chatgpt. And giving you the results they aren't. There's no digital nativeness because they don't understand your business. So they're giving you a great memo that would blow you away from an intern, but also isn't useful for a senior person.
- [29:09]
A
Exactly. And so how do you. How do the best managers combat that? What are you having interns do? This goes back to how we deal with not actually learning by doing anymore.
- [29:18]
B
So I. So we don't have. I mean, I know some people starting to think through this stuff, but it's very early days. I mean, a lot of it is trying to ban AI content. Like this is the general problem that also we have, I think, which is just, you know, work slop. Right. Like a lot of the AI tools out there just optimize Brutus work slop. Oh, you want 50 power points? Great. Here's 100 PowerPoint points. Here's a thousand PowerPoints. Nobody wants a thousand PowerPoints. But that means somebody in the organization has to sit down and say, what do I actually want? What was the goal of the PowerPoint? Was there a goal? Was the PowerPoint goal that we have an argument with each other about it? Is it a way of compressing information? I think about your analyst reports too, right? Going back to that. Is it the idea that does the analyst report matter or is it a signal to your clients that they should call you so you can have a deeper conversation and understand them? A lot of those symbolic pieces are going to float away. And I think, think if you start by focusing on what the real intellectual work is, you can bring someone along in that journey as an intern, right? Which is like, I want you to write something and if you turn to AI to do it, it's going to be less bad because we're going to sit in a room after, I'm going to grill you so I know the world better. But that changes how you do the work. Now you're a teacher rather than just a manager, and a lot of people aren't ready for that.
- [30:21]
A
No, that's exactly right. And that's exactly where I came out when I did the AI journalist newsroom is that the world does not need more articles. It just doesn't. I open my six publications every day. I read one or two of 20 articles I would love to read if I had time. Like, there's just no time. And so the value is something else. And it's the same way with analyst reports. Like, I remember one of the very stark moments I had as an analyst was going into somebody's office. Big investment manager. Oh, I sent out a report last week. Did you read it? He's like, yeah, it's probably in there. And he points behind him and there's a stack, stack literally 4ft high of all the reports he's gotten in the last month. It's like, maybe it's in there. Oh, okay. So my value is not actually producing analyst reports. That doesn't have anything to do with it. So yeah it is, it's knowing what to do and, and so forth. And so I like how, what's that? Like how do you, what are the smart managers doing? And I'll give you another thing you've written about recently is how when you give an open ended creative task to Claude or ChatGPT, you can do a great job. It's when it's a very specific thing that you're looking for, where it struggles and that's where actually a good manager comes in. So talk about that.
- [31:39]
B
Yeah, I mean, I think so we're moving to this sort of management world with the AI piece which is like it can execute a fairly good close ended task, but then you have to think about it like an actual close ended task. Right. If I was writing an RFP or PRD or you have any field like a marketing brief, like a standard operating procedure, you know, the army does five paragraph orders, like they're all the same kind of thing, which is how do I commit my intent to someone else so that they execute on a task. And that is a skill in itself that's required in a deep way. Right. Like the information gathering is a piece of that. Right. And by the way, they're very good at reading a lot of analyst reports. And you know, but like you want to figure out, do I want to report to me about where the sections of the analyst report were that it read, that it found were interesting. Do I want it to have, you know, do I like, what's the intermediate step? Do I want to have a second AI review these results and push back work? Design becomes a really big deal. Making sure that we have the right steps that carry us through the process becomes a big deal. Where are the tests? Where do I want input? Like these are hard problems to solve that are very management problems. Right. And so it's not automation, it is where do I, you know, it's almost like thinking about outsourcing to an organization, to another company. Like they're going to be like anytime anyone outsources for the first time, they think it's going to Be great, because someone's going to do all the work and then they realize like, oh, no, actually this person isn't good at all of these things that I thought they were, and I have to give them more instruction and my job shifts to an instructional one. I think that's. That becomes the challenge, which is an interesting one. Right? Like, I think that a newsroom, you don't want an automated newsroom, but you may want six really good editors. Having AIs do first passes at things to see whether or not they're interesting or not before assigning someone good into it. I'd want every journalist I talk to to run a deep research report before we have a conversation. But then again, that gives you 40 pages. Anyone going to read all that? So maybe you turn into a movie. Like, we don't know yet. Because suddenly we have all of these new affordances, all these new ways of working with information, and we don't have an answer to which ones work best yet. Yet.
- [33:31]
A
And we see that with humans delegating work all the time. As you're saying, like, some people are really good at it. They're really good at specifying exactly what they want and what the evaluation criteria will be. Others are not good at it at all. And that's a management skill that you have to learn over time. So is that a big part of your own learning to use AI effectively has been like, actually, how do I manage this helper?
- [33:57]
B
Yeah, I mean, and it's an evolving process, right? That is absolutely, like a huge amount of this comes. So you have to kind of do two things at once. Not be overawed, but also be overawed. And what I mean by that is, like, you have to realize these technologies are limited. They can do only certain things, but they can actually do kind of impossible things. So what I worry about a lot when I talk to companies is that they try and make this technology normal, right? So their desperate goal is like, how do I put this in it? And uses a fuzzy logic processor or natural language processing or, you know, fit into an existing workflow or how do I add automation so that before your analysts report you, it looks up all the articles for you and then gives you a summary of them. Like, that's a failure of imagination in some ways, because you could ask the AI. You know, the story I was just telling you a little bit earlier was, and I think I have a copy of it here. But like, I've always been kind of obsessed with this idea of what if? I mean, always being a Few years of what if you made LLMs physical? So what if we printed out all the physical weights for the LLM like in books? You could in theory do all the math that a large language model does. It would just take you about a year to calculate a single sentence or, you know, part of a sentence from the AI systems. And I think about that and I was like, you know, it'd be kind of fun to do that. So recently Claude Code got good enough that I just asked it actually. And here's the book I actually asked it, could you just do this? And it created an 80 volume set of all the weights of GPT1, which I. And it formatted it beautifully, right? And made little covers that are actually visualizations of the data inside. It made a website, it hooked up to Stripe and to one of the instant on demand presses. And so I sold a 20 edition print run of volume one of this, this that I didn't even know if it would arrive. People got it and then I sold out. I'm like, I'm done selling this. But like I didn't have to touch anything, right? The AI did all of it, including the beautiful formatting, right? And like I was like, well, maybe the text should be lighter. But I didn't even bother intervening. So the idea is you should aim for both impossible tasks, but realize that there's going to be limits when you find those.
- [35:47]
A
And so, I mean, tell us more about that. So were you providing things like credit cards to buy accounts and was that scary? Because we all hear the horror stories already.
- [35:57]
B
I absolutely gave, I gave it. I had to do two, only two things during that process. It told me, I didn't tell it how to do this. It said, okay, great, you need to log into your Stripe account, which I use for other things and you have to give me the API key. I'm like, all right. And then you have to log into, you have to create an API and here's how to do it. API connection to, I think it was Lulu Press and here's how to do that and I'll handle everything else. And it already had my Netify account and that's why I did a limited run of 20 and I was spending a lot of time checking the dashboards to see if anything went wrong. But that's part of what. But then you think about risk assessment. Like, okay, so I put hard limits on the. I went back and later on put hard limits on what credit card it could use and how it could spend on that. So I won't get into trouble. I took down the website after a day, but also if I delegated that to a person, I'd be nervous. Right? So there is a little bit of like, you should be anxious. You can get into trouble. I mean, I have my open claw instance running on this computer over here, and it makes me very nervous to use for anything serious. There is risk in this, and I think balancing that risk and reward is important.
- [36:59]
A
Foreign. It's not just something you made. It's the privilege that you get to work with your hands. It's building something that serves a purpose, proof that you have the grit to keep going. At Timberland, we understand you take your craft seriously, and we do too, which is why our products are built to the highest quality. We put in the work so you can perfect yours with perfect purpose, in every detail, and crafted with intention. Timberland built on craft. Visit timberland.com to shop.
- [37:34]
B
Get in the game with the college
- [37:35]
A
branded Venmo debit card.
- [37:38]
B
Wreck your team with every tap and earn up to 5% cash back with Venmo Stash, a new rewards program from Venmo. No monthly fee, no minimum balance. Just school pride and spending power. Get in the game and sign up
- [37:51]
A
for the Venmo debit card@venmo.com
- [37:54]
B
the Venmo MasterCard is issued by the Bancorp Bank NA Select Schools available Venmo stash terms and exclusions apply at venmo.me stash terms max $100 cash back per month. Going outside is so in during spring fest at Lowe's for a limited time, get extra big deals on select Holland Pavers. Three for $1 plus save $70 on a char broil performance four burner grill now $179. And chef up shareables for your whole crew. Picture perfect patios and good food. Yes, please. Our best lineup is here at Lowe's. Valid through 3:30 while supplies last. Selection varies by location. Paver offer excludes Alaska and Hawaii.
- [38:34]
A
And so you've been watching this for since the beginning. Have we progressed faster than you thought?
- [38:41]
B
Yes. I mean, I've known we're on an exponential for a while. It's hard to feel an exponential, especially in the kind of early days. I think we're further. I mean, the models are better than I thought. I wouldn't have guessed two years ago that we'd have agents that could revolutionize coding by now. Like, I think we're off in the timelines a little bit. Like, not so much that I'm like, I had no anticipation of it. I think I've been pretty good at my predictions, but I think that I thought there'd be more options for slowdowns.
- [39:05]
A
And so where do you think we're going? What does exponential actually look like? Help us understand.
- [39:09]
B
It's a really hard question. It is a really hard question to have a clear vision of what that future looks like. I think jaggedness gets like. So there's the two sides of this. On one side, the AI labs think we'll have AGI and machines smarter than human at every intellectual task in the next three years. They keep talking about it. They think it's sooner than that. You know, there's a question about whether or not we already have AGI. Are the current systems good enough? We're just lacking the tools, the harnesses, the equipment that makes them work well. Just like the AI models that are really good at coding could be really good at design or journalism if they had the right kind of tools packed in with them. So maybe we're really in a place where that can happen over several years as people think about how to make these systems operate, but they want to replace all humanity work. I think AI is more jagged than people think. It's harder. There are lots of rough edges where it interacts with the world that are complicated. And I think that change will be slower than we think in some ways. But I do think people aren't ambitious enough about what these systems can do. And I think the world will be very different in a few years in hard to predict ways.
- [40:03]
A
And one of the things that folks who are optimistic about the job impact, and let me say I'm optimistic, but I also think there will be a lot of disruption and pain and a lot of people will have to learn to do new things and so forth, which is never a painless process at all. One of the things we say is every technology to date has created way more jobs than it has destroyed. And are you seeing any evidence, like what kinds of new jobs are being created here?
- [40:33]
B
The problem is because of general purpose technology that does everything and has general effects. I don't think we're seeing the same kind of thing as like steam engines come out and you have engineers whose job it is to attach a steam engine to maintain the steam engine engine. It's very different. What's happening is the marketing job changes, right? So the job is a different job. The coding job is a different job. You're now managing agents doing stuff that hasn't changed the title to make coding agent manager. The job of the future. Right. It's not like, you know, if you look at sort of 1970s future handbooks that would be like, oh, you know, nuclear engineers will be the job and robot psychologists will be the job. Instead the job is you're a marketer, but what you do is different than what you did as a marketer before. And in some cases that will expand the number of people you need. There's a very plausible argument that we'll need more coders than ever before because suddenly you have the ability to like solve a lot of problems. Like you probably could automate hundreds of newsroom tasks and build code that you always wanted to build, but you would never hire anyone to do because it's too expensive. But if one coder can generate a program for you every two days that solves a problem, suddenly you want to have a coder in your office that you didn't before. So maybe there's more of those jobs. It's still called coder, but it's not what a coder was doing when you need to be part of a team of 50 people following an agile development process than a two week sprint cycle.
- [41:45]
A
All right, so last question or topic. So you mentioned AGI. It seemed to me that a few years ago we were hearing a lot more about that and that one of the justifications for this just complete arms race is to get to the RSI recursive self improvement moment faster than the rest of the world. It seems to be what all the model builders are doing. Um, is that still likely to be a moment that changes everything? And I know AGI is different from RSI and, and tell. Talk to us about both of those.
- [42:19]
B
I mean, so we are seeing. So all the, the three big AI labs have all, you know, Google and OpenAI and Anthropic have all reported some form of recursive self improvement. Right? To the extent they reported that they're using models to build models. We know that you're building products like there's a reason why coworkers shipping every week with new. Like it is being built by AI, right. And, and it has some of the rough edges of something being built by AI and being shipped every week, but they are using the tools to build the tools at this point. Is that enough to create some sort of accelerating process where the tools can make the tools better and the next generation that we haven't seen yet? We haven't seen that sort of like, okay, significant day versus night kind of changeover. But they report self improvement. That's where they and if you listen to the labs, they haven't changed their timelines on when they. I'm not seeing them adjust their view about when they think the disruption is going to happen.
- [43:11]
A
And so when we get there, what does the world look like then? And are you worried at all? I mean, there are some voices, not just cranks, saying, like, look, we are creating entities that are going to be smarter and more capable than we are. And show me an example of any natural system where the dumber, less capable entities aren't ruled by the smarter and more capable ones. Are you, Are you at all worried about any of this?
- [43:41]
B
I mean, I think there's a lot of smart people worried. So I'm worried, right? Like, I have difficulty putting like a marker down on how worried I am. I don't think we're seeing regulation in an effective way that's going to stop any of this thing from happening anyway. And to me, a lot of what I spend my time worrying about is more the intermediate steps that I feel get overlooked. Like, how are we going to constitute work in the right kind of way around this? There's a whole bunch of immediate problems, right? Like every social media site is collapsing under the weight of AI bots that all use that can't be filtered out easily. Obviously, education is disrupted, there's deep fakes and there's involuntary images of creative people. There's a lot of problems that need to be regulated and mitigated that I really worry about more because we're not doing things around those things, right?
- [44:23]
A
And is there regulation that would be effective?
- [44:26]
B
I think that the best evidence is economist Josh Gans at the University of Toronto, who has, I think, a good piece about fast regulation. What I really want to see is tighter partnership between industry and AI. They're building stuff. We need regulators to move very quickly in response to harms that are occurring, because otherwise it's really hard to know what the harms are and you may actually just force more harms down the line. But I would like to see fast reaction. We don't have a regulation system built for that, right? So things are happening. I mean, it's hard enough for me to keep track and I'm in this all day. The people in the AI labs don't even know what's happening on a regular basis. So it's hard to say, but that would be the right way to do it. But I also am a realist, I'm a pragmatist. Know if there's an appetite for doing this kind of thing, right? Is there an appetite for real regulation? I haven't seen it yet. And the regulation tends to either be very, you know, let's stop all of this development, which I think could be problematic, or it tends to be, let's let everything rip. And there has to be some sophistication there. I mean, I don't think there's enough positive energy either. Like, I worry a lot, like, we have a tool that could transform education. Where are the universal tutor systems? I mean, we built some stuff at Wharton, you know, Sal Khan's bought some stuff. But, like, this feels like it should be national editing efforts. If we're going to transform science, that should be a Manhattan Project style spend. To do this, we don't need to build the models. We should be thinking about what we use these things for. And I think that there's a lack of interest in both the good and bad piece. And it's going to end up being, you know, I think it's becoming politicized in ways that are really complicated and will be hard to make it harder to respond to AI rather than easier.
- [45:56]
A
Professor Malik, it's terrific to talk to you. Thank you so much. And thank you for your substack again, everybody. It's one useful thing. You will be joining hundreds of thousands of other grateful subscribers if you subscribe. So thank you very much for that and good luck. We'll check in with you soon.
- [46:10]
B
Thank you very much. And I'm also a big admirer of your work over the years, so I'm glad I could be here.