AI is great at predicting text. Can it guide robots?

Summary

---

Short Wave: "AI is great at predicting text. Can it guide robots?"

Host: Regina Barber (NPR)
Guest: Jeff Brumfiel (NPR Science Correspondent), with insights from Moojin Kim (Stanford), Chelsea Finn (Stanford), Ken Goldberg (UC Berkeley), Pulkit Agrawal (MIT), and Matthew Johnson Roberson (Carnegie Mellon)
Date: February 11, 2026
Length: ~15 minutes

---

Episode Overview

This episode explores the intersection of artificial intelligence and robotics, focusing on whether the recent leaps in AI—especially language-based models—can truly revolutionize how real-world robots learn, act, and assist us. While AI chatbots excel at generating human-like text, translating those capabilities to physical robotic actions in a messy, unpredictable world is a very different challenge. Host Regina Barber and NPR science correspondent Jeff Brumfiel walk listeners through current breakthroughs, persistent obstacles, and expert perspectives, with a lively, curious tone and a bit of skepticism about hype versus reality.

---

Key Discussion Points & Insights

1. The Hype and Reality of AI-Powered Robotics

• Context: Recent events, like Tesla’s announcement of the Optimus humanoid robot and Google’s unveiling of an AI-driven robot, have hyped up expectations for intelligent, autonomous machines.
  • [00:20] Regina Barber notes AI is "everywhere"—from email to cars, and now, robots at major tech events.
• Historical Letdowns: Despite decades of promises, robots repeatedly fail to live up to the sci-fi vision.
  • [01:33] "The robots... have always disappointed compared to the vision." — Jeff Brumfiel
• Central Question: Can the advances that made text-based AI so powerful finally make general-purpose robots possible, or are there unique obstacles?

2. AI Robotics at Stanford’s IRIS Lab: Teaching by Doing

• Lab Tour: Jeff visits Stanford's IRIS lab, where researcher Moojin Kim demonstrates a simplified robot (just mechanical arms) powered by an AI model called OpenVLA.
  • [03:58] Jeff describes not a humanoid, but "a pair of mechanical arms with pinchers," emphasizing the practical, non-flashy nature of real research robots.
• Neural Network Approach: Instead of programming every little action, the robot learns "by example."
  • [05:28] "Whatever task you wanna do, you just keep doing it over and over. Maybe like 50 times or 100 times." — Moojin Kim
  • [05:40] Smiling robot anecdote: "It's exactly the same thing, except this robot's actually doing stuff." — Jeff Brumfiel
• Real-World Demo: The robotic arms are trained to pick out green trail mix pieces on command, exemplifying how AI can interpret text and translate that into action—if properly trained.

3. The Dream of Generalist Robots

• Ultimate Goal:
  • [06:25] "In the long term, we want to develop software that would allow the robots to operate intelligently in any situation." — Chelsea Finn
  • Includes basic tasks like "making a sandwich, cleaning a kitchen, or restocking grocery store shelves."
• Current Achievements:
  • Chelsea Finn’s spin-off shows a robot folding laundry—a mundane but surprisingly complicated affair.
  • [07:38] Regina anthropomorphizes the robot as it folds: "It looks like it's like, oh, I just gotta fold another one."
• Limits: Although impressive in lab conditions, these robots still make mistakes, get stuck, and need human intervention.
  • [08:00] "It might be able to fold laundry 90% of the time or 75% of the time, but the rest of the time, it's going to make a big mess..." — Jeff Brumfiel

4. Why Isn’t Robotics Evolving as Fast as ChatGPT?

• Big Data Bottleneck: AI chatbots are trained on vast internet datasets, but robots lack a similar “big dataset” for physical actions.
  • [09:00] "There's no examples online of robot commands being generated in response to robot inputs." — Ken Goldberg
  • [09:24] Goldberg quips, "At this current rate, we're going to take 100,000 years to get that much data."
• The Problem of Physical Data Collection:
  • Training robots in the real world is time-consuming and logistically tough.
• Simulation as an Alternative:
  • [09:48] "The power of simulation is that you can collect... very large amounts of data." — Pulkit Agrawal
    • Example: 3 hours in simulation can generate 100 days of "robot experience."
  • Sim-to-Real Gap:
    • [10:20] Simulations can teach basic locomotion, but tasks like picking up fragile objects are much harder to simulate correctly.
    • [10:34] "Your robot will fling things across the room if it doesn't understand the weight and size..." — Jeff Brumfiel

5. Deeper Challenges: Problem Framing and Learning Limits

• Predicting Text vs. Doing Things:
  • [11:09] "The question is not do we have enough data? It is more, what is the framing of the problem?" — Matthew Johnson Roberson
  • Text prediction is linear and well-bounded; robotics requires adapting to endlessly variable physical situations.
  • [11:37] "Next, best word prediction works... It's not clear I can take 20 hours of GoPro footage and then produce anything sensible with respect to how a robot moves around." — Matthew Johnson Roberson
• Teaching Robots to Teach Themselves:
  • [12:06] "Or have the robots teach the robots." — Regina Barber

6. Optimism, Cautiously Applied: Where AI Robotics is Already Working

• Narrow AI Applications:
  • [12:24] AI image recognition is already helping in package sorting—robots pick objects efficiently with AI's help.
  • [12:52] "It's working really well... and I think we're going to see a lot of that AI being used for parts of the robotic problem. You know, walking or vision or whatever. It's going to make big progress. It just may not arrive everywhere all at once." — Jeff Brumfiel
• Memorable Demonstration Closes the Episode:
  • [13:20] "Usually that, that spot right there where it identifies the object and goes to it. That's the part where we hold our breath in." — Moojin Kim
  • The robot successfully grabs the right trail mix—showcasing real incremental progress.

---

Notable Quotes & Memorable Moments

• On Disappointment:

  • [01:33] Jeff Brumfiel: "The robots... have always disappointed compared to the vision."

• On Teaching Robots:

  • [05:28] Moojin Kim: "Whatever task you wanna do, you just keep doing it over and over. Maybe like 50 times or 100 times."

• On Failing in Real-World Robotics:

  • [08:00] Jeff Brumfiel: "...it might be able to fold laundry 90% of the time or 75% of the time, but the rest of the time, it's going to make a big mess that then a human has to get in there and clean up."

• On Data Limitations:

  • [09:24] Ken Goldberg: "At this current rate, we're going to take 100,000 years to get that much data."

• On Simulation:

  • [09:48] Pulkit Agrawal: "For example, in three hours, you know, worth of simulation, we can collect 100 days worth of data."

• On Framing the Challenge:

  • [11:09] Matthew Johnson Roberson: "The question is not do we have enough data? It is more, what is the framing of the problem?"

• On Incremental Progress:

  • [13:42] Jeff Brumfiel: "Nobody really programmed the robot. Exactly. This is all neural network, learning how to move the claws and respond to the commands on its own. And to me, it's pretty wild that that works at all."

---

Important Timestamps

• 00:20: Introduction to the episode’s big question: Can AI move from chatbots to real-world robots?
• 03:58–05:35: Inside the Stanford IRIS lab; teaching a robot by demonstration.
• 06:25: Articulation of the dream: robots that understand plain instructions.
• 07:21: Demo: AI-powered robot folding laundry.
• 08:25: Ken Goldberg on realism versus hype.
• 09:48–10:34: Simulated training for robots—opportunities and limits.
• 11:09–11:51: Expert on the fundamental differences between text prediction and physical robotics.
• 12:24–13:14: Successful narrow applications; AI in industrial robots.
• 13:14–13:42: The trail mix test and the emotional tension of “will it work?”

---

Tone & Style

• Friendly, curious, and skeptical—humorous banter between Regina and Jeff keeps things light even as complex issues are discussed.
  • [12:09] Regina Barber: "Okay, so Jeff, you've taken me from like optimist to pessimist. It's the, you know, the road I take every day."
• Experts balance caution with real enthusiasm about incremental progress (especially in narrowly defined tasks).
• Encouraging close to the episode focuses on progress and continued discovery.

---

Summary

The promise of AI in robotics is immense, but we're only beginning to bridge the gap from virtual text predictions to physical-world mastery. Successes are real, if limited: robots can now learn some tasks with less rigid programming, especially in controlled environments or using carefully designed simulations. Massive hurdles remain—especially the lack of vast, diverse real-world “training data” for robots, and the fundamental leap from predicting text to manipulating a chaotic, physical world. Still, researchers’ incremental advances hint at a future where AI-powered robots can increasingly assist in everyday life, if not quite with the flawless grace of science fiction.

---

Transcript

[Capella University Announcer]

This message comes from Capella University. That spark you feel, that's your drive.

[Advertisement Announcer]

For more.

[Capella University Announcer]

Capella University's flexpath learning format lets you earn your degree at your pace without putting life on pause. Learn [e-mail]. you're listening to Shortwave from NPR.

[Regina Barber]

Hey, Shortwavers, Regina Barber here. I don't know about you, but for me, recently, it seems like artificial intelligence is everywhere. It's, it's in my search results, on everyone's phones, being pitched in my email, trying to read my emails. So I thought it would be a perfect time to revisit an episode we did last year with NPR science correspondent Jeff Brumfiel. He noticed that AI isn't just showing up online anymore. It's creeping into reality, like at Tesla's big marketing event in 2024.

[Jeff Brumfiel]

Yep, AI was there.

[Ken Goldberg]

Speaking of robots.

[Jeff Brumfiel]

Tesla is obviously a car company. Elon Musk, Tesla CEO, made a big part of the event about a humanoid robot powered by AI and called Optimus.

[Ken Goldberg]

The software, the AI inference computer, it.

[Jeff Brumfiel]

All actually applies to a humanoid robot. And Google unveiled another humanoid robot that operates using AI. We're bringing Gemini 2.0's intelligence to general.

[Pulkit Agrawal]

Purpose robotic agents in the physical world.

[Regina Barber]

Okay, Jeff. But even before AI came along, people and companies have been making like big claims about robots.

[Jeff Brumfiel]

They have, they have. And the robots, as I'm sure you know, Gina, have always disappointed compared to the vision.

[Regina Barber]

Yeah, that's true.

[Jeff Brumfiel]

And that's why I set out to understand the truth about AI and robotics.

[Regina Barber]

The truth.

[Jeff Brumfiel]

And I think I kind of found it in a bowl of trail mix.

[Regina Barber]

Today on the show, what happens when artificial intelligence moves out of the chat and into the real world world?

[Jeff Brumfiel]

We're looking at how AI could maybe revolutionize robotics.

[Regina Barber]

You're listening to Short Wave, the science podcast from npr.

[Advertisement Announcer]

This message comes from LinkedIn ads. One of the hardest parts about B2B marketing is reaching the right audience. That's why you need LinkedIn ads. You can target your buyers by job title, company role, seniority and skills, or the professionals you need to reach in one place. Get a $250 credit on your next campaign so you can try it yourself. Just go to LinkedIn.com nprpod that's LinkedIn.com nprpod Terms and conditions apply only on LinkedIn ads. This message comes from Wix. Nothing beats seeing your ideas turn into cold, hard cash. Well, if you use Wix Harmony, you better get used to it. Wix Harmony makes it unbelievably easy to create a Fancy new website that's built to sell. Get the perfect blend of AI and drag and drop tools that puts you in control of every detail. Plus an AI agent to help you every step of the way. Try it for [e-mail] Harmony this message comes from Kachava. That wellness goal you set at the start of the year is. It's not too late to stick with it and make your future self proud. Especially with The all in One Nutrition Shake from Cachava with 25 grams of protein, 6 grams of fiber, grains adaptogens and more. No fillers, no nonsense, just the highest quality ingredients. Stick with your wellness goals. Go to kachava.com and use code NPR for 15% off. That's K-C-H-A-V A.com code NPR.

[Regina Barber]

Okay, so, Jeff, you are interested in finding out more about how AI works in robots. Where did you start?

[Jeff Brumfiel]

Well, I didn't go to Tesla or Google, but I did drive right by them on my way to Stanford University.

[Regina Barber]

Okay.

[Jeff Brumfiel]

And specifically the IRIS Laboratory, which stands for Intelligence through Robotic Interaction at scale. I got a tour from a graduate student named Moojin Kim. Moojim works on a new kind of robot powered by AI, similar to the AI used in chatbots.

[Moojin Kim]

It's one step in the direction of, like, ChatGPT for robotics, but still a lot of work to do.

[Chelsea Finn]

Okay.

[Jeff Brumfiel]

All right, well, you want to show me how it. Show me what I can do for sure.

[Regina Barber]

So, Jeff, what did the robot look like?

[Jeff Brumfiel]

Well, this wasn't some humanoid robot that the big tech companies are rolling out. It's just a pair of mechanical arms with pinchers.

[Chelsea Finn]

Okay.

[Jeff Brumfiel]

But what made it interesting was that it's powered by an AI model called OpenVLA. So first we should probably just say quickly. A regular robot must be very, very, very carefully programmed. An engineer has to write it detailed instructions for every task you want it to perform.

[Regina Barber]

Yeah, and AI is supposed to change that.

[Jeff Brumfiel]

Exactly. And that's what's going on here. This robot is powered by a teachable AI neural network. The neural network operates kind of how scientists think the human brain might work. Basically, there are these mathematical nodes in the network that have billions of connections to each other in a way similar to how neurons in the brain are connected together. And so when you go to program this sort of thing, it's simply about reinforcing the connections that matter between the nodes and weakening the other ones that don't. So in practice, this means Mugen can just teach OpenVLA a task by showing.

[Moojin Kim]

It so basically, whatever task you wanna do, you just keep doing it over and over. Maybe like 50 times or 100 times.

[Jeff Brumfiel]

The robot's AI neural network becomes tuned to that task, and then it can do it by itself.

[Regina Barber]

Yeah, it makes me think of this smiling robot story we did. And that robot just watched a lot of videos of people smiling. Then it lear how to do it.

[Jeff Brumfiel]

Yeah, it's exactly the same thing, except instead of just smiling, this robot's actually doing stuff. So to show me, Mujim brought out a tray of different kinds of trail mix, and I typed in what I wanted it to do. Okay, so scoop some green ones with the nuts into the bowl, see what happens.

[Regina Barber]

Okay. So, Jeff, personally, I've been waiting for something like AI in robotics, because you can teach it to do something, you can ask it to do something, to make me an ice cream sundae or something without any fancy programming or special knowledge.

[Jeff Brumfiel]

That's exactly it. And this really is the dream of the researcher who runs this laboratory. Her name is Chelsea Finn.

[Chelsea Finn]

So in the long term, we want to develop software that would allow the robots to operate intelligently in any situation.

[Jeff Brumfiel]

And by intelligently, she means the robot could understand a simple command like scoop some green ones into a bowl or make me a sundae, and then execute in the real world, even just to.

[Chelsea Finn]

Do very basic things like being able to make a sandwich, or being able to clean a kitchen, or being able to restock grocery store shelves.

[Jeff Brumfiel]

These are simple tasks that could help humans do their jobs or do tasks at home. Now, Chelsea also has co founded a startup called Physical Intelligence. It recently demonstrated a mobile robot that could take laundry out of a dryer and fold it again. This robot was taught by humans, training its powerful AI program.

[Regina Barber]

Okay, so ice cream sundaes, is that too advanced? Is folding an easier start?

[Jeff Brumfiel]

I mean, I'd actually argue, Gina, that folding is harder. Okay, let me show you a video.

[Regina Barber]

Okay. It's going to the dryer. It's pulling stuff out, putting it in a basket. It has the concentration I have when I'm going to do laundry. It almost looks like annoyed with folding like I do. Oh, my God. It's doing really well, actually.

[Jeff Brumfiel]

Yes, it is. Right? And this is a complicated task. It's got to pull these clothes out. It's got to figure out what they are.

[Regina Barber]

It doesn't even have a head, but I'm like giving it personality. It looks like it's like, oh, I just gotta fold another one. Okay, so is it really as simple as just teaching a robot what to do? Because if it was, wouldn't these robots be everywhere?

[Jeff Brumfiel]

Yeah, I mean, right? It looks cool on the video. The truth is that when you get out and these robots are trying to do these tasks over and over again, they get confused, they misunderstand, they make mistakes, and they just get stuck. So, you know, it might be able to fold laundry 90% of the time or 75% of the time, but the rest of the time, it's going to make a big mess that then a human has to get in there and clean up.

[Regina Barber]

Got it. Okay.

[Jeff Brumfiel]

I spoke to Ken Goldberg, a professor at the University of California at Berkeley, and he is pretty emphatic that AI powered robots weren't here yet.

[Ken Goldberg]

Robots are not going to suddenly become the science fiction dream overnight.

[Regina Barber]

Okay, so, like, tell me why. Because, like, AI chatbots have gotten, like, way better super fast. So why are these robots getting st.

[Jeff Brumfiel]

Okay, so it's true that AI has improved massively over the past couple years, but that's because chatbots have a huge amount of data to learn from. They've taken basically the entire Internet to train themselves how to write sentences and draw pictures.

[Ken Goldberg]

But Ken says for robotics, there's nothing we don't have anything to start with.

[Moojin Kim]

Right.

[Ken Goldberg]

There's no examples online of robot commands being generated in response to robot inputs.

[Jeff Brumfiel]

And if robots really need as much training data as their virtual chat bot friends, then having humans teach them one task at a time is going to take a really long time.

[Ken Goldberg]

You know, at this current rate, we're going to take 100,000 years to get that much data.

[Regina Barber]

Okay, that's so long. Like, are there any alternatives? There must be.

[Jeff Brumfiel]

Yeah. Well, scientists are exploring them right now, and one might be to let the AI brain of the robot learn in a simulation. A researcher who's trying this is a guy named Pulkit Agrawal. He's at the Massachusetts Institute of Technology.

[Pulkit Agrawal]

The power of simulation is that you can collect, you know, very large amounts of data. For example, in three hours, you know, worth of simulation, we can collect 100 days worth of data.

[Jeff Brumfiel]

So this is a really promising approach for some things, but it's much more of a challenge for others. So, for example, let's talk about walking. When you're just dealing with the Earth and your body, the physics of walking around, it's actually kind of simple.

[Pulkit Agrawal]

When you're doing locomotion, you know, you're mostly on Earth, you know, there's no amount of force you can apply which will make the Earth move.

[Jeff Brumfiel]

And so the simulation can do that reasonably well. But if you want your robot to say, try and pick up a mug off a desk or something that's a lot more complicated.

[Pulkit Agrawal]

More forces, you know, if you apply the wrong forces, these objects then fly away very quickly.

[Jeff Brumfiel]

Basically your robot will fling things across the room if it doesn't understand the weight and the size of what it's carrying. And there's more. You know, if you robot encounters anything that you haven't simulated 100% perfectly, then it won't know what to do, It'll just break.

[Regina Barber]

Okay, so it sounds like these like, simulations have limits and real world training is going to take like a while. I can begin to see why AI robots aren't going to like, be here tomorrow.

[Jeff Brumfiel]

Exactly. And some researchers think there are even deeper problems actually with trying to put AI into robotics. One of them is Matthew Johnson Roberson at Carnegie Mellon University in Pittsburgh.

[Matthew Johnson Roberson]

In my mind, the question is not do we have enough data? It is more, what is the framing of the problem?

[Jeff Brumfiel]

So getting back to AI chatbots for a minute. Matt says for all their incredible skills, the task we're asking them to do is actually relatively simple. You know, you look at what a human user types and then try to predict the next words that user wants to see. Robots have so much more that they're going to have to do than just compose a sentence.

[Regina Barber]

Right.

[Matthew Johnson Roberson]

Next, best word prediction works really well. And it's a very simple problem because you're just predicting the next word and it is not clear right now. I can take 20 hours of GoPro footage and then produce anything sensible with respect to how a robot moves around in the world.

[Jeff Brumfiel]

So in other words, the sci fi tasks that we want our robots to work so complicated compared to sentence writing, no amount of data may be enough unless researchers can find the right way to teach the robots.

[Regina Barber]

Or have the robots teach the robots.

[Jeff Brumfiel]

Yes, that's also an option they can teach themselves.

[Regina Barber]

Okay, so Jeff, you've taken me from like optimist to pessimist. It's the, you know, the road I take every day. I'm starting to think that AI is like never going to work that well in robots or like, it's going to be a really long time.

[Jeff Brumfiel]

You know, I'm sorry if I've like turned you into a pessimist here, Gina. And then it happens and I'm going to have to sort of whipshaw you back because AI is already finding its way into robotics in ways that are really interesting. So, for example, Ken Goldberg has co founded a package sorting company and just this year, they started using AI image recognition to pick the best points for their robots to grab the packages.

[Regina Barber]

Ooh, okay.

[Jeff Brumfiel]

Yeah. And it's working really well, he told me. And I think we're going to see a lot of that AI being used for parts of the robotic problem. You know, walking or vision or whatever. It's going to make big progress. It just may not arrive everywhere all at once. And to really end on a high note here, let's get back to that Stanford lab. Remember, I asked it to grab some trail mix, right?

[Regina Barber]

Yeah.

[Jeff Brumfiel]

So the robot correctly identified the right bin to Moojin Kim's relief.

[Moojin Kim]

Usually that, that spot right there where it identifies the object and goes to it. That's. That's the part where we hold our breath in.

[Jeff Brumfiel]

And then very, very slowly and kind of hesitantly, it reached out with its claw and picked up the scoop.

[Regina Barber]

It's doing it.

[Jeff Brumfiel]

Mujin, did I just program a robot?

[Moojin Kim]

You did. Looks like it's working.

[Jeff Brumfiel]

And to my mind, it's incredible. Like, remember, nobody really programmed the robot. Exactly. This is all neural network, learning how to move the claws and respond to the commands on its own. And to me, it's pretty wild that that works at all. And I think it's going to lead to some very cool developments.

[Regina Barber]

I'm excited to hear more. Jeff, thank you so much for bringing this reporting to us.

[Jeff Brumfiel]

Thank you very much.

[Regina Barber]

We'll link Jeff's full story, which has robot videos in our episode notes. This episode was Produced by Burleigh McCoy, edited by our showrunner, Rebecca Ramirez, and fact checked by Tyler Jones. Jimmy Keeley was the audio engineer.

[Jeff Brumfiel]

I'm Jeff Brumfiel.

[Regina Barber]

I'm Regina Barber. Thank you for listening to Short Wave from npr.

[Advertisement Announcer]

This message comes from vanta. What's your 2am Security worry? Is it do I have the right controls in place or are my vendors secure? Enter Vanta. Vanta automates manual work so you can stop sweating over spreadsheets, chasing audit evidence, and filling out endless questionnaires. Their trust management platform continuously monitors your systems, centralizes your data, and simplifies your security at scale. Get [e-mail] that's V A N T A dot com. This message comes from BetterHelp. February is full of relationship talk, making it feel like everyone has it all together in their love lives. But most people are still figuring it out. Therapy can take that pressure off. Visit betterhelp.com NPR for 10% off. This message comes from Greenlight. Parents say financial literacy is the hardest life skill. To teach. Greenlight's debit card and money app for families makes it easy for kids to learn to earn, save and spend wisely. Start today risk [e-mail] NPR.