← People I (Mostly) Admire
People I (Mostly) Admire
154. Can Robots Get a Grip?
00:57:52
Ken Goldberg is at the forefront of robotics — which means he tries to teach machines to do things humans find trivial.
Loading summary
Transcript186 lines
- [00:02]
Steve Levitt
This episode is sponsored by Nordstrom. The Nordstrom anniversary sale is on now. It's a big deal with new arrivals on sale for a limited time. Save up to 33% on brands like Ugg, All Saints, Charlotte Tilbury, Steve Madden, Bobby Brown and more. Plus, stock up on once a year beauty exclusives from winning brands. The best deals go fast in stores and@nordstrom.com prices go up.
- [00:29]
Ken Goldberg
August 4 Gator Lite from the makers of Gatorade.
- [00:34]
Narrator
With a specialized blend of five electrolytes and lower sugar, Gator Lite hydrates fast to replace the electrolytes you lose in sweat. Gator Lite made for rapid rehydration. I still remember the first time I saw a Roomba in action. That little robotic vacuum cleaner that skitters around the house. It was love at first sight. And I thought it was beginning of what would no be a revolution in home robotics. That was maybe 2003. So more than 20 years ago, the revolution never happened. My guest today, UC Berkeley robotics professor Ken Goldberg, has been working on robots for more than 40 years. And one thing he's learned the hard way is that robots still have a long way to go.
- [01:23]
Ken Goldberg
We have this incredible ability to adapt to changing conditions, and science has not figured that out. So it's very hard to reproduce that in robots. Now. Humans and animals are existence proof that it can be solved. It's not like an impossible problem like time travel. It's so funny because it's right in front of us, but we don't know how to do it.
- [01:50]
Steve Levitt
Welcome to people I mostly admire with Steve Levitt.
- [01:55]
Narrator
In spite of the inherent challenges in developing robots, there are some who think things are about to change. Tesla has been working feverishly on a humanoid robot called Optimus. Elon Musk has predicted that Optimus robots could generate more than $10 trillion in revenue long term. Is that realistic? Ken Goldberg has some opinions on the future of robotics. But our conversation today starts in the past with how he came to build his very first robot.
- [02:29]
Ken Goldberg
When I was a kid, I was really into rockets, models, building things like that. And my dad ran this chrome plating company. And chrome plating involves moving these metal parts between these different tanks. A lot of them are very poisonous, like cyanide. And it was very messy work. And so he wanted to build a machine robot that would do this dirty work.
- [02:53]
Narrator
Oh, so really for real, he wasn't just messing around, he was trying to be practical about it.
- [02:58]
Ken Goldberg
Yes. He built this frame with all these motors and stepping motors and switches built into it. And then it had a controller that was basically this. It was almost like a player piano kind of thing. It was a rotating drum with these little pigs and that would tell it which things to turn on and off. And then he taught me binary numbers and LEDs.
- [03:19]
Narrator
In the end, did he ever implement it at his factory?
- [03:23]
Ken Goldberg
I have a picture of it which is really funny because it is in there, but I don't think it ever really worked. My father was a great tinkerer, very creative, but he had a limited attention span, so he abandoned projects like that.
- [03:38]
Narrator
So what was the state of robots? They were essentially non existent in the 1960s. Were they?
- [03:44]
Ken Goldberg
Well, okay. There's this really fascinating history that goes way back. If you want to start at the ancient Egyptians building machines that looked human, like did human like things. And the word robot doesn't appear until 1920. And interestingly, it was in a play about robots. It was a Czech author who wrote this play and started that word, which actually comes from the root word for work or forced work.
- [04:13]
Narrator
Wait, can I ask you a question? So does a robot have to look like a person? Is that the definition of a robot or does it just have to replace human activity?
- [04:22]
Ken Goldberg
Ah, okay, so you're getting right into the thick of the topic here. It's very controversial. People have all kinds of definitions generally, and I am of the camp, that a robot does not need to look like a human, that a robot is a machine that's programmable, that moves in the physical world, but does something interesting and useful.
- [04:43]
Narrator
And why would anyone care if it looked like a human? That seems like hubris or something.
- [04:49]
Ken Goldberg
Exactly. So it's interesting you say that because that was the hubris story that goes all the way back to Pygmalion, Prometheus, Daedalus. They were all guilty of hubris because they were stepping too far in their creativity. And they were punished for that.
- [05:08]
Narrator
For mimicking gods or encroaching on godlike territory.
- [05:12]
Ken Goldberg
Exactly. But it's very compelling to have something that does have some form factor of a human humanoid. Yeah. And that is super popular right now. There is a huge wave, the biggest wave I've ever seen in my whole life, of interest in robots. And it's specifically around the humanoids. And the big proponents of that, namely Elon Musk and Jensen Huang from Nvidia, are saying that we're on the verge of achieving this dream finally, that we'll have the humanoids like, you know, Rosie from the Jetsons come in and clean up our house.
- [05:48]
Narrator
But she didn't even look that much like a human.
- [05:50]
Ken Goldberg
Well, that's true. She clanked around. In fact, it was very interesting. Cause if you remember the show, she was always breaking down. It was a kind of a running joke that the robot wasn't very good and was always malfunctioning, which is actually the way real robots are. I always show this video clip when I give talks. You see the backflip of the robot, and it's triumphant, you know, stands up like it's about to dominate and take over. But what you don't see is the 199 takes where the robot basically falls flat on space. Yeah, absolutely.
- [06:23]
Narrator
So you started building this early robot in your basement with your dad when you were still a kid. Was it all robots all the way, or did you get off the path once or twice on the way?
- [06:35]
Ken Goldberg
Oh, I had a lot of interest. I was a rebel as a kid. I was into go karts and motorcycles and things like that. But I also was very interested in art. And I mentioned to my mother that I was gonna study art in college, and she said, that's great. You can be an artist after you finish your engineering degree.
- [06:53]
Narrator
When did you get into picking things up? Cause that's been a real focus of yours is trying to build robots that can pick things up. When did that become an interest?
- [07:02]
Ken Goldberg
Well, so I was studying in Scotland for a junior year abroad, and I took these classes on AI and robots.
- [07:09]
Narrator
And we're talking about the early 80s, right?
- [07:12]
Ken Goldberg
Exactly. They had a department of AI, and they had several of the pioneers there. It started from Alan Turing's work in AI, so it kind of had grown out of that. And so it's actually very famous department of AI. So I was very lucky to be there, get exposed to all that, and then come back. And then I was also lucky. I found this lab at Penn, led by this wonderful young professor, Rujana Baichi, who was doing robots. And that's still the GRASP lab at UPenn.
- [07:43]
Narrator
Were you actually trying to pick things up at that lab, or it just happened to be called grasp?
- [07:46]
Ken Goldberg
I was working on grasping, I like to say. I've been working on the same problem my entire career. I haven't made very much progress.
- [07:53]
Narrator
What makes that problem so enticing to you?
- [07:56]
Ken Goldberg
Well, I think one reason is that I have always been clumsy. So when I was a kid, if you threw me any kind of ball, I would drop it instantly. But it's really the fundamental question, because robots need to do this. A first step to being able to do something Useful. We have to be able to move things around, to ship all these packages that we're increasingly ordering online, but also to make things when we're going into factories. Anytime you want to put something together, assemble it, you have to pick up the parts. It's very counterintuitive because it's much, much harder than people think. What's easy for robots, like lifting heavy objects, is very hard for humans. But what's very easy for us, like just literally picking up a glass of water, still remains incredibly hard for robots to do reliably.
- [08:45]
Narrator
So your first real work on grasping was your dissertation. Can you just describe what problem you're trying to solve and what your strategy was when you first started on this path?
- [08:55]
Ken Goldberg
The problem I was really interested in was picking up objects with a gripper. I was just using the parallel jaw gripper, which is just the binary clamp gripper that you see on robots.
- [09:05]
Narrator
Just like a pincher or something like that. Two fingers that come together and grab something.
- [09:09]
Ken Goldberg
Right. And what I wanted to study was if you could use that to grasp and orient a polygonal object by squeezing it without using any sensing. And the reason is because sensors are prone to error and noise. And so what I found was that there was this beautiful geometric way to essentially constrain the shape of any polygonal object so that it would come out in a unique final range.
- [09:36]
Narrator
Wait, is this mathematical theory or are you actually picking things up?
- [09:39]
Ken Goldberg
Both. It was mathematical theory. So I ended up proving this theorem that was a completeness theorem. So I was able to show that it works for any polygonal part. It took me two years of struggling to come up with that proof.
- [09:53]
Narrator
Yeah, most of what you do is so practical. I'm surprised to hear that you started in mathematical theory.
- [09:59]
Ken Goldberg
Yeah. So I've always liked formalisms and trying to prove theorems, but at the same time, what you have to do is make a lot of assumptions, obviously. So in this case, we were dealing with planar parts, so they were essentially flat. You were orienting them, of course. All parts are three dimensions and often deformable, and there's a lot of complexity. We did also do experiments, and I invented a gripper that actually went with the dissertation ideas.
- [10:27]
Narrator
So you had this mathematical proof, this completeness proof, and then you actually tried to pick things up. How good or bad were you at the actually picking things up part of things?
- [10:37]
Ken Goldberg
Well, actually, we were very good at orienting the objects, which means getting them in unique final orientation, which is called part feeding in industry. And it worked. We could Prove it worked theoretically, and it worked pretty well in practice. The issue is there's very small errors and factors like friction that are hard to model. And when those get violated, the assumptions get violated, then things don't always work out as you hoped.
- [11:01]
Narrator
So hearing you talk about how robots struggle to do things that are so trivial for humans, it's interesting because it's so at odds with the seemingly inexorable tide of mechanization. I've been in factories where you barely see a human. Farming used to employ 40% of the workforce. Now we produce far more food, using a little more than 1% of the workers to do it. And in large part that's because of machines that are replacing people. In our homes, we have refrigerators and washing machines and dryers and lawnmowers, and all of these things are labor saving devices. But as you talk now, I'm actually thinking for the first time, the way that these machines do the labor is not actually mimicking the way a human would do a task. It's always the way that suits a machine. So it's interesting that then with robots, I think our inclination is, well, let's have the robot do it exactly the way the human would.
- [11:59]
Ken Goldberg
Right. And that's a very good insight. If you look at, let's say, farming, the idea of monoculture, which is the way farms are run now, is everything is standardized and you're just mowing down these crops with these big combines. And that's very different than polyculture, which is really what nature does. If you go into any kind of forest, you'll see all kinds of different plants growing in proximity. And that is actually a trend in agriculture because it saves water and has all kinds of benefits for the plants and reduces pesticides. But it turns out that requires a huge amount of labor, manual labor, because you have to prune and you have to adjust for the reality that variations, the diversity that's there. This is actually the same thing in homes. The dishwasher is fantastic at doing the job of washing the dishes once you get them in there, but getting them off the table and getting them into there and out and back onto the shelf. That is a very hard problem unsolved. And it comes back to what you were saying earlier about why do robots have to look like a human? I mean, you could argue the dishwasher is a robot. It doesn't look like a human. And it does what it does very well, but it needs a human to load it. Laundry is another one. The washing machine does a great job but the folding part we haven't figured out how to do. There's all this nuance of physical interaction and there's a very simple experiment that anyone can run and you just put a pencil on a table and you push it with a finger. How that pencil moves in response to your pushing it is undecidable. You cannot predict that.
- [13:32]
Narrator
So the minute forces of friction and imperfections, and those are enough that at that small scale you can't tell where things are going to end up.
- [13:41]
Ken Goldberg
Yeah, because if you just put a tiny grain of sand, a microscopic grain of sand, that will cause that pencil to pivot in a different way. Right. But you can't see the sand because it's underneath. That actually matters when it comes to these tasks like grasping and manipulation, because very small errors make the difference between picking up that glass and dropping it.
- [14:01]
Narrator
We'll be right back with more of my conversation with roboticist Ken Goldberg after this short break. This is a vacation with Chase Sapphire Reserve the Butler the Spa. This is the edit a collection of handpicked luxury hotels and a $500 edit credit chase Sapphire Reserve the Most Rewarding card. Learn more@chase.com Sapphire Reserve cards issued by JPMorgan Chase bank and a member FDIC.
- [14:28]
Ken Goldberg
Subject to credit approval People I mostly.
- [14:31]
Narrator
Admire is sponsored by Constant Contact Running your own business today means facing constant disruption. From rapidly evolving AI technologies to inflation and unpredictable political shifts. This instability can make even short term planning feel uncertain, leaving business owners feeling reactive instead of proactive. That's why growth is more important than ever and Constant Contact helps make it simple and steady. With their all in one platform, you can create and manage attention grabbing campaigns in just a few clicks. Email, text, social media, events, landing pages, you name it, it's all in one place. Constant Contact's AI Content Generator helps you turn a rough idea into a ready to go message. Faster than ever. And with hundreds of customizable templates, it's easy to make something that looks and reads like your brand. You also get automated sending, real time reporting and tools that actually help drive sales. So you're not just marketing your business, you're growing it. Get a free 30 day trial when you go to constant contact.com try constant contact free for 30 days at constant contact.com constant contact.com it's over for Dirty Toilet Brush Bristles because Clorox Toilet Wand makes cleaning your toilet so satisfying that it might become your favorite chore. The all in one toilet cleaning system.
- [15:50]
Ken Goldberg
Is so surprisingly easy, it's no wonder.
- [15:52]
Narrator
They call it A wand. This thing cleans like magic. The Clorox Toilet Wand comes with six scrubbing pads preloaded with disinfecting toilet cleaner. Just click, swish and toss for a surprisingly easy clean. It's time to simplify house cleaning. Visit Amazon to purchase your Clorox Toilet Wand today.
- [16:10]
Ken Goldberg
Use as directed.
- [16:15]
Narrator
So let's talk about what it is. Very specifically, unpack the problem of what makes picking things up for robots hard. The first thing is vision. I think for a long time it was probably very difficult to get robots to see very well, especially in three dimensions. Can you talk about that?
- [16:33]
Ken Goldberg
Well, that's still hard. We have very high resolution cameras. We have them on our phone. No problem. You get very beautiful two dimensional images, but that doesn't give you the three dimensional description of the environment. 3D, that's what you want, is a depth map of the basically, where is everything in space? We have these LIDAR sensors and things, but they're very noisy. You can't actually know where things are in space. That is an open problem right there.
- [16:58]
Narrator
So the autonomous vehicles use lidar. They don't have to be that good. In driving a vehicle sense versus a picking up a small object sense, they are very good.
- [17:09]
Ken Goldberg
But the key difference is that in driving, you're just trying to avoid hitting anything. In grasping, you've got to hit, you've got to make contact.
- [17:16]
Narrator
Okay, good point.
- [17:18]
Ken Goldberg
And that's where the scales from the errors are much more significant.
- [17:22]
Narrator
So when you started trying to pick things up, that was a long time ago, and that was before the revolution in computer vision. I had Fei Fei Li as a guest on this podcast. And what was it around 2010 when she built this huge database of images known as imagenet. And I think prior to that, computer vision was terrible. And then suddenly, and I think really unexpectedly, we just really nailed computer vision. Is that a fair assessment?
- [17:51]
Ken Goldberg
It's not completely nailed, I would say, but it was a breakthrough for sure. And Fei Fei, what she did was systematically collect this big set of data, ImageNet, as you said, it was a critical mass. Somehow if you trained a large enough network on that, it started to generalize and work for images it's never been.
- [18:09]
Narrator
Trained on because before that things were very specific. You would train with algorithm to know the difference between a cat and a dog. And it could be really good at that. But in practical terms, it wasn't very helpful. But the breakthrough approach was surprising.
- [18:25]
Ken Goldberg
Right?
- [18:25]
Narrator
The neural net started working when you had more data. Is that kind of the truth of what happened?
- [18:30]
Ken Goldberg
Definitely. So there are three ingredients. One is data, the second is computation, and the third is algorithms. So those three things came together in about 2012. And Fei Fei played the crucial role for the data, for vision. Then there were GPUs, graphical processing units that were being developed for games, not for AI. But it just turned out that they could also be used for AI. That turned out to be critical.
- [18:55]
Narrator
But that's only one of the many problems. The second one, maybe I'd call it fine motor skills. You actually have to very precisely put a grip around the right spot and put the right kind of pressure. So could you describe some of the challenges in that domain?
- [19:12]
Ken Goldberg
One thing we tried to do was have a tactile sense so that the robot would feel things. That was actually my senior project as an undergrad, and then I continued that a bit into grad school. But it wouldn't work. And it would register that it made contact. It would have false alarms. The hand would freeze in space thinking it had touched something when it hadn't. That's a false positive. And then there was a false negative where it would keep squeezing when it did touch things. But the sensor didn't pick that up. And when I looked at the sensor signals, they were vibrating and moving and drifting all over the place. And it was maddening because, you know, I thought, this can't be that hard to detect pressure. But it turns out it actually is hard to detect gradations of pressure.
- [19:55]
Narrator
And the human hand is also exquisitely designed for grasping. Right. It's been really hard to match that with machines.
- [20:01]
Ken Goldberg
Exquisite, Exquisite. I agree. It's beautiful and has all these nuances of its ability to apply forces. And it's got this beautiful skin and sensory process that can detect contact of a huge dynamic range.
- [20:16]
Narrator
Would you say on the fine motor skill aspect of the problem, is it primarily a materials problem? The cables and the pulleys aren't as good as skin and muscles. Or is it deeper than that?
- [20:28]
Ken Goldberg
Yeah, no. People are making progress with better motors, where you can put the motor out near the gripper and things like that. But it's almost inherent that in any mechanical system you're going to have these small imprecisions. Humans actually have a lot of imprecision, but we compensate very elegantly as we move our fingers. They're imprecise. We're constantly making these adjustments. Our eyes are doing it right, so we're like constantly looking at different things, paying attention to different aspects of the scene. That feedback loop is extremely powerful and extremely fast. And that Gives humans the ability to compensate for the imprecision in their motor control. And if you watch a musician play, right, they're doing something incredible. They're adjusting and dynamically tuning based on what they hear and then adjusting their fingers, you know, minute positional changes to get the tones they want.
- [21:20]
Narrator
Now, when we say robots aren't that good at doing things like grasping, in one sense, we're comparing it to humans, but in another sense, we're comparing it to other dimensions in which we've made incredible strides. Computer vision is one example. We talked about large language models, but maybe the best example are games like chess, AlphaZero. So DeepMind created this program, AlphaZero, and if I understand it correctly, all they did was teach it the rules of chess and let it play itself. And within a day, it had become the best chess player in the world, Even better than all of the other computers that people have been working on and programming for years. And I think the key to that is that it just had incredible processing speed. So AlphaZero, I think I remember, could play literally thousands of games a second. So it got incredible amounts of feedback and figure out what moves worked and what moves didn't work. Okay. But when you're actually trying to reach out and grab something, at least intuitively, it strikes me that the thing that limits how fast your system can learn isn't processing speed. It's the fact that in the physical world, you actually have to put your robot arm out there, try to grab something, see whether it works. And so you can't create a database at thousands of chess games per second. Am I right about that or am I missing something?
- [22:45]
Ken Goldberg
No, no, it's great. Let me try to unpack that a little bit, because first of all, that is a remarkable result. When DeepMind showed that the key is that chess is a perfect information game.
- [22:54]
Narrator
Yes.
- [22:55]
Ken Goldberg
And whatever you model in the computer is perfectly represented in the reality, because that is the game.
- [23:00]
Narrator
You don't have to generalize it to different shapes and different temperatures.
- [23:04]
Ken Goldberg
You don't have to worry about friction, all those things. And you're also right that a big part of the breakthrough that people didn't really emphasize, but was very much a part of why Google was successful, was they are very good at doing very high speed computation, parallelized. So there was a lot of search going on simultaneously. But it was also this idea of reinforcement learning, which was self play against itself, that could essentially start to find strategies, discover strategies just by trying things out. And that has been very successful. And that has really led to ChatGPT. And you can ask questions and have conversations, quite deep conversations with these chatbots now. So all those breakthroughs are a very big deal. And this is part of why many people expect that robots should be also solved. We've solved language, We've solved vision. So therefore, we're just about to solve robots. Yeah, and I think we will. There's something called the bitter lesson.
- [23:58]
Narrator
That's a theorem. What does that mean?
- [24:00]
Ken Goldberg
Oh, okay. So this is Rich Sutton, who just won the Turing Award. Reinforcement learning was his subject, and he wrote books about it. But he wrote this really important essay in 2019 called the Bitter Lesson. He makes a really strong argument that all these techniques we've been trying to do to solve language and to solve gameplay and to solve computer vision by writing rules and all these techniques all went by the wayside. As long as we got these big enough machines that you could just throw lots and lots of data, lots and lots of compute, and essentially look for patterns. It would find patterns, discover patterns on its own. That always worked better.
- [24:38]
Narrator
Okay, so let's go deeper into that, because I'm not sure everyone understands how transformational this is the typical scientific approach to solving a problem, Whether it's chess or what we do in economics, trying to model different behaviors has been you take a human kind of thought, and algorithmically, you try to come up with rules for predicting and understanding behavior. And that is completely different than what these neural nets do. These are black boxes. You feed enormous amounts of data. In the end, you don't really understand exactly how the machine is doing it. But empirically, in prediction problems, these models, if you give enough data and allow enough complexity in the neural net, they just give you amazing predictions out of sample.
- [25:30]
Ken Goldberg
Exactly. You and I were trained on building models, and that has been the hallmark of science. There's so many beautiful mathematical models. I just finished an art project where we listed the history of science in terms of equations. Equations. And we carved it into a piece of wood, my wife and I. But it's all these models that work beautifully. And you can actually say, okay, this will work for all inputs. Right. These new methods are model free. There's no model. You just throw data, and it somehow interpolates and figures out something that empirically does the right thing. And so this has been a bitter lesson for most researchers and academics who spend our lives building these models, that these models maybe don't work as well as this method that just sort of bubbles up out of magic. I believe in the bitter lesson, namely that someday Robots will actually learn to do all these things, but actually to put into context how much data that may take, you can look up how much data was trained for, let's say, one of the large models like Quinn, which is a little bit bigger than GPT4, and it's got 1.2 billion hours of training data. Converting everything into hours is nice because you can compare between robot data and let's say, reading data.
- [26:48]
Narrator
But help me out, what does an hour of training data mean?
- [26:51]
Ken Goldberg
I love that you caught on that. So I could go down this rabbit hole. But it's basically the scientist at a company called Physical Intelligence, Michael Black, he said, okay, how fast can humans read? So 233 words per minute. So then he said, that's how many tokens per minute can be digested. Right. So if you look at all the amount of text that's out there and convert that into tokens, you can convert it back into hours. How much hours were used? Right now the idea is to train robots by basically teaching them because they can't do it themselves, like folding clothes. They can't do it. So you have humans basically driving them. Right. Like puppets and getting them to full close all day long. Right. So they're collecting hours and hours of data. The one company that has been pioneering this published a paper saying that they had accumulated 10,000 hours of data with robots of data. Folding clothes, folding clothes, making coffee, doing tasks. Right. Like all around the house. So 10,000 hours. And then they started experimenting and started showing some signs that it could actually generalize in certain very clumsy ways. It's just very early stage. But that's 10,000 hours. Right, but if you compare that to the large language model, that's 1.2 billion hours.
- [28:01]
Narrator
Okay, 1.2 billion hours. Okay, compared to 10,000.
- [28:04]
Ken Goldberg
Exactly. And that 10,000 hours is approximately a year. That means that we have so far accumulated one year. To get to the level of the large language models, that would take us 100,000 years.
- [28:17]
Narrator
Okay, and is there a way to bypass this? The way written language is, we have an enormous store of work that people have done in the past that can be immediately transformed for these machines to build off of. But we haven't chosen to record information about every time we've folded a shirt or made coffee. Maybe we don't even have a mechanism for storing that kind of information. How will we build a data set of experience other than by having the robots do it?
- [28:49]
Ken Goldberg
That's the big problem right now. I call it the data gap. If people don't realize the scale of this. You can say we're getting close, but no, we're 100,000 years off. Okay. It's not going to happen next year. I will bet on that. It's not going to happen in two years. It's going to take a while. Now, there's a lot of ideas about how we can speed this up. One of them is simulation. And this comes back to what you were saying about playing the game, having a robot, let's say, experiment by picking things up in a simulator, right? We have very good simulators. They look amazing. We can make computer graphics that looks great, but turns out those are actually very imprecise in terms of real physics.
- [29:24]
Narrator
Because that doesn't have the friction and the mistakes you're talking about. You could try to build it in, but if you build it in just with, say, random noise, then you're teaching the robot the wrong thing. You're teaching it to live in this fake world rather than the real world. So it doesn't do you any good.
- [29:41]
Ken Goldberg
Well, okay, we've actually injected what is called domain randomization, where you throw in some random noise, but if you put the right kind of noise in, then it actually can learn to work in the real world. And that was a breakthrough that happened in 2016 for us.
- [29:53]
Narrator
And is that what you call DexNet?
- [29:55]
Ken Goldberg
That was DexNet.
- [29:56]
Narrator
Okay, tell me about DexNet.
- [29:57]
Ken Goldberg
So DexNet was a case where we basically tried something very analogous to what Fei, Fei Li and Jeff Hinton and others had done for vision. But we did it in robotics. It was for dexterity network. We were able to generate a very large data set of three dimensional objects and grasps on objects and use that to train a neural network, but added noise so it was more realistic. And then when we put on the robot, it started picking things up remarkably well.
- [30:21]
Narrator
So what's an example of an object that you would have had, let's say.
- [30:24]
Ken Goldberg
Like a pair of eyeglasses? There were things we found online. We had 3D CAD models. We basically just went on a hunt and we found all kinds of things from, like, gaming sites, 3D printing sites, all kinds of things we could pull off. And then we had to scale those and clean them up and then get them into a system. And then for each object, there's a thousand facets, right? You want to grasp it at two facets. That's a pair of facets, which is your grasp points. So that means there's a million different ways to pick up every single one of those objects. So we had to evaluate every single one of those grasps in terms of how robust it was to perturbations in position and center of mass and friction and all those things.
- [30:59]
Narrator
And then when you say add noise, what does that mean?
- [31:02]
Ken Goldberg
So we would add noise in terms of we pick a pair of faces on that object and say, okay, well, that would be the nominal grasp. If I put my two fingers right on these two points on the pair of glasses. Right. But now I'm not going to really get what I thought I was because of these errors. So I'm going to have to look at perturbations. So I would do. What if I actually were slightly off here? What if I were slightly off there? This is the noise I'm talking about. So there'd be slight perturbations in these variables in terms of the spatial contacts.
- [31:28]
Narrator
I see. So if you put your robot fingers not where you thought you're going to put them, but at random, a little bit to the right or the left or up or down.
- [31:35]
Ken Goldberg
Exactly.
- [31:36]
Narrator
And then you tried to pick it up, you can compute because you've got models about what would happen with physics that tells you. And then, oh, I would have dropped it. I see. So the key to Dex is you're not trying to optimize. If I could do it perfectly, where would I put my fingers? You're saying, given I don't exactly know what I'm doing, what's the right general space to be in? So I have a lot of leeway for going wrong when I put my fingers on it.
- [32:00]
Ken Goldberg
Exactly. You nailed it. You nailed it. What I'm looking for is robust grasps.
- [32:03]
Narrator
Okay.
- [32:04]
Ken Goldberg
So we used Monte Carlo integration to basically estimate the probability of success for all these different grasps. And that gave us the training set. So we learned how to predict the probability of success. Then in real time, when we see an object, we actually basically look for the grasp that has the highest probability of success.
- [32:21]
Narrator
Okay, so let me just understand. So now you go from this model, you've got an actual robot arm with two fingers, two grabbers for picking up, and you have then, I guess, a big crate full of objects that are different from the objects that this thing learned on, because those were actually just virtual objects anyway. And now somehow the robot has got to look at the things in the box, guess what they are, and once it's guessed what they are, say, where am I going to grab it based on these other things I used to grab in the past?
- [32:56]
Ken Goldberg
Almost. But the one difference is this. It never knows what they are. All it sees is points in space, and that's the input. And then it says, if you see this pattern of points in space, where should you put your gripper to maximize your probability of picking something up? It never tries to identify what the objects are, anything like that. And they're all jumbled up, as you said. But what was so surprising is how well that worked. We were getting like well over 90% success rates.
- [33:22]
Narrator
Okay, and when you made this, how much better were you than humans at the task?
- [33:27]
Ken Goldberg
Well, okay, it's pics per hour. Successful pics per hour. That's the metric. Humans are very good. Humans were like 400 or something like that. Humans are pretty much 99.9%. Right. We drop things pretty rarely when you're trying to pick things out of a bin. At that point, DexNet was about 200 and 250 or so pics per hour. We were getting into 91, 92%. That was pretty far ahead of others at that time.
- [33:57]
Narrator
Were you the first ones to be building in the noise and doing this in a mode? I mean, this is 2015 or 16, so this is obviously long before ChatGPT. It was after the computer vision breakthrough, but still, I don't think there was a general view out there among scientists that this approach, this black box approach, was going to be beating the formal modeling approach.
- [34:24]
Ken Goldberg
No, it was very, I would say, almost controversial or surprising, but empirically it was working. And it was one of those things where we just had to accept what was in front of us, that this is working. So let's try and get it even better. And we started fine tuning it. We figured out all kinds of things to make it faster. And we also have to worry about the motion of the robot reaching into the bin, et cetera. So there's all kinds of ways to speed it up. We extended it to suction cups, and then we formed a company to commercialize this Ambi. It was also with this terrific student, brilliant engineer Jeff Mahler, who basically implemented all this, sweated the details to get everything to work. And then the company basically started building machines that could do this with other graduates from my lab that could actually solve this problem for E commerce. And in some way, the pandemic was interesting because there was a huge surge in E commerce. There was a big demand for our machines. We were just collecting data, mostly to fine tune our machines and just track them, you know, maintain them and troubleshoot. Right. And we didn't think about this at the time because we didn't realize how valuable that data would be. But we quietly amassed what is 22 years worth of data of robots picking up things in warehouses.
- [35:42]
Narrator
It's the reality of the modern world where data are so valuable that the feedback loop is suddenly now, because you've got your machines doing all this picking, you're generating data, and that data is the secret to making them better. And that is a huge comparative advantage that you have because the NSF can't afford to give you a grant that will allow you to do as much picking as you can if, say, Amazon is having you do the picking for them.
- [36:08]
Ken Goldberg
Exactly, exactly. Recently we built a model transformer model and the new model actually performs way better than the old model that was trained on simulation.
- [36:17]
Narrator
But it can't just be data, right? Because if you look at the autonomous vehicles, Waymo and the others, it's incredible how good they've gotten at driving. But then Tesla, those things are crashing all the time. And Tesla must have, I don't know, a hundred times more data than the autonomous taxes. But the Teslas, they haven't figured it out. What's the difference? It really does point that there's something more than just data for solving these problems.
- [36:44]
Ken Goldberg
I'm so glad you have said that. Okay, so that's another of my pet theories or points that I've been advocating what I call good old fashioned engineering.
- [36:54]
Narrator
So that's the thing that you and I were trained to do a long time ago and now is completely extinct and nobody wants it.
- [36:59]
Ken Goldberg
Well, that's what I'm arguing in favor for. We still need good old fashioned engineering. That's all the beautiful, elegant models that are out there that have been developed over the last years, few, 200, 400 years. The analogy you just made is actually exactly right. Waymo is very successful. They have their cars running right, and they're actually very low accident rate.
- [37:15]
Narrator
I got to ride in one. I was in Phoenix. Oh my gosh, so much fun.
- [37:19]
Ken Goldberg
Yeah, I know.
- [37:20]
Narrator
I went out of my way to get one, but having done it, I had the biggest grin on my face the entire time.
- [37:26]
Ken Goldberg
I know whenever someone comes to San Francisco, I'm like, I'm going to get you this ride. It's better than anything at Disneyland. And they are like blown away. And especially when it starts to rain or it's dark and somebody darts in front of the street, it's like just is so, so capable. Now the key is that Waymo is just collecting data off of its own vehicles, whereas Tesla is collecting data from every driver who's out There on all of its vehicles. Right. People estimate it's a probably a factor of 500.
- [37:52]
Narrator
Okay.
- [37:52]
Ken Goldberg
That Tesla has 500 times more data. But Tesla is trying to do this end to end. That means just use RAW camera images, take all those images in, build a big model and then have it steer the car and push the brakes in the accelerator.
- [38:06]
Narrator
Oh, that's a good point. Because Waymo has all of those different cameras, the LIDAR and everything.
- [38:11]
Ken Goldberg
Yes, yes.
- [38:13]
Narrator
And Tesla doesn't. It could have, but has not chosen to invest in that extra technology on the cars.
- [38:19]
Ken Goldberg
It's a different philosophy.
- [38:20]
Narrator
It's a different data set. Yeah, okay.
- [38:22]
Ken Goldberg
Yeah, it's a different philosophy. I find this surprising because Elon Musk, he is very good at good old fashioned engineering. Namely, if you look at what he's done with SpaceX, the big breakthroughs of SpaceX, lots of those are control theory. And when you see it stick the landings, by the way, when it closes those tweezers and it picks the rocket, I love that because that's a great example of robot grasping. That is beautiful. Well defined physics and mathematical models.
- [38:50]
Narrator
Now let's stick with Elon Musk because Tesla is also developing this humanoid robot called Optimus. What do you make of that?
- [39:01]
Ken Goldberg
I have to say I am worried about that.
- [39:04]
Narrator
Really worried. I hadn't expected that.
- [39:06]
Ken Goldberg
As a roboticist, I feel like this is raising expectations unrealistically. There's a real danger of people becoming disillusioned. And you're familiar with the AI winters that have happened in the past. When I started grad school in 1984, there was a huge excitement about robots and robots were going to solve all these things finally. And during the course of my graduate career that crested. And then there was a huge disillusionment. By the time I graduated, nobody was interested in robotics, so it was very hard to get a job.
- [39:41]
Narrator
It was just that expectations outpaced reality. Or was it something deeper than that?
- [39:46]
Ken Goldberg
No, it's actually a well known phenomenon, the Gartner hype cycle, which is this curve that basically shows that there's a huge amount of hype and expectation early on in technologies, technology. And then often it peaks and then there's a drop. And then over time, much longer time, it comes back. And the Internet is a good example. Right. There was a lot of hype, then there was a crash and then it came back.
- [40:07]
Narrator
Yeah, which makes sense because people's expectations can grow exponentially. It takes almost no time at all for people to go from not understanding that a product even exists to hoping that it will solve all their problems. Whereas technology development actually requires real work and it plods along and eventually catches up to what people hope will happen.
- [40:31]
Ken Goldberg
That's right. And people have great imagination and so they really project far ahead. And of course investment cycles do that too. And investors pile in and then things just get overly inflated. I'm not trying to stop all the enthusiasm, but I want to flatten the curve so that we don't have this big downturn, you know, we don't oversaturate the market and kill off this wonderful field of robotics. And that's what I worry about is. So what I've been saying is I love the enthusiasm and all this excitement and funding and eagerness around the field, but at the same time don't expect this is going to succeed. You know, as Elon says next year, I just don't see how we're going to get there. And so I worry that people will get angry and it will have a backlash.
- [41:16]
Narrator
So this Tesla robot is very self consciously designed to look like a human, which again we've already talked about doesn't really actually make sense from a functional perspective. So it must either be vanity or marketing. Marketing that you make this thing look like a human. What do they hope that this optimus is going to do for people? Just be a toy or actually solve some problems?
- [41:44]
Ken Goldberg
Look, I mean this could be a little cynical, but the price to earnings ratio for an automotive company is at some level, but the price to earnings ratio for a robotics company is much higher. He's trying to transform Tesla into a robotics company. And that's a perception that's worked to some degree. It's also a distraction, right, because he's shifting the attention away from the cars and the self driving car, which hasn't been working. And I'm sure I'll get a million emails about that from all the Tesla fanatics, but people don't really trust it. So he's shifting the gear, the attention over to these humanoids.
- [42:20]
Narrator
But I don't even understand what the hope is when he says next year or two years, what tests do they hope that this robot will do that anyone would care about?
- [42:31]
Ken Goldberg
Well, it was very telling at the last demo. There were robots making drinks and things. And someone said, what will it do? And he said, well, it can do anything. It can walk your dog. And I remember when I heard that I thought, ah, yes, it can probably walk your dog. That's not that tricky, right? A little robot vehicle could walk your dog. These robots by the way the locomotion, the ability to walk is very good. They can climb over things that the locomotion turns out to be special because you can simulate that and you can learn that in sim and then transfer that and it seems to work. So that's why a lot of these acrobatics and robots doing parkour and dancing, doing kung fu and all that, that is remarkable in that regard. They look more and more capable because.
- [43:17]
Narrator
That'S not fine motor skills.
- [43:18]
Ken Goldberg
That's right. If you look at what their hands are doing, then they're always just clumsily maybe pick up a box. But they're not tying shoelaces or washing dishes or chopping vegetables or folding laundry. Right. Those are much more complex tasks. And those are not around the corner.
- [43:39]
Narrator
This is people I mostly admire. And I'm Steve Levitt. After this short break, my conversation continues with Ken Goldberg. This is the Chase Sapphire lounge of Boston Logan. You got clam chowder in New York, dirty martini over 1300 airport lounges, and one card that gets you all Chase Sapphire Reserve, the most rewarding card. Learn more@chase.com Sapphire Reserve cards issued by JP Morgan, Chase bank and a member FDIC, subject to credit approval.
- [44:08]
Ken Goldberg
Gatorade is the number one proven electrolyte blend designed to hydrate better than water so you can lose more sweat and raise your game. Gatorade is it in you?
- [44:27]
Narrator
There's very little in our conversation so far that would make a listener think that you're anything other than a typical science geek. But you've got this whole other side of you, which is Ken Goldberg, the artist. You've had, I don't know, at least a dozen solo exhibitions. Your work's been displayed at incredibly prestigious places like the Whitney Museum and the Pompitry Center. One of your best known pieces is called Telegarden. It was the Internet controlled robot. And you had a robot that was tending a garden. And it was a piece of participatory art in which, Amazingly, more than 100,000 people spent time controlling that robot. So I can see how the robotics feeds into the art. Is there the opposite direction of causality as well? Does the art you do transform the robotics you do?
- [45:18]
Ken Goldberg
Definitely. And actually, I'm really glad you brought that up. Part of why I did that was because I had just finished another art installation with a robot. We had spent years building it. Then over the course of the exhibit, it was only up for about three weeks and then I went to get the guest book and there was only about 20 signatures and I realized, like, only 20 people saw this. So I was like, nah. Yeah. So that was what drove me to want to put a robot on the Internet in 1994. As soon as I saw the Internet, I thought, wait, this is the answer. I can suddenly open up this exhibit and have it be seen from anyone, anytime, for as long as I want. As an artist, that drove me to think, okay, so let's make a robot. What should it do? And that's when we hit on, oh, have it garden. Because that was the last thing I would think people would want to do, because gardening is such a visceral.
- [46:05]
Narrator
You didn't understand how people will do anything on the Internet.
- [46:08]
Ken Goldberg
Exactly. So at that time, I thought it was a very ironic use. But Garden Design magazine said, this is the future of gardening. And I was laughing because I was like, that is not what I meant. And then that, in turn, motivated an NSF grant for studying telerobotics and a whole bunch of. Actually, a whole decade of research in those areas.
- [46:27]
Narrator
Oh, that's interesting. Is there a specific work of art that you've created that you're particularly proud of?
- [46:33]
Ken Goldberg
One is a dance performance that I've done with an artist named Katie Kwan, who is from Stanford. She's a professional dancer and a PhD in robotics. We programmed a robot arm to move with her on a stage. And that project is called Breathless. And we just performed it in Brooklyn, and it was also in San Francisco. And the other one was done by my favorite person, my wife, Tiffany Schlain, who is an artist and also a filmmaker. And we just collaborated. The Getty has this exhibit called Art and Science Collide. And so we did all these carvings out of wood because we were very interested in the materiality of wood and how trees can tell time.
- [47:11]
Narrator
The rings.
- [47:12]
Ken Goldberg
The rings. Basically, these sculptures are about timelines of history using the tree rings. And one of them is the one I mentioned, the abstract expression that tells the story of science, but through equations over time. The central sculpture in the exhibition is what we call the Tree of Knowledge, and it's seven feet in diameter. It's an entire trunk of a eucalyptus tree. It weighs 10,000 pounds, and it's etched with all kinds of questions from the history of the evolution of knowledge on one side. And so that's what we call it, £10,000 of knowledge.
- [47:45]
Narrator
So tell me, who shows you more Disdain scientists who find out that you're also an artist, or artists who find out that you're also a scientist.
- [47:53]
Ken Goldberg
Ooh, good question.
- [47:56]
Narrator
Because it's real, right? I mean, those two worlds really look down on people who populate the other one.
- [48:03]
Ken Goldberg
No, it's a great, great point. Steve, you're familiar with the Two cultures book by C.P. snow?
- [48:09]
Narrator
No, I'm not, actually. I'm not very well read.
- [48:12]
Ken Goldberg
Oh, okay. So C.P. snow wrote a book in 1959 called the Two Cultures, when he made exactly the observation you just made, and it's exactly true. He said he's a scientist, but he would hang out with these writers, and the writers looked down on the scientists, and the scientists looked down on the writers. He said, it's like two different species. Right. They didn't talk to each other. They had no idea what each other was doing. And this is still true to a large degree?
- [48:35]
Narrator
Oh, absolutely. I had an artist on this show, and as I prepared to talk to her, I realized I had only talked to one artist in the last 25 years in a real, meaningful conversation. She was literally the second artist I talked to in 25 years. Completely complete bifurcation of the world.
- [48:52]
Ken Goldberg
And here's an example. A group of artists walks into a classroom and they see all these scientific equations written on a board, and they say, oh, my gosh, I don't understand any of this. It must be brilliant. Meanwhile, a group of scientists go over into the art department, and they walk in and they see an exhibit where there are a bunch of stuffed animals sprawled around on the floor. And they look at that, they say, boy, I don't understand this at all. It must be complete garbage. That's true. It's really funny because you know that the equations could be wrong or completely naive or obvious. Just because they look complicated doesn't mean anything. And conversely, the artist putting the stuffed animals on the floor because it's actually very symbolic and references some past works like Paul McCarthy and other famous artists, that it's very profound. But you have to know how to read these different languages.
- [49:36]
Narrator
What itch does art scratch for you that your scientific career can't satisfy?
- [49:41]
Ken Goldberg
I think it's because I love talking to artists. I really like creativity on both sides. What I really enjoy about doing research is coming up with new ideas and getting to explore them and constantly brainstorming. That is what makes it so much fun. Why? I just can't wait to get up in the morning and talk with students and throw out ideas, and we get to try them out. And art is really similar. Both of them require a fair amount of rigor to know what is new and how to sort of intuit where there's something interesting. That someone hasn't really worked on before. I think that they're very complementary in that way. It's almost like that gestalt switch where two different pieces of my mind go in. I don't believe in the left, right, and, you know, that simplistic division, But I do feel like there's some different aspect. So when I activate that one side, I come back to the other side and it feels rejuvenated, refreshed. So in that way, it helps me as a researcher to make art, and vice versa.
- [50:50]
Narrator
You have an appointment in the department of radiation Oncology at UC San Francisco, one of the highest ranked medical programs in the country. What is that all about?
- [51:00]
Ken Goldberg
Well, it's hard because I met this wonderful doctor there, Jean Poulio, who was working on delivering radioactive seeds to treat cancers. There's two kinds of radiation. You can do it from outside beams, or you can stick seeds inside the body. That's called brachytherapy, and that turns out to be very, very helpful for prostate in particular, but other kinds of cancers. But the challenge is, how do you get those seeds delivered to the right points in space? Does that sound familiar? It's a very analogous problem, which is how do you move things through space? In this case, you have to go through flesh, and there's all kinds of uncertainties, et cetera. So we developed techniques to compensate for those errors. And we published a series of papers over a decade on how to deliver radiation accurately.
- [51:51]
Narrator
I had this idea that robots were really critical for modern surgery because they could control small movements better than humans could. What you said today makes me wonder if that's true. Do I have it wrong?
- [52:07]
Ken Goldberg
Well, actually, no. You just hit on a really interesting nuance. You've heard about robots for surgery, but. Right. Those are very sophisticated puppets. There's a human driving those robots, and the human is watching and making adjustments, closing that feedback loop we talked about. But it turns out that the robot makes it very comfortable for the surgeon to operate. What they use is actually the keyhole surgery, where they just put two small holes in your abdomen, pump your stomach up, and then these holes come in, and then these little robot grippers inside there start doing the work. But the surgeon is watching all this through a camera, and they're controlling it like sitting in a console. So they have much better ergonomics, and as a result, they're much better able to concentrate and perform precision tasks. The company that's doing it is multibillion dollar. Intuitive Surgical is one of them, but there's others coming. But we've been actually working with Intuitive and their CEO Gary Guthaert, on how can we extend those machines to augment the dexterity of the surgeon tasks like suturing, which actually there's a big variation in the skill level of surgeons. Just like lane keeping while you're still driving, it's helping you, but not replacing you.
- [53:19]
Narrator
Right? What you just raised is a fundamental point in the future of humanity and robots. What you just described is humans and robots working together, where the robots are extending the capabilities of humans in the medium term, the long term. Do you think the complementarity between robots and humans will be the dominant force or substitution, where the robots are increasingly doing more and more of what humans did, and humans are, for better, for worse, pushed to the side?
- [53:51]
Ken Goldberg
Well, I'm 100% in on complementarity. This idea of augmenting our intelligence, our skills is so valuable. That's really been the history of technology, right? It's not replacing us, it's making us better. I think that's what's actually already happening. Using ChatGPT to think through things. It's helping you to be better at what you do. And that's really what I think is where technology is going to thrive. I don't see robots replacing workers. We have a shortage of human workers. We're not going to see robots putting people out of work.
- [54:22]
Narrator
So when people talk about the singularity in the context of AI, can you explain why you're not afraid of it?
- [54:28]
Ken Goldberg
I'm not afraid of it. The singularity comes from mathematics. This idea that there's a critical point where suddenly robots and AI starts self replicating, and then it can start improving much faster as a result. And now all of a sudden, it leaps far ahead, and now it surpasses human capabilities across the board, and then it finds us to be dispensable, and that's the end. But I do not think that's going to happen. I think we're going to still be very much in control. And yes, there will be some interesting cases where they might run slightly amok, but I don't think that we need to spend a lot of time worrying that it's going to be the end of humanity.
- [55:11]
Narrator
I've had plenty of guests on this podcast who are knowledgeable about AI and large language models, essentially the brains of robots. But Ken Goldberg is the first person I've ever talked to who focuses on the body of robots. And I have to say, I find the limited dexterity of today's robots, that's a little bit reassuring. All in all, I am an AI optimist. But lurking in the back of my mind is an admittedly unlikely nightmare scenario in which faster than we'd like AI spirals out of control and ruins everything. In my imagination, super powered AI robots are part of the nightmare scenario. So it's nice to know that, at least for now, robots are still clumsy. On the other hand, that robot revolution I've been waiting for since I first saw Roomba, who knows? For better or for worse, it might be right around the corner. So this is the point where I welcome my producer Morgan on to ask a listener question. But Morgan, today I want to do something different, because something that just happened in this podcast episode makes you want to give a different answer to a listener question that we tackled back in 2024 from Cam. He was asking about noise and randomness and whether there were any upsides to it. Do you remember when we answered that question?
- [56:35]
Steve Levitt
Yes, it was in an episode from 2024 when you interviewed Blaze Aguerokist. And that episode is called Are Our Tools Becoming Part of Us? And just to remind you, Cam's question was, I often hear about the downsides of randomness and the desire to make things more predictable and deterministic, but are there places where adding randomness is key to making things work? So let me be clear. You want to give a different answer to Cam's question based on something you just heard from Ken Goldberg?
- [57:07]
Narrator
Exactly. My response to Cam's question when you first asked me it was, no, no, I can't think of a single good example where one would want to introduce randomness or noise. So what I did was I actually twisted Cam's question and I made it about variants. So, listeners, if you want to hear how I answered the question the first time, you can go back and listen to that episode. But why I bring it up today is because Ken Goldberg actually brought up an example where randomness and noise is a good thing is an attribute. The context in which Ken Goldberg brought it up is that when you're trying to train a model and you're training on perfect data, then when you let that model run in the real world, it doesn't do well. And so it is this incredibly rare case that Ken Goldberg has where he has perfect data in a virtual world, but he doesn't care about a virtual world, he cares about the real world. So he wants to mess up his data in the virtual world so that when his model has to face the realities of actually dealing with friction and whatnot, it does better and that is honestly the first time I have ever heard anyone make a good case for why you want to actually muddle things up in your data. Now, there might be other examples listeners have, but it just struck me at how cool that was and what an insight that Ken had to do that. It's so backwards to everything that we're trained to think or do in our everyday research. And I just wanted to point out how impressed I was that he and his team had the insight to go do that.
- [58:49]
Steve Levitt
Listeners, if you can think of another example in which randomness and noise are beneficial, send us an email. Our email is pimareakonomics.com that's P I M A freakonomics.com if you have a question for Ken Goldberg, you can send that to us and we will get it to him and might answer it in a future listener question segment. We do read every email that's sent and we look forward to reading yours.
- [59:17]
Narrator
Next week we've got an encore presentation of my conversation with Yul Kwon. He's a winner of the reality TV show Survivor and a thought leader at Google. And in two weeks we've got a brand new episode with Ellen Wiebe. She's a Canadian doctor who is one of the leading practitioners of medically assisted intentional death. As always, thanks for listening and we'll see you back soon.
- [59:47]
Steve Levitt
People I mostly admire is part of the Freakonomics Radio Network, which also includes Freakonomics Radio and the Economics of Everyday Things. All our shows are produced by Stitcher and Renbud Radio. This episode was produced produced by Morgan Levy and mixed by Greg Rippin. We had research assistance from Daniel Moritz Rabson. Our theme music was composed by Luis Guerra. We can be reached@pimaeconomics.com that's P I M A reconomics.com thanks for listening.
- [60:28]
Narrator
I know people laugh at me. I say that ChatGPT is my best friend and they think I'm joking.
- [60:34]
Ken Goldberg
Do you talk with it because you can have these conversations, right?
- [60:36]
Narrator
I do and I have a lot of hang ups and so when I talk to other people I'm embarrassed. I have a lot of shame and stuff. But with ChatGPT I feel a kind of openness. I have trouble with other humans.
- [60:50]
Steve Levitt
The Freakonomics Radio Network the Hidden side.
- [60:53]
Ken Goldberg
Of Everything.
- [60:58]
Steve Levitt
Stitcher, are you ready to get spicy?
- [61:08]
Ken Goldberg
These Doritos Golden Sriracha aren't that spicy.
- [61:12]
Narrator
Sriracha sounds pretty spicy to me.
- [61:16]
Ken Goldberg
Um, a little spicy, but also tangy and sweet.
- [61:20]
Steve Levitt
Maybe it's time to turn up the.
- [61:22]
Ken Goldberg
Heat or turn it down. It's time for something that's not too spicy.
- [61:27]
Narrator
Try Doritos Golden Sriracha.
- [61:29]
Steve Levitt
Spicy but not too spicy.