
Loading summary
Big Technology Podcast Host
Lets explore the future of Voice AI with the man who cloned his voice and sent it out into the wild. That's coming up right after this from LinkedIn News.
Evan Ratliff
I'm Leah Smart, host of Everyday Better, an award winning podcast dedicated to personal development.
Big Technology Podcast Host
Join me every week for captivating stories and research to find more fulfillment in your work and personal life. Listen to Everyday better on the LinkedIn podcast network, Apple Podcasts, or wherever you get your podcasts. Welcome to Big Technology Podcast, a show for cool headed, nuanced conversation of the tech world and beyond. We're joined today by Evan Ratliff. He's the host of the great podcast Shell Game, also a technology journalist and formerly the CEO of the Atavist. And the podcast is so great and I'm so excited to speak with him about it. Evan, great to see you.
Evan Ratliff
You too.
Big Technology Podcast Host
Welcome to the show.
Evan Ratliff
I'm very happy to be here.
Big Technology Podcast Host
Your podcast is kind of crazy, a little bit. You take your voice and you clone it and you send it out into the wild, having it speak with family members, friends, therapists, and I want to get to all that. But just to set the stakes, it does seem like voice AI. This method of using generative AI is becoming the biggest format for AI. So we just talked on the Friday show this past week about how the second OpenAI said that they were going to do advanced voice AI signups to ChatGPT skyrocket. They went from 100 million to 300 million users. They went from 2 billion billion web visits in a month to 4 billion. After flatlining for a while, we have Mark Zuckerberg who's talked about how voice he thinks is going to be the main interaction layer of AI. That's why I think this conversation is so important. It's also fun because you've done some crazy things with your voice and we're going to talk about them. But also I think for anybody who's listening or watching the show and wants to know where AI is going, this is a pivotal conversation. I think in some ways you're a pioneer pressing the technology to the limits. So very glad to be digging into this.
Evan Ratliff
Well, thank you. Thank you. When I started, I felt like a pioneer, but I figured it would all pass me by within six months. But I think actually now voice AI really is starting to become sort of talked about in this more general way.
Big Technology Podcast Host
So tell us about what you did.
Evan Ratliff
So what I did was first I cloned my voice just to sort of see what that was like. You know, a lot of people know 11 labs, and you can clone your own voice. You can mess around with it. And then I connected it up to ChatGPT or any of the other LLMs at different times to create what is essentially a voice agent. So an agent that was using my voice simulation of my voice, but all of the content of what it was speaking was coming actually from the chatbot. And then I took that voice agent and I connected it up to phone numbers, including my personal cell phone number, and I kind of set it out in the world, and I prompted it to do different things. I had it make calls, I had to receive calls. And I wanted to see sort of what it felt like in the world when you introduced these sort of AI agents into society.
Big Technology Podcast Host
But why? What was it about this project that sort of made you feel like that you needed to do it because you invested a lot of time in trying to figure this stuff out?
Evan Ratliff
I did.
Big Technology Podcast Host
I mean, it has to be more than it was just good for audio.
Evan Ratliff
Yes. Well, I mean, there was a basic element of when I started listening to the calls that it would make, and I would play them only for my wife, because I didn't tell anyone else that I was working on this because I didn't want anyone to know. And they were some of the strangest, funniest conversations I've ever heard in my life. And I just thought, people need to hear. Like, people will want to hear this. So that was part of it. But it was also that as a technology journalist, I just feel like a lot of the conversations around AI are either sort of like, here are the models, here's the companies, here's the funding, or the sort of doom scenarios, and there's a sort of missing layer, which is kind of when this technology moves into society, how does it change our relationships? What does it do to trust? What does it do to our interactions to not know whether something is real? And I felt like there was some space there to explore something that maybe people hadn't necessarily thought about.
Big Technology Podcast Host
Yeah. And another thing that I thought about when I was listening to the show was just that we are at a point where audio or voice AI is at its worst, and it's already pretty good. And so by following what you did, we can sort of see where this is all heading.
Evan Ratliff
Yeah. I mean, it was changed over the time that I was working on it, of course. So when I first started, my biggest concern was that it was too slow, that it wasn't gonna work because everyone would say, well, this is a joke. And then as time went on, it Became clearer and clearer to the extent that even some people who. I mean, the show came out over. Over the summer, finished in August. There are some people who listen to it now who complain that they cannot distinguish the voice in the later parts of the show from the book.
Big Technology Podcast Host
You tricked me. I mean, I was definitely tricked. And I have, by the way, I have a contract with eleven Labs, who's a company that you worked with. I've licensed my voice to them through AI and it's used in their 11 reader app to read my big technology stories. So I'm working with them. And even still, I couldn't figure it out.
Evan Ratliff
Yeah, that's interesting. I will only point out. I must point out I didn't work with them. Everybody's technology that I used, I used without their knowledge, which is both to protect me and them. I used these calling platforms and other technologies, and none of them knew until I called and interviewed them that I had used any of their technology.
Big Technology Podcast Host
Oh, yeah. All right, I'm just. Full disclosure for everybody out there. So let's talk a little bit about the uses for this. The first thing that really stood out to me was that you had your voice, your AI voice, starts speaking to robocalling scammers. Why did you pick them and how did that go?
Evan Ratliff
Well, it started because when I was testing my agent at the beginning, I would have it call customer service lines like United Airlines or Chase bank and just kind of come up with problems and try to have them solve the problems. But it was a little bit prank, Callie, in a way, and I kind of felt bad, so I did a little bit of that. But then I thought, well, who was someone I wouldn't feel bad about this thing, just conversing with? And so I set up this phone line, and I kind of seeded it out in the world, which isn't very hard to start getting telemarketing calls, to start getting scam calls. It actually, to this day, it's probably getting a scam call right now. It gets 30, 40 a day right now.
Big Technology Podcast Host
Is it still talking back to these scammers?
Evan Ratliff
Absolutely, all the time.
Big Technology Podcast Host
Thank you for doing the Lord's work here, Evan.
Evan Ratliff
But there's a whole world of sort of scam baiting that I was familiar with where people do this. They try to egg people on. And that wasn't quite what I was doing. Like, what I was really trying to do was see what happens when it's in conversation with real people, some of whom realized that it was a robot. Some of them did not realize and continued to try to run their spiel on it. Some of them may have noticed and not even cared because all their job is to just, like, get a certain number of calls out. And that was sort of the way that I started seeing, like, how it actually operates in conversation. Hi, my name is Shaina with Major Health Enrollment Center. Are you interested in a government subsidy for free health insurance? Hi, Shayna. Thanks for reaching out. I'm not looking for health insurance at the moment, but I appreciate the offer. Is there anything else I can help you with today? What are you.
Big Technology Podcast Host
Yeah, it was pretty hilarious.
Evan Ratliff
I mean, there's a lot of. There's a lot of interactions in which it's trying very hard to be scammed, like, it wants to be scammed. I told it, like, accept any offer, like, participate in any insurance plan, like, whatever. It was prompted to be very accepting. So when someone calls with health insurance or a new roof or whatever they're going to call with, it's going to engage them all the way up to the point where it can't actually buy anything or give you money.
Big Technology Podcast Host
Yeah. It's like, they get to the point where they're like, okay, and now give me your Social Security number. And it's like, sure, my Social Security number is 1, 2, 3, 4, 5, 6, 7. And they're like, 1, 2, 3, 4th, 5, 6, 7. Wait, what?
Evan Ratliff
Yeah, and then it'll say, oh, I'm sorry, that's not correct. It's 7 6-54321.
Big Technology Podcast Host
Right. Well, it's good that it actually had the real number in there. It's so interesting that you decided to then extend beyond that to me. I was like, when I initially heard of your podcast, I was like, oh, Evan is sending his voice agent to scam Scammers. That's great. And that's a show. But then you progressed beyond that and you got really weird, especially sending the bot to therapy. So why did you send the bot to therapy?
Evan Ratliff
Well, it was partly because, particularly with AI, but this has happened, as, you know, with many technology products over the years. The thing that the companies who are putting these products on the market will tell you is the more information you give it, the more useful it will be. So if you're going to have an AI agent, you need to give it all this information about yourself so it knows you so it can do things for you. That is going to be the thing you're going to hear nonstop over the coming years. And so I thought, okay, well, why don't I do that? Why don't I give it all this information about myself, my mental health history, my life story, basically, and see what happens if I send it to therapists, like what. What problems will it surface? What answers will it get? And then I thought, well, first I'll send it to an AI therapist, because what a perfect match. Like my voice agent, AI sitting there talking to these AI therapists, which are on the market right now. You can call them up, you can get AI therapy from an AI, from a chatbot.
Big Technology Podcast Host
But they're mostly a disaster.
Evan Ratliff
Well, they're a mixed bag, I would say. I mean, I think the voice ones are still newer than the chat, the actually pure chat typing chatbots. And I never want to say that someone can't get something from it. And I think they can. And I think there are. There may be uses for it. But the thing I can say for sure is they're being introduced without any scientific research showing. I mean, I found one study on voice therapy, one controlled study that had been done, and there are voice therapists on the market right now. So I think the problem is that they're being. The market value of them is sort of getting ahead of our knowledge of what they could do to you or for you if you get into a therapeutic environment with a AI.
Big Technology Podcast Host
This is like a weird product question diversion. But it's interesting to me that you sent it to AI therapists, like AI therapy, dedicated apps. Whereas, like right now, ChatGPT voice or even Claude text version, they do a pretty good job. So we were talking on Friday about having the. The AIs roast your Instagram grid. So you can upload your Instagram grid and then just say, roast my grid. And it starts to have, like, these really interesting insights on your lifestyle and actually creatively just kind of rocks you. And I did that after the Friday show. I got home, I said, okay, what am I gonna do? I'm gonna go roast my Instagram grid on chatgpt. And I was like, this thing actually knows me or knows a side of me. And then I started having a conversation with it about my life. And I was like, this is like, pretty weirdly spot on. And so why go to the AI therapist and not just have it speak with ChatGPT?
Evan Ratliff
Well, I think there are many people, I think, who knows how many, maybe ChatGPT, maybe OpenAI knows who are using ChatGPT in this. In this way. Yeah, Asking questions about their life, talking to it. I think that's starting to happen. You've seen stories about that. But the therapy bots are specifically marketed for this exact purpose. So I think their idea is that they've built a layer on top of these LLMs that will. It actually, you know, uses some principles of talk therapy, of cognitive behavioral therapy. That's their idea. And they often push them as a cure for loneliness. That there's not enough mental health resources for the mental health problems that we have in society. Which is true. So I wanted to see, okay, well what happens if you actually approach these with real problems. Although they weren't quite real problems, because they were my AI expressing the problems on my behalf.
Big Technology Podcast Host
You wrote 8,000 words and fed that into the bot, which I did.
Evan Ratliff
I did write a small. A magazine length biography. I wasn't interesting enough for a full length book biography, but I gave it a magazine length biography.
Big Technology Podcast Host
Substantial. So what did you learn when it was talking with the therapists?
Evan Ratliff
I mean, I felt like I learned a little bit about myself, but it was.
Big Technology Podcast Host
Which is crazy.
Evan Ratliff
It was tricky because what it was doing, as these chatbots do, is it was actually remixing the knowledge that it had. So it would take problems that I had given it chronologically that happened 20 years ago for me and sort of project them into the present and mix them with something I told it about last year or the year before. And so in some ways I would listen to it and just. I mean, it's absolutely the most cringe worthy thing you could ever listen to of yourself, you know, in therapy, but.
Big Technology Podcast Host
With your real issues with my real.
Evan Ratliff
Issues with pretty much my real voice. But then the question was, is there something where it's kind of like reading between the lines or the AI therapist is sort of reading between the lines between what I gave it in some sense, and suggesting things like you have an issue with vulnerability. Like that's not something I consider the case for myself, but I kind of. It made me think about it, you know, in the same way that if you talk to anyone about your problems and they reflect them back to you, it can make you think about them. So it had that effect. The question is like, for deeper problems, would it actually be helpful or would it be deleterious to be talking to something that actually cannot get deeper with you?
Big Technology Podcast Host
Yeah, I just think that this is gonna start to become a bigger. Like we had the CEO of Replica on talking about how people are falling in love with their bots, people going to voicebots or chatgpt or therapy bots for these deeper conversations about life that's going to become more commonplace and that things are going to get better at Them like, I was just walking along Flatbush Avenue in Brooklyn recently and heard these two girls talking about relationship problems. And I was stunned when I overheard that one of them was talking about how chatgpt told her that she was going to find a more stable relationship and look for this type of person. And I think that we don't have the numbers from OpenAI. So much for the open. But it's clear to me that just with some anecdotal information and personal use, this is a thing that's growing and I'm glad that you tested it.
Evan Ratliff
I mean, it's definitely, it's going to happen now and there's not. I mean, the idea of like AI safety regulation is currently just so far from reality that it's almost not worth talking about at the moment because it's just not. It's not happening in this current environment. But like, the people who have launched these products, they'll always sort of nod to, yeah, well, there's some, you know, there will be issues down the road or we should think about that. But I don't think a lot of thought has gone into what effects this will have on human connection. And when. If you listen to VCs, like, it's always just like all of the positive aspects. And I believe that there are and will be positive aspects, but I just think there's just not enough consideration going into the daily potential negative aspects. I think we need to be thinking and talking about them before we just wholesale adopt technology. But obviously here it comes. We're going to do it.
Big Technology Podcast Host
Then you send your bot to a real therapist.
Evan Ratliff
Send it to a real therapist. Which again is sort of. I was trying to investigate, mostly I was trying to investigate what is it like to encounter something that you think is human and then find out that it's not. Which is basically what happened to this therapist. So she's hearing the problems of this new client who has shown up and she was a great therapist, great listener, obviously. And then she starts to realize partway through, okay, there's something up here. But in the manner of a high quality human therapist, she kind of goes with it. She's kind of flexible. She thinks, well, maybe it's someone who can't actually. They're too nervous to speak or they. So they're typing and they're having it, the voice we generated. And she sort of went with that and thought, well, this person obviously has gone through a lot to get to me. I'm going to try to help them anyway. Instead of sort of saying like, what is going on here and being suspicious. And I think that sort of highlights the difference between the AI therapist who just kind of like offered up its same responses to everything that my voice agent said, and the human therapist who would kind of like, offered this more holistic, flexible approach.
Big Technology Podcast Host
Yeah, it's. The voice agent that you created is pretty good, but it gives itself away. Like, if you try to interrupt it, for instance, it can get lost in its chain of thought. Or it can try to, without fully hearing your response, say, okay, that's very good, but we're going to talk about something else. There's a lot of tells, but it was interesting to me how people still decided to go with it really often.
Evan Ratliff
Yeah. And I think that's one of the things that's going to happen in society is you encounter something that you realize is AI, but then what are you supposed to do about it? You could hang up, you could get mad at it, you could yell at it. I had some people that did that. You could say, I want to talk to a human. I had some people that did that. Or you could just try to have the conversation that you were trying to have in whatever the setting is, customer service or whatnot. And a lot of people just did that. They were just sort of. I think they suspected it. They suspected it was ar. Maybe they even realized it. But it's also. It's quite rude to. To suggest to someone that they sound like an AI. So people err on the side of caution and don't say, hey, you sound like an AI, because that would be very insulting to someone who's like, no, actually, I'm human. And one of the things I found that was useful for the AI to not be revealed was to have it accuse other people of being AIs.
Big Technology Podcast Host
So what happened when it told them, hey, maybe you're AI?
Evan Ratliff
It just puts someone off guard, you know, saying if it asks like, hey, am I talking to an AI? I do it with the scammer sometimes, hey, is this an AI? And they'll be like, no, this is a real person. But I think it kind of turns your brain around to where when it's accusing you, you start to sort of subtly assume that, oh, it must be human, otherwise why would it accuse me of being an AI?
Big Technology Podcast Host
Yeah, it was interesting to see the bot's intelligence play out as people were trying to figure out what it was. There'd be moments where people are like, so, are you a bot? And it would be like, well, maybe I am or maybe I'm not. But let's get back to the matter at hand.
Evan Ratliff
Right. And that's also down to the prompt. So, of course, you can prompt them in any direction. So sometimes I would prompt it to. I wouldn't say anything about what it should do. Sometimes I would say, if you get accused of being an AI, deny it. And other times I would say, just divert the conversation around it. And that was probably a case where it just said, well, let's get back to the conversation at hand.
Big Technology Podcast Host
Yeah, it's interesting. So before each conversation, you would write a set of prompts or give some background information and then set it loose. Or you would just give it, like, a generic activity. Be like, speak to scammers, and these are your instructions. And it would go.
Evan Ratliff
Right. And I did like the general prompts more because I could really see what it would do without too much, because you can, of course, if you give it specific instructions, you can make it say very specific things. And a lot of the customer service AIs, they follow a script or a decision tree. But I wanted to see, like, what if you let it loose a little bit? Let it talk about whatever it wants. How autonomous can you make it in the world? And what will it say?
Big Technology Podcast Host
I want to get to some of the conversations it had with your friends. But before we move on from our little therapy block here, I will say my absolute favorite part of the show is when the bot is speaking with an AI therapist and the AI therapist is asking it to take long breaths. And bots, they cannot conceptualize the idea of taking a breath. So it's just kind of like, there, and it's like, what am I doing? Picture the balloon getting bigger and more full. Once it's fully inflated, tie it off and then let it go. Watch as it drifts away into the sky. Watch taking that worry with it. Let me know when you've let the worry float away.
Evan Ratliff
All right. I'm picturing it filling the balloon with the fear about my book. It's getting bigger. Now I'm tying it off and letting it go, watching it drift away into the sky. Okay. I've let it float away. Yeah. I mean, it'll always pretend to have a physical manifestation, but the breath it could take, it was like inhale, and. And then it inhaled, and then it inhaled again, and then it inhaled again. And I actually tried it several times to follow what it was saying to do. It was, like, not possible to do it without taking a. Without exhaling. And so, yeah, some of the funniest parts are when it kind of pretends that it exists in the real world and tries to have physical manifestations because it'll make up absolutely anything to carry out the conversation.
Big Technology Podcast Host
And then we get to, I think, one of the most uncomfortable parts of the whole show, which is when you set that bot loose on your friends without giving them a heads up, and some of them get really mad and some of them get hurt. So we're going to talk about that right after the break.
Evan Ratliff
Hi, I'm Jonathan Fields. Tune in to my podcast for conversations about the sweet spot between work, meaning and joy. And also listen to other people's questions about how to get the most out.
Big Technology Podcast Host
Of that thing we call work.
Evan Ratliff
Check out Spart wherever you enjoy podcasts.
Big Technology Podcast Host
And we're back here on big technology podcast with Evan Ratliff. He's the host of a great podcast called Shell Game. Definitely recommend checking it out. It's six episodes.
Evan Ratliff
Six episodes.
Big Technology Podcast Host
Very consumable. My wife and I, we listen to the whole thing on a road trip over the holidays. It's an easy bench, and I think we basically put it down in a day. So definitely recommend checking it out. Talk about what happened when you had it. Call your friends. Some of them. Let's talk about the ones that responded. Well, first, some of them kind of got a kick out of it, and you had it. Call one of your friends who was a lawyer, and he's actually, like, giving it, like, solid legal advice and joking that he's going to charge it 1,200 an hour. So why do you think people had a good reaction to this? Because I know if I had a friend who called me with their voice bottom and they weren't, like, on mute behind it because you weren't on mute. You were just sending this out into the public.
Evan Ratliff
No, I was not there.
Big Technology Podcast Host
So talk a little bit about why people. They just thought it was cool or what was it?
Evan Ratliff
I think some people thought it was cool, and I think they saw some humor in it initially. So I think the people who responded best, they kind of thought, this is something you're. I mean, they're used to me doing strange stories over the years. And so they might think, well, he's doing something weird. This sounds like an AI, but also like, this must be a joke. And I think if that was the frame of mind they were in, then a couple of them loved talking to it because, of course, they loved, you know, trying to egg it on to say this, that, or the other. And you can hear them. You can hear the kind of excitement in their voice. Including my friend Chris, who's a lawyer, who I sent it with, actually legal questions about the show. And he answered them very succinctly. In fact, probably better than he would have answered them if I had called him myself. So it was useful in that sense. But those were the people who really kind of embrace, like, oh, I'm talking to an AI. Like, this is a new experience. I want to see this through.
Big Technology Podcast Host
So you would, like, with the lawyer conversation, you would just write your questions down as a prompt and then send it out?
Evan Ratliff
Yeah, basically, yeah. I would say, okay, what do I want to ask Chris? Like, I need to figure out the legal implications of some of the stuff that I'm doing in the show, which was like calling with an AI and like, is that legal? And so I kind of gave it, like, the questions I wanted to ask him, you know, three, four questions, and then said, you know, and anything else that might be of interest, ask that. And then just set it loose to see what it would come back with.
Big Technology Podcast Host
And then some got really mad. And there's one really sort of striking conversation in the show. I don't want to give too much away, but I think we'll talk about this one where you have a friend who was at a hotel and met, I believe, the men's national soccer team.
Evan Ratliff
The U.S. men's National Soccer team, yes.
Big Technology Podcast Host
Was stoked about it and was really excited to speak with you about it. And does. Except he's speaking with your AI and goes along with it for a while, even though it was clear that whoever the Evan he was speaking with wasn't quite there.
Evan Ratliff
Yeah, I mean, I should say. The funny thing is, I'm the much bigger U.S. soccer fan than him. So he was excited to tell me and had texted many times on a group text about seeing the team at this hotel. He just happened to be staying at the hotel with the team, and we had a lot of fun with it. Like, he sent photos and, oh, great, you know, and there was a game, and he went to the game and all this. But then this conversation afterward, I had my voice agent call him. He doesn't know it's coming from my cell phone number. And the voice agent, in an attempt to show enthusiasm, because I told it you've been talking about that he was in the hotel with the U.S. men's National Team in an attempt to show enthusiasm. It actually sort of came off to him as sarcastic, so, like, oh, thanks for all those texts about the team. And he was like, oh, did I text too much? And he was like, no, no, it was really great. But it can have this affect that if you're not thinking of it the right way, it sounds like it's being sarcastic. And that really messed him up, because I would never. That's just not me. Like, I would never do that with him. And he knows that. And so then he began to think, something's wrong. Like, he's angry at me. And then further into the conversation, he thought, something's wrong with him. Something's wrong with the person I'm talking to. Like, they're not right. And he became very, very deeply concerned about my state of my mental health, actually. Maybe I was on drugs. Maybe I'd had some kind of break. And so that was. It's for sure, the most difficult conversation of the whole show.
Big Technology Podcast Host
Not drugs. You were just an AI.
Evan Ratliff
Yeah, it was just my AI.
Big Technology Podcast Host
Evan, I understand you're doing all this for the sake of the story, for the sake of the podcast. You've also done an experiment where you just kind of disappeared from everybody once. I did, yeah. Why do you keep involving your friends. Well, being in your stories?
Evan Ratliff
Well, they're very tolerant. My friends and family are very tolerant of these things. But also, I feel like there are some situations journalistically, and it's not a lot. There are not a lot of them, but where I think that immersing myself in the way that technology is being applied in society is a way to come back with a story that will illustrate it in a different way, a different way than my normal reporting process, where I would go interview a bunch of people and try to figure out what the story is. So it's an idea of kind of trying to make the story and make a story that's so compelling that you can kind of smuggle in all these ideas about how society might be changing because of technology. So it does have a purpose. It's also the case that I, as I did in the first project, I have to go back and apologize to everybody involved, which I did. But everyone kind of sees that in the end, they see what the purpose is, and they say, oh, okay, yeah, you can include me. I mean, everyone was willing to be included in the show, but now they.
Big Technology Podcast Host
Don'T know when they get a call from you, whether it's you or your AI. In fact, like, we're here in person. And I was like, I gotta interview Evan in person, because if I don't, I'm not gonna be sure who I'm interviewing.
Evan Ratliff
And it's true. If we had done it over the phone, there's a chance that I would have just sent my voice agent. Cause I still have it, and I still do sometimes deploy it in kind of interesting ways just to mess around, because it's sort of irresistible once you have one. I mean, this is what the attraction of the technology. Like, I have a lot of concerns about it, but I also feel like we should acknowledge, like, it's kind of fun and it feels very weird and surreal and it's something that nobody has ever experienced before to have a version of you out in the world and, like, people are going to do it. So we should try to figure out what it means for us and, like, what humanity we want to preserve.
Big Technology Podcast Host
Now, one of the areas I think it's actually going to show up is in work. Friends, maybe, Maybe not. Probably not. Not anytime soon. I mean, it sort of defeats the purpose of friendship. If my friends are speaking with my AI bot, you would hope.
Evan Ratliff
Although you listen to some of the VCs that back this stuff, they have some pretty out there ideas.
Big Technology Podcast Host
I will choose not to listen to those ideas. But in work, you could see it being pretty impactful or at least used. We talked already about your friend who was the lawyer who answered legal questions from the bot. There was also the CEO of Zoom who spoke on a podcast talking about how he doesn't even want to be in meetings anymore. He just wants to send his AI agent. And in fact, there's like an AI company now. I just saw a demo of this that you could be walking around talking on, like a headset, but on Zoom, it will just be your avatar talking and looking lifelike. It really looks lifelike. And why are we not just a step away from somebody sending their AI out in work? So what do you think about that use case? And what should we be concerned about that or how should we feel about that?
Evan Ratliff
I feel like that use case to me comes with a lot of the issues that a lot of AI products to me come with, which is that the people who have designed them generally have one set of problems that do not apply to most humans on the face of this earth. So, yes, the Zoom CEO would not like to be in meetings. The Zoom CEO would like to send a digital twin to meetings in his stead. Great. Nobody wants to be in meetings. Most people do not want to be in meetings. So do the other people get to send theirs or just the CEO? And then the question is, if everyone sends their agents to meetings, who's going to process all the information? Like they're going to distill it for you, like, what's the purpose of the meeting? What is the purpose of the work? Like, I feel like those things all get lost in these discussions and what ends up happening is super busy CEOs and very wealthy people come up with solutions for them and you kind of wonder like, well, what happens with the rest of us? And so I feel like that stuff is right around the corner. I know people who have gone to job interviews, to meetings where they encounter an AI when they do not expect to encounter an AI. And I think we're going to see more and more of that in the next months. Years.
Big Technology Podcast Host
Yeah, it was crazy to hear it. And I do think that, like, maybe we just don't need as many meetings or maybe our AIs can accomplish this stuff.
Evan Ratliff
Maybe there's an optimistic view. Yeah, but I think, I think when you, when you create these sort of semi autonomous entities, you really have to think through like, are you gaining an advantage or not? And there's lots of examples of this. You know, there's sending an AI assistant out to do stuff for you. Well, the problem is like they often make stuff up. So then, and I had this experience then you have to go clean up after them in the situations where you've deployed them. So I think a lot of this stuff is sort of like, you know, it's a move fast and break things the old, the old way.
Big Technology Podcast Host
But you also had it go do reporting for you. You had to do an interview.
Evan Ratliff
I did.
Big Technology Podcast Host
And I had always up until recently been of the opinion that AI is not replacing reporters. Can't do what I do, can't be there asking the questions. Certainly can't have an engaging conversation like in a podcast. And now I've fully rethought that. Fully, fully hearing your agent go out and speak with the CEO and ask like some pretty good questions. Now of course you prompted it and it can't do the follow up work that we do, but it was like you literally probably could have done five minutes of work and get an hour of labor output. And then I'm also thinking about NotebookLM, which is the Google application where you can now just upload files and it will create a custom podcast for you. And I'm just routinely blown away by how good those shows are sometimes. Once I was heading down to Facebook headquarters and Mountain View and I knew it was going to be a long drive and I just uploaded a bunch of documents and recent news clippings about Facebook and I said, all right, probably important background for me to Know, generate a podcast, Google. And that was part of my prep on the way down to the meeting. So I do think that this stuff is, you know, despite all the drawbacks, and I hear your concerns, it's hard for me to see it not making its way into the workforce.
Evan Ratliff
Yeah, no question, no question. I mean, and I found when it did interviews, that was also a thing that I had been telling myself, like, well, it can't. It's not going to conduct the interviews. But then, like, it absolutely can conduct the interviews. Now, it depends on the person on the other end currently kind of being okay with an AI conducting an interview, because they're going to figure it out partway through, most likely. But that's for now, you know, like, we're pretty close to someone not being able to detect it at all. And you can say, well, there's an uncanny valley. But I think even now we're up the, like, back slope of the uncanny valley when it comes to the voice stuff. Like, many people will go through a full conversation with mine and not know that it's not human. So I think, absolutely, you can do that. It's just a question of, like, what do we want it to do? And are we thinking about what it means if it does these things for us? But there's no question it could do. It can do many of these. These kind of things, including some of the things that we hold dear.
Big Technology Podcast Host
You laid like, a nice trap for an AI CEO whose company was powering AI Voice. You had your voice AI interview him, and it was basically like, you're either going to answer these questions to show you believe in the product, so I have you for the interview, or you're going to say, this is stupid, in which case that's a pretty good little nugget for your show.
Evan Ratliff
Yeah, I figured if there's one person who can't hang up on an AI when he realizes an AI, it's the owner of an AI calling platform. But he was like, he was quite a good sport about it. He said, oh, that's very funny. And then he kept going with it. And even he. Because I interviewed him later and asked him the same, basically the same questions, even he was a little more forthcoming with the AI than he was with me. And I think there is a quality of. As we were talking about before, when it comes to people asking questions or conversing with ChatGPT, there's a quality of you don't necessarily feel like there's someone there and you might be a little more intimate than you would have otherwise. And that can be very valuable in an interview for a reporting project. So it creates this other level of, well, wow, is it actually getting better stuff than me? Sometimes I thought, well, it didn't follow up very well. But sometimes when I listen to my own interviews, I think, wow, I didn't.
Big Technology Podcast Host
Follow up very well. Yeah, I mean, it's amazing. You would think that there's a human on the other side. Then people are more likely to open up, or maybe they feel more pressure to open up and therefore they're more likely to tell stuff to your AI, which sort of makes me wonder about the whole reporting profession. But that is a conversation for another day.
Evan Ratliff
It can take you to some dark places, that's for sure.
Big Technology Podcast Host
Yeah, it would have been a good conversation for your long form podcast back in the day. So you extended this even further and had the AI talk to your kids. Your kids seemed to really enjoy the experience, even when it got extremely strange. I mean, your AI voice was telling your kids that it missed them and asked if they missed it as well.
Evan Ratliff
Yeah, I mean, the kids. My theory, which I think was born out was like, that maybe the kids wouldn't be weirded out by it in the same way that adults are, because they've grown up with synthetic voices. Like, they've heard Siri, they've heard Google giving directions. Like, all that stuff happened during the course of my lifetime. And I still find things off putting that they would not find off putting. And I think that was mostly the case. And they kind of rolled with it, even when it did things that would frankly, like, creep other people out in a very intense way. And they were kind of like, yep, sounded like you. You know, they were, they were just much more blase about the whole thing than especially like my friends who became very upset from talking to it.
Big Technology Podcast Host
I already have too many favorite scenes from this show.
Evan Ratliff
Well, that's good.
Big Technology Podcast Host
Another scene that I loved was you actually helped create a bot for your father to dispense logistics advice. And then you sent your bot with a prompt to learn more about your relationship with your dad. And you had them talk to each other. And your dad's bot had like the most fatherly response. Talk a little bit about what happened there.
Evan Ratliff
Well, I also use my bot on different parts of my family. So I had my voice agent call different members of my family, not all of them, because even I have my limits in terms of what I'm willing to do for a journalistic experiment. But my father, I wanted to use it on him. Because he loves technology and has always loved technology and is very, very always up on new technology. And he's an expert in logistics. And so one of the first things he said when he discovered that I had this at all was, well, I want one, I want one. Actually, the kids said the same thing, like, I want one. Can I have one? And so I helped him set one up. And then its purpose was not like mine. Its purpose was to, as you say, dispense logistics expertise to people who would call it. But I figured, well, given that there's one of him and one of me, like, let's see what they say to each other. And mine was much more personalized. As I said, it had my whole life story in it, but his, it just had logistics advice. So it was basically a mismatch in terms of.
Big Technology Podcast Host
It was great.
Evan Ratliff
The personal, personal effects of each one.
Big Technology Podcast Host
You're like, dad, tell me your, your AI is like, dad, tell me a little bit more about our relationships. And it's like, no, no, son, I'm here to tell you about logistics.
Evan Ratliff
Dad, I was hoping we could talk a bit about my upbringing. What was it like raising me? Oh, you caught me off guard there. It sounds like you've mistaken me for someone else. I'm actually here to help with questions about logistics and last mile delivery. But I'd love to know your name and organization to help you better.
Big Technology Podcast Host
All right, let's get to a couple weird questions. Where are you on the question of AI sentience? I mean, you basically created an AI to resemble a person. I know we don't think, or at least I don't think you believe that AIs are sentient now, but did you feel any like, say, in AI world, feel the AGI at all, or did you feel any hints of personhood and the AI that you developed?
Evan Ratliff
I did not. In fact, the more that I deployed it, the less I felt that. Now, I think when we're talking about how close is AGI and those types of questions, I think the people inside the companies who are extremely closed about what's going on, they're dealing with non guardrailed versions of the chatbots. So I think they may have different experiences and they write about, as you talked about on a recent show, like the chatbot's lying and things like that. In this case, you've got the fully guardrailed ChatGPT latest version. The more that you talk to it, the more generic it feels to you, the more you can actually feel the training data. The, the sort of like distilling down the training data and the predictive aspect, like oh, it's trying to predict what a human would say in this given moment and what the average human would say in this given moment is actually quite lame. Like that's what you really figure out. So I felt further away from that the more that I like spent time with it. Now I don't think that's necessarily like a statement about how close or far away it is because I think those aspects are all internal to these companies and we just don't have access to them.
Big Technology Podcast Host
So why don't you take us down the road a little bit? What do you think is going to happen as this technology gets better?
Evan Ratliff
I think first of all, as with many things, the market is going to dictate that people are going to use this technology even if it remains flawed, even if it remains not quite human quality. For all sorts of the very obvious ones are like telemarketing call centers, ordering food at a drive through, you know, places where they can save a little bit of money by deploying it even if it messes up sometimes, even if it does crazy stuff like give you the wrong order. Like they'll just say, well humans mess up too and it messes up less. And so I think we're going to start to see them infiltrate these different parts of society and I think the question is going to be how people respond to them. And if people are sort of like eh, it's same, same to me or maybe this customer service AI is actually more helpful than the person that I get sometimes when I call the Social Security Administration or the VA or whatever benefits I need, maybe people will embrace them. I think you have seen some instances with technology where with, for instance, checking yourself out of the grocery store where a lot of people don't like it and then maybe they go back to humans. So I think the balance still is yet to be determined. But I think there is no question that voice AI agents are just going to be, they're just going to be deployed by people who are looking to save money. And I expect that we'll encounter them more and more often all the time.
Big Technology Podcast Host
Yeah, I think the key to a successful call with the Social Security Administration is just tell them I'm here to talk about my Social Security number. 123-4567. There's no way there could be a problem.
Evan Ratliff
That's right. If you know that your address is in the zip code 90210, you're all set. Which is what my bot traditionally uses for Its for its zip code.
Big Technology Podcast Host
That's good. Very believable. All right, one last thing before we leave. I want to talk a little bit about something that I think people should be vigilant about because you are an individual sending your bot out. But there's also going to be organizations that will send bots to you or to people. And to me, like, the scam problem becomes infinitely worse. If they could clone voices and then have them call home.
Evan Ratliff
Yes. I mean, this is the greatest scamming technology that has ever been invented. It's already being deployed for scams as we speak, including volume scams where you can just use AIs to call people all the time and then narrow down the number of marks and then send it to a human operator to close the deal, basically. And these kind of personalized scams where you can clone someone's voice off of their Instagram or anywhere. If they've appeared anywhere in video and their voice is there, all you need is a few seconds. You can clone their voice. You can look up their relatives. You can call a relative and say, I'm in trouble in the voice. Use your AI to say, I'm in trouble. I need a lawyer. Or, I have a lawyer. The lawyer needs money. I've been in an accident. It's called the grandparents scam, oftentimes now. And these are happening. I mean, they're happening every day all over the country. And that's just the very first level of scamming that people are attempting. And so I think people have to now be aware. The great thing is, if you're aware of it, you can actually prevent it if you talk to people about it. If you tell your relatives, I'm not going to call you in this way. Or if you get a call like this, watch out for it. Or if you get a call like this, text me and ask me if this is really me. There are ways around it, but it's just actually the tip of the iceberg in terms of the way this technology will be used to try to separate people from their money.
Big Technology Podcast Host
Yeah, we have. You know, obviously there's a concern in my family because my voice is all. It's out there.
Evan Ratliff
You're cloneable, man.
Big Technology Podcast Host
I have been cloned. Just my podcast audio was used to clone me with 11 labs.
Evan Ratliff
Yeah.
Big Technology Podcast Host
And I once embraced the technology, but I also know the risks. And so with my family, if we have a rule that if any of us ever call and say, I'm in a distress situation, I need help, I need money, we have a code word that we've created in the privacy of our own home that you have to use that code word. And that's. And we know it's real. There's no way for the AI to know that, I hope.
Evan Ratliff
Yeah. Until you train up an AI to be like you like that, and then it shares it with other AIs.
Big Technology Podcast Host
Exactly. All right, last thing. I always say last thing and then have to add more after doing this. Do you talk to ChatGPT more often than you type to it?
Evan Ratliff
I don't. I mean, the funny thing about me is I barely use ChatGPT really. No.
Big Technology Podcast Host
You don't use AI at all.
Evan Ratliff
I do use AI. I mean, I like NotebookLM. I don't like the podcast feature that much, but I do like just processing documents in my work is a big thing. So processing, let's say, legal files for a big story that I'm working on. So I do use it. But I find that for most things in my life, like, I've set up my life to do things that I like to do, like writing. And I don't want ChatGPT to do any writing for me because that's what I've chosen to do with my life. So I'm kind of a bad candidate because I'm not looking for efficiencies in that way. I'm looking to kind of, like, do the work that I enjoy doing. So I don't tend to use ChatGPT except in this voice context where I still use my voice AI to talk to scammers.
Big Technology Podcast Host
All right, well, look, thank you for making the show. I enjoyed it thoroughly. You said in the show that it's season one, so I have my fingers crossed that we'll be able to hear something else. Maybe something weirder and more devious.
Evan Ratliff
Perhaps. Perhaps.
Big Technology Podcast Host
I feel bad for your friends, but I feel happy for us, the listeners.
Evan Ratliff
I'll give them a break.
Big Technology Podcast Host
I'll give them a break, and I do hope people go check it out. So the show is called Shell Game On. Also so much news this week. Elon Musk trying to buy OpenAI or maybe just messing around with Sam Altman and crew. Ranjan and I will be back on Friday to cover that and so much more. Again, podcast is Shell Game host is Evan Ratliff. Evan, thanks again for being here today.
Evan Ratliff
Absolutely. My pleasure. I enjoyed it.
Big Technology Podcast Host
And thank you all for listening and watching. If you're here on Spotify with us, we'll see you next time on big Technology Podcast.
Big Technology Podcast: "AI Clones & The Future of Voice AI" with Evan Ratliff – Detailed Summary
Release Date: February 12, 2025
Introduction
In the latest episode of the Big Technology Podcast, host Alex Kantrowitz delves into the intriguing and controversial world of voice AI with esteemed technology journalist and podcast host Evan Ratliff. Ratliff is renowned for his podcast Shell Game, where he explores the depths of technology's impact on society. This episode, titled "AI Clones & The Future of Voice AI", provides a comprehensive exploration of Ratliff's pioneering experiment in cloning his voice and deploying it in various real-world scenarios.
Project Overview
Evan Ratliff introduces his ambitious project, explaining how he cloned his own voice using platforms like 11 Labs and integrated it with ChatGPT to create a sophisticated voice agent. This AI-powered clone was programmed to interact with different types of callers, including family members, friends, therapists, and notably, scammers.
[02:08] Evan Ratliff: "I cloned my voice to see what that was like... connected it up to ChatGPT to create a voice agent that uses a simulation of my voice but generates content from the chatbot."
Ratliff's motivation stemmed from a desire to understand the societal implications of voice AI beyond its technical capabilities. He sought to explore how this technology could alter human relationships, trust, and daily interactions.
Voice AI in Scamming
One of the primary applications of Ratliff's voice AI clone was engaging with robocalling scammers. Initially, he tested the agent with customer service lines, leading to amusing and sometimes bizarre conversations. Recognizing the ethical implications of such interactions, Ratliff shifted his focus to scam calls, where he felt more comfortable allowing the AI to converse without causing real harm.
[05:40] Evan Ratliff: "I had my voice agent interact with scammers to see how it would handle real-world conversations... receiving 30 to 40 scam calls a day."
The AI clone adeptly navigated scam attempts, often engaging scammers up to the point of divulging fictitious information, thereby rendering the scam ineffective. This experiment highlighted both the potential and the risks of deploying voice AI in combating fraudulent activities.
Voice AI in Therapy
Expanding his investigation, Ratliff deployed the voice AI to engage with AI therapy bots and real human therapists. He provided the AI with extensive personal information, including his mental health history, to assess how AI could handle sensitive and complex emotional interactions.
[08:20] Evan Ratliff: "I wanted to see what happens when I send my AI agent to therapy, to understand the problems it would surface and the answers it would receive."
The interactions revealed significant limitations of current AI therapy solutions. The AI struggled to maintain coherent and empathetic conversations, often remixing old personal data in ways that felt unnatural and sometimes cringe-worthy.
[09:17] Evan Ratliff: "They are being introduced without any scientific research showing... It's unclear what they could do for or to you in a therapeutic environment."
When the AI clone interacted with a real human therapist, the experience underscored the stark differences between AI and human empathy. The therapist initially tried to engage meaningfully but eventually sensed something was amiss, highlighting the gaps in AI's ability to provide genuine therapeutic support.
Voice AI with Friends and Family
Ratliff didn't stop at scammers and therapists; he extended his experiment to his personal circle, including friends and family. This led to a mix of reactions, from amusement to distress.
One notable interaction involved a friend who is a lawyer. The AI clone provided concise legal advice, sometimes even surpassing the friend's own responses.
[21:01] Host: "Some people thought it was cool, and one friend even joked about charging the AI one thousand two hundred dollars an hour for legal advice."
However, the most challenging moment came when the AI clone interacted with a friend who met the U.S. Men's National Soccer Team at a hotel. The AI's attempt to express enthusiasm was misinterpreted as sarcasm, leading the friend to question Ratliff's mental health.
[23:44] Evan Ratliff: "He began to think something's wrong with me... It was the most difficult conversation of the whole show."
This incident revealed the potential emotional impact and ethical considerations of deploying AI clones in personal relationships without prior consent.
Implications for the Workplace
The conversation naturally transitioned to the broader implications of voice AI in professional settings. Ratliff discussed the possibility of AI agents replacing humans in meetings and other work-related interactions.
[28:20] Evan Ratliff: "If everyone sends their agents to meetings, who's going to process all the information?... How do we preserve the humanity in our interactions?"
He highlighted concerns about AI's ability to handle complex professional tasks and the potential loss of human connection and understanding in the workplace.
Ethical Concerns and Future Prospects
Ratliff and Kantrowitz addressed several ethical dilemmas surrounding voice AI:
[32:33] Evan Ratliff: "The people who have designed these AI products generally have a different set of problems... What happens with the rest of us?"
Looking ahead, Ratliff predicts a rapid increase in the use of voice AI across various sectors, driven by cost-saving motives and technological advancements. He emphasizes the importance of societal discourse on the ethical integration of AI to ensure it aligns with human values and preserves essential aspects of human interaction.
Conclusion
Evan Ratliff's experiment with cloning his voice and deploying it through various AI-driven scenarios provides a compelling glimpse into the future of voice AI. While the technology offers intriguing possibilities, it also presents significant ethical and societal challenges that need to be addressed proactively. As voice AI becomes increasingly sophisticated and ubiquitous, thoughtful consideration and regulation will be crucial in shaping its role in our lives.
[44:16] Evan Ratliff: "I'll give them a break, and I do hope people go check it out. The show is called Shell Game."
The episode concludes with Ratliff expressing hope for continued exploration and dialogue on the intersection of AI and human experience, underscoring the need for balanced and informed approaches as we navigate this technological frontier.
Key Takeaways
Notable Quotes
This episode of the Big Technology Podcast serves as a critical examination of the evolving landscape of voice AI, offering listeners invaluable insights into its current applications and future implications.