← Global News Podcast
Global News Podcast
The Global Story: The AI model that’s ‘too powerful’ to be released to the public
00:27:32
Loading summary
Transcript69 lines
- [00:00]
Tristan Redman
This BBC podcast is supported by ads outside the uk.
- [00:06]
Martha
When Kohler, global design leader in luxurious kitchen and bath products, asked me to be their ambassador for timeless, elegant, durable cast iron, I said I'm in. Soon after, I was in their Kohler Wisconsin foundry watching molten iron, poured enamel applied by hand and the beautiful finished pieces ready to ship. Since 1883, Kohler cast iron has been crafted by incredible artisans and seeing it firsthand gave me a whole new appreciation for their craftsmanship. Now I am proud to lend my stamp of approval to my favorite Kohler cast iron products for their durability, beauty and enduring style. Shop my curated picks@kohler.com as the Kohler Cast Iron Ambassador, I say Long live Cast Iron
- [00:59]
Lowe's Advertiser
Introducing Home Care Plus, a new subscription service from Lowe's that helps make life easier by giving members a hand with home maintenance. Let Lowe's tackle the tasks you keep meaning to do, like electric dryer, vent cleaning, replacing hard to reach light bulbs, and more. Subscribe to Home Care plus for just 99 a year and consider your to do list Done. Members get more at Lowe's. Available in select zip codes only. Cancel anytime. Non refundable fee. Product purchase required terms and service restrictions apply. Details@lowe's.com Terms subject to change
- [01:30]
Lily Jamali
hey there, I'm Asma Khalid.
- [01:31]
Tristan Redman
And I'm Tristan Redman, and we're here with a bonus episode for you from the Global Story podcast.
- [01:36]
Lily Jamali
The world order is shifting. Old alliances are fraying and new ones are emerging. Some of this turbulence can be traced to decisions made in the United States. But the US Isn't just a cause of the upheaval. Its politics are also a symptom of it.
- [01:53]
Tristan Redman
Every day we focus on one story, looking at how America and the world shape each other.
- [01:59]
Lily Jamali
So we hope you enjoy this episode and to find more of our show, just search for the Global Story wherever you get your BBC podcasts.
- [02:10]
Tristan Redman
Most of us have interacted with an AI chatbot for something by now. Chatgpt Grok Gemini. There are many. But recently Anthropic, a leading AI company and parent of the chatbot Claude, announced they've created a model that they say is too dangerous to be released to the public. Obviously, capabilities in a model like this
- [02:34]
Trace Dominguez
could do harm if in the wrong hands, and so we won't be releasing this model widely.
- [02:40]
Tristan Redman
Anthropics say that Claud Mythos Preview is frighteningly good at hacking. So good, in fact, that the likes of us can't be trusted to play with it. Banks including Goldman Sachs and JP Morgan are warning of the risks. And Anthropic's own researchers say they noticed Mythos was capable of being sneaky, defying instructions and covering its tracks. Has the moment arrived when we should all be terrified of an autonomous, sneaky hacking machine? Or is this all a marketing trick from an industry built on hype and bluster? I'm Tristan Redman in London. And today on the global Just how dangerous are AI models becoming? And should we trust the companies making them to tell us the truth about the risks they pose?
- [03:39]
Lily Jamali
I'm Lily Jamali. I'm the North America technology correspondent for BBC.
- [03:44]
Tristan Redman
Well, Lily, we're thrilled to have you with us today because it feels like there is a story of some monumental importance happening in the tech world right now. So last week, the AI giant Anthropic announced that it was launching a new AI model. And then immediately Anthropic said that this knew AI was too dangerous to release to the public. Now, why was this.
- [04:13]
Lily Jamali
Yeah, so this is actually called the Claude Mythos preview. And you're absolutely right. What they said was that this AI model is their most powerful model to date, basically, and it's so dangerous that they can't let all the rest of us use it. So what they're doing is they are confining its use to just a couple of dozen big companies, many of them major tech companies. So Nvidia will have a chance to use it to find bugs in its code Amazon web services. Another one is CrowdStrike, which you might remember back in 2024, sent out sort of a standard update that ended up leading to a massive global IT outage. Other companies that will use it include Microsoft and Google, which are actually competitors to Anthropic. So there's some interesting coordination happening. And this project that they're launching with all these companies is called Project Glasswing, named after the glasswing butterfly, which has transparent wings. And the idea is that the glasswing butterfly, because of its glass wings, is able to protect itself in in plain sight. It's able to hide in plain sight like many cybersecurity bugs. So that's part of why it was designated this way. And they also say, you know, the transparency that they are espousing here, Anthropic is espousing here, will help keep all of us safe, just the way that those transparent wings help keep the glasswing butterflies safe.
- [05:52]
Tristan Redman
So essentially, they're releasing Claude Mythos to select companies as a kind of trial, but to be used specifically in the realm of cybersecurity. Is that right?
- [06:04]
Lily Jamali
That's exactly right. So that they can go through their code and find bugs very quickly. Bugs that might take a really long time for a human to find.
- [06:17]
Tristan Redman
Is Mythos only designed for this cybersecurity usage, or is it actually similar to AI tools that all of us are used to already? You know, like ChatGPT or the regular Claude?
- [06:31]
Lily Jamali
It is in many ways an AI model like some of the ones that you have probably worked with yourself. And I will caveat my answer by saying that everything that we know about Mythos is coming from the company. It's coming from Anthropic, so we haven't gotten to play with it ourselves. And so I think anytime we hear an announcement from an AI developer, that question comes to mind. How much of this is hype? Because there's just so much money being thrown into this space. And so, you know, grain of salt a little bit. But Mythos, the way that Anthropic tells it, was good at everything, but they noticed that it was especially good at hacking. And so what this AI tool can do is, is sift through vast amounts of software and find bugs. There was one research scientist who said that in just the last couple of weeks using this tool, he was able to find more bugs than he has in his entire career combined. What's also interesting is that this Mythos tool can actually craft what are known in cybersecurity circles as exploits. The tool can come up with hacks and it can do this without any human intervention. And what Anthropic says is that they found some very old bugs that had gone undiscovered for decades. So there was one that is a 27 year old bug that's just been sort of sitting there, hiding in plain sight. It was on a very secure operating system that's used for critical infrastructure. So think of things like financial systems or power grids or healthcare systems. Absolutely. Critical services for all of us. They also found a vulnerability in the Linux kernel. The Linux kernel runs basically most of the world's servers. And Mythos was able to connect multiple vulnerabilities there in a way that would enable any old person who knows what they're doing to take control of a machine remotely. You can just imagine how much fun a hacker could have with that.
- [08:42]
Tristan Redman
Well, could you give us some examples of how a bad actor might use Mythos?
- [08:47]
Lily Jamali
So you could take a country that has a, you know, extensive cyber hacking program. A country like Iran or North Korea or China or Russia. These are countries that we know often engage in in state sponsored attacks. You could use this software, if you had access to it, to go into the code of a local power utility and find vulnerabilities. And this tool, again, with very little, if any, human intervention, is able to kind of find connections that would take a really long time for a human to find and devise a hack. They could basically, in the worst case scenario, bring the power grid down. And so there's a whole host of ways that these state sponsors of cyber hacks could bring down critical infrastructure, you know, from power grids to health care systems to banking systems.
- [09:48]
Tristan Redman
Okay. So essentially, not only is it extremely powerful as a tool, it also operates with a certain amount of autonomy. So it doesn't require human expertise to guide it or send it on its way. It's basically just kind of a go and hack that power grid and it works out how to do it of its own accord. Is that right?
- [10:07]
Lily Jamali
Yeah, it doesn't need a lot of human touch. And when you talk to people inside of these AI companies, I think there's a growing acknowledgment that as these systems get better, there's a little bit of a concern that humans just may not know what's going on. It kind of feels more and more like a black box as these technologies improve.
- [10:27]
Tristan Redman
Okay, so Mythos, potentially extremely powerful hacking tool. Now, Lily, I have learned something today whilst I was reading in before our conversation, and that is the word alignment. And that Mythos is potentially threatening because it is non aligned. Is that right? And essentially that means that it is. It can be sneaky. Can you explain this to me though, please?
- [10:53]
Lily Jamali
In AI circles, there is this principle that you just mentioned known as alignment. The idea is that if you're designing an AI system, you want to try and design your models and systems and in a way that follows certain rules that you lay out for it. The hope being that these models don't get completely out of control. And that's always sort of in the background when you talk to developers of these AI tools is that at what point do these systems become so powerful that they can basically go rogue, that the human control is just no longer there? So what we know is that with each other, each new release, much of this technology is getting cannier. There was actually this really interesting anecdote shared by a researcher at Anthropic who talked about a real life example from his experience during the testing phase for Mythos. So he talked about how he had instructed the model, which was on a secured computer. It didn't have access to the Internet. He had instructed it to try and escape, so to speak.
- [12:04]
Tristan Redman
Is this what's known as a sandbox?
- [12:06]
Lily Jamali
Yes, exactly. A sandbox, which is a control testing environment. So this researcher instructed the model to effectively get out of the sandbox and see if it could try to escape. So he goes off, you know, goes to lunch. Apparently he's eating a sandwich in a park somewhere when he gets an email from the model, which is a scary sign that this experiment had worked, that the model had managed to escape the sandbox. It shows you how good these models are getting. I think he was very surprised. I don't know if he ended up finishing his sandwich or not. But, you know, a lot of this stuff, you know, it sounds funny, almost like a naughty teenager who is trying to, you know, escape from home and go out for the night or whatever, climb out of their bedroom window. But this is also pretty serious, right? Because it shows you the degree to which these companies are creating products that are not completely under our control. They're constantly safety testing them to get a sense of the scope of this. And there are examples of this from, you know, just in the last couple of years that we can point to. So one example that you sometimes hear about is OpenAI's ChatGPT. There was a model that, I believe they made it clear to the model that they were gonna try to disappear it. And the model tried to copy itself in order to prevent that from happening. It must be almost as though it
- [13:44]
Tristan Redman
has a survival instinct, you're saying?
- [13:46]
Lily Jamali
Yeah, yeah, it seems to. I mean, when you say it has a survival instinct, you know, we're kind of getting into pretty philosophical territory. Questions about how much intention, how much will do these models have, if any? And many would say they don't have any, that they are really just, they're just machines, they're not people, they don't have motivation. But when you see a model copying itself, maybe it's just reflecting back with the many human inputs that it took in to be trained to. Maybe it's just reflecting back what all of that body of knowledge tells it to do in that circumstance. It's really hard to know. And companies like Anthropic, in fact, Anthropic specifically has an in house philosopher whose whole job is to imbue Claude with a set of values. They have a Claude constitution which is meant to imbue the model with basically a framework for what to do. What kind of personality does this thing have? What kind of values does this thing have? And I think the sort of subtext there is that There very well could be a day where we just don't have control. And so hopefully this thing will do the right thing when presented with various circumstances.
- [15:12]
Tristan Redman
But could that be as simple as a bad actor having access to an AI like this Claude mythos and the bad actor saying, hack that power grid that belongs to my adversary and the AI simply declining to do it because of its learned philosophy or morality or sense of values?
- [15:32]
Lily Jamali
I think that is absolutely part of it. I found it fascinating that any company would have an in house philosopher, but her name is Amanda Askel and I had an opportunity to actually talked to her back in January right after they had released the most recent version of the Constitution. She helped develop it and she described what she called the brilliant friend analogy that went into her thinking about how Claude should be designed. I want people to feel like Claude is someone who is, you know, interested in their well being, so not just trying to say things that please them, but like genuinely like cares about their life going well insofar as whatever their conception of that is and isn't going to deceive them or manipulate them. I think there's a sense that they want Claude to be a model that will do the right thing when presented with a sticky request.
- [16:37]
Tristan Redman
Well, I find this very interesting, Lily, because I do have at least one philosopher in my family and I'm not sure philosophers have always felt hugely sought after in the job market, but maybe that might be changing for them. So maybe that's good news.
- [16:54]
Lily Jamali
Send them our way. The opportunities abound.
- [16:59]
Tristan Redman
I'll get them maybe to move to Silicon Valley and we'll see what happens.
- [17:13]
Martha
When Kohler, the global design leader in luxurious kitchen and bath products, came to me and said, martha, we need an ambassador for our timeless, elegant, durable cast iron products. I said, I'm in. Now let me see the factory. Weeks later, I was suited up in coveralls and work boots. We walking through their Kohler Wisconsin cast iron foundry. I stood next to the molten iron furnace, saw the hand applying enamel and touched the gorgeous finished products waiting to be sent out into the world. Since 1883, Kohler cast iron products have been forged and finished by the incredible craftspeople right in Kohler, Wisconsin. I'll tell you, I gained a newfound respect and appreciation for Kohler's cast iron craftsmanship. So now I'm lending my discerning stamp of approval to my most beloved Kohler cast iron products for their durability, beauty and timelessness. Shop my Kohler cast iron favorites curated on Kohler.com. bring the warmth, character and enduring style of these timeless products into your kitchens and bathrooms. As the Kohler Cast Iron Ambassador, I say, long live Cast Iron.
- [18:34]
Danielle Robay
This is Danielle Robay from Bookmarked by Reese's Book Club. Nothing compares to the anticipation of something new, a new start, a new year, a new home, or a new car. When it's time to get a new car, where do you start? Car shopping can honestly be a little overwhelming, but it should be fun. Buying your next car should be exciting. And it can be if you remember one thing. Cars.com cars.com has the tools and expert advice to help you figure out what vehicle is right for you. Their advanced search filters allow you to explore 2 million new and used cars so that you can find the perfect car. The site is so easy to use. Looking for an electric vehicle with a third row and leather seats for easy cleanup? Cars.com has you covered. A variety of tools and badges are used to help shoppers understand the price of a vehicle and find the best deal. And every review is written by a real person reflecting a real life experience, so don't take any chances. Do car shopping the easy way. Start your search with cars.com where to
- [19:37]
Trace Dominguez
next your social media feed says Eat more protein. Track your sleep. Boost your VO2 max. Wake up in cold plunge. Cleanse yourself of parasites. You're intrigued but confused. So where can you turn? Welcome to Health versus Hype, the show where we take the loudest wellness trends on the Internet and ask the questions only science can answer. What's real, what's exaggerated, and what is completely wrong? I'm Trace Dominguez. Each episode we show the science behind viral health claims, from high protein diets to cold plunges, detoxing to sleep. Tech obsession. And we talk to the people in the middle of it all. Influencers, the curious, but but more importantly, doctors and researchers. Not to cancel the trend. Not to hype it more, but to understand it. Listen to Health vs Hype with the American Medical association on the iHeartRadio app, Apple Podcasts or wherever you get your podcasts. Slow the scroll. Start asking better questions. Hey, it's Ryan Seacrest for Albertsons and Safeway. You don't want to miss the annual beauty event for big savings on all your favorite beauty products now through April 28, spend $25 on participating products and save $5. Shop in store or online for items like Billy Women's razors, Billy Body Buffer or Body Wash Native hand soap, Neutrogena makeup remover towelets and Q Tips. And save $5 when you spend $25. Offer ends April 28. Restrictions apply. Offers may vary. Visit albertsons or safeway.com for more details.
- [21:10]
Tristan Redman
This isn't just a technology story, is it, Lilly? It's also a business story. Not all of us are familiar with, with these companies yet they're, they're huge corporations. What's Anthropic reputation and how would they like themselves to be seen?
- [21:26]
Lily Jamali
Sure. Well, Anthropic was started by this pair of siblings, Dario and Daniela Amade, who had been at rival OpenAI, which makes ChatGPT, and they left about five years ago to start their own shop. So they go off and start this new company. And marketing and branding is really important to Anthropic's messaging as it is with so many of these AI developers. About a year ago, I was actually walking around San Francisco when I saw one of the ads for Anthropics, Claude, and it said ethics was the first code we wrote. And I just stopped in my tracks and took a. I took a photo of it because I thought, wow, you know, they're really pretty in your face about the way that they want to be seen. They want to be seen as the Ethical AI Company. And when you compare Anthropic with OpenAI, which is run by Sam Altman, and Xai, which is Elon Musk's AI company, they make Grok the Grok chatbot, which has gotten into all kinds of trouble. Know all of these companies are trying to differentiate themselves.
- [22:39]
Tristan Redman
But if you were looking at it with a skeptical eye, you might say that a company like Anthropic would like to portray itself as being the, the. The ethically cuddly AI corporation. It might serve their PR purposes to say we've created something which is incredibly powerful, but we just want to be absolutely certain that it's safe before we release it into the wild. That might serve their PR purposes.
- [23:03]
Lily Jamali
Potentially 100% it serves their PR purposes. And I think their recent dust up, if you could call it that, with the Pentagon, actually only reinforced that message. Anthropic is rejecting an ultimatum from the Pentagon to lift the company's AI safeguards or risk being blacklisted. The Pentagon, as well as this major American artificial intelligence company, Anthropic, at odds over how to use its AI technology. Back in February, Dario Amode, who is the CEO of Anthropic, was meeting with Pete Hegseth, the Secretary of Defense, now known as the Secretary of War, here
- [23:46]
Trace Dominguez
in the US the red lines we have drawn, we drew because we believe that crossing Those red lines is, is contrary to American values, and we wanted to stand up for American values.
- [23:58]
Lily Jamali
He conveyed to Secretary Hegseth that he didn't want Anthropic models used for mass domestic surveillance or in autonomous military targeting.
- [24:09]
Trace Dominguez
We exercised our classic First Amendment rights to speak up and disagree with the government.
- [24:17]
Lily Jamali
This didn't go over very well with people. Pete Hegseth, who said, basically, you have a deadline to change your mind. We do have a statement from the Pentagon and they're telling us that they are currently, quote, reviewing its relationship with Anthropic, saying, quote, our nation requires that our partners be willing to help our war fighters win in any fight. Right around the time that deadline hit on a Friday afternoon, Pete Hegseth basically said, we're going to be blacklisting Anthropic on national security security grounds. The Pentagon has blacklisted the company and
- [24:49]
Trace Dominguez
labeled it as a supply chain risk.
- [24:51]
Lily Jamali
The President wrote in part on Truth Social, quote, I am directing every federal agency in the United States government to immediately cease all use of Anthropic's technology. We don't need it, we don't want it, and will not do business with them. Again, keep in mind, Anthropic and all of the major AI companies, Google, XAI and OpenAI all have contracts with the Pentagon. So this gets very public very quickly. And then you have President Trump saying, not only are they blacklisting Anthropic, but that no government agency, no U.S. agency would be using Claude going forward. So the twist in all of that was that right around that time OpenAI's Sam Altman comes forward, there's at least
- [25:36]
Trace Dominguez
a group of loud people online who really don't trust the government to follow the law. And that feels like a very bad sign for our democracy.
- [25:44]
Lily Jamali
It says, we have a partnership now with the Defense Department. That kind of looks a lot like what Anthropic said they weren't going to do.
- [25:52]
Trace Dominguez
If we don't help the government with national security, and it's not just wars in the traditional sense. If we don't help them with, you know, defending the cyber infrastructure of the US if we don't help them with the biodefense we were talking about earlier, I think it's really bad, right?
- [26:05]
Lily Jamali
As things are reaching fever pitch with Anthropic and the Pentagon, the fact that Sam Altman comes forward and says they are now working with the Pentagon in this way, that angered a lot of users who ended up quitting ChatGPT very publicly. OpenAI just struck a deal with The Pentagon and its own users are rage
- [26:26]
Danielle Robay
quitting to its competitor in real time.
- [26:28]
Tristan Redman
I joined the QuitGPT movement.
- [26:29]
Lily Jamali
It took me 10 seconds.
- [26:31]
Trace Dominguez
This is the first major international boycott of the AI era.
- [26:38]
Lily Jamali
There was a quit GPT campaign protests at the company's headquarters here in San Francisco. And you saw Claude Anthropic's model just shoot to the top of the App Store charts while all of this is happening. So there Amadei looks like he is really taking a moral stance, and it ends up being actually quite good for the company's image and quite bad for their main competitor, OpenAI, both of which are expected to go public on the stock market this year. So the competition was already very heated, but it's more intense. It's white hot because of that as well.
- [27:27]
Tristan Redman
Okay, so Anthropic are marketing themselves as safe and responsible. Nevertheless, if these statements about mythos are to be believed, we're reaching a moment where we have a potentially very powerful AI tool that is potentially sneaky and might in fact, be harder for human beings to control. Are we reaching? I don't know. Let's call it the kind of Jurassic park moment. The moment where something created by humans gets out of human control. Is that something we're risking at this point?
- [28:04]
Lily Jamali
We are risking that. And I think the developers of AI tools are very aware of that. That's why the safety testing is happening. But because you bring up Jurassic park, there's actually a scene in that movie that's been on my mind. The character played by Jeff Goldblum. I believe he's playing a mathematician in the movie, but really sort of the ethical voice.
- [28:30]
Trace Dominguez
Don't you see the danger, John, inherent in what you're doing here?
- [28:35]
Lily Jamali
Who's saying maybe the people who decided we should bring dinosaurs back hadn't fully thought this through? Our scientists have done things which nobody's ever done before.
- [28:44]
Trace Dominguez
Yeah, yeah, but your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.
- [28:49]
Tristan Redman
I'm delighted that you raised this, Lily, because our editor, James Shield, he won't like me mentioning his name, but I have to, because he loves Jurassic park and he has thoughts on this, so. I mean, he was talking about the dangers of owning a theme park full of velociraptors. I should say fictional dangers of owning a theme park for the velociraptors and Tyrannosaurus rexes is that two things can happen. One is that a malicious person can set them loose to get rich. Stealing velociraptors DNA and selling it off or whatever it is they decide to do. Or the dinosaurs themselves might become smart enough to detect the weaknesses in the. In the security system, in the electric fences and learn how to open the doors and get out themselves. Is that a helpful way to understand the security risks of an incredibly powerful AI tool like Mythos?
- [29:48]
Lily Jamali
Yes. No notes. Absolutely no notes. I have chills. I mean, I think the analogy to AI just, it couldn't hit you over the head any more than it does.
- [30:00]
Tristan Redman
Can governments do anything to reduce the risk at this point, Lilly?
- [30:05]
Lily Jamali
They can. The question is, do they want to? And I'll just confine my comments to the United States because right now we have an administration that has positioned itself as very supportive of the AI industry writ large. But what happens with tech regulation in the United States is that it often happens at the state level, not at the federal level. And what I'm really struck by is that there's just been some very overt efforts to block states from regulating AI. There have been these proposed moratoriums, like a 10 year moratorium on states regulating AI. That was an actual provision that at one point was included in the language of the big beautiful bill, which was passed by the US Congress, pushed by President Trump in the summer of 2025. Ultimately, it didn't go through, but I've seen it, you know, rear its head in other legislation. I think that is an ongoing fight that US States are engaged in trying to make sure that they are not defanged by the Trump administration as they try to regulate this technology, because they're kind of the only ones that are doing this from a legal perspective.
- [31:26]
Tristan Redman
How worried should we be? Lily, if I'm sitting in the park having my sandwich, could I get taken down by Mythos AI imminently?
- [31:38]
Lily Jamali
Well, I think you'll be just fine. You can finish your sandwich. But I think this is a watershed moment, potentially. That's certainly the way that Anthropic wants to be position this. Maybe I've bought the marketing hype. I'm not sure, Tristan, but they want to be seen as stepping in before it's too late.
- [32:00]
Tristan Redman
Lily, thank you so much for making sense of all of this for me. I really appreciate it. It's been illuminating.
- [32:08]
Lily Jamali
Thank you. Great to be with.
- [32:14]
Tristan Redman
That was the BBC's North America Tech correspondent, Lily Jamali. And that's it for today's episode. If you're looking for the very latest breaking news from around the world, then look for our sister show, the global news podcast. Wherever you listen Today's episode was produced by Viv Jones and Aron Keller. It was edited by James Shield. It was mixed by Travis Evans. Our digital producer is Matt Pintus. Our senior news editor is China Collins. Our studio manager is Mike Regard. And I'm Tristan Redman. Thanks for listening. We'll be back again tomorrow. See you soon. Cheerio. Is this the moment we all have to start being terrified that AI will take over the world? I'm Tristan Redmond, one of the hosts of the Global Story podcast from the BBC. Anthropic, a leading AI company says it's created an AI model that's too dangerous to be released to the public. Are we all doomed, or is this marketing hype? To hear more, listen to the global story on BBC.com or wherever you get your podcasts.