← The Times Tech Podcast
The Times Tech Podcast
AI safety meets OpenClaw – what India’s AI summit tells us
00:39:23
AI safety, OpenClaw and the future of AI
Loading summary
Transcript91 lines
- [00:00]
Paige Desorbo
Hey, this is Paige desorbo from Giggly Squad, and today I want to talk to you about Boost Mobile. Quick question. Why are we letting our phone bills bully us? Here's a money tip. Stop paying a carrier tax when you bring your own phone and switch to boost mobile's $25 unlimited forever plan. You can unlock up to $600 in savings. That's real life money, not money trapped in a pricey phone bill. $600 is a trip, a shopping spree, or paying something off. Your money belongs in your life. You get unlimited data, talk and text for $25 a month with no contracts and no minimum line requirement. Your phone head to boostmobile.com to switch today and unlock the savings you actually deserve. After 30 gigabytes, customers may experience lower speeds. Customers pay $25 per month while active on Boost Mobile Unlimited plan savings claim based on a January 2026 Boost Mobile survey comparing average annual payments of major carrier customers to 12 months on the Boost Mobile Unlimited plan. Visit boostmobile.com for details.
- [00:56]
Danny Fortson
This episode of the Times Tech Podcast is sponsored by ServiceNow.
- [01:00]
Katie
Danny, one thing we keep hearing from business leaders right now is AI sounds great, but how do you actually make it work inside a company?
- [01:08]
Danny Fortson
Exactly. Because most organizations aren't neat, shiny systems. They're layers of software, legacy tech and teams, all doing things slightly differently.
- [01:17]
Katie
ServiceNow sits across all that, acting as a control tower for making work move seamlessly through the organization, connecting people, systems,
- [01:25]
Danny Fortson
data, and increasingly AI agents so that things don't happen in silos.
- [01:30]
Katie
Learn how ServiceNow puts AI to work for people@servicenow.com.
- [01:40]
Danny Fortson
Hello and welcome to the Times Tech Podcast, where every week we unpack how technology is reshaping business culture and everyday life. I am Danny Fortson out here in Silicon Valley and Katie is unfortunately not with us this week. She's off, but representing the great city of London. I am pleased to welcome back to the POD my very able colleague, Mark Selman, the technology correspondent for the Times.
- [02:08]
Mark Selman
Thank you for having me back, Danny. Great to chat.
- [02:10]
Danny Fortson
Great to have you. So, our big question this week is around AI Summits. Because as we are speaking, India is hosting the AI Impact Summit. Politicians, thinkers, tech CEOs are all gathered there to discuss the future of tech. So we're asking, who gets to shape the future of AI, Silicon Valley or the rest of the world? And more to the point, what is the purpose of AI summits anyway?
- [02:37]
Mark Selman
Well, Danny, this is the fourth AI summit. Yep. Count them. One, two, three, four. And it's starting to feel like what? G20, G7, ASEAN COP, every kind of Alphabet soup where leaders get together, they discuss geopolitics, lots of pictures and family photos, and we're wondering if we're ever the wiser or safer as a result of it. So, yeah, I'm keen to know if this summit gives us clues into how AI is evolving too.
- [03:08]
Danny Fortson
That's right. And later you're going to hear an interview that Katie and I did right before she left with one of the lead authors of the 2026 International AI Safety Report, Karina Prunkle, which is being presented at the summit. So that should give us some insight into the topics being discussed. But before we get into that, it is worth giving the broader context because not just these AI summits, which we'll get to shortly, but also what is happening out here in my neck of the woods, especially around the AI project that has taken the world by storm. And that is, of course, openclaw.
- [03:47]
Mark Selman
Yeah, I can't stop reading about this or seeing videos about it. It's absolutely everywhere this week. Peter Steinberger, who actually is based in London, he created this incredibly viral AI agent called claudebot, and then it had to change to Multbot and now it's called openclaw. But essentially the big news is it's been bought or poached by OpenAI. There was a bidding war between Sam Altman and Mark Zuckerberg for this technology. Essentially both billionaires wanted Steinberger and Altman won the day.
- [04:24]
Danny Fortson
Yeah, of course. And this is important, right, because it's just been a few weeks since it went viral. An open clause forced everyone out here certainly to take a breath and rethink how they're building AI. When you're talking about hundreds of billions being put into kind of AI data centers and into these companies and people are having to rethink, you know, what form the technology will take next. Because for those who have not been following along, OpenClaw is a personal agent that you can run on your own machine, on your own computer, and you can give it as much access as you want to your life, your email, your schedule, give it a WhatsApp number and you can chat with it. And it effectively becomes your employee, your co worker, your digital butler. And it's really a glimpse of this agentic future we've all been hearing about for probably the last 18 months, realized by this one guy and this open source project. And I'm guessing not reporting here, but I'm guessing that Mr. Steinberger was handed, I'm guessing at least a nine figure pay deal to join OpenAI. Because of course, Zuckerberg has been throwing around insane amounts of money, including giving one guy from thinking machines lab $1 billion to join Meta. So if Sam Altman beat out Zuck, you know, there was some serious money being thrown around.
- [05:47]
Mark Selman
Yeah. Now Altman does say that OpenClaw will live on as an open source project under a separate foundation that will be supported by OpenAI. But it's been a dizzying few weeks and OpenClaw going from nowhere to millions of users and it has forced the AI labs to think hard about their own plans. I feel like I been hearing about agentic AI for ages and you've heard Mark Benioff from Salesforce talk about it, but I've never really seen it deployed in the wild like it has been by OpenClaw. And I think the other side of that is there are implications for security and, you know, jobs obviously, because once you see these things go, they go. And they work all day and all night and all you got to do is pay for the tokens. That is an incredible and both frightening prospect at the same time.
- [06:46]
Danny Fortson
It reminds me a little bit of the ChatGPT moment three years ago when Google had this, this capability for years and years and then they didn't release it because they're like, hmm, this is a tricky problem because it hallucinates, it's not always right and it's also very powerful. But ChatGPT was the kind of upstart, it had nothing to lose. So it puts it out there and then Google freaks out, they declare a code red and then we're off to the races with this kind of AI revolution. And here we are three years later. People have been talking about Agen, but again, it's a really tricky thing because handing the keys to your life, to an AI and then just letting it run is not something that kind of a lot of grown up companies have figured out how to do in a safe way. But then you have this open source project by one guy and it shows the potential. And so now everybody's trying to rush to get on board.
- [07:41]
Mark Selman
Yeah, I think it will be interesting to see whether this particular viral trend, I would call it a viral trend within Silicon Silicon Valley and the tech and coding community and essentially early adopters breaks out beyond that because it's still not incredibly user friendly. And the implications of what you're doing are absolutely huge because you do have to give these agents pretty full access to your computer. I've had as many notes from Cybersecurity companies about it by AI companies pushing the whole agentic AI thing. But it does show that this thing is moving to deployment. And once you start to see that and people start saying, hey, I did it, and it does the work of people, and then another company goes, well, if they're doing it, maybe I should be doing it. And you get this rolling kind of snowball effect.
- [08:35]
Danny Fortson
Yeah. And I think one of the biggest questions, beyond making it more user friendly, is also just liability. And it reminds me of like having a dangerous dog. Like if my dog bites somebody, I'm liable. Right. If my agent does something stupid or buys something on my behalf that I didn't want or install some malware, whatever it may be, am I liable? Because it's not me, it's my agent. And so that's a big thorny question. I think that still has to be sorted.
- [09:06]
Mark Selman
Yeah. And you know where the companies are going to be pushing this one exactly down to you and me. I certainly, speaking to some people at the AI summit going on in India, the agents is top of their list that they are particularly worried about.
- [09:20]
Danny Fortson
I wanted to talk about that initially because of course, if we step back right, to where we started, AI summits. Right. And I think given this context of just how fast things are moving, I think it's worth talking about what these AI summits are, why they're necessary and the challenges that they face.
- [09:39]
Mark Selman
Yeah, I think it's probably worth going back into how it all started, really. And yeah, it did start in the UK. You know, ChatGPT came out, everyone was going, wow. And then suddenly we had this slew of researchers who quit, like Geoffrey Hinton from Google, famously, who then got the ear of the British government and said, look, this is really worrying. You guys need to start talking about safety. And the then Prime Minister Rishi Sunak picked up on that and created this idea of an AI Safety summit at Bletchley Park. There was a lot of buzz around it. I was there and you know, the British government was seen as doing something quite unique at the time. And then they came out this thing called the Bletchley Declaration and all the governments there sort of signed up to these warm words of treating AI as a shared international risk area, global dialogue, you know, keep going with AI safety research. But the big thing that they trumpeted was this, the fact that the companies signed up to do this pre release testing, so the AI Safety Institute as it was in the uk, would be able to test their AI models. So that's kind of how it started. Then it rolled onto Seoul, which was like an interim summit and then it went to Paris and then it became
- [11:00]
Danny Fortson
the AI Action Summit.
- [11:02]
Mark Selman
Oh yeah, yeah.
- [11:04]
Danny Fortson
Gotta rebrand these things. And now we're here in Delhi. Do we know what the theme is this year?
- [11:10]
Mark Selman
So it's the first time that a country from the global south has hosted it. And Narendra Modi, who's the Prime Minister of India, he is really championing this. As you know, India wants to be the leader of the global south in AI. He wants to democratize AI. There are banners all around the summit saying democratizing AI for a billion plus Indians. So this is really spreading the technology beyond the wealthy nations, letting everyone have it and just really making sure this doesn't become an American wealthy nation story, essentially. And a big part of that is obviously championing the open source movement which develops the technology for everyone to use and adapt as opposed to those closed systems developed by OpenAI, Anthropic and others.
- [12:02]
Danny Fortson
Yeah, I'm out here and everybody's so kind of anthropic, OpenAI Google focused. Right. It's almost like a league where people are watching the standings as people kind of rise and fall against each other. But it feels to me that the open source aspect of this is going to be huge, especially if you're talking about democratizing this for a billion plus Indians or a billion plus whomever all over the world. And there's models like Kimi K25, which is this Chinese model, which if you talk to people in the industry, they're like, this is basically as good as the leading models and it's free. So I'm out here on the west coast, I'm going to put on my west coast techie hat and just be completely skeptical of all attempts to regulate the brave new world that's being built for us by our guys out here. And it's mostly guys. Is there a point to these AI summons? Like at Bletchley, everybody committed, you're like, yes, we're going to give you access to our models before we release them. Has any of that happened or are people just doing whatever they want to do? The move fast and break things approach and just completely blowing past any soft commitments that they've given at these summits.
- [13:21]
Mark Selman
There are two questions there. Is it important that the world talks about AI? I think it's crucial that these issues are being discussed at the highest level of governments, industry, civil society. Now obviously, is it important to act? Well, yes, but as we've seen, when Governments or blocs like the EU do act and you sometimes act too fast, then you end up with egg on your face. So it is a difficult balancing act. But what I find interesting is that a lot of those that follow this area, they always look at social media as a sort of template and they go, okay, well we had all this social media, we ignored it and, and we had to retrofit it. And even in the area of social media, you are obviously starting now to
- [14:17]
Danny Fortson
see regulation appear 20, 20 years in.
- [14:21]
Mark Selman
Yeah.
- [14:21]
Danny Fortson
But when you look at AI is arguably going to be more powerful and it's developing much more quickly because unlike 25 years ago, 5 billion people have a supercomputer in their pocket and they can access it at any time. Hopefully it doesn't take a quarter century. And I do think it's wor worth just so listeners understand they've got all the big hitters here. So Sundar Pichai from Alphabet, Sam Altman, Bill Gates, Demis Hassabis from Google DeepMind, Daria Amadei, friend of the pod Mistral, AI CEO Arthur Mensch, all of these folks are here as well as several national leaders from around the world. So people are at least showing up and taking this seriously. But again, it's a question of after everybody leaves and does the family photo, what happens?
- [15:16]
Mark Selman
Yeah, I think also it depends what you're trying to address now. If you're trying to address a safety issue, do you basically regulate the company saying you can't release that product until you show us what it's like so they can test it? A sort of mandatory version of what we got already. But if you're trying to address 4 million people are going to be out of work in this country in the next eight years. That is not something the international corporation is going to help you out. You're on your own. You've got to work out what you are going to do for your people.
- [15:53]
Danny Fortson
And I think that also gets to vital conversations around tax because if even a small percentage of the predictions are true around the kind of reorienting of the entire economy and how much is going to be sucked up into agents and machines effectively, how do you redistribute the wealth that is generated from that? What does that look like? What do you to your point around what does society look like? Because if I'm a big CEO employing 100,000 people, if I can employ 80,000 and be more productive, I'll probably do that.
- [16:30]
Mark Selman
Yeah, the value of this technology is going to accrue to capital, not labor, let's put it, put it simply, we've seen what happened after 2008, you know, that particular crisis. And I think, you know, we could see a huge societal backlash against the technology if the worst fears about jobs are done. Now, obviously, even Modi is talking about transformation more than displacement, because his people will be very worried. They have an incredibly skilled IT population, but they're all working for companies that could be disrupted by AI. They're Serv AI, they're SaaS companies, software as a service. So if you believe the current market disruption, they are on the front line to be disrupted by AI. He'll be worried for his people for sure.
- [17:22]
Danny Fortson
And we just had Nicola Merksich on last week from Poly AI, and they're doing customer service agents. And if you combine that with coding, there's that whole universe, millions and millions of people in India who work for these kind of outsourcing companies that could be in the firing line for this. And also, again, just circling back to our friend Peter Steinberger and openclaw, it also, just when you think about these summits, again, it just comes back to the question of how quickly this stuff is changing and how do you regulate something that. We're three years into this and things have already changed so dramatically. I mean, if you go back to November 2022 and ChatGPT was like telling you to put glue on pizza and saying bears walked on the moon, we're really far from that now. When you have the top coders in the world being like, I don't write code anymore. The machine does it. So this sets us up for our upcoming guests. So let's take a quick break, and when we're back, we're going to hear from Karina Prunkle, the lead author of this big AI safety report that is being discussed at the summit. Today's episode of the Times Tech Podcast is sponsored by ServiceNow.
- [18:41]
Katie
ServiceNow describes itself as an AI control tower for business reinvention, connecting people, systems, data and AI so workflows more smoothly across an organization.
- [18:52]
Danny Fortson
The platform integrates with different clouds, models and data sources, bringing them together in one place to help your team.
- [18:59]
Katie
To learn More about how ServiceNow puts AI to work for people, visit ServiceNow.com
- [19:08]
Progressive Insurance Announcer
this episode is brought to you by Progressive Insurance. Fiscally responsible financial geniuses, monetary magicians. These are things people say about drivers who switch their car insurance to Progressive and save hundreds because Progressive offers discounts for paying in full, owning a home, and more. Plus, you can count on their great customer service to help you when you need it. So your dollar goes a long way. Visit progressive.com to see if you could save on car insurance, Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states or situations. Blood donation is now more inclusive. More people are able to donate blood with the American Red Cross through FDA guidelines that eliminate eligibility questions based on sexual orientation. The Red Cross celebrates this historic change and welcomes those who may be newly eligible to donate blood. There's a place for everyone in the mission of the Red Cross. The Red Cross is committed to achieving an inclusive blood donation process that treats all potential donors with equality and respect, while maintaining the safety of the blood supply. Join us and help save lives. To learn more and make your appointment to donate blood, visit redcrossblood.org LGBTQ that's redcrossblood.org LGBTQ.
- [20:31]
Danny Fortson
Welcome back to the Times Tech Podcast where we've been discussing India's AI Summit taking place this week. So as world leaders and tech bosses gather in New Delhi, there's another group in the room that arguably matters just as much, and that's the researchers trying to answer a much harder question. What can these AI systems actually do? And what could go wrong? So behind the scenes at the summit is the team responsible for the International AI Safety Report. It's led by Turing Award winner Yoshua Bengio, and it brings together more than 100 AI experts, is backed by more than 30 countries and organizations such as the UN and the EU. And this is their second major report looking at what today's AI systems are capable of, the risks they pose and the safeguards that might keep them in check. And so we are very lucky to have one of the lead authors of the report on the podcast. Her name is Karina Prunkle. She's a researcher at France's National Institute for Research in Digital Science and Technology and a research affiliate at Oxford's Institute for Ethics in AI. Katie and I caught up with her last week just before she headed out to India. It's a 221 page report. Why do we need this? Is the idea that this will be used as a reference for policymakers, for government leaders, et cetera. Like if we just step back and look at the moment, you know, because we're week in, week out, we're talking about the power of this technology, the latest advances, people worried about jobs, worried about kind of AI slipping the leash, whatever it may be, from where you sit, why is a report like this necessary right now?
- [22:18]
Karina Prunkle
Well, general purpose AI is now really deployed at scale. It's rapidly becoming more capable. But the evidence that we have on the risks and on potential mitigations to address them is still emerging and much slower than systems Advance. And it's also quite hard to assess. And so this creates a fundamental governance challenge for policymakers who have to make decisions on the basis of this limited emerging evidence. And so this is where the International AI Safety Report comes in. The idea is that it just synthesizes the best available scientific evidence we have on. Well, on three questions. One, the capabilities, what general purpose AI systems can do, then what risks are emerging and what safeguards exist. So by having this shared international understanding, at least we have something that we can base our conversations on. We can cut through the hype, we can cut through the fear. We can just have this common minimal denominator. And then governments and policymakers can go away and base their decisions on the basis of this evidence.
- [23:36]
Danny Fortson
And what is the involvement and kind of attitudes of the frontier labs, the googles, OpenAI's anthropics, et cetera?
- [23:47]
Karina Prunkle
So we had several feedback rounds. We had a feedback round with academics, with civil society organizations, but also with for profit organizations such as the big AI labs. And their attitude has been quite positive. I mean, they were very constructive, they gave good feedback. In the end, what we put into the report is under the discretion of the chair and the writing team. So we're really keen to emphasize that this is an independent report. But when it comes especially to technical safeguards and capabilities, then it is often useful to just have some input from people who work in the big AI labs.
- [24:28]
Katie
So they let you test their stuff, I guess, is the question.
- [24:31]
Karina Prunkle
What do you mean by testing?
- [24:33]
Katie
Well, like looking inside their models and seeing what they're up to?
- [24:38]
Karina Prunkle
No. So the report is really a collection of scientific evidence. So we didn't go out and collect our own data and do our own tests, but we were synthesizing the scientific evidence that has been published officially. And also, just to stress again, this independence, we do flag when papers have been written predominantly by people working in the tech sector or in private companies, or when people are from an academic background.
- [25:07]
Katie
So what are the top lines in the report about risk? And what emerged this year that's particularly changed?
- [25:14]
Karina Prunkle
So we see that some risks have really manifested in a way. So the evidence now has become quite obvious. And thinking about cyber attacks, for example, now AI systems are regularly used in cyber attacks or deep fakes and synthetic media. So we also see harms that have, where we now have really robust empirical evidence, but for other Risks, we have much more mixed or incomplete evidence. So these risks rely on evidence from lab studies or modeling or theory. But for those risks, we also see development and movement. So one development we see on biological risks is that several deployers have released their models with additional safeguards because they could not rule out that those models would be able to assist novices in the creation of biological weapons. So this seems to be an indication that we should take these risks serious, even though there is a lot of uncertainty that remains about how does this translate into a real world risk. Is it just theoretical or is it actually feasible that we can create biological weapons on the basis of outputs from general purpose AI models? But it is something that we need to take serious at this point and also something that, that might be worth monitoring.
- [26:37]
Danny Fortson
Yeah, I mean, Dario Amadei in his recent 1920,000 word blog post highlighted that as a big fear of his. And I guess stepping back. So just thinking again about just the process here. How long were you guys working on this? Was this. I presume this is many months or like the whole last year before you published it.
- [27:00]
Karina Prunkle
So we've started working on the report in summer and just because it takes time and we have so many feedback rounds and there is a lot of back and forth also with writers and internally, and there's just a lot of scrutiny. So it does take time to write such a long report.
- [27:19]
Danny Fortson
Which leads to my question of how useful will these reports ultimately be? Because not to be like use a trite Silicon Valley phrase, but is this moving at the speed of the technology? Like if you're taking whatever six months to write this thing, bringing in people's ideas and analysis from all over the world. And in the meantime, claudebot happens or the latest anthropic model comes out that is leaps and bounds beyond maybe what your team was analyzing. How do you guys navigate that challenge? Because that feels like particularly what's been happening the past few years is things have been developing so quickly that oftentimes people try to pass laws and by the time they pass them, they apply to something that is no longer relevant.
- [28:10]
Karina Prunkle
Yeah. So this is certainly a challenge. We navigated it by really adding content up to the last minute. So the final, final deadline was in December. So I think we've got most of the things that happened in 2025. But more general, as an institution, we tried to address this fast moving landscape by publishing these interim reports. So we had a couple of key updates. The first one was on capabilities. So what has happened since the publication of the last Report that was published in October and in order to keep up with this rapidly moving AI landscape. But in general, I completely agree that a yearly report is already pushing it by the time it's published. There are so many new developments. Developments. But this is exactly why we had the key updates. In order to counter this.
- [29:03]
Katie
You talked about some of the risks there. I'd love to hear what really jumped out for you while you were doing the research for this. Was there anything that particularly surprised you?
- [29:13]
Karina Prunkle
Yes. So one thing that surprised me was that we do have a number of safeguards on our hand and none of these safeguards works by its own. But when we layer them, and we call this in agreement with the rest of the field as a defense in depth approach. So once we layer these safeguards, then we can actually, it gave me hope that we can manage the risks appropriately.
- [29:43]
Katie
I love the way that you say it gave you hope. Do you think we are trying to. Are governments or policymakers listening to you when you talk about the safety risks of AI? Yeah.
- [29:55]
Karina Prunkle
So I mean, it gave me hope, but I mean, I should of course also say that a lot of the safeguards and especially the technical safeguards have their individual limitations. And I guess coming back to this question about whether governments will listen to us, it's not so much, you know, we're not. The report doesn't give any policy recommendations. The report really is just there to inform. And maybe. Danny, this goes back to your previous question. You know, who is going to read this? We've got really good feedback from various governments who, some of which have established their own reading groups going through the report section by section. And this really indicates that it can be useful to inform decision making processes on the governmental level.
- [30:43]
Danny Fortson
And I mean, I think one of the risks, because I think this is a safety report, so it's about bioweapons and cyber attacks, but also I think people that are thinking about IR much less deeply or just kind of your man on the street is like, am I going to have a job? What do I do? What do I tell my kids in a world where the ship has sailed in terms of any human being smarter than a machine ever again, theoretically, do you guys get into that around the kind of societal risk as these systems continue to advance and what that could mean for kind of societal cohesion and it's not a kind of a technological risk as much as a social one?
- [31:26]
Karina Prunkle
Yes, we do that. We actually categorize our risks into misuse risks, malfunction risks, and what we call systemic risks. And Those systemic risks really are capturing the societal aspect that you're mentioning. So we do have a section on labor market and we also have a section on human autonomy which also falls into this category that it might be this broader societal phenomenon that we see emerging in the wake of general purpose AI.
- [31:55]
Danny Fortson
What's the answer? How bad is it going to be in terms of that kind of that societal upheaval? I mean, obviously you can't predict the future, but what did you find?
- [32:08]
Karina Prunkle
Yeah, so what we found is that currently we don't see any aggregate labor market effects. So when we look at the most recent studies from the United States and Denmark, both national level, we find no aggregate employment effects. But what we do find are more targeted effects on the job market. So for example, there's less demand for writing and translation, but at the same time there's more demand for machine learning programming, for example. Another effect that is quite marked is that it seems that junior workers in AI exposed jobs are also seeing a decline in employment opportunities. But senior workers don't see that same decline. In fact there it's stable or there's even an increase. So the labor market effects currently are quite targeted. But yeah, as you said, I can't predict the future. So we'll have to see how this continues.
- [33:08]
Katie
That's so interesting. It really tallies with a study that we looked at on the podcast last year called Canaries in the Coal Mine. Do you remember that? By Stanford University. Exactly. I wanted to ask about the vibe shift. So I remember being at the Bletchley Park Summit and it was the AI Safety Summit in 2023 and now we've got the India Impact Summit. Have you felt that vibe shift from the focus on oh my goodness, AI is going to take over the world and we need to think about safety to actually there are other more pertinent or more pressing things to worry about.
- [33:43]
Karina Prunkle
I wouldn't frame it in that way. I think there certainly have been different focal points in the two summits. For example, you write the, the Bletchley Park Summit was much more safety risks associated with emerging capabilities focused. And now the India Summit has this big theme of integrating the global south and topics that are relevant for the global south. I think that's absolutely fair enough. Different world regions have different focal points. We found that across the line all of the summits were really interested in the, in the work of the International AI Safety Report because the report addresses risks that are really border crossing.
- [34:28]
Katie
But it's right, is it that America hasn't signed up to this?
- [34:31]
Karina Prunkle
Yes, that's right.
- [34:34]
Danny Fortson
Just like with the climate change protocols, who needs it? You know, it's like, ugh.
- [34:40]
Katie
Yeah, but I mean, is that a concern? If you think, you know, this is. This tech is mainly being developed by a small handful of American companies. Is it a problem that the US Government doesn't sign up to it?
- [34:53]
Karina Prunkle
So I'm afraid I can't comment on that question. So I would refer you to the secretariat.
- [34:58]
Katie
What do you think, Danny? He's your president.
- [35:02]
Danny Fortson
I do. I do think it's a problem. I mean, I think it's great that the Frontier Labs are taking part and being proactively involved and kind of being collaborative, but, you know, again, it's a little bit like climate change. If the biggest polluter does not take take part in an international agreement, how good is that agreement and how impactful will it ultimately be? So that is worrying. But again, the other aspect is just how quick moving this all is. If one guy can create this cloud bot or open claw thing that kind of takes the world by storm. It does feel like, as we talked so often on this podcast, just the. The autonomy, the empowerment that these things allow, both good and bad. Somebody who is really determined may be able to make some kind of chemical or biological weapon with the aid of these things. It does feel like. I'm just wondering, as you pulling this all together, do you walk away feeling hopeful or filled with dread or both about just where things are and how quickly they're moving?
- [36:16]
Karina Prunkle
I think both. I mean, I already expressed my hopefulness regarding at least the possibility of containing and mitigating certain risks. But I see there are just also a lot of shortcomings, and some of them are quite fundamental. So we emphasize something that we call the evaluation gap. So, you know, sometimes it's just almost AI systems have a specific benchmark performance, but this performance really doesn't really translate into the real world. Same with safety. So we already don't know exactly how to best evaluate these systems and how to ensure reliability of the systems. And I think this is like a fundamental scientific challenge that we're facing. So I think the challenges are multifold and we talk about them extensively in the report as well. But. But some of the challenges are at a very basic level, like how do we evaluate all purpose AI systems? And those seem quite fundamental.
- [37:18]
Katie
Fascinating. Yeah, I hadn't quite realized that was so impossible.
- [37:21]
Danny Fortson
Yeah, you can't. The first step is measuring something to then be able to kind of regulate it or whatever before you go. I think it's worth. This may end us on a positive note or not, I don't know, it depends on your answer. But one of the kind of areas that. But there is a lot of optimism around or a lot of curiosity around us around healthcare, whether that's the delivery of healthcare or also drug discovery, treating diseases that have been intractable, et cetera. When you think about how these AIs can kind of step in, do critical thinking for humans, do this kind of analysis involved with some of that, are there examples that come to mind when you're looking at this aspect of this technology?
- [38:04]
Karina Prunkle
Yeah. So especially in the last year we've seen a lot of progress in scientific reasoning, in troubleshooting lab protocols and in these sort of problem solving skills that are in general required to advance science and medicine in particular. So I think there will be really big opportunities. But at the same time, very often the same capabilities that allow us to make progress in those fields are also capabilities that then can be dangerous or misused in other contexts. So going back to the biological weapon example, you don't want somebody who is trying to develop a biological weapon to use their general purpose AI to troubleshoot their virology lab protocol. So it's a bit of a double edged sort when it comes to the scientific capabilities of AI systems. But maybe just to end is that again, I think to have general purpose AI systems deployed in real world medical contexts, we still have a way to go. So the systems might be passing medical exams and medical benchmarks, but at the same time there was one study that showed that a system gave potentially harmful answers to 19% of realistic medical questions asked. So again, there is this gap between the pre deployment evaluation and real world performance of those systems.
- [39:43]
Danny Fortson
As you say, I think it's the jagged edge. Right. It's excellent at some things and just really terrible at others. And it's kind of hard to predict how that looks.
- [39:51]
Katie
Yeah.
- [39:54]
Danny Fortson
So what do you think? Are you filled with hope? Are you filled with dread?
- [39:57]
Mark Selman
Both like she was. It's always both in this job, isn't it? I think she was being very careful and trying to be balanced. But the fact that we've run out of evaluation tools, I mean the System Card for anthropics Claude Opus 4.6. Just pull out that document if you really want to get scared. Have a look at that. They too say that their system evaluations cannot evaluate their own product, that the product knows when it's being assessed. It's over eager, it does things it's not supposed to. I mean this is why people get worried about this stuff.
- [40:36]
Danny Fortson
Well, that's what's so interesting. She was like, they spent six months on it and you know, the last bits they included were from December. And then the whole open claw moment happens in three weeks, this last three weeks. And you're kind of like, oh, well, that's a completely new dynamic that is not contemplated just because it can't. And it does make you wonder, like, how do you regulate and how do you not throw out the baby with the bathwater, not be overly aggressive in terms of regulation, but also be like, there are some things to worry about here. But it's all happening so fast.
- [41:12]
Mark Selman
I am increasingly of the belief that, that something like that it's like cyber attacks, you know, it's like something bad happens and suddenly everyone wakes up and says we've got to have more cybersecurity. So I, I kind of believe that that's unfortunately probably what's going to happen. But the jobs thing is really incumbent on. I mean, they don't have to wait for an international agreement to do that. These governments have just got to get, get their ass in gear about it. And. Yeah, which is why I think actually India is very well placed in the sense that their population really wants to learn and do and act. And actually if you have a population that wants to do that, you will succeed at this technology because it's incredibly empowering and that is, does act as a supercharger that it will, you know, but you need to have a population and a government that is fully on board to embrace it. So I think that's where they're in a good place.
- [42:06]
Danny Fortson
Well, that's a positive note to end on More hope, less dread. That is it, I think for this week's episode of the Times Tech podcast, if you are enjoying the show, please do follow or subscribe and leave us a rating or review. It genuinely helps other people find us. And you can find more about all the stories we discussed today and tech news in general@thetimes.com and you can email us with ideas or questions at Tech Pod at the time. Times.co.uk, thank you very much for stepping in to the fire this week. It was a, it was great having you on.
- [42:42]
Mark Selman
Pleasure.
- [42:43]
Danny Fortson
Until next week. See you later. Bye bye.
- [42:48]
Katie
This episode of the Times Tech Podcast is sponsored by ServiceNow.
- [42:52]
Danny Fortson
There's a lot of excitement around AI right now, but the problem is what happens after the demo when you have
- [42:58]
Katie
to plug that technology into a real company.
- [43:01]
Danny Fortson
Different clouds, different data, different systems that were never designed to talk to each other.
- [43:06]
Katie
ServiceNow's platform is designed to help people by connecting these pieces, enabling organizations to coordinate work across departments, tools, and increasingly, AI agents.
- [43:16]
Danny Fortson
In fact, the company says more than 80 billion workflows run on its platform
- [43:20]
Katie
every year, which gives you a sense of the scale of operations it's designed to handle.
- [43:26]
Danny Fortson
Learn how ServiceNow puts AI to work for people at ServiceNow.
- [43:33]
Progressive Insurance Announcer
This episode is brought to you by Progressive Insurance. Do you ever think about switching your insurance companies to see if you could save some cash? Progressive makes it easy. Just drop in some details about yourself and see if you're eligible to save money when you bundle your home and auto policies. The process only takes minutes and it could mean hundreds more in your pocket. Visit progressive.com after this episode to see if you could save Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states.