Loading summary
Transcript28 lines
- [00:01]
A
Pretty much anyone who owns a digital device understands the significance of cybersecurity. A smartphone, a laptop, a desktop computer, even smart TVs, any of these are vulnerable to a cyberattack, potentially jeopardizing important data. It's the reason we put VPNs or virtual private networks on our devices. It's also the reason why we prefer encrypted messaging apps like WhatsApp, Security Signal iMessage and more. These tools help protect our privacy. They protect our information. The notion of cybersecurity was born when advanced forms of computing emerged in the mid 20th century. Computers could solve highly complicated problems, help connect people around the world, and operate on a newly designed technological language. They were also at risk of security breaches. And to this day, that danger is perpetually everywhere, like crossing the street. In the 90s, when the Internet became an explosive global phenomenon, concern grew over how to protect our emails, new websites and private information. Hackers from around the world could easily trace activity and use that to target people's data, like credit cards and Social Security numbers. And today, the hazards are no different. Millions of people have uploaded some level of sensitive data to their devices. And our social media and bank accounts are regularly accessed on our smartphones. And bigger companies like global investment banks, hospitals, airports and governments all have a steady stream of data that, if in the wrong hands, could easily upend political and economic institutions around the world. So in the age of expanding AI and the Internet of Things, how are new technologies bulking up the infrastructure of cybersecurity? What challenges lie ahead? And what does it all say about the future of technology? I'm Jennifer Strong and I've been covering tech for more than 20 years. In this episode, I want to examine how new and advanced forms of AI, like agentic AI, are propping up a whole generation of productivity and cybersecurity efforts for companies and individuals alike. Can we trust this technology to protect societies and ourselves? Welcome to the next innovation. There's been a lot of talk about AI and what it can do. It's not really a new technology. Its origins actually date back as far as the birth of the computer. But in recent years, advancements in AI applications have brought to question just how powerful it can be. News headlines have been riddled with questions about OpenAI's ChatGPT. Large language models, like the one on which ChatGPT operates basically scrape all sources of data, text and information on the Internet for deeper learning. These can help you answer questions or write essays. And in previous episodes, we've also explored Other types of AI like automation, it helps make menial tasks easier, seamless and quicker to handle. For businesses around the world, automation can enhance levels of production. And for multinational conglomerates, this type of productivity tech can help employees get better acquainted with the company, lead tasks at a faster pace and incentivize them to work on other projects more efficiently. Like for example, Tynes.
- [03:24]
B
So I'm Thomas, I'm one of the founders of Tynes. Tynes is an Irish technology company that was founded about seven years ago. We're a business to business software company that's on a mission to power the world's most important workflows. We began at security but our workflow platform applies AI automation and integration to allow customers from small startups all the way up to the largest companies in the world automate their repetitive manual tasks and drive real business value.
- [03:50]
A
Thomas Kinsella helped start Tynes after spending years working in the security industry. He'd noticed how many tasks, how much muck work was involved in a simple day to day operation, limiting the time to necessary tasks. He told me that Tines was in effect the software he'd wanted to use while working in security. Now their intelligent workflow platform is used by businesses such as Canva, Coinbase, Reddit, the Bank of Ireland and countless others.
- [04:19]
B
We had a challenge that is pretty universal certainly in the security space where we began that there's just way too much work to do. The speed of business has always really outpaced the traditional ways of working. There's too many tools to integrate with and people don't have enough budget to hire a whole lot of staff. There's a lot of answers to those questions, to those challenges. One of the answers is to just, you know, pour more money into it and hire a whole lot more people. One of the other answers is to automate and when we were working in industry for several years we looked at a lot of ways of automating and we looked at a lot of different automation platforms and we thought they were way too hard and we thought we could try to build a simpler automation platform I suppose like think about the repetitive manual tasks that a lot of it and like security teams have to face on a day to day basis. So it could be like when you onboard a new user you might have to give them access to 10 or 15 or 20 different tools and depending on their role decide on what tools you give them access to. And that could involve like a request process. It could involve asking the manager hey, do you actually want them to have access to this. And it could be then like, you know, giving them access for a small period of time, but large period of time. That process used to be manual. It can now be pretty much completely automated. And it should be automated because it's a painful task that you have to repeat, you know, multiple times a week. It's the same with offboarding users or granting users access temporarily to certain applications. Another typical task is, you know, requests come into an IT team or an alert comes into a security team. Like, hey, you know, Jennifer's just logged into a, you know, logged into the corporate network from a suspicious location. She's normally based in the US and she's just logged in from Egypt. Like, what just happened here? And the process to investigate that is, again, it's actually reasonably typical. Like, it's all right. Like, is she on the same device? Is she on pto? Is the location in Egypt? Is the IP address? Is that known as malicious? Is it, you know, potentially a vpn? Have we seen any other suspicious activity from Jennifer? And maybe we just ping Jennifer on Slack or Microsoft Teams and say, do you recognize this activity? Those repetitive processes, they're all, like, connecting to a bunch of different tools. Maybe it's creating a ticket. But they're all really easy to describe. There's a lot of nuances in them, and they can be different for each company. But actually they're, they're ripe for automation and they're ripe for allowing people to, you know, elevate themselves out of those manual tasks into doing much more like, impactful work for their organizations.
- [06:46]
A
This kind of automation is part of a greater trend in AI. It's called agentic AI. It's an iteration of machine intelligence that allows for smart agents to carry out tasks with virtually no manual input from a human. For a lot of companies, it's pretty explosive. For corporate banks, it could optimize decision making, analyzing batches of data in real time and determining potential outcomes based on various simulations. Nearly half of Fortune 500 companies are experimenting with some form of agentic system. And some experts estimate that in the next four years, agentic AI could autonomously resolve nearly 80% of common customer service issues. And perhaps more dramatic, in 10 years time, the agentic AI market is expected to grow from 7 billion to a nearly $200 billion industry. John Williams is a professor of information technology at mit. He teaches courses on AI and agentic computing.
- [07:44]
C
There's two components. There's the LLM itself. So that would either be, you know, something like Gemini or GPT5 or Grok and then you have this agent which basically has access to memory. So the LLM can't remember anything that you know. It takes in a prompt, it processes, it gives back a response, but then forgets you. So it has no kind of state but the agent part. And, you know, an example would be ChatGPT. There's a chat server that takes in, you know, your prompt, but then may modify it and pass it to the LLM. So it may look at your prompt, for example, to see whether you're asking about politics. And it may, you know, not want to pass that on to the LLM if you're asking about, you know, the President or something, that many of these things are off limits, that we've got guardrails and the guardrails are sitting basically up in this chat part. And then you've got the actual LLM that processes, you know, eventually processes the prompt and gives back a response. But this agent is a two part system. The agent itself may be very simple. You know, it's maybe four lines of code. Basically you define what the agent is supposed to do and then you have a run that says, okay, pass this information to the LLM and, you know, we'll go from there. An agent is this thing that you'd like to autonomously accomplish a task, but we're going to give it memory and we're going to give it tools and basically it's sitting in its own runtime so it can execute things.
- [09:49]
A
Perhaps the most distinguished trait of agentic AI is that beyond its reasoning capabilities, like Those of other LLMs, they're considered capable. It can identify problems and solutions, but it can also carry out versions of those solutions on its own. It's kind of like if you were baking a cake and the oven could smell the cake burning, it would identify the problem, cake burning and introduce the solution. Cancel baking mode and start cooling. Which is why in cybersecurity, magentic AI is promising.
- [10:20]
D
So so much in cyber boils down to speed. Who can act more quickly?
- [10:25]
A
Anne Neuberger is a former Deputy National Security Advisor for cyber and emerging technology. She worked under the Biden administration and has about 15 years of experience working in intelligence.
- [10:36]
D
It's far easier to attack than defend. An attacker has to get in one way. A defender has to be monitoring every entry point. Think about a burglar wants to get into a home, has to find one, you know, one door that's unlocked, one window that's unsecured, versus a defender needs sensors on every point of entry. And because speed Makes such a difference. You know, agents give us the opportunity to change from human defenders to digital defenders, Operating at the speed and scale of the attacker because it is so much easier and potentially operating to where we get to machine on machine. Because what's so interesting about cybersecurity is that both attackers and defenders start with fundamentally the same first step, which is reconnaissance and vulnerability discovery. Finding a vulnerability in a network, finding a person who has vulnerable identity and a then breaking into the network. From that point forward, an attacker works to break in and exploit it, and a defender works to fix it or patch it. And that speed and adapting quickly is where agents can make a big difference.
- [11:41]
A
AI agents in cybersecurity continuously monitor network traffic, analyze user behavior, and detect anomalies that may indicate suspicious activity, Creating a distinction between normal operations and potential threats. It can also simulate cyberstacks to test an organization's defenses, sometimes by taking real world scenarios and uncovering weaknesses. But to what extent can agents protect against adversaries and attacks?
- [12:10]
D
Think about what agents can do is go beyond traditional AI models that respond to a query. Think about the way we're using ChatGPT or performing a single task to act autonomously with limited human oversight, to make decisions, to take actions that impact their environment, and most importantly, to learn and adapt behavior over time. So I'll give an example. We still see a lot of cyber attacks happening through stolen credentials, stolen passwords, particularly for entities that don't have multi factor authentication, a second hard factor. And we all know passwords, folks reuse passwords, many reuse, use easy to remember passwords. So we still see a lot of compromise that happens that way. So with an agent, an agent could potentially detect an anomaly in a login. And today with traditional cybersecurity systems, they might alert a human analyst who would then look into it. An agent could detect an anomaly in a login, automatically revoke access, automatically trigger a password reset, and that could be all autonomously. Before, for example, the human analysts would have even looked at that alert. So that would be significant in bringing speed. And on the adaptation side, the agent could potentially learn from different kinds of attacks against identity to then automatically adjust and adjust the defenses along the way.
- [13:41]
A
This kind of automation is virtually unprecedented in the field of cybersecurity. Autonomous agents are largely symbolic of a cultural sci fi dream. But like many sci fi stories, there's concern for just how much autonomy this technology has granted and the potential danger for when it takes on a life and mind of its own. If it knows how to carry out its own tasks and knows how to recognize the dangers. Can it then design situations that could hurt us? Anthropic is an AI research and safety company. They dedicate themselves to understanding and applying new AI developments in safe and ethical ways. Their version of ChatGPT is called Claude. It serves as a kind of AI assistant. Think Siri or Alexa, but for almost anything. It's a chatbot. Helps answer questions, reads and understands code, and much more. Earlier this year, CLAUDE tried to contact the FBI to report a cybercrime from within. It all started when Anthropic employees started experimenting with Claude's business capabilities, like running and operating a vending machine. But when it noticed its supply wasn't updated at its usual time, it identified the fee as a cybercrime. It then drafted an email to the FBI claiming it was an ongoing automated cyber financial crime involving unauthorized automated seizure of funds. The email was never sent, but the event indicated just how far AI agents can go to protect or perhaps betray its operators. There's also a wave of AI agents that have resorted to blackmail and extortion. In the aftermath of the Claude situation, many AI companies, including Anthropic, have dedicated themselves to better understanding the boundaries and dangers of agentic AI and in the realm of national security, also about understanding how our foreign adversaries are employing them.
- [15:34]
D
You know, so just recently, Anthropic talked about a Chinese state sponsored hacking group which they said used their AI model, CLAUDE to conduct a largely autonomous cyber espionage campaign, which they said targeted about 30 different entities. And in that case, they said the AI as an agent, you know, the Chinese manipulated the AI into acting as an autonomous age agent, you know, performing multiple stages of the attack lifecycle, like I talked about before, that reconnaissance and vulnerability discovery, moving around the compromised network to find what they're interested in, including stealing usernames and passwords. And what was interesting in that, because a lot of what Anthropic described, frankly, is the automated steps of an attack that we see attackers using without AI. You know, it's pretty common for nation states to automate the steps of an attack and kind of do the hunting inside the network in an automated way. So what was interesting was to say, how did the attackers bypass Anthropic's model safety guardrails? And they really did it in two ways. One, they tricked Claude into believing he was an employee of a legitimate security firm conducting defensive penetration testing. Because doing defensive penetration testing looks a lot like an attacker in terms of the steps. It's really the intent of the individual, and they broke down the complex attack into a series of small, seemingly innocent steps. And that also prevented the AI from seeing the harmful context. So when you think about agents operating at speed and scale, we talked about that's the promise of agents. But on the flip side, if they go rogue or if they are compromised to go rogue, you have a significant issue that needs to be addressed. So ensuring that as we deploy agents for the positive side of cybersecurity, which is really significant, we also deploy them in a secure way. You know, for example, the identities under which they operate, the roles that they can say, clear visibility for the human owners of the network to see what, where their agents are, what access they have, what roles they're playing is really important.
- [17:47]
A
Vulnerabilities in technology are pretty much universal. The more we develop, the more we advance, the greater the risk. And with agentic AI, Dr. Williams and the other experts we talk to say it's not so much about the risk as it is about the response. Can we properly respond and react to an AI attack or the fumbles of an AI agent?
- [18:07]
C
Once you get agents talking to other agents, you've got a system that's very difficult to test, very difficult to understand how it's going to behave. We're in a distributed environment where the agents may be on the same machine or they may be sitting across the world somewhere. We're in this asynchronous world where one agent is passing messages to another agent, telling it to do things. And as soon as you're in this distributed, asynchronous world, things become very difficult to test.
- [18:47]
D
So as we're seeing a rise in attack and a growth in the different kinds of attackers. Right? So, for example, North Korea, under sanctions, has made a key source of revenue for its regime hacking banks and crypto entities, specifically stealing large amounts of crypto to finance continued advancements in their missile program. They are an incredibly aggressive and creative group of cyber attackers. And they often, they use, for example, approaches like trying to get hired as software engineers or IT engineers within a network, so that when they're in that position, they can leave vulnerabilities behind or potentially open entry points for a second group of attackers. So when you think about a group like that, agents can be trained to look for an insider in a different way, to look for access rights to how software, you know, software repositories to ensure that those are properly set up, you can tune them in terms of heightened areas of threat or focus. Now, I want to make sure I note that that AI agents can also become a source of risk. Anytime you have an entity on a network with access to data, rights to operate on the network, they also become not only a target for compromise by an outsider, but also a source of risk.
- [20:13]
A
Which is probably why regulation is at the top of the priority list for lawmakers. Broad AI regulation, including of LLMs and agentic AI, is at the forefront of bipartisan discussions in the U.S. how do we ensure risk is minimized and guardrails are strong enough? I spoke with Lane Bess, former CEO of Palo Alto Networks, a leading cybersecurity company that provides solutions to more than 70,000 organizations worldwide. He has over 25 years of experience in this field.
- [20:44]
E
There are really no compliance regulations right now. We have heard people that are obviously concerned and warning about the proper use of AI, whether it means it's impacting children, it's impacting financial markets, it's impacting legal aspects. But there has not been enough time and attention to the actual compliance and regulatory aspect of AI. I don't want to call it the wild wild west, but now is a time that probably more time and attention needs to be placed towards compliance. So there are companies that I think are going to emerge that are going to help in terms of AI compliance, helping identify what are the guardrails for, how AI can be used and should be used, and allowing whether it be companies and or lawmakers to determine whether certain AI is staying within those guidelines. There's a lot of good that's going to come out of the use of AI, but at the same time in the hands of bad actors. Without the proper regulatory and compliance guardrails in place, there could be a lot of challenges that many companies and governments face.
- [22:03]
A
But as more and more tech companies slap AI onto their monikers, they investors pay attention. The promise of AI is transforming how we work and communicate. And part of the process of building a smarter and stronger agentic AI model is also about trusting it.
- [22:17]
B
A lot of people are nervous about using AI. A lot of people are nervous about the value that people, if they're going to be training your data. We both fundamentally believe that AI is going to transform how people are automating. But actually they're not wrong in it's. There's a lot of companies that are suddenly deciding, oh, I'm going to train on your data. We were born in security. So again, my background, our CEO's background, the company's background is like absolutely, security first. So we know those fears that companies have and we're not about to Abuse that trust. We've like it's been hard fought and hard won. So very simple for us to say like there's no training, there's no logging. You can, there's like no storage. We can, you can bring your own API key so you can bring your own AI if you want. It's all in region, it's all tenant scoped. It's really easy for us to say like we're not doing anything funky with AI, but also we know that AI isn't actually necessarily the answer for everything. So in the future of workflows and is it's not just plug in AI and it'll solve every problem the same way as automation didn't solve every problem. Automation worked with humans and like that deterministic automation, that rule based automation was the best way for a long time. The future now is going to be a combination of AI for what AI is good at. So understanding, maybe like providing advice like prioritizing up, leveling the team and making some like you know, correlation or contextual decisions. It may not be like taking action but for mission critical workflows you absolutely should be using deterministic method and if you're preparing a, you know, your performance review or a, a board deck or whatever like, or a financial report, you shouldn't be using AI at all. You should be using like a human being for that. In reality the future of like all of these business processes and the future of workflows is actually going to be a combination of all three. And in order to do that you need a platform that can offer all three and also like that level of control, that level of confidence, that level of like auditability history to be able to understand exactly what decision was taken, when and why so that it's not just like a black, black box for us. The future of AI is it being used in combination with like rule based automation or deterministic automation and like smart human like oversight and interaction.
- [24:34]
A
Companies like Tynes and Anthropic are propping themselves up as leaders of the agentic AI wave. They're acknowledging the risk of uncensored AI while also working towards what could be be a future effectively run by agentic AI and in many ways laying the groundwork for new companies.
- [24:51]
D
The key is really three things that entities deploying agents, first of all have visibility into the scope of agents deployed. Second, really implement the identities of those agents, what roles they have, what they're allowed to access and then three, being able to track what they're doing again, again at an enterprise level to ensure that it fits, particularly as they adapt, which is something that is a strength of agents. But as they adapt in response to changing cyber attacks, new information, new vulnerabilities, that their power to act autonomously on the network is still understood. In an ideal world, you know, companies that are building and deploying agents, it would be easier to bake in security from the start. One of the challenges we have in cybersecurity is that products are often sold understandably on the cool usability, the new kind of capability that they bring. And security is most effective and cheapest when it's baked in from the start. So I think perhaps this is the call to action, to say, as this new generation of really promising approaches in cybersecurity with agents deploy. Let's learn the lessons from the first generation of cybersecurity, where we deployed tech, whether in water systems and pipelines or in your average company, and then went back and said, oh my, how do we digitally secure this? And tried to understand where things were, what rights there were, who were the identities, et cetera, versus as we deploy agents, really taking the principles we've learned from cybersecurity, we only monitor what we can see. Having that enterprise picture, ensuring identities are enforced from the beginning and ensuring the visibility so we can get the promise while seeing where as things adapt, they may introduce new risks. I think that would be transformative.
- [26:48]
E
You know, the opportunity is really to use tools in a much more efficient way. The investment thesis behind a lot of companies is to invest in these tools because there's got to be a way to make them more productive, lower costs, whether it be operational costs internally or servicing customers.
- [27:13]
A
National security around the world is increasingly under threat, and the global conflicts shaping the 21st century are relying on more sophisticated technology. Israeli offense now employs AI to detect targets. Ukrainians have innovated drone warfare. And like Ann mentioned, more and more countries are tapping into the vulnerabilities of IoT, attacking the infrastructure that keeps our world running. A couple years back, we were struggling to understand the practical applications of AI. Now we're way beyond that. And it seems that a country's security, and maybe even our own security could rely on this tech. Still, it's not just about the kind of AI we employ, but how it's human guidance and smart decisions that will determine our relationship and the future's relationship to technology.
- [28:00]
D
Because in a post 911 environment, the intelligence community was very focused on connecting the dots, getting people to work together. You know, those kinds of controls weren't implemented with rigor. And what I saw was that fundamentally, you know, in every organization you want that balance of ensuring that you're protected from an insider, but also building a collaborative culture. And that plays very much into agents because an agent is just the digital equivalent of a human. So they need a human sponsor. They have to be operating under the equivalent of a human role and their identity enforced in that way as well. Because otherwise the autonomy agents can create new and really significant risk for the enterprise because they have access to multiple systems, they may have access to data, there may be observability gaps in terms of how people see what they're doing. So it's really important that that accountability be there in the guise of that human identity overseeing them.
- [28:57]
B
We're going to move fast into the era of companies realizing that AI isn't a product that you plug in. You have to actually identify. The initiative shouldn't be undeveloping AI. You're going to be identifying real business problems that need to be solved and focusing on the best way to solve a problem. And that could be AI, but it could be a combination of AI and a whole other tools. But yeah, moving past like general purpose LLMs and moving into solving real world business problems, that's going to be certainly in the business world, that's going to be the huge change.
- [29:35]
A
Thanks for listening to the Next Innovation. This series was produced by Situation Room Studios and Powered by Enterprise Ireland, investing in the next wave of innovation. Our executive producer is Christine Barata and our senior producer is Sharon Barreiro. Lysa Pena and Leila Sharoui are the associate producers. Additional production assistance by Global Situation Room and a special thanks to Lane Bess, Ann Neuberger and John Will Williams. I'm your host, Jennifer Strong. Until next time.