← Future of Life Institute Podcast
Future of Life Institute Podcast
Defense in Depth: Layered Strategies Against AI Risk (with Li-Lian Ang)
00:55:47
Loading summary
Transcript57 lines
- [00:00]
Lilian Angel
You only need, like, one person and a very powerful AI system to be able to do this from end to end. If AI systems are so good at spinning up apps, like designing web pages, doing the whole code review, what's going to happen to all the software engineers? At first, you become reduced to sort of like the managers of AI systems, and then over time, why don't you just automate what the managers do actually will own all of the value that gets generated by business. It will be like a handful of people. And there's a version of this world where it's, like, kind of dystopic. If I think too hard about it, I get kind of sad because I'm like, oh, my God, there's, like, so many things happening and it's all really horrible, and I wish I could solve it. The whole world feels like it's kind of on fire. So I tried to focus just on the problem that I have, like, right in front of me.
- [00:49]
Gus Ducker
Welcome to the Future of Life Institute podcast. My name is Gus Ducker and I'm here with Lilian Angel. Lilian, welcome to the show.
- [00:58]
Lilian Angel
Thanks so much for having me, guys.
- [01:00]
Gus Ducker
Fantastic. All right, you work at bluedot Impact. Maybe give a bit of introduction to your work there.
- [01:09]
Lilian Angel
So with my work at BlueDot Impact, I started working with BlueDot about three years ago. Back then, when I first joined, BlueDot was a small reading group that was being hosted in Cambridge in the uk, where people are coming together to discuss about what is this risk with AI that people keep talking about. Seems to be something like LLMs that were being created. And this was at the beginning of when ChatGPT was just starting to become a thing. But before that, there were many people who were already writing about the potential impacts of such a technology. And what this group of people in Cambridge were doing was compiling all of these writings from these different people and working together with Richard Noem to create a kind of curriculum or kind of, like, structure for just thinking through what could be the potential impact of AI on society. And then ChatGPT kind of came out, and more and more people started using these AI systems, and people started to get a lot more worried and feel like this is a bit more real. But at the time, it was still a lot more of a fringe concern. And then this group of people who were like the founders of Blue Dot Impact, were then given some money to scale this up and continue to train and bring more people into this. And over time, one of the things that we realized that was really important was that because AI is such a general purpose technology and has such a transformative impact on many aspects of society. We are going to need just like one way more people thinking about what we can do to ensure that the way that we build and roll out this technology actually goes well. And so this is kind of like turned into more of like our current mission statement, which is to try to build the workforce that protects humanity from these risks from AI.
- [03:05]
Gus Ducker
Yes. How would you say your concerns and the concerns of Blue Dot impact have changed over time here? What were you concerned about in the beginning and what are you concerned about now?
- [03:18]
Lilian Angel
Yeah, in the beginning, a lot of the risks felt kind of speculative in the sense that people were predicting that these AI systems would be able to do the kind of things that you see it able to do now. Just like code, like whole web apps from scratch, be able to very realistically synthesize people's voices, like generate images and videos. And now it's being used a lot in things like warfare or cyber attacks. And there's more like all the crazy, dangerous and scary stuff that people only thought were maybe possible in 10 years, 15 years time. So a lot of the work in the beginning was very focused around how do we get more clarity around what the risk actually is and measure it and then convince people that this is something worth paying attention to. And because of how rapidly AI systems have developed, a lot of now, the focus has shifted to, okay, we don't need to convince people anymore that this is happening because it's clearly happening. It's more of like, okay, now mobilize people. Let's really do something about this now.
- [04:23]
Gus Ducker
Yeah. And what about specific risks? Which risks are most salient to you now?
- [04:31]
Lilian Angel
Yeah, I think there are like four types of risks. Risks or ways, things that could go wrong that I and probably the rest of the team feel most concerned about. One is with engineered pandemics. So now with the ability of AI systems to help with synthesizing or discovering ways of synthesizing different kinds of molecules or pathogens, being able to help a lot with moving through how to do wet lab experiments, giving people a lot more biological knowledge than they would have otherwise needed to have, which lowers then the barriers for entry for people who might have previously wanted to develop these kinds of bioweapons and now are able to do it much more easily. So this is becoming an increasing threat where the barrier was low. And this is similar with the second thing which we're worried about, which is large scale cybersecurity attacks on things like critical Infrastructure where previously you'd need a really well resourced team of experts, but now Claude can just code up anything. You've already seen this being used with China in the Threat Intelligence report from Anthropic, how they were using AI agents to launch a full scale cyber attack on parts of country in Southeast Asia's infrastructure. And we are likely to see a lot more of this, especially for countries who don't have as robust cybersecurity practices.
- [06:19]
Gus Ducker
The third is with on the second one. How is this different from hiring a bunch of people to carry out these cyber attacks? What is the addition advantage from the perspective of the people perpetrating these attacks?
- [06:37]
Lilian Angel
I think it's like you need a fewer personnel and the speed at which you can do it is a lot faster. Where there are lots of steps that are required for you to be able to launch a cyber attack, you need to first be able to find the vulnerability, be able to exploit this vulnerability within a system and then deliver some kind of payload that destroys, takes down or extracts data from that system for you to cause some kind of harm. And previously for each of these steps, it's a puzzle almost to solve. And when you only had human hackers, you would need them to just figure out themselves. Okay, for every task you need a person on hand. But if you have an AI agent running in the background who can already automatically do these tasks by itself and maybe even coordinate a group of agents to just do this, you only need like one person and a very powerful AI system to be able to do this from end to end. Yeah.
- [07:31]
Gus Ducker
Yes. I'm wondering if it's mostly a speed advantage or if it's a cost advantage where we are lowering the cost of carrying out cyber attacks or we are enabling these cyber attacks to happen so quickly in a way that they couldn't before.
- [07:50]
Lilian Angel
Yeah, I think it's a bit of both. Well, one of the speed aspect is because as much as the attackers are going to have access to AI agents to be able to launch the attacks, the defenders are also going to have access to similar technology. And then over time it will be a matter of whose agent is better, which is kind of similar to just how the whole cybersecurity space is just overall it's not totally novel. And it then moves to a question of who is able to attack or defend much faster and who has more resources to be able to strengthen each side much better and protect this kind of attack defense balance, as people say.
- [08:38]
Gus Ducker
Yeah, I'm more hopeful if it's mostly a matter of cost. Because if the defenders can also say large corporations or large nation states are often able to out compete sort of smaller access when it comes to just spending a bunch of money on security. And if that's, if that's the main advantage of using AI agents for hacking, then it seems like the current balance will hold. But if it's the case that AI agents can move much quicker in a way that's more difficult to defend against even with your own agents, then maybe the picture changes. What's your sense there?
- [09:27]
Lilian Angel
Yeah, yeah, I'm not, I'm not super sure. Yeah, I'm not super sure. Partly because like there is like a cost aspect to it in the sense of you want to be able to run these models and that might require, I guess, what would be the cost factors involved in this one. There is an existing kind of zero day or a security vulnerability market that just kind of exists that people can exploit and then you can pay certain brokers or people who are hackers online to be able to find these vulnerabilities and try to exploit them. But at the same time there already exists many known vulnerabilities in many open source software. And oftentimes it's a matter of figuring out what are all the different vulnerabilities that do exist that haven't been patched yet, and how can you string them together. And to me the compute cost for this doesn't seem terribly high or it isn't going to be in the millions. It's probably too out of reach for a non state actor. The trouble will probably be with actually trying to string together the complexity of all of these different zero days to actually have to mount a proper attack. So I think AI agents as they are today may not be able to uplift to the point of doing it autonomously, but I think every day they are becoming more and more effective at doing this, such that at some point you may just be unlucky and they will find the right combination or be able to find the right opportunity to be able to do this.
- [11:23]
Gus Ducker
My intuition here is also just that this is a task like patching together different vulnerabilities is like a task that AI agents would be pretty well suited for solving. It seems like they would be good at writing a code or sort of glue code that makes the pieces fit together. And so that makes me a bit more worried now that you're describing the problem like you are. Do you think AI agents are especially good at that?
- [11:58]
Lilian Angel
I think they are good at finding patterns in large amounts of Data that is their comparative advantage against people. I think what has made AI agents a lot more powerful today than they were before is just like we have poured a lot more resources or a lot more training techniques into them being able to do the computations more effectively. And if it's a matter of solving these puzzles, the better that we make their thinking processes, the better that we teach them how to solve problems and split apart into different agents who think and break down the problem to different things because it's useful for them to help us with other work. It's just going to therefore increase their ability to then solve these problems overall, which is kind of crazy.
- [12:57]
Gus Ducker
Yeah, we can't really get the nice benefits we would like out of these models, helpful code writing and so on, without potentially also getting harmful capabilities, is what you're saying. Yeah, yeah. That's sort of a quirky snacker, actually it is. Is there some hope of separating the two or containing the abilities that allow AIs to do cyber attacks in a way, for example, where we tune the model, so to speak, so that they are better at defending than at attacking.
- [13:44]
Lilian Angel
This is a strategy that some people feel excited about, which is like, if we can differentially accelerate or put more resources into defenders being able to better use or develop AI systems that are much better at defending, then they will always have the upside against an attacker. And the kind of limitation to this strategy is like, where is the limit of this? At what point will will reach a ceiling where it just can't have that much of advantage against an attacker? And will we need something way more robust, like formally verifiable mechanisms that make a system secure no matter what kind of attack? Because lots of cyber attacks rely on the fact that you will never meet this edge case. Or this is just really unlikely, because the more secure you try to make a system, the less kind of user friendly that it becomes, the harder it is for you to edit the code or understand what it's doing. And so you're making a trade off with how usable this thing is or how easy it is for a person to interact with it against some security risk. And will this mean that one day there will be a paradigm shift with how we think about cybersecurity overall? I think this kind of ties into this overall idea of what makes this dual use technology really difficult is that it's not so much of there is a definite solution that all we need to do is find this golden rule for how we should be developing AI systems and then everything will work out just Fine. Because that's just not how it works. It's way too complex for you to just say that's just like one path forward. It's more of like what are the trade offs that we're willing to make to be able to use this technology in such a way where we gain a lot more of the benefits than kind of the harms. Very hand wavy, so to say.
- [15:53]
Gus Ducker
No, no, this makes sense. And this also, this aligns well with Blue Dot's defense in depth framework for sort of, we have multiple layers of defense. Some of them will fail. Hopefully all of them put together will allow us to stay safe. Because AI will impact everything. Basically the risks will pop up everywhere. Everywhere. And we will be, will be sort of, we need to take a route where we are defending ourselves in various different ways in a layered. In a layered way. And so, okay, so those are the two risks that you're worried about. Bio and cyber attacks. What are the other two?
- [16:39]
Lilian Angel
One is about what people call sometimes gradual disempowerment. Or you can also conceive of it as massive job loss. Or what are people going to do when AI systems do all the jobs that people usually do. And it's kind of like the way things feel like they're going where, if AI systems are so good, I mean, you see this mostly with software engineering right now. If AI systems are so good at spinning up apps, designing web pages, doing the whole code review, what's going to happen to all the software engineers and extrapolate this other fields. What's going to happen to this entire field of people? What are they going to do? Or like, where will they go?
- [17:24]
Gus Ducker
The worry here would be that if you, if you say you're a coder right now, maybe you feel like you are becoming, you're being reduced to sort of like a rubber stamp of saying, okay, Claude code, please proceed or implement these changes. And perhaps, you know, Claude code is so good that maybe you don't even need to check the code and then extrapolate that to lawyers. Okay, this draft of a contract, it's actually good. I trust the AI doctors. This diagnosis is probably good because the AI is almost always right. And so suddenly we are not being active participants in the economy in the way we are now. And we are sort of atrophying cognitively perhaps also. What is your main worry about disempowerment or how would you frame disempowerment here?
- [18:14]
Lilian Angel
Yeah, I think kind of happens in the stages. Right, but where at first you become reduced to sort of like the managers of AI systems. And then what happens is that you see, like a lower. You just don't need to hire as many people. Like, if you've got. You're like managing a team of interns who do most of the work, and you're like, as productive as, like, the other staff. And the next business is not going to be hiring just as many people. And the biggest cost usually of any business is like, hiring staff. For you to compete and to be able to thrive in the market, you're going to need to be cutting down your own staff costs. And so this is like reducing kind of like the bottom, and you're having more people to be more like, of a managerial layer. And then over time. Why don't you just automate what the managers do? They go through a process, right? They just check the code, they look through these things. Okay. Then you just have fewer managers, and then you have just the C suite. And the C suite's job is also something that you can kind of encode over time and figure out the processor that it runs. So then what happens at the end? Who actually will own all of the value that gets generated by a business? It will be, like, a handful of people. And all the businesses are going to be paying money to these different AI developers and AI providers, and a lot of that money is going to be pouring into them. And then what happens to the mass of other people who suddenly find themselves struggling to find a thing to do with their time? And there's a version of this world where it's like, kind of dystopic. It's like, oh, God, I have no money. I have like. There's like a massive unemployment and there's a lot of just like, upheaval. But it's also like a positive vision of this, which is like, you know, like, I don't like all these jobs that have to be done in the world if people don't have to do them. You know, like, what other exciting things would they be doing if they didn't have to work, if they could choose what they spend their time with? And how can we design for that kind of future instead of the dystopic one? This is the kind of thing that I'm most excited about.
- [20:18]
Gus Ducker
I mean, just say the future economy goes well and we have maybe. Maybe because a lot of people in the present world have a lot of investments, and maybe those investments just skyrocket because of AI productivity gains. And so there's concentration of power, there's concentration of capital, but there's also widespread benefits because again, many people have investments in the stock market. Even in that situation, I do worry about us losing control of what happens just because we've now handed over power to these automated firms. Where what happens in the future? Well, that is a product of what these automated firms decide to do. The AI AI CEO is trying to optimize for profit and impact and future power and so on. And this is a process where we are not really a part of it, but we are maybe sort of relaxing and the money is coming into our accounts and so maybe we've sort of handed over the future to the AIs. Is that an accurate description of the the kind of disempowerment you're worried about?
- [21:39]
Lilian Angel
Yeah, that's also another version of it. There's one version that I was describing earlier which is dystopic where it's like a few people hold most of the power and they make all the decisions. And a different version of that as well. It's like what you mentioned, where if it's an AI system instead, who is then making all the decisions? We are then relying on the fact that these AI systems are actually than running all these processes or running all these jobs in a way that is actually beneficial to us and that they won't someday do some quirk of the way that we design them or some way that they end up deciding to act because we don't fully understand how these AI systems work, decide to go against us or to do things that do more disempowering to us rather than helping us live lives that we feel more satisfied by. And I guess we don't really know the answer to how it will happen, but we can't say either way. We're certain that it will or it won't. But there is a real possibility that this could happen.
- [22:44]
Gus Ducker
Yeah, we could expect sort of a crowding out effect where human interests are sidelined. And you see the automated firms, the AI CEOs deciding to spend more and more resources on data centers and energy production, the stuff that makes AI more productive and gives AI their ability to sort of impact the future. Because that is, there will be market pressure to optimize in that direction. And perhaps suddenly we're not that important for the economy going forward. What options do we have for remedying this?
- [23:27]
Lilian Angel
There is one branch of thinking that if we could ensure that we encode the things that we value into AI systems, we can make sure that they always make decisions that match the kind of values that we have as Humans. So if we value happiness, freedom, autonomy, and in what priority order to what portfolio of things and what edge cases kind of has the human spirit in the same way that we do, then perhaps it's fine for an AI system to then be making these really complex decisions for us because we know that it will act in our best interests. And this is one branch of work that people are doing. Another option is to ensure that we design a future where humans are always going to have a say. So we very intentionally use these AI systems in a way where we never lose touch with how they are being used. Somebody always needs to be signing off. There are people whose jobs it is or task it is to ensure that the way that we use these systems in our daily lives are actually going to have people's inputs and will benefit us ultimately, or to keep check on these AI systems power and have kill switches with them if they ever do turn tail on us. But it's obviously like it feels kind of far fetched right now or it feels kind of hard to imagine what that future might be. And I think a lot of people's pushback might be. Of course we would never allow this to happen, or of course we would always want to, to be in charge and we would never just allow an autonomous system to make decisions for us. But we forget that oftentimes we're very happy to give over control of hard to do things and just mindlessly scroll on our phones or have people decide things for us. If it's easy.
- [25:34]
Gus Ducker
Yeah. I worry with the second sort of possible solution there where we are, we are constantly asked by the AI, do you want to do this or do you want to do that? So we are sort of, you could see us as the, you know, maybe there's some, this is a cartoon version of it, but maybe, maybe we have a law that says that CEOs must remain human and they must sort of make the crucial decisions. Even in that situation, there will be market pressure to create automated firms. Right? Because the automated firms will be able to. If the AI CEO is just more competent and just can act faster because the human CEO becomes the bottleneck, well then the economy will probably end up with more automated firms versus firms with human CEOs. And on the first solution, I guess this is just sort of the problem of loading our values into the AI is just a classic problem in AI where we need to solve the technical aspect of the alignment problem. But then we also need to decide which values we need to have the AI sort of act upon and so of course you know that there are problems with both of these approaches. But where are you most hopeful and do you see a path forward here?
- [27:06]
Lilian Angel
Where am I most hopeful? I think the difficulty or the sheer complexity of thinking through this is one of the reasons why Blue Dot strategy is not like, let's figure out what the plan is and execute on this perfect plan. And once we execute on this perfect plan, everything is going to go well. There's just so much I feel like I don't fully understand about the way that AI could impact the economy, social relationships between people, how I expect different countries to use it, the really technical nitty gritty of what's possible. I feel like a lot of the role that I have or a lot of the role that I feel like blued out is trying to play is more of enabling more people to have the right kind of skills and knowledge and community to be able to make really good decisions and to bring together expertise from across fields, bring together people who care and are committed to this, to really think hard about all the different permutations of this, work together and collaborate on what are different ideas for solutions that we could have, test out small versions of those and then iteratively work towards a future that works well with people who both people who dream really big and expansively about. These are all the different ideas of things that we can pursue to people who think in the more present day. What's the next step that we could take? What are initiatives that we could try right now that could give us better clarity? And this isn't super answer your question from before, of which it is, and that's because I also feel quite stuck on. It just depends on so many different things. It's hard for me to say which I feel more bullish on.
- [29:03]
Gus Ducker
What about your fourth sort of risk area that you're concerned about?
- [29:09]
Lilian Angel
Yeah, we talked a bit about the power concentration. So graduate empowerment is about people kind of not feeling like they can they meaningfully contribute to society or have meaningful meaning to their lives at all. And then the power concentration aspect of it is people could be using these AI systems to try to gain more power for themselves. And we see this in the use of AI in military and warfare, but also in misinformation campaigns or trying to disrupt elections where if a lot of the basis of power is democracy, if you kind of disrupt people's ability to make informed decisions based off of evidence or informed decisions, I suppose, on what they want and you kind of distort their vision of reality, then that Kind of breaks the foundation of what a democracy is and therefore the kind of basic foundation where most institutions have their power from, which is like the trust of the people and the vote of the people will fall apart. And this will lead to a lot more just like instability. And with warfare, it's just like if one country is way better military than the other, that's kind of like game over for a smaller country. And you kind of have to give in. And this is just like with people. And you can think of this as like AI systems as well. Like if you were to concentrate power into an AI system and if you. We were wrong that these AI systems actually do things that we believe are good for us and instead decide maybe it's better if people no longer exist on Earth and are able to enact that kind of plan because we were not careful. That could also be quite bad. That could be quite bad, yes.
- [31:14]
Gus Ducker
So one underlying concern here is the issue of just the pace of change in AI. So many of these risks we've been discussing here, in some sense this is already happening, right? Anthropic is already encoding values into Claude via their constitutional approach. So AI is already quite capable in cyber and in bio, and there's arguably already some power, concentration and disempowerment happening. So what's the role of an organization like Bluedot here, where you're not inside of any of the AI corporations, What can be done from the outside?
- [32:04]
Lilian Angel
So these like frontier AI companies, so companies who are building these extremely powerful models. When you work inside of them, some of the levers that you can pull is that you have access to these models. And so perhaps you'd be able to run more different kinds of experiments because you have direct access. You could potentially have some amount of sway with what kinds of features go into the model, what kinds of safety precautions or break glass measures that you have. But a lot of these companies are limited by also market pressures. The company still needs to survive and they have to navigate things that are more to do with keeping their company afloat and in good standing over time with the investors and less to do necessarily with purely altruistic or good reasons. I think this is where the role of independent labs or independent think tanks and other organizations come in. Where you are not where you can give a more like rigorous evaluation or rigorous like research on this, where you are where you don't feel like the work that you're doing is being governed by needing your research pitch to bring in more revenue or to capture more of the market. And this kind of research can then be fed back into these organizations and hopefully influence the kind of things that they do or feedback into policy work to inform policymakers on how they should address these risks based on how much certainty that we have over them. And not just people who work on technical versus governance, but it's also people who work on a third layer of the defense in depth approach, which may be, I should explain, but work more on like if we are unable to train models to be, train the dangerous actions out of models and if we are unable to prevent these actions from being taken in the world, what are the kind of defenses or societal resilience mechanisms that we can build that would still ensure that, that we are safe or that we live good lives? And these are things like pandemic preparedness measures, better cybersecurity practices, different kinds of economic or social policies, different ways that we maintain the integrity of the information environment and things like this. That is not technical work or governance work, but it's still extremely important.
- [34:52]
Gus Ducker
Yeah, yeah, yeah, that's, that's interesting. I, there's an argument to be made that right now is a very sort of. You have a lot of influence if you're a researcher at one of the frontier AI corporations and perhaps if, if they succeed in automating AI research fairly soon, then your influence is going to disappear. And so maybe you, you are. Right now, if you're able to influence the companies from the inside, you have sort of peak influence right now. This will be reflected in their salaries and in the fact that you see the companies doing a lot to attract talent and being very careful about how they navigate sort of scandals in order to not lose talent and so on. Do you think right now we are at sort of peak influence for the researchers inside of the corporations?
- [35:51]
Lilian Angel
I would push back on that. I guess. I feel like if you've ever worked in an organization, it's really not that easy to change what the leadership level wants. There are micro decisions that you can make, there are things you can advocate for within the company. But I think it's a lot harder than most people think than just joining OpenAI or XAI and saying like, I'm going to change how they approach safety from the inside. I don't think it's impossible, but I think it's a lot harder than some people might think. I mean, I've, you know, like, you know, like caveat. I've not worked in one of these, you know, like frontier companies so far, but this is just like things that I've heard from other people who have worked in these places is that the kind of political games that you might need to play within these companies and the kinds of efforts that you might need to put in to be able to really steer it in a way that you want is not something trivial. And so with that kind of effort that's required within companies itself, I think it lowers to me how necessarily impactful it is to be joining a company like a frontier company, rather than say like an independent lab, for example.
- [37:19]
Gus Ducker
Yeah, yeah. Do you have any. So you can anonymize these? Of course. Do you have any interesting stories of how it is to try to influence these companies from the inside?
- [37:31]
Lilian Angel
Yeah, I think I've heard from one of these companies that the way that they operate is quite chaotic. So it's kind of hard to know who are the people working on particular projects or who are the people who actually have a say or will give the final verdict on whether this project goes through or how much resources are going to be devoted to it.
- [37:57]
Gus Ducker
Yeah. This is why people are now just member of the technical staff as opposed to say, working on this specific. Exactly.
- [38:04]
Lilian Angel
Yeah, exactly. And you don't always have the ear of senior leadership to tell them, I want this done and to make a case for that. Because it's growing. These companies are growing to be thousands of employees and having offices all around the world. It's tough.
- [38:27]
Gus Ducker
Yeah. So we're bringing up a bunch of problems here. What, what would it look like for us to be on a better path? What does the world where AI development is going well, what does that world look like?
- [38:46]
Lilian Angel
Super idealistically, which is hard, obviously. But there's one proposal which is like, oh, if AI companies make some kind of safety commitments, that they will definitely develop their things in a safe way and have all agreed on what this is and that there is some accountability mechanism from a government or some other body that will ensure that they do these well, then we can ensure that this kind of technology will be built in a so called safe way. But this, I think is. It feels very ambitious and hard to achieve. I think it's a good direction to be on where some of the motivation for doing this comes from the companies themselves. It will work. And if there is some other outside mechanism that keeps them in check, that will be good. But I think realistically there will be cases where we just do not catch certain kinds of dangerous capabilities that models are able to do because we have not properly estimated or not properly evaluated the risks that these models could have or that One of these companies just ends up defaulting on their agreements anyway because they get a crazy leader and they just decide to do whatever they want. And if it gets rolled out, it gets rolled out. Right. It's kind of hard to just pull something back, especially if they make a model open source. That's kind of it. How are you going to delete that from every single person's hard drive in the world? Which is then why we have this kind of defense in depth kind of thinking at bluedot, where we obviously want to plan for the best case scenario where we're able to actually control how these models are developed and they're developed in a safe way. But realistically it's not going to happen perfectly. So then can we make sure that we're able to detect if a dangerous capability or if something like dangerous actions are being taken in the world and to do this in a robust way and barring our ability to be able to detect these. Well, this is where the kind of withstand or societal resilience kind of layer cancer kind of comes in. Where given that these two layers have lots of quite big holes in them where things can just kind of still slide through, can we make sure that we have good layers at the end that kind of prevent the worst things from happening still? No.
- [41:26]
Gus Ducker
Just remind the listeners here, what are the three layers?
- [41:30]
Lilian Angel
Yeah, so the first layer is to prevent the training of dangerous AI systems. The second is to be able to detect if these systems. Sorry. The second layer is to be able to detect if these systems have in fact still taken dangerous actions anyway, despite our efforts to try to train it out of them. And then the third layer is to be able to withstand any kind of dangerous actions that we weren't able to stop in the first two layers.
- [42:07]
Gus Ducker
Yeah. And so I'm genuinely in doubt here about what the right approach is when we see that there are certain lines that have been crossed. For example, we have the best open source model is sort of a lower bound for which capability will be out there for anyone to use. And the best open source models are quite good now. Right. And so this, this sort of, this means that some, some plans for say developing AI in a, in a very controlled environment with international input and only releasing models when we are entirely sure they're safe and so on. That is, that is, that's probably not, that's not going to happen now. And so I'm, I'm sort of watch. What, what do we, what do we make of that? What do we do with that? Do we then Just go to the next layer and, and sort of downgrade for how safe the world is going to be, or do we try to. Yeah. What's the right approach here? I generally don't know.
- [43:06]
Lilian Angel
Again, yeah, it's hard to find the right approach. Earlier we talked a lot about if all the development of AI systems were constrained to these five companies. If we could control these five companies, then maybe things will be fine. But then open source, open weight models, all these things add an additional complexity to this because you don't even know how powerful a person's model is because there's a proliferation of these around. And one of the proposals that I've heard around this is in a more decentralized fashion, can we ensure that all the open weights models, the people who maintain these open weights models are all bought in on the idea that we need to do these kinds of safeguard mechanisms on these. And we have people who go around and test all the open weight models that are available and make sure that they follow these guidelines and that they're always able to be safe. But obviously this is not, this is not a robust method at all. And this is why I also think that as much as we try to minimize the risk in this area, we're still just going to need to build out the rest of the layers of defense because it's just unrealistic to believe that we could ever get 100% guarantee on these earlier fronts. We can try to reduce it as much as possible, which is why people try to do methods like trying to remove specific capabilities from AI systems from doing cyber attacks, or to be able to like build unlearning mechanisms, remove particular knowledge from them, or to build it into their personalities such that they would never take these actions because they have it encoded in them that this is morally wrong or something. But they're not 100%, but they're helpful to a degree.
- [45:16]
Gus Ducker
How do you. And how does Bluedot think about betting on interventions that might be ready in time? Perhaps some of the AI CEOs are saying that we will have AGI this year or perhaps next year. This constraints which research lines might be worth working on? How do you guide people and think of where it's worth spending money and allocating talent.
- [45:52]
Lilian Angel
I feel like the timeline that people have or how long people feel it will take until we have fully automated AI or some level of AI ability, or how quickly AI systems will be able to do some percentage of all human work differs from person to person. And it also just changes as we get more information and it's a very constant thing. And I feel like people. I have lots of chats with our graduates, trying to help them figure out what to do next. And lots of them do ask me stuff like this. It's like, oh, you know, like, how long? More like, what timeline should I be operating on? Like, what do you think I should focus all my energy on? Like, you know, like, what kinds of techniques are going to work best? And to be honest, I don't know. Again, it's like a really, like, complex picture. And I don't. No one knows. No one knows. And I don't have all the answers. And the best thing that I feel that I can do for people is to. To point them towards resources or ways of thinking that can help them get closer to the answer, to help to connect them with other people who are thinking about this. It's like last year, bluedot, we spent a lot of time trying to figure out, surely there must be somebody who has a plan who can tell us what is the priority of work that should be done. Should we go all in on macinturp or more just trying to align models. Should we just do more control? What's the percentage? Somebody should know what's the plan for how all these things mix together. And in talking to tens of experts in AI safety, no one has a plan because no one knows, because it's a hard problem to solve. And I think the way that I've gone about trying to advise people on this is to basically just gain more information, connect with people who are working on this, and then to work out from the information that they have what is the best next thing that we could be doing and then to constantly iterate on this. And I wish I had a better answer that would make my life so much easier. I would just know exactly what kind of courses to run, what kinds of training to provide and where to point people to. But
- [48:31]
Gus Ducker
I guess a general piece of advice here is just also to stay sort of stay flexible in what you're working on, adapt to what's happening, be able to sort of abandon projects that might not seem helpful given the circumstances now, and sort of be prepared for things to change and change quite quickly. On that note, how do you just, in a personal sense, how do you deal with the pace of AI and uncertainty about the future? We've brought up so many sort of unsolved questions here and. Yeah, how do you deal with that?
- [49:17]
Lilian Angel
Yeah, I think day to day, if I think too hard about it, I get kind of sad because I'm like, oh my God, there's like so many things happening and it's all really horrible and I wish I could solve it. And if I think even just like beyond AI safety, where AI safety is just like one class of problems, the whole world feels like it's kind of on fire. But I think it's kind of debilitating or makes me feel unable to act if I think this way. So I tried to focus just on the problem that I have right in front of me, or the scope of work, or the area that blued out has chosen to, to invest its time in, which is trying to build the workforce, bring more people into the field, equip them with the right knowledge, skills and opportunities, and to get more people working full time on this problem and things outside of that. I touch on as much as it takes to help us on this, but ultimately if I try to think of the whole thing all the time, then I'll just be very depressed.
- [50:19]
Gus Ducker
That's good. We sort of covered a bunch of ground. Okay, that's good. As a, as a, as a final topic here, I would like to talk about some of your other sort of non AI related projects. You at one point wrote a novel called Purple is the Noblest Shroud. Tell me about that. And tell me, why did you, why did you engage in that project?
- [50:45]
Lilian Angel
Yeah, I think, yeah, some wild, wild times. So when I was in my third year of uni, it was Covid time. Like Covid had just kind of happened and I was feeling like, oh, what am I going to do with all my time? And I found this like, writing program that promised that, you know, like, they would help you publish a book. And it was always been like a childhood dream of mine to write novels because I love reading. I'd like spend every night, every night without fail, I spend like 30 minutes to an hour reading a book. It's like one of my greatest joys in life. And I love stories. And I thought, you know, I've got time, why don't I just do this? And I had watched this YouTube series about this Emperor Justinian, who is like the emperor of the 6th century Byzantine Empire at the height of this like, so called, like Eastern Roman Empire. And in the story of him, he had married this woman called Theodora, who came from a very like low class birth. Standing and marrying her didn't really help much with his social standing at a time when he was still kind of like unstable when he first got onto the throne. And the most striking thing to me is that there were many, like little tidbits in his history that indicated that he really, really loved her. Like she passed away and he lived on 20 years after her. And even though she never bore him like an heir, he never remarried anybody. And to me, and, and like every year he would have a big procession to like pay respects to her in this like, big funeral, a big funeral ceremony. And I was like, why? Why does this man love her so much? There must be something really amazing about this woman. So I decided like, okay, I'm just gonna like, figure out everything I can about her life and try to reconstruct her entire life and finish this book project in a year so that I can graduate from university without having to think about having this like, loose start. So that was like the challenge I gave myself. I'm going to write and published his
- [52:57]
Gus Ducker
book in one year and that's quite an achievement. On just returning to AI for a second here, how does it feel? Sort of. I'm assuming you wrote this with little or no AI assistance? Yeah. How does it feel to see AI becoming sort of better, at least at writing, potentially be able to write a book quite soon?
- [53:20]
Lilian Angel
Yeah, it's crazy. I always think back also to my degree. I did double major in data science and history and I think my entire degree experience would have been so different if I had AI tools. One of the tools I use most in my work right now is a speech to text translation tool, because I think better out loud. And that was basically the whole way that I wrote my novel was to just blab out all the different sections in not thinking too hard about my word structure or trying to wordsmith things to make sure that I had a really concrete structure for the entire book before filling it in with prose. And I think I could have gone through that process so much faster if I could have a really reliable speech to text translator other than the one that I was using over four years ago.
- [54:15]
Gus Ducker
Yeah, I sometimes think back to my university experience, which was quite a while ago, but still, I mean, just taking like an intro to Python course is just so. Just kind of learning everything by hand, constantly failing to get the syntax correct and then just so, so difficult compared to what you can do with AI today. But. But yeah. Anyway, do you want to leave our listeners with some places they can go if they want to learn more about Bluedot and perhaps are interested in what you have to offer there?
- [54:56]
Lilian Angel
Yeah. So if you are concerned about AI and the way things are going and you want to do something to ensure that we have a much better future with AI and to play an active role in shaping the kind of future that we might have. I would encourage you to take a look at our AGI strategy course@bluedot.org it's a free course that we offer where you can apply for it and you'll go through a kind of like structure, together with discussions facilitated by AI safety experts and a group of other peers who are as excited as you about helping to make the future go well. Yeah, so check that out if you're interested.
- [55:42]
Gus Ducker
Fantastic. Thanks for chatting with me. It's been great.
- [55:46]
Lilian Angel
Thanks.