← 80,000 Hours Podcast
80,000 Hours Podcast
GPT-7 may democratise bioweapons. But we can defend ourselves anyway. | Andrew Snyder-Beattie
02:31:11
Loading summary
Transcript511 lines
- [00:00]
A
This is an elastomeric respirator. I could wear this six months in a row without needing to change the filters. The shelf life is 20 years. If a government were to purchase a bunch of these, that gives you basically 20 years of protection, that you're protecting your population. The protection factor for something like this is about a factor of 100. So that means it filters out 99% of particles coming in. This right here is propylene glycol. It's a chemical that kind of disrupts the membranes of pathogens. It also dehydrates them. The we have enough of it to cover basically all industrial floor space in the US plus a wide variety of residential space. 24. 7. It's in fog machines at, like, Broadway shows. It's extremely safe. Hypochlorous acid is also an interesting method of sterilizing things. You can make this at home using salt water and electricity. You don't want to get it in your eyes, but it's like, totally safe for skin. You can, like, put it on your face or hand sanitizer. Philanthropists, like, potentially, like, cover their entire countries. You know, if there's like someone in Norway, it's like $50 million. And like no pandemics in Norway, there's another type of offense, dominance, which is no matter how much the defender spends, they get through. And I think nuclear weapons are a good example of that biology. I do not think that that is the case. There are tractable things we can do. I think some of them could be surprisingly affordable, at least to buy time for the more expensive countermeasures to come online.
- [01:24]
B
Today I'm speaking with Andrew Snyder Beatty, who runs Open Philanthropy's biosecurity program, which has so far dispersed hundreds of millions of dollars and is looking to disperse hundreds of millions more. Andrew has spent many years, I guess at least eight, possibly, possibly more, thinking about how to prevent human extinction from the worst biological catastrophes. And his team at Open Philanthropy has come up with a concrete plan that they think can drive down those risks by at least half, possibly a whole bunch more. They call this plan the four Pillars strategy. But it does have quite a lot of crazy sounding components to it. Crazy sounding components that we're going to interrogate today. Thanks so much for coming on the podcast, Andrew.
- [01:56]
A
Great to be here. Thanks for having me.
- [01:58]
B
What are the threats that keep you up at night that really make you worry?
- [02:02]
A
Yeah. So one interesting place to start might be thinking about the historical Soviet biological weapons program. So in the 70s and 80s, the Soviet Union Basically had tens of thousands of scientists, in violation of the Biological Weapons Convention, coming up with the craziest possible weapons that they could come up with. So things like combining smallpox and Ebola into a chimera virus, creating strains of plague that were resistant to 16 different kinds of antibiotics, or putting things that would give pathogens an autoimmune reaction which would make it very difficult to counter. And all of those things were things that the Soviet weapons program was doing in the 80s. And so you can imagine with 40 years of additional technology, the possible biological weapons of the future are much scarier.
- [02:54]
B
If a bunch of those biological weapons had been released, what would it look like?
- [03:00]
A
Yeah, so, in fact, we don't need to have hypotheticals here. There were a number of accidents in the Soviet Union. One of them, they were testing a smallpox weapon over the ocean, and it hit this fishing boat. And then when the fishing boat got in, they had to quarantine and vaccinate everyone. And it's very fortunate that it ended up getting contained within that. There were a number of other accidents as well. You know, there was this big Sverdlovsk accident of anthrax, that anthrax plume hit over a city. But all of these were relatively contained because, again, these were kind of weapons from the 80s. You can imagine much worse things happening in the future as well, if there was something that was contagious and very deadly.
- [03:48]
B
Yeah, I mean, if you managed to release an Ebola that was much more contagious and actually was able to spread using the respiratory tract rather than only on surfaces, I guess. Do you think these things could plausibly cause actual extinction? Could they kill everyone or close to everyone sufficiently? You kill enough people that eventually civilization just falls apart, and then it's questionable whether humanity continues?
- [04:10]
A
Certainly, yeah. So I think there are a few things going on here. One is you might think that civilization is kind of like riding a bicycle, where you need to have momentum, and then if everything kind of falls apart, it's very hard to put things back together again. But you might also think that biological weapons could directly kill very, very large fractions of, you know, the population on Earth. There are a number of ways that this could happen. One is that perhaps the pathogen is spreading for a long time before we even know that it's spreading. You could take HIV as an example. You know, we discovered that in the 80s, but it had been circulating for well over 50 years before we discovered it. And, you know, fortunately, it was not some airborne virus. But had it been, you know, maybe we'd be in a much more dire situation. Situation. Similarly, you could have pathogens that are not just spreading from humans to other humans. You could have pathogens that are kind of persistent in the environment. There have been a number of species extinctions, like frogs that go extinct due to these various fungal infections that are kind of pervasive. And then finally, and this sounds a bit more sci fi, but Ryan had mentioned it in the previous podcast.
- [05:19]
B
Ryan Greenblatt.
- [05:20]
A
Yeah, exactly. You could imagine that an AI system might, you know, be willing to gamble on some very risky strategies, including kind of knocking down humanity with biological weapons in order to rebuild from the ashes. And so even if the system can't get everyone in the first go, you know, maybe there are follow up attacks following that.
- [05:41]
B
Yeah, this is something that people have speculated about that one advantage that an AI would have is that it's not vulnerable to any of these biological catastrophes at all. So if it really is something that destroyed humans or most biological life, then that could potentially put it in a much better position to take over. Or even just threatening to saying, I've gone rogue, I've got biological weapons that could kill you all puts you in a very strong negotiating position if you're trying to not get shut down. I guess it's all a bit speculative, but I guess worth at least considering.
- [06:06]
A
Yeah. And I think this is an interesting argument for why even some of these response things could at least help with a deterrence by denial strategy to make these biological weapons less appealing for future AI systems.
- [06:17]
B
Are these biological weapons programs still going on? Does Russia still have one now?
- [06:21]
A
So the State Department has publicly stated that they believe that Russia and North Korea both have active, ongoing offensive weapons programs.
- [06:29]
B
Ok. But I guess it's not public exactly what they're doing. We just have to speculate based on what has come out about what they were doing in the 80s.
- [06:35]
A
Yeah, yeah. So, yeah, you can read about the history of the weapons programs and occasionally there'll be publications that scientists come out. One interesting thing is that a lot of the Soviet program, a lot of people in the weapons program didn't even know that they were part of the weapons program. There were kind of these like concentric circles. And so, you know, they had this giant institute that was studying plague. And most of those people were just studying plague, thinking that they were, you know, kind of reducing the risk of plague. And then there was this inner circle that was using all those research results to, you know, figure out how to weaponize it.
- [07:09]
B
I guess a lot of people have Said it doesn't ever really make sense to use biological weapons because they'll almost always, you know, if you release some super plague, it's going to blow back on you. Right. So what's the tactical or strategic, the use of these things? Do you think that is a strong argument to think that North Korea or Russia possibly that would develop these things, but that there's no way that they would ever release them?
- [07:28]
A
Yes. So I do think this is a strong argument. I think the strongest counterargument, I think there are two, one is you might think that it would be useful as sort of a second strike weapon. So you might have kind of like a layered defense. And that was, I think, how the Soviet Union was thinking about it. And the second thing is, just empirically speaking, the Soviet program was investing a lot of money into sponsoring smallpox, which is a very contagious virus. And so even if this was somewhat irrational, the fact is they were in fact doing it. And the other interesting thing, if you read about the history of these weapons programs, is that they somewhat get divorced from what you would think would be the rational kind of strategic move. These weapons programs get bureaucratic interests of their own. They want funding for biological science, they want funding for other things. And so there's this kind of runaway culture where they're just trying to come up with the nastiest possible things, even if it's somewhat divorced from the strategic usefulness of such a weapon.
- [08:31]
B
If I recall, Gorbachev wanted to shut down Biopreparat, which was the Soviet bioweapons program. But he was told, I think, that there would be a coup to remove him if he did so, because so powerful was this interest group and they didn't want their program to be shut down. That was their livelihood. Right?
- [08:44]
A
Right. Yeah, I don't know about that exact fact, but yeah, certainly something like maybe up to 1% of the Soviet defense budget was spent on biological weapons. Yeah. Quite a large significant undertaking. Yeah, exactly.
- [08:59]
B
Okay, I guess that's biological weapons. There's this whole other cluster that's come on people's radar in the last year, which is mirror bacteria or mirror life. Can you explain what that is?
- [09:09]
A
Sure. So many molecules on earth can exist in one of two forms, a left handed version and a right handed version. A common example of this is sugar. So glucose can exist in the right handed version and that's the version that we eat as well as a left handed version that you cannot digest, which is pretty interesting. These two molecules are identical. If you put them in a Mirror. So it's similar to your hands. Your hands in some sense are identical, but they are mirror images of one another. So there are lots of properties where it's the exact same and there are lots of properties where they're different. For example, you can't put a left handed glove on a right hand. Now what's interesting is that many of the molecules in your body, and in fact all of the big, most important molecules have this chiral property. So if you imagine a strand of DNA, all the little A Ts, Cs and GS, all of those use the right handed version. And all of the proteins in your body, like the bigger molecules that comprise the bigger machines, all use the left handed version. And so if you're a scientist in a laboratory, in the same way that you can create the mirror image version of sugar, you can also create the mirror image version of those little ATCs and GS. And if you put the mirror image version of those little ATCs and GS, you can create a mirror image DNA strand that looks, you know, it spins in the opposite direction and it looks like the mirror image of regular DNA. Now one interesting thing is that this is not just true of, you know, the human biology, this is true of basically all life on earth. So bacteria, humans, plants, everything, you know, all animals use right handed DNA, left handed proteins. And so a lot of scientists were thinking, wouldn't it be interesting if we could create the mirror image version of not just DNA or proteins, but what if we created an entire mirror image version of a bacteria, like a whole mirror image organism? And there were a number of labs that were looking into this as a possible exciting project. The NSF even funded about a $4 million grant to look into this. But there's a major problem with this, and that is that your immune system has been trained on molecules that it recognizes. And if you flip that molecule to the mirror image version, your immune system is not going to be able to detect or break down those molecules. And so what that means is that if this bacteria were to get into your lungs or get into your bloodstream, there is a decent chance that it would grow on achiral nutrients and it would cause a lethal infection. Now you might then be asking, okay, there are plenty of bacteria that cause lethal infections. What makes this so bad? The reason that this is bad is because it's not just true of human immune systems. This is true of most immune systems on the planet, have been trained on a certain chirality. And so this would not Just potentially infect and kill humans. It would potentially infect and kill many, many species of animals, possibly even species of plants. Plan immune systems work in a very similar way. And what that means is that this could be very persistent in the environment. It could be kind of pervasive. And this would be a lot less like a human to human pandemic, but it would be something that is persisting in the soil, persisting in the environment. If there's a tree that's infected outside of your house and the wind blows in, then that would potentially infect you. And so it would be much more akin to living without an immune system. And people that have genetic diseases that have certain receptors broken typically die in childhood. It's like a very, very nasty disease. And this would be like the whole world ending up in that situation.
- [12:54]
B
So if this theory is right, then these myrobacteria would have an enormous competitive advantage against every other organism because they would have potentially able to evade the immune system of basically all other organisms, and they wouldn't have any natural competitors in that sense. Or maybe, maybe sort of.
- [13:09]
A
Yeah. So they would have a big competitive advantage ins side of like an animal where it's evading the immune system, but other pathogens are not evading the immune system. So it would have a competitive advantage there. I think it's a lot less clear how big of a competitive advantage it would have, say, in the soil or, you know, in the dirt or something like that. It would have some advantages. For example, viruses would not be able to infect it. So phages, they typically cut down on bacterial populations. Other protists that graze on bacteria also wouldn't be able to eat and digest it. And so it would have some fitness advantages, but it would also have some big disadvantages. For example, it wouldn't have horizontal gene transfer. And so it wouldn't be able to adapt as quickly to different environments. It would also be relatively limited in the types of nutrients that it could get. It would be probably persisting on just achiral nutrients, which are a lot less abundant. And so I think this would not be something that would, you know, kind of literally take over the whole ecosystem and like all bacteria suddenly turn into mere bacteria because it outcompetes everything. It would not look like that. I think instead it would look like there's a tiny trace amount of it, but there might be a tiny trace amount of it kind of everywhere.
- [14:24]
B
Yeah. When I first heard this idea, I think the objections that jumped to mind for me were it doesn't really make sense that it would be able to kind of. Well, we wouldn't be able to attack it or digest it, but it would be able to kind of attack us and digest us. Shouldn't there be some sort of symmetry there?
- [14:38]
A
Yeah, so it's not necessarily attacking or digesting all of you. The way it would work is it's only digesting very, very small parts of you. In other words, the achiral nutrients in your bloodstream. So these are things like pyruvate, glycerol, glycine. These are nutrients that don't have this handedness property. And there are enough of that in your bloodstream such that it could grow and persist. And so it would basically just be growing in your bloodstream and eating a small fraction of you, if you will. And that's enough to cause potentially big problems like sepsis or blocking up blood vessels and causing stroke and stuff like that.
- [15:14]
B
Yeah. I guess another objection you could have is why couldn't we just treat these bacteria the same way that we treat other bacteria using antibiotics? I guess you've got a flag why that would necessarily work. That we wouldn't just have to treat people. We'd have to basically coat the entire earth in antibiotics in order to stop it growing through all of the plants and animals.
- [15:29]
A
So the first objection is you're not going to necessarily be able to save the crops or the ecosystem this way, because that would just require far too many antibiotics. But the second, I think there are two other objections to this. First is that even if we were to pivot 100% of US antibiotic production, including agricultural antibiotic production, we'd only be able to cover maybe about 10% of the US population. And so just the scaling of this would be quite grim. The second thing is that people that have these immunodeficiencies, they tend to die even if you give them antibiotics. So they need to be on the antibiotics prophylactically. And so this isn't something like where you just treat an infection. It means that, like, all of us would have to be taking antibiotics every single day, like for the rest of our lives. Life. And that puts us.
- [16:11]
B
And we'll be doing that while watching the natural world probably die out.
- [16:14]
A
Right. And this puts us in a very precarious situation because, you know, then if the power plant gets cut or, you know, the antibiotic production facility goes out, then it's game over.
- [16:25]
B
So I guess we suspect that if we did create myrrobacteria, there's a good chance that I guess most like many of the plants around the world would just start gradually dying off as this bacteria spread and began infecting them. Do we know if that would take like months or years or longer?
- [16:38]
A
Yeah, well, I should also say that there are still a lot of uncertainties here. Like, it's very hard to predict exactly how this would interact because we don't.
- [16:45]
B
Even know what species of bacteria it is.
- [16:47]
A
I would say probably more than a 10% chance that if mere bacteria were released tomorrow, it would be catastrophic. But I don't necessarily think it would be like more than 80%, for example. I think there's still a lot of uncertainty, but I think more than 10% chance is still like, this is kind of a doomsday scenario. On the question of speed, I think, interestingly, bacterial pathogens tend to spread quite slowly if they only infect plants. So there are these studies where an orchard, orchard will be infected with something and it won't even necessarily get to the other end of the orchard, you know, within a year. That being said, animals can spread bacteria very quickly. So if insects are infected with mere bacteria and then they're spreading it from plant to plant, that could end up, you know, saturating a forest quite quickly. Also, just human travel can, can move things very quickly. I mean, Covid was basically everywhere in the world, you know, quite quickly. And so there's no reason to think that wouldn't be true of mere bacteria as well.
- [17:41]
B
Yeah, I guess you might think in order to tackle these high tech problems that we're potentially creating by being able to make mirror bacteria or whatever else, or having an advanced bioweapons program, you need really advanced technology in order to combat it. And I think that actually has been a bit of a mindset that people have been stuck in looking for really advanced technology to counter this stuff. But I think you actually think because we need to scale our counter measures to everyone or as many people as possible almost instantly, you need to figure out what is the most low tech thing that you can potentially use, something that you might already have around, or something you can like manufacture on a completely mass scale within weeks, ideally, or possibly even less. I think you've brought an example of one of the low tech things that you think might be able to eventually stop here. Any of these.
- [18:22]
A
This is an elastomeric respirator. And I think this would help in a wide range of scenarios, perhaps even including something as bad as a mere bacteria scenario. So maybe one thing I'll say on, yeah, I do think probably relatively sophisticated countermeasures are going to be required to get us out of some hole if there's a big catastrophe. But I think there's a question of how do we buy time during COVID It took a long time to get those vaccines up and running, and we need to do work to bring that timeline down. But I think we also really need to know how are we going to protect people for the short run while we're getting those countermeasures up online?
- [19:02]
B
Yeah. So the virtue of something like a mask is that they're not specific to any particular pathogen. In principle, a sufficiently strong or effective mask could basically stop any bacteria or potentially any virus from reaching you if you wore it properly, consistently. But we had N95 masks during COVID and they didn't manage to stop the pandemic to any significant degree, and they were reasonably effective in principle, at least, stopping people from catching it. Why would we think that something that's so low tech that it kind of hasn't really worked in the past would be able to make a radical difference on stuff that is way more dangerous than Covid?
- [19:35]
A
Yeah, so the first thing is that we didn't have enough N95s. In fact, the Strategic National Stockpile had a big stockpile of N95s, but they were out of date. They were expired, and the elastic band was just broken. And so you couldn't even wear them. If you compare that to something like an elastomeric respirator, an elastomeric respirator has something like a shelf life of 20 years, so it survives a lot longer. The other thing is that Covid was in this kind of dangerous sweet spot of being dangerous enough to be deadly and kill people, but perhaps not dangerous enough that people would take, you know, very draconian measures and, you know, you know, do you know, whatever it takes. Yeah. And so, yeah, I think there's some irony, which is, you know, if Covid had been substantially more lethal, perhaps we would have actually been able to contain it because there would have been more motivation to do so. And you saw this with sars, which was, you know, another coronavirus that was successfully contained in places like South Korea.
- [20:31]
B
I guess that was about 10xs as deadly. And that, I guess, prompted a more significant response. People were actually willing to wear masks more often.
- [20:37]
A
Yeah, I guess.
- [20:39]
B
What do you think is the chances that within our lifetimes there is a biological catastrophe such that we basically want everyone who's leaving their house to be wearing one of those masks or better?
- [20:47]
A
Yeah. So the center for global development has the number at basically a 50, 50 chance within the next 25 years of a pandemic that would be as bad as Covid or worse. Then there's the separate question of what's the probability of a catastrophe where it would actually threaten human survival? And I think that number is also quite high. I would say that's something like 1 to 3%, which sounds like a low number, but what that means is I think there's a higher probability that I die to a biological catastrophe than I die, say in a car accident.
- [21:19]
B
Well, that everyone dies in a biological catastrophe than that you die in a car accident. Right, yeah. Why have you chosen to work on biological catastrophes in particular over all of the other risks? I mean, I guess we tend to talk more about AI on this show. It's particularly salient at the moment. But there's other things you could have gone and worked on as well.
- [21:35]
A
Yeah, so this is interesting. So I was kind of working at this existential risk institute way back in Oxford with you, actually way back. And basically at the time everyone was focused on AI and there were very, very small numbers of people focused on biosecurity. And the thing that attracted me to biosecurity is although there was a big biosecurity community focused kind of in general on biosecurity, there was a very small number of people thinking about catastrophic risk in these worst case outcomes. And so the short answer would be, I think it is very tractable. I think it is extremely neglected. I think there are basically fewer than 100 people working full time on strategies that would encompass the worst case scenarios. And I don't think the importance is, say more than an order of magnitude less important than something like AI risk.
- [22:27]
B
Yeah. Why do you think it's not an order of magnitude less?
- [22:31]
A
Yeah. So I think basically it depends on how much risk you put on AI. If you put a 30% chance of extinction from AI, then a 1 to 3% chance of extinction from bio is still within the realm of the tractability. And the neglectedness arguments can still outweigh that.
- [22:51]
B
I thought you were going to make an argument that the risk from AI and risk from bio are connected. I suppose we've mentioned one reason already why the two are intertwined, which is that that making a biological weapon is one way that an AI might threaten humanity in order to get its way. Because there's also the possibility, I guess people think that there's a high risk that especially open weighted AI models might be extremely helpful for bad actors. Who want to run their own biological weapons program, basically they can get much better expert advice and assistance. And so something that previously would have required something like the Soviet program with thousands of workers, maybe could be done with just dozens.
- [23:24]
A
Exactly.
- [23:25]
B
At some point in the future. That's quite troubling.
- [23:27]
A
That is quite troubling, yeah.
- [23:29]
B
Is there anything we can do to stop that from happening or is it kind of just a foregone, foregone conclusion that sooner or later these open weighted models are going to be jailbroken and they're going to be extremely helpful at doing this sort of work?
- [23:38]
A
I don't think it's a totally foregone conclusion. You can imagine different things like better DNA synthesis screening. So even if a model is giving you terrifying instructions for how to make a biological weapon, ultimately you still have to get physical things in order to conduct that attack. And it's not trivial to get those physical things. And so perhaps there could be various checkpoints to prevent terrorism.
- [24:02]
B
Yeah. To what extent do you think it helps us that? I mean, even if you're getting good advice from a computer kind of telling you what to do, it's just like difficult to run the experiments, it's difficult to do the work in a lab. I guess biology is very fiddly, kind of famously so.
- [24:13]
A
Yeah, absolutely.
- [24:15]
B
I think you've mentioned in your notes that you kind of want to do a crash program to put in place your four pillar strategy, which we're going to talk about later in about like two and a half years. Is that sort of set by this AI deadline where you feel that the threat is getting worse because of AI advances?
- [24:27]
A
Possibly. I mean, I think in some sense it's because I think we could do it in two and a half years. And so why not set a very ambitious deadline? But yes, with AI progress being what it is, the possibility that it could help create biological weapons faster. I think even if there's only a 10% chance of that, that's something that we should be really sprinting towards and working to close that risk.
- [24:48]
B
So I guess ideally, right, we would stop these pandemics or these biological catastrophes.
- [24:52]
A
From the plan A.
- [24:55]
B
So prevention is ideally better than the cure.
- [24:58]
A
That's right.
- [24:59]
B
What are you all funding on the prevention side and what do you feel most excited about?
- [25:03]
A
Yeah, a lot of great things. Funding work to think about better DNA screening mechanisms and funding work to think about how policymakers should be getting better DNA synthesis screening into regulation. We're thinking a lot about AI models and the ways that you could Put guardrails on those that would prevent them from divulging ways of creating biological weapons. So just like a lot of very common sense things that. That basically everyone agrees would be good.
- [25:33]
B
Yeah. Are you spending most of your budget on prevention or. Okay, yeah. And do you think that's going to remain the case?
- [25:40]
A
I think it depends. Maybe some of the strategies that we'll talk about could absorb a lot more funding, and I think could involve a broader coalition of philanthropists. So I think that remains to be seen, but for the time being, most of our work is just on prevention.
- [25:56]
B
Yeah. I guess the Mirror Bacteria and Mirror Life stands out as a pretty unique risk in its profile and how much damage it could be potentially do. What's being done to make sure that no one ever makes that damn thing?
- [26:07]
A
Yeah, well, I think a lot of it is driven by a number of scientists who have really kind of been on the forefront of this, including most of the scientists that used to think that they wanted to build mirror bacteria because it was interesting to them. And I think after considering the risks and talking with other scientists, yeah, they've really been leading the way in kind of setting a norm and setting a taboo that this is not something that we should be doing and we should not be building mirror bacteria. I think there's now an open question about, like, where do you want to draw the red lines? If we all agree that we don't want to be making mirror bacteria, what about mirror proteins or really complicated mirror proteins? Most of those are going to be totally fine to make, but maybe a mirror ribosome, for example, or an entire mirror proteome might be crossing the line. And so I think there's still going to be a discussion among the scientists, and we're supporting those discussions.
- [26:59]
B
I guess the challenge there is that while a mirror ribosome might not be dangerous in itself, if we develop the science and the technology to really easily do that, then it's like reducing the breakout time that it would take for some rogue actor or some crazy, reckless person in the future to actually jump from what is safe and permitted to actually making a full mirror bacteria.
- [27:17]
A
Yeah, exactly. So, yeah, the question is, how far do you want to set that threshold back from what a terrorist could potentially do? And right now, that would be a very tall order. But one interesting thing is that there have been some terrorist organizations with relatively large biological weapons budgets. So Aum Shinrikyo was this doomsday cult. Their chemical and biological weapons program had something like $3 million per year allocated to it. So, you know, it would cost a lot more than that to make a mere bacteria. Some scientists estimate that it would be like $500 million. So you're still, you know, two orders magnitude away. And it would also require some very, very talented scientists, which typically terrorist organizations don't have. But, yes, I think that that's a good reason to keep the, the threshold high.
- [28:05]
B
So my understanding is that there's, like, close to unanimity, I guess, among the scientists in this area. They don't almost. Yeah, I guess. How much do you think that has reduced the risk that there could be some sort of myrobacteria release over the next 50 years?
- [28:16]
A
Quite a bit. Maybe a factor of two, factor of four, something like that. And for context, I had maybe a 1 to 2% chance of extinction due to mirror bacteria previously. And so maybe that's down to quarter of a percent, which is still, you know, a terrifyingly large number. And we need to drive that risk down further.
- [28:34]
B
Given that there's, like, no one who really wants to do it. How would we end up with my bacteria?
- [28:39]
A
I think it would be, you know, we don't set the threshold far enough, so we go right up to the brink. And even if all the scientists agree that we don't want to make mere bacteria, perhaps all the components are there, all the precursor materials, perhaps, you know, AI systems make it so that it's quite easy to take all of the things that are right against the threshold and like, talk you through how to. How to build that, how to test it. And so the barrier to entry just keeps getting eroded steadily in the future decades until the point where it becomes quite accessible. So that would be the nightmare scenario.
- [29:12]
B
I guess North Korea would be another possibility, that they might develop it for deterrence purposes, maybe.
- [29:18]
A
I mean, it's quite a wild thing. You'd be developing a weapon that would be killing the political leadership of your own country.
- [29:25]
B
It was released.
- [29:26]
A
Right. I think there are other arguments against it as being a good doomsday weapon. It would be very hard to test it and know with reliability that it would work. The advantage of a nuclear weapon is you can test it. It's like, you know, you can run it through a bunch of tests. You know exactly what you're going to get. With something like a biological weapon. It's actually quite hard to make a credible deterrent because it's, like, very difficult to know exactly how it would work in the environment.
- [29:51]
B
I think a lot of people have this idea in their heads that we can't really Go extinct from biological threats because they're somewhat self limiting. Because a disease that kills people really quickly or kills like almost everyone who gets infected with it is, so it has a really difficult time spreading. I suppose you've discussed one way that that can fail to be the case, which is if the pathogen is spreading through the environment, through all of the plants, through the soil, and so you could catch it that way. Are there any other ways that we could end up going extinct despite this factor that more lethality can often reduce spread?
- [30:19]
A
Yeah, if the lethality is delayed. So I think HIV again is a good example here, where the lethal symptoms don't typically appear until eight to ten years later. And that's far outside of the window in which the virus is getting spread. So that could be another way that this could be very bad.
- [30:36]
B
Okay, so what are the four different pillars of your defensive strategy here? Can you explain? Basically name them all and explain what each one is at a high level.
- [30:44]
A
The first thing I should say is that the four pillars plan is basically a plan to buy time. You want to basically intervene at the earliest possible stage and make sure that society and civilization can keep running while we buy time for medical countermeasures, which I think is eventually the way that we would need to get out of really bad scenario. The four pillars in order would be, the first is personal protective equipment. So things like really good elastomeric respirators, masks to keep people protected from respiratory pathogens, possibly other types of ppe. The second pillar is basically protected buildings. So you know, eventually you're going to need to take off your PPE and you need to do that in a clean, safe environment. The third pillar is detection. You want to make sure that nothing is spreading without you knowing about it. And so having really good detection systems is very important. And then finally medical countermeasures. And I think there's a question about where the offense, defense, balance eventually bottoms out with medical countermeasures. And I think hopefully it bottoms out with the defender winning. And if that's the case, then I think medical countermeasures would be very promising.
- [31:47]
B
Okay, so pillar one is personal protective equipment to protect individuals. Pillar two is bio hardening of environments like homes and offices to prevent anything from getting in that might infect people. Pillar three is detecting the pathogen as early as possible and I guess being able to observe if it's spreading where you had prevented it from doing so. And I guess the earlier we know, the sooner we can put in place these other measures and the less it's killed people in the meantime. And the fourth one is, I guess, trying to escape from this situation where everyone is in their bio hardened homes, where ultimately we want some kind of more permanent medical fix that can actually just get rid of the pathogen or protect people forever.
- [32:22]
A
Exactly.
- [32:23]
B
So we're going to dive into the plan in a second. But I guess one of the reasons you've come on the show is that you're trying to hire for a whole lot of roles, including some pretty senior roles, to kind of make this plan happen. We're going to talk more about that at the very end of the episode. But I guess I thought it would be useful to just flag what some of those positions are now so that people can be thinking in their heads. Maybe I'd be suitable for running one of these four pillars of the program. What are the roles that you're trying to recruit for at the moment?
- [32:45]
A
Yeah, recruiting for a lot of roles. So at open philanthropy itself, we're going to be on the hunt for grant makers, people who could deploy tens of millions of dollars to reduce these biological risks, who can learn about a new field, get information, talk to people. I think it's a common misconception that grant making is about sitting in a chair and reading grant applications coming in. And we make very few of our grants that way. The vast majority of our grantmaking is you kind of have to spend the first 5% of a project doing it yourself to understand what it is. You have to get to know people in the community to understand who's doing good work here. And then you want to be headhunting the top people to do the things that you need to have done in the world. And so on that note, we're also looking for many roles outside of open philanthropy. For example, on the PPE project that we'll talk about, we need someone to lead a really good nonprofit who's going to basically run that, run the manufacturing, run the distribution, think through all the things that need to be done there. It's a very senior role on some of the other things that we're looking for. Yeah, people to basically just own very large components of the problem. We're looking for more researchers and we're also looking for even people who are just pivoting their career and don't know quite what they want to do. We have a scholarship and fellowship program for people to transition into biosecurity. So we're going to have a Google form. I assume there'll be a link to that. And yeah, we'd encourage you to fill that out if you're interested in transitioning into biosecurity.
- [34:23]
B
So I think up until now most people have thought of biology as kind of the archetypal case where offense is stronger than defense. And it's going to be potentially just extremely difficult to protect us from these threats to a degree of fatalism. About, like, maybe the situation is just hopeless or, you know, all we can do really is try to prevent people from creating these things in the first place. But if they do, then we're kind of screwed. What do you think the balance lies on offence, defence and bio?
- [34:47]
A
Yeah, so I think there are a lot of different things that you could mean when you talk about an offense defense balance. One thing you could talk about is the cost of the attack and the cost of the defense. And there is an example where I think the attacker has a huge advantage. One very concrete example of this is that after 9 11, we bought, you know, the United States bought well over 300 million doses of smallpox vaccine to basically cover the entire US population. That cost well over a billion dollars. And then there's the question of like, well, how much would it cost to create smallpox? And I think one number here is there was a postdoc that synthesized horsepox, a very similar virus, for about $100,000. And even if you added an order of magnitude onto that for evading synthesis screening mechanisms and acquiring the expertise, you're still looking at an offense defense ratio of 1,000 to 1. If you take the $100,000 number, that's like 10,001. So it's very skewed and it's hard to think about any other area of national security where there's a 10,000 to 1 cost ratio. So that's one thing that's quite scary. Like, I think that's what we mean when we say biology is potentially offense dominant. I think there's a separate question though, which is that take any given person or any given city, could you protect that city or the majority of the people in that city if you really had to and you were willing to spend that money because you could imagine there's another type of offense dominance, which is no matter how much the defender spends, they get through. And I think nuclear weapons are a good example of that. Maybe we have missile defense now, but before that, basically there's almost nothing the defender could do. Biology. I do not think that that is the case. I do think there are tractable things we can do. And in fact, I think some of them could be surprisingly Affordable, at least to buy time for the more expensive countermeasures to come online. Another way you could think of offense defense balance is through kind of a silly thought experiment. And the thought experiment is imagine there's a person in a box and a both the attacker and the defender get to release some sort of biological or chemical agent onto that person in the box. And the question is, can the defender successfully keep that person alive? And here I think the answer is probably not. I think the attacker has the big advantage, but I don't think that's the case. I don't think that's a realistic thought experiment because people are spread out and it's quite hard to get physical things delivered to people. Right. And I think that's fundamentally the thing that the defender has the advantage on. So you could imagine, imagine an alternative thought experiment where There are like 10 people in 10 different boxes that are connected and the attacker gets to infect the first person, but the defender is trying to protect as many of the other people as possible, maybe trying to protect the first person, but maybe that's hopeless. But what the defender can do is just build a wall and prevent the disease from spreading. And I think that's an area in which the defender has a real fundamental advantage.
- [37:44]
B
Yeah. Is that basically the core weakness that you have? I guess if you're a bacteria and a virus trying to kill everyone, basically you have no way to penetrate through walls or through physical barriers. And I guess you're also vulnerable because you're so small. That creates the advantage that people can't see you. You can sneak into people's lungs without being noticed. But on the other hand, you're so small that you're completely vulnerable to heat, to uv, to chemical attacks that would just disintegrate you. And you basically just aren't large enough to defend yourself.
- [38:10]
A
Basically, yes. The other thing I would add to that is just like straight up filtration. And maybe you would say, well, maybe in the future we could have nanotechnology and these little microscopic robots that could burrow through your filters and bur through your walls. But that also has a lot of other constraints, like just the simple amount of energy that each of those things would need to be holding. And it's also not clear that you couldn't use similar countermeasures, like you could have your nanobot pesticide that kills it in a similar way. And so I think these are like a lot of ways that the defender basically is fundamentally protected.
- [38:42]
B
I suppose the other way in which they have a Disadvantage against humans is that we're kind of intelligent and we can use science and technology to think up specific, I guess, countermeasures that can target them in particular. Whereas a bacteria can't be doing science in order to figure out how to outwit us. It does have evolution as an option to try to move in a more dangerous direction or to evade our countermeasures, but that probably would be slower, basically.
- [39:04]
A
Yeah. And evolution is not going to be optimized for killing people. And so if there's some pathogen that's evolving, probably it's going to be evolving in a direction that's less lethal. And, in fact, this is interesting. During the Soviet weapons program, they were generating these gigantic vats of anthrax. And what they would find is that the anthrax would evolve to get very good at growing in these giant containers and less good at killing people. Right. Which is, like, exactly what you would expect from evolutionary pressure. And so I think there would be a similar dynamic if there's something spreading through the environment.
- [39:33]
B
All right, pillar, one of the plan is personal protective equipment. Do you want to bring out the mask that you've brought again?
- [39:40]
A
Yeah. So this is an elastomeric respirator. Elastomeric respirators are really excellent for a variety of reasons. The first thing is that I could wear this six months in a row without needing to change the filters. So this is not like an N95 where it's disposable. You know, you end up needing to change it every single day. I could wear this six months. Protects me for the full six months. The other thing is that the shelf life is 20 years. So if a government were to purchase a bunch of these, that gives you basically 20 years of protection, that you're protecting your population. The other thing is that the quality of protection that you get is higher on average than you would for an in number 95. The protection factor for something like this is about a factor of 100. So that means it filters out 99% of particles coming in. And interestingly, that also applies to particles coming out. So sometimes with an elastomeric respirator, you'll see it has, like, a valve. This one does not. So this also filters the breath coming out. So what that means is that if both of us are wearing one of these, that would be a factor of 10,100 reduction for me, and then 100 reduction for you.
- [40:51]
B
Is that enough to stop even extremely contagious diseases?
- [40:54]
A
We think it is. In fact, you can actually Put an upper bound on the amount of aerosols that a human generates. And basically our understanding right now is that basically nothing gets through a factor of 10,000. That's basically an upper bound. So even measles, which is the most contagious virus, would not be getting through a factor of 10,000. And so assuming it fits properly, and the nice thing about this is that you don't need training to fit it. This will fit 90% of faces, just like first go. Basically, it just has a much wider margin of error. Whereas with the N95s, you typically, if you're a healthcare worker, need to get the fit testing and whatnot. And I don't know if you had this experience, but, yeah, if you're wearing an N95 and your glasses are fogging up, then that means it is not fit it, you know, properly.
- [41:45]
B
Yeah, there's definitely air getting in on the sides, I guess, this thing. So it has. If you look. I guess if you look on the inside, it's got this part around here. I guess it has a much bigger buffer, I guess, where it's like, it can fit different face shapes. And it's still going to, like, have a seal around here.
- [41:58]
A
Yeah, yeah. So if I wear that, it's like very sealed. Yeah, you can. All right.
- [42:02]
B
If I recall, if you. Yeah, I think the audio people are not going to love this. You can do like a fit test. Right. Where you can tell. You can. So you push on. Yeah, so you push on those things there.
- [42:14]
A
Yeah.
- [42:14]
B
And you can tell that it sucks in. You can tell that there's no air that can get in on. Around the. Around the side.
- [42:19]
A
Yeah, exactly.
- [42:20]
B
I guess for the people who don't have video, I guess you should go.
- [42:23]
A
Check out the video. Right.
- [42:26]
B
But I guess it looks. I suppose you mentioned Darth Vader earlier.
- [42:30]
A
Yeah, yeah.
- [42:31]
B
Fortunately, it's a nice blue color, so it doesn't look too weird. It's like a sort of latex siliconey thing, I guess, like. Yeah, fits over. It's got like two things on the side, a little bit like a fallout mask.
- [42:41]
A
Right, right.
- [42:42]
B
And it's got these, like. I mean, it's got these elastic bands that kind of go around the head. And you think that these things would last 20 years or that you could wear them.
- [42:51]
A
That's what the current testing suggests. I think, you know, more tests could be good. The interesting thing about this specific model is that. Yeah, so this is called the EM Pro. And this actually was the result of a grant that Barda. So, like, the government agency in the US was funding a lot of research on like mask innovation. And they funded. Yeah, they funded the research that went into this. And it's really cool for a variety of reasons. So this, this particular mask is made of one piece of medical grade silicone. So that means that you don't need any labor to assemble like pieces like other, other masks. You know, you need to put together different pieces. But this, you manufacture it in a silicone injection mold. So like you can imagine like a waffle press, you know, type thing and.
- [43:35]
B
It just like injects the hot. Yeah, yeah, Silica or the silicone in there and it just like. And I guess cools off solid, solid, solid, solid.
- [43:43]
A
And then basically this is the thing that comes out of it.
- [43:45]
B
So I guess it has like no weak points or no moving parts that can break.
- [43:48]
A
And you can pop this in the dishwasher to sterilize it. You know, it's like one piece. You'd have to pull the filters out.
- [43:55]
B
And the filters are replaceable. If they get damaged.
- [43:57]
A
The filters are replaceable, but you don't need to replace them that often. So people are maybe used to thinking about the electrostatic filters. So these are some of the ones that are disposable where they rely on this electric charge and that they lose the charge over time. These are just mechanical filters. So, you know. Yeah, you could be using these for six months and I guess eventually they get clogged up.
- [44:18]
B
But it takes ages for that to happen.
- [44:19]
A
Yeah, yeah.
- [44:21]
B
So I heard on the grapevine that these were good, so I bought some. I think I paid $50 per one. I guess you hope to get the price down to $5.
- [44:30]
A
That would be ideal. I think we're reasonably confident we can get the cost down to $10 per mask.
- [44:36]
B
Yeah. What are the different ways to shave off the cost?
- [44:38]
A
Yeah, so one thing you can do is you can replace the medical grade silicone with just food grade silicone. Medical grade silicone you typically need if you're going to have something inside of you. But if it's just on your face, probably food grade silicone is fine. That also means that the speed of manufacturing might be faster. The logo, you don't need the logo. You could get rid of that. There are other various innovations. Like right now, maybe there are two filters. Maybe you could replace that with one central filter and a lot of different things like that to get the cost down.
- [45:15]
B
Guess also if you're just making 100 or 1,000 times as many, presumably you'll find all Kinds of ways to make it easier.
- [45:20]
A
Yeah.
- [45:20]
B
Is it hard to get the food grade silicon?
- [45:23]
A
No, it's like there's. Yeah, it's pretty cheap and you can get a lot of it in the US So maybe one interesting implication is that if we can actually get the cost of this down to $10 or maybe even $5, this becomes one of the most cost effective ways of preventing respiratory transmission. If the shelf life of this is 20 years for $10, that means basically 50 cents per person per year of protection, which is just outrageously cost effective. So this is substantially more cost effective than an N95. And yeah, basically we'd be blocking most respiratory pathogens or basically all of them. And so yeah, if you're a government government, I think it makes a lot of sense to just stockpile enough to cover your entire population. Right now we spend about $10 billion a year on missile defense per year. Stockpiling one of these for every single person in the US would be two orders of magnitude cheaper than that and depends on what you think the risk is of a nuclear missile launch versus a pandemic. But yeah, this is extremely cost effective. I think any rationale, national security person should think that a population should have one of these for every person.
- [46:37]
B
You said that this is the result of a BADA grant.
- [46:42]
A
Biomedical Research. Biomedical Advanced Research, I think authority. Yeah.
- [46:49]
B
Okay, but it's like DARPA or iarpa.
- [46:51]
A
But for biological stuff kind of DARPA and IARPA fund earlier stage research. They that's somewhat new. BARDA will fund more intermediate stage stuff. So they'll actually fund companies to develop things. They're responsible for a lot of the really good vaccine development and stuff like that. So yeah, they fund a lot of really important stuff.
- [47:07]
B
So they had a prize or a competition for people to come up with the best new mask. I guess this was in response to Covid and this is one of the winners. I guess it's got a lot going for it, I guess. So the advance here is it's much cheaper per day of use, much cheaper per year of coverage in storage. I guess it requires even less space. Right. Because you don't have to stockpile like an N95 for one person every single day that they're going to be using it. So I guess it's like a ten hundred fold reduction in cost relative to the.
- [47:34]
A
Basically just better.
- [47:35]
B
I guess the N95s were like maybe even were still worth stockpiling if that was all that we had, but at a hundredth the price.
- [47:40]
A
Yeah, this is just like the common sense thing to do. I mean, I could talk a bit about the disadvantages. I don't think it dominates on every dimension. One thing is that healthcare workers don't like wearing these because they make you look a bit more scary and, you know, it's harder to talk through them compared to the, the cloth of like the N95. So, you know, that's like one kind of disadvantage. You know, if you wear this for long periods of time, the condensation will get kind of gross, you know. So I'm not saying, like these are perfect in every way, but I think in a life and death situation, it's pretty obvious thing that I want. Yeah, yeah.
- [48:16]
B
I guess depending on how bad the pandemic is, you could potentially like take it off in the bathroom or so, I don't know. You could like find some.
- [48:20]
A
Yeah. The other interesting implication of the $10 cost per mask is that this starts to get within the budget range of, say, a group of philanthropists that wanted to do this even if the governments were not making the rational decision and buying a lot of these. And that's something that we're in the early stages of thinking through. But, you know, you could totally imagine a philanthropic effort where, you know, if you were just trying to cover, say, the people that had to go outside during a catastrophe, a group of philanthropists could come together, stockpile, say 30 million of these. That would be $300 million, maybe $150 million if the cost is cheap enough, and hire a bunch of really good people to do the shipping and logistics. And the next time there's a pandemic, if you need to work in a power plant or you need to work in a water treatment facility. Yeah, you wake up one morning and there's one of these on your door doorstep for every single essential worker. And yeah, this is something that private philanthropists could just do to protect people.
- [49:22]
B
Yeah. So I guess in the US and uk, I mean, most houses get a delivery every day anyway, so it's pretty straightforward that you could deliver it to almost every property within a few days. Certainly if everyone was clamoring for them, then I don't think it would take long to distribute them.
- [49:36]
A
Yeah, this is not some hard technical problem that we need to solve. It's just like we need to make this and we need to give it to all the people that need it. And we can kind of map that out and it's like a very tractable problem.
- [49:48]
B
Okay. So if we imagine it costs $10 and I guess so globally, you might have a billion essential workers or something. So it costs like $10 billion globally. That's, I guess, like a tiny. And I guess that would cover you for 20 years. Right. So it's more like 500 million per year. That's, like actually a pretty small fraction of global philanthropy, really, like a negligible fraction. And I guess in the US it's just even a lot more very obviously within the budget.
- [50:14]
A
Yeah, yeah. I mean, philanthropists could potentially cover their entire countries. If there's someone in Norway who wants to get $50 million and no more pandemics in Norway, it's pretty outrageously cost effective.
- [50:26]
B
So what sort of staff do you need in order to make that vision a reality?
- [50:30]
A
Yeah, so we need people to basically run this nonprofit. We need people to, yeah. Think through the manufacturing of this. So manufacturing experts, product design people, people who've successfully gotten products across the finish line, people with silicon injection molding experience, people who've worked in the global health space and could think about ways of integrating this into global health systems. Yeah, we need a ton of different roles. We're basically establishing a new nonprofit to explore this idea. It's still very early stages, but, yeah, I think there are a lot of great things that we could do to explore this idea.
- [51:10]
B
Is it possible to mess up wearing it that easily? I guess it seems like it has more of a margin for error.
- [51:16]
A
Well, your beard is actually going to be a bit of a problem.
- [51:20]
B
I'd rather die than shave off my beard.
- [51:24]
A
It's much harder to screw it up. But, yeah, I think, yeah, you could definitely. I mean, during COVID you'd see people with a mask, like, you know, under their nose. So, like, it's definitely possible to screw up wearing it, but it's.
- [51:36]
B
I guess maybe the room for error.
- [51:38]
A
Is much harder with this.
- [51:39]
B
I guess I'm thinking about that because during COVID people were so lackluster in their effort. But I suppose we're imagining here we're trying to protect against pandemics that kill almost everyone who gets the disease. So the motivation is going to be pretty high.
- [51:49]
A
I think that's right.
- [51:49]
B
Yeah. So to what extent do we need to separate in our minds the kind of classic respiratory virus that only spreads between people and something crazy like mirror bacteria that would just be everywhere, potentially ambiently in the environment. When considering whether this would be sufficiently effective to protect people, not just, you know, most of the time, we really need these essential workers to be surviving until we can come up with medical countermeasures. To end it. So you need them to like consistently be wearing the mask properly every single day.
- [52:15]
A
Yeah, absolutely. So either way you're going to need respiratory protection. And so I think the strategy, whether or not it's a more traditional respiratory pandemic or something more exotic like mirror bacteria, I think the strategy is the same. You need to protect people, you need to protect their lungs. Lungs and respiratory protection, I think is actually the weakest link in this chain. So, yeah, I think there's a question of whether or not these masks would be sufficient. And there, I think there are a lot of uncertainties as to how much mere bacteria would actually be in the environment if a catastrophe like that were to occur. I think there is a decent chance that two orders of magnitude would be sufficient to keep people alive and protected and so that this mask might be sufficient. There's also some chance that there would be higher concentrations of mirror bacteria in the environment and you would need to get additional orders of magnitude. I do think there could be ways of doing that, but to get that at scale, you would need to improvise. So you could do things like hook up vacuum cleaners to air filters and basically create positive pressure and then have that over this, which would then give you additional, like, you know, logarithms of reduction or additional orders of magnitude of reduction.
- [53:24]
B
So that sounds like a significant step up in complexity potentially.
- [53:29]
A
Yeah. And I think, I think that's why we need people to be kind of doing this early research and like thinking about this and like engineering things to see what works and what doesn't work, so that in an emergency we're not trying to figure that stuff out.
- [53:39]
B
Yeah. In a worst case scenario, are people also going to need goggles basically, or something to protect their skin from getting exposed?
- [53:44]
A
Yeah. So this is really interesting. The respiratory pathway is hands down the weakest link. If you compare the surface area of your eyes to the surface area of your lungs, it's a factor of 10,000. So the surface area of your lungs is common factoid that it's like basically the size of a tennis court court. And then you compare that to the size of your eyes and actually the respiratory route is like much, much more vulnerable. Now that's to aerosols that are kind of ambiently in the air. There could be like droplet spread, in.
- [54:12]
B
Which case, or against touching your eye.
- [54:13]
A
Yeah, or touching your eye. Exactly. So, you know, I think there are other reasons to think, you know, you would want to protect your eyes. But the other important thing to know is it's like, much easier to improvise protection. Like, you just wear glasses, wear goggles, like plastic face visors, stuff like that. Whereas improvising respiratory protection is, like, a lot more difficult. And so this is, like, the type thing we want to have in advance, I guess.
- [54:35]
B
And we're imagining that most of these pathogens wouldn't be able to get into people through the skin, because, I guess, I mean, most.
- [54:40]
A
Yeah, your skin is really good. Your skin is just like, you know, hundreds of millions of years of evolution have, like, you know, made your skin, like, quite difficult to penetrate. Yeah, exactly. Good against fighting pathogens. It's extremely dry. There's, like, not many nutrients. Like, it's just. Yeah, it's. It's. It's. It's good stuff.
- [54:57]
B
All right, let's push on to, I guess, bio hardening of houses and offices and things like that. Yeah, I guess. How do we make safe living spaces for hundreds of millions of people when they're not doing this sort of essential work where that forces them to go outside?
- [55:10]
A
Yeah. So there are a lot of possible things that we could do here. One would be things like ultraviolet light, which sterilizes pathogens and could be quite good, although we don't have that widely deployed right now. And so the question is, what would we do in an emergency if there was something that happened tomorrow? Yeah. So one interesting option is kind of various vapors that could potentially sterilize pathogens. So this right here is propylene glycol. There are other vapors as well, like triethylene glycol. Yeah, it's interesting. I learned about this from a professor at Johns Hopkins at a Happy hour in D.C. and it turns out that this is a chemical. It's in fog machines at, like, Broadway shows, or it's also. Yeah, it's in vapes. And the interesting thing is that we produce a huge amount of triethylene glycol. So it's used in natural gas processes processing, which means that we have enough of it to cover basically all industrial floor space in the US plus a wide variety of residential space 24 7. And there are a number of studies in the 40s done on this in military barracks and things like that that showed that it got basically a factor of 10,000 reduction of pathogens in the air. It's a chemical that kind of disrupts the membranes of pathogens. It also dehydrates them. And interesting. There are plenty of chemicals that we could put in the air that would kill pathogens. The problem is that most of Them are also quite nasty for you, too. And this is quite interesting because it's extremely safe.
- [56:48]
B
And that's true even if you're. I mean, I guess, like fog machines aren't dangerous, but I wouldn't want to necessarily live next to a fog machine constantly. But you're saying, like, even then it basically does almost nothing to the lungs.
- [56:57]
A
Yeah, that's right. And I think probably we should be doing even more studies for chronic exposure. But my understanding is that. Yeah. Is just very, very safe. Like, just outrageously safe.
- [57:08]
B
Okay.
- [57:09]
A
And so, yeah, this is. This is one example of a chemical that's already, like, very widespread. We already have a lot of it. And so this is the type of thing that you could do. You know, if you. You're in a situation where you need to take your mask off, but you still need to be like, reducing the concentration of pathogens in the air.
- [57:24]
B
Yeah. How much of it do you need? Like, is that what was. I guess that would only cover a very, very small room.
- [57:29]
A
Well, yeah. So interestingly, it works when it's totally, like, invisible. Right. So. So there. There is like propylene glycol, like outside of the fog that's, like, covering us. So the fog is actually, you know, it's mostly invisible when it's working. Yeah.
- [57:44]
B
I guess for people listening who can't see, it's kind of a small handheld device that's producing, I guess it kind of looks like a vape sort of. Or it's producing about the same amount of smoke that someone would if they were breathing out through. Through a vape. And I guess it's the same. It's literally exactly the same chemical.
- [57:57]
A
Yeah.
- [57:59]
B
And how does it kill viruses and bacteria that are in the air?
- [58:02]
A
Yeah, so we think it works by basically disrupting the membranes of these pathogens, and it also works by dehydrating them. And the reason that it kills pathogens at a higher rate than it damages you is you already have a ton of water in your lungs, and so it comes into your lungs and it's not going to affect anything. Whereas for pathogens getting dried out, just a tiny bit is going to.
- [58:24]
B
Yeah, and it affects all of them. There's no sort of bacteria or viruses that are not common ones that are resistant to.
- [58:30]
A
Yeah, I think this is interesting. Like, basically all biological organisms share certain vulnerabilities here in terms of, like, either membranes. So. Yeah. Interestingly, this works against envelopes, and I think it also works against non enveloped viruses. So I think it's doing a lot of different Things against them.
- [58:44]
B
I guess it's a little bit like asking, are there any animals that can walk through an open flame without dying, sit in an oven and yeah, because bacteria and viruses are just so small that even like tiny amounts of this stuff is just very large relative to them.
- [58:57]
A
Yeah, exactly. And the same thing is true of ultraviolet light. And interestingly, you can think of this kind of from first principles. If there's a certain amount of energy that's getting put into a very small organism that's going to start breaking stuff. It's not specific to just DNA. So this kind of general introduce energy to a small object to destroy. Destroy. It should be fully generable all the way up into crazy nanotech or something like that.
- [59:23]
B
Okay. So the plan is to get this chemical that we previously were using in gas extraction. We're going to distribute it to all of the different houses and offices and then have everyone. Do they need a machine like this? Because we won't have that many machines.
- [59:35]
A
Yeah. So interestingly, there are a ton of machines like this in the US So there are tons of humidifiers and things like that. So I think we have actually enough to basically cover every U.S. household. So does it just evaporate also? Just evaporates. So I could douse my towel in this and just hang up a towel. I wouldn't need like a fancy nebulizer like this.
- [59:56]
B
And it just hangs around in the air until the air like goes outside.
- [59:58]
A
Until the air moves out. And I think that's actually the major constraint. Like it's hard. So. So although it gets four orders of magnitude pathogen reduction, like in, you know, a fixed environment, the fact that there's air moving in and air moving out means that realistically this is probably only getting you Maybe about a 1.5 magnitude. Sorry, order of magnitude reductions. Maybe about 30x. Yeah.
- [60:20]
B
Okay.
- [60:21]
A
Just still good. And importantly, we already produce enough of this, so we wouldn't need to like on ramp production. I think if there was going to be a project here, it would be someone looking into the supply chains and like getting to know people and figuring out like in an emergency, what are we actually going to do? Like, this might work in theory and on paper to help out a lot, but, you know, what are you going to do in a real emergency is still like an unsolved problem and someone needs to start tackling that.
- [60:45]
B
Has this ever been used in hospitals or other environments to stop infection before?
- [60:50]
A
Yeah, so it was experimentally used, I think in these military barracks briefly. I don't know about hospitals. Though, yeah, I'm not sure.
- [60:57]
B
Yeah. Does this work for surfaces? Although is this just cleaning the air?
- [61:01]
A
Yeah. So this works for surfaces. In fact, for surfaces you have a lot more options because you don't have to breathe in the stuff. And so you can use just like ethanol, use bleach. The United States produces an obscene amount of ethanol due to the farm subsidies. And so we actually already have enough ethanol to basically sterilize all of the major surfaces in the US and residential at least once a day or something like that.
- [61:27]
B
Maybe we should have paused and said, what is the actual threat to people who are staying at home, say that they're not going out as essential workers? Is the risk that I guess it comes in on some object that they have to get delivered in order to survive. I guess if it was mirror bacteria, you could imagine it coming through air vents or coming in through the soil or coming in on insects that come inside. I guess that's a really nasty case.
- [61:48]
A
Yeah, exactly. So with the standard respiratory transmission pathway or something that's like human to human, you don't need to worry about this. But with the kind of things that are environmentally persistent, yeah, there'd be this concern that, okay, you need to eventually take off your mask, you need to eat, you need to sleep, you need to drink water. And so this is why it's so important to have just physical space where people can live that's like free of pathogens. Pathogens. And so that's what the second pillar is basically focused on.
- [62:17]
B
Yeah. So if someone was an essential worker, they're going out to operate a power plant, they're getting exposed potentially to my bacteria or whatever else, and then they have to come home to their family. How do they make sure that they don't bring home the pathogen on their clothes or on their body or something like that.
- [62:31]
A
Yeah. So one thing is that in a situation like that, you might have limited movement. Like if you're working in a power plant, probably it's safest just to stay at the power plant overnight and just be living there. And so I think in a really worst case scenario, there's going to be a lot less movement of people. People like in and out of houses. But let's say you have to like move in and out. I think there are still a number of things you can do. You can basically take off your clothes and like sterilize them. You could have like, you know, improvised airlock type situation, but I'm not even sure you would necessarily need that. I think as long as you have lots of filters, like, set up in your house. The air changes could. Could make it so that. That's, like, not as big of a deal if it's like, two temporary.
- [63:11]
B
Yeah. And we're gonna sterilize surfaces and clothes just using all of the ethanol that.
- [63:15]
A
We have lying on the. Yeah, ethanol. This is actually another really interesting thing. So this is hypochlorous acid. You know, spray it. So hypochlorous acid is also an interesting method of sterilizing things. You know, you can buy it online. It's actually advertised as, like, also an anti acne thing. It's pretty. You don't want to get it in your eyes, but it's, like, totally safe for skin. You can, like, put it on your face or hand sanitizer. And the thing that makes this really interesting is that you can make this at home using salt, water, and electricity. Right. And it's actually kind of interesting because I thought, like, during COVID you might remember when there was this big hand sanitizer shortage, like, we were producing enough ethanol, like, in aggregate, but it took time to, like, reallocate that supply chain to making hand sanitizer. This stuff, you know, you can make it at your own house with salt water, basically.
- [64:08]
B
What do you. Do you, like, get salt water and then run a current through it?
- [64:10]
A
Yeah, exactly.
- [64:12]
B
And. Yeah, for, like, very long.
- [64:15]
A
Not that long. I mean, it's. It's. It's kind of crazy. Like, you don't. You don't think of salt as having chlorine in it, but, like, that's. That's the chemical formula in acl and. Yeah. So basically you. You separate out the chlorine ions and just.
- [64:26]
B
Yeah, and then it becomes.
- [64:27]
A
Sterilize.
- [64:28]
B
Antiseptic. Sterilize the stuff.
- [64:29]
A
Yeah.
- [64:29]
B
Okay. Why? Has this ever been used before?
- [64:32]
A
Yeah, this is used all the time. Like, I got that. You know, it's like an antibacterial thing. Yeah. Yeah.
- [64:37]
B
And I guess people don't have. Don't produce it at home because there's never been any reason.
- [64:40]
A
It is really cheap. Another fun fact is that the chemical reaction that this uses to kill pathogens is actually the same chemical reaction that your white blood cells sometimes use. So. Yeah.
- [64:52]
B
Come full circle.
- [64:53]
A
I know, right?
- [64:55]
B
Okay. So you've got the gas, you've got the. I guess these surface disinfections. That is relatively easy to scale up massively. In your notes, you talked a whole lot about, I guess what's kind of more like a bio containment strategy where you're putting, like, duct Tape over all of the windows. I guess you were talking about using, I think, was it air blowers or something just like blow air into the house through a filter? Is that sort of just like the plan C or the plan B in this?
- [65:21]
A
So it turns out that just filtering air is sometimes the simplest and most effective thing. And so actually just getting air filters is like, probably the best thing. And yeah, you see this in. Yeah. One interesting thing is that people will have these facilities that grow laboratory mice that don't have immune systems. Right. Because you need to run experiments on like immunocompromised mice. And so they will have these facilities, these kind of clean rooms. And basically all it is is like plastic and air filters and like people being very careful about, like, where they go and like putting, you know, touching things. Things. And so, yeah, that's like one example of like a proof of concept that you can use of like, you know, people, you know, keeping animals alive without immune systems using very simple techniques. And the cost of one of these immunocompromised mice is only about 10x the cost of like, you know, a regular lab mouse.
- [66:16]
B
I guess we all know how much that costs.
- [66:18]
A
It's about 330 at least, according to Gemini.
- [66:21]
B
For one mouse.
- [66:22]
A
Yeah.
- [66:23]
B
Yeah, I guess so again, if this was a respiratory virus, something more like the flu or Covid, then we're not really worried about it blowing in through the window. Or you probably don't need almost any of this stuff except for someone's coming home. PPE is all that you need, basically. But I guess so this is kind.
- [66:39]
A
Of temporarily, to be clear.
- [66:40]
B
Yeah, so this is all kind of focused on the more dramatic sort of mirror bacteria or something along those lines. Scenario where you're worried about it being ambient in the environment and coming up in through all kinds of different surfaces. And so you just want to be vigilant all the time.
- [66:53]
A
I think some of it is. I would say that some of the glycol vapors are still very useful in the human to human transmission pathway. I think having ultraviolet light could be really helpful if you're in hospitals and you need to be extra safe or you're dealing with people that are coughing and generating even more aerosol. So all that stuff is still back.
- [67:11]
B
Up in an office or a hospital.
- [67:12]
A
Yeah, it's still super useful. You don't need the exotic, mere bacteria stuff in order to think that the stuff is important, but in those scenarios it's strictly necessary rather than just icing the cake.
- [67:26]
B
Yeah, I guess you haven't talked about uvc. So there's like UV C lamps that I think can disinfect surfaces and disinfect the air as well. But I guess we don't have many of those. They're quite expensive and I suppose very difficult to scale. So we wouldn't see that as like a core part of the plan.
- [67:40]
A
I think it should be part of a longer term plan. I think there needs to be more studies on it. I think I would like to see people experimenting with it and adopting it more. But yeah, I don't think it's the type of thing that we could get everywhere within two years. Whereas these other options might be very scalable and tractable on very sharp turnaround.
- [68:02]
B
Okay, so I guess I was already to give you a bit of a hard time about the idea of, I don't know, turning these leaf blowers into air filters in every single house and trying to make them almost air contained like a BSL2 facility. I guess what is the weakness of using this glycol suspended in the air? And I guess these surface disinfectants, does this actually just solve the problem?
- [68:24]
A
I think in many scenarios it might be sufficient to solve the problem. I think in situations in which the outdoor concentrations of some sort of pathogen are very high, then they might not be sufficient just because you have some vapor in the air, but it's getting blown out of the house or out of the building that you're protecting. And so you want to be also in the worst of the worst case scenarios, protected against air coming into the building. And so the way you can do this is you can just have basically a fan and a filter and you can blow air in. And this is what they do for standard hospital clean rooms and other things like that, or the facilities where they grow mice without immune systems. This is like a very common thing. So there's a question of like, well, how many people could you cover with a strategy like that?
- [69:13]
B
How many answers with a strategy like that?
- [69:15]
A
Maybe most people. So this is really interesting. About 60% of houses in the US have furnace fans that are powerful enough to push air through a HEPA filter. You can also use things like leaf blowers or vacuum cleaners or other improvised things. The power consumption starts to add up. But you could potentially do this 24, 7. And then you might ask, well, we don't have enough HEPA filters to cover huge populations. But interestingly, HEPA filters, the first HEPA filters were made during the Manhattan Project. There were worried about radioactive dust, like getting into workers and Hurting them. And so they had these fiberglass fibers and they stacked a bunch of those together and that was the first HEPA filter. And it worked. And it turns out that the fiberglass in household insulation are even better than the ones that were used in the Manhattan Project. So potentially you could just rip out the insulation in houses and kind of improvise HEPA filters and stack those up again. This is all still working on paper and I think it would be really interesting to have a team of people that's actually trying this in the real world and figuring out what the weaknesses are. So this is still early days, but at least on paper it looks like maybe we could even in the worst of the worst case scenarios, protect a lot of people just using these kind of improvised methods.
- [70:39]
B
Yeah. So I think that there's two different reasons, I guess, why you need to be letting air in. One is just that we need some additional oxygen to breathe and to get rid of the carbon dioxide. And I guess you want as clean air as possible to be coming in. But I think in your notes you also talked about the benefit of having positive pressure, where if you're actually actively pumping air into the house, then that.
- [70:58]
A
Means that to prevent wind and stuff from getting in, so there are inevitably going to be some cracks in the building. You're never going to fully seal something. That's just impossible. And so if you have a little bit of positive pressure, that makes sure that stuff can't just blow in easily because everything is blowing out and the stuff that's blowing in is going through the HEPA filter.
- [71:18]
B
So if you are able to maintain the AI being like pushed into the house, then you can basically think, well, maybe you would basically stop anything blowing in through any cracks that are remaining.
- [71:27]
A
Right.
- [71:27]
B
Would this be as foolproof as it sounds if you could actually pull it off?
- [71:30]
A
I mean it sounds, someone needs to test it.
- [71:32]
B
I mean it sounds mental.
- [71:33]
A
I mean, of course. But you know, the thing is like it's because it sounds mental, there are zero people working. Right. Like this was like, you know, a small fraction of one of our researchers time. Just kind of like looking into the role of materials and what you could use. And so I'm not saying that there should be a massive effort to look into this, but there should at least be one full time person looking into this as a backup option.
- [72:00]
B
Yeah, I guess. What is the work that has to be done now? I suppose it's figuring out whether any of this stuff works and then also I guess writing a guide to say, should the Time come. Here's all the things you might have in your house that you could use to seal it up.
- [72:12]
A
Exactly. So the ideal kind of two year goal would be you have a team of people, they've tested, tested out all of the vapors, all of the different improvised methods, and they've run it through really, really rigorous tests. And then at the end of that, you could have a guidebook that explains how to do this in an emergency. Or you could imagine an LLM that's trained. You could take pictures of things in your house and the LLM is telling you how to tighten up the cracks and adjust the filter or things like that. And so this seems doable. It's like a research project that's taking advantage of the fact that there could be a lot abundant materials all around the world that could protect us, even against the kind of worst case scenarios.
- [72:55]
B
Yeah. I guess as we can remember in the early stages of COVID I guess once it really hit home what was happening, everyone was spending almost all of their time learning about this thing and trying to figure out how to react to it. So you have a lot of people who would be willing to try to figure out how to protect their house at that point. They're not going to be willing to do anything right now. But if you can figure out, I guess, how to repurpose all the stuff that they happen to have lying around anyway, then you would have a huge labor force force basically to scale it up.
- [73:18]
A
Yeah, exactly. And I should also mention it might not necessarily be the case that houses are the best thing. Like maybe you want to be converting office buildings or malls or other things like that to protect as many people as possible. But yes, basically you take advantage of the fact that in an emergency there's a lot of motivation and people are going to be using the information that's already been produced.
- [73:37]
B
Yeah. So you're hiring someone to lead on this task of trying out all these different.
- [73:40]
A
I would like to find.
- [73:41]
B
This sounds like a pretty cool job.
- [73:43]
A
I mean, it could be. It could be a cool job. I mean, there needs to be like at least one full time person thinking about this. It's crazy that, like, for all of humanity, like there's not one person who's like, you know, digging into.
- [73:53]
B
We can't scrounge up one human.
- [73:55]
A
Yeah. So I'm looking, I'm looking.
- [73:57]
B
Yeah. All right, let's switch on to pillar three, which is detection, I guess. What is the point of detecting these things A little bit sooner than we like eventually we're going to figure it out because everyone will be dying.
- [74:05]
A
Right.
- [74:05]
B
But what's the benefit of doing it a day or a week earlier?
- [74:08]
A
Well, I think the benefit could be huge. In a lot of scenarios, you might want to be able to stop the spread of something. And even in instances in which you're not able to stop the spread, if you really imagine a truly catastrophic or kind of existential risk, it's important to think about how much governments are willing to spend and willing to go all in when they have their back up against the wall. One of my favorite statistics is that at the end of World War II, Japan was spending 76% of its GDP on the war effort and the United States was spending about 41% of its GDP on the war effort. And so if you take the current US GDP, it's like what, $30 trillion now, and you say that the US spends half of its GDP fighting some sort of catastrophic risk. I think that comes out to like 100 billion per day of acceleration in countermeasures. Now that's the kind of hypothetical best case scenario where governments are taking things really seriously. But yeah, basically detection. If you don't know that something's spreading, you can't do all these really dramatic countermeasures.
- [75:14]
B
So I guess the earlier you find out, the sooner people will be able to turn that GDP onto focusing on this issue. And I guess also at an earlier stage, there's some options that might be available to you that the door might be closed a bit if you waited too long. There's also one case where you really do have to find it out because you might not figure it out for a very long time, is if you have a disease that's spreading that doesn't really show any symptoms for a very long period of time. Exactly. Something more like the HIV case, right?
- [75:38]
A
Yeah, yeah, exactly. That's where the detection really, really helps a lot because you might be able to contain it. And it also allows you to get started working on medical countermeasures, which might take a bunch of time. So you might need that time for the medical countermeasures.
- [75:50]
B
All right, so what should we do on detection to find out about new diseases? Soon enough?
- [75:55]
A
Yeah, so I think there's a lot of stuff to do on detection. One of the things that open philanthropy is funding is kind of pathogen agnostic metagenomic sequencing. And I can get into that in a moment briefly. The way we detect diseases now is that people show up in a hospital and they have symptoms and maybe there's a Cluster of people. And the doctors will, you know, well, first of all, in some instances, if the symptoms look like the flu or Covid or something, people just get sent home. So in Seattle, during the early days of COVID it was like, peak flu season. People were showing up to the hospital or, like, you know, clinics and doctors would just be, like, sending them home, saying, like, I'm sure you have the flu, and not testing. But let's say that there's a cluster of people, they're showing some symptoms that the doctors can't quite explain. Typically, the doctor will run a panel of tests. If those tests come back negative, then the doctor is like, okay, gosh, this is a bit weird. Maybe we should look into this. They will then probably take a sample, send it to the cdc. The CDC will then run metagenomics over it to sequence all the stuff that's in the sample to determine if this is some new pathogen that we have haven't seen before. And I should say, in some situations, the system actually works surprisingly well. With COVID and Wuhan, the virus was sequenced within two weeks of the first cluster showing up. So that's pretty good. I think the thing that's dangerous, though, is if there's something spreading that does not show symptoms. So for hiv, for example, typically once you're infected, you're not gonna be showing symptoms until eight to ten years after you were infected. And so if, you know, and it took us a very long time to discover HIV, it wasn't discovered until the 80s, and so it had already been spreading for, like, many decades. And so if there was something that was spreading quickly, like that was airborne, that would be far too slow. And so, yeah, there's this question of, like, well, why don't you just skip to the last step and start doing some metagenomic sequencing on presumably healthy people just to make sure that they're not, you know, harboring some sort of of, like, pathogen that has a long latent period. And this is something that people are doing. So we're funding the Nucleic Acid Observatory. They're one of our grantees, and basically they're swabbing people in the Boston subway and, like, sequencing that. They're also analyzing a lot of wastewater. So lots of times, if you're infected with something like a virus, it'll come out in your wastewater, and then all of that goes to the water treatment plant, and they take a sample out of that, and then they sequence all the viral, like, RNA and DNA in that sample, and so they can pick up on Interesting things that people have missed by doing that.
- [78:37]
B
I guess there's going to be an enormous number of DNA sequences in people's feces, but I guess they can just get a good sense of what the baseline is and then they can just flag anything that they've never seen before.
- [78:47]
A
That's right. And in fact this is actually one of the big engineering challenges. And I should say there's a very small team of 10 people working on this specific problem. And so I think more talent directed to this problem would be good. But yes, there's a huge amount of DNA in these samples. And so typically what they'll need to do is they'll filter it out. So like bacterial DNA that's like, with like very common bacteria, they'll want to like get rid of that. Like human DNA, they obviously want to get rid of that. And then there are different filters. You know, you can just run it through a filter. And so that also gets rid of like the chunky bacteria so that you're like left with like the viruses and like free DNA. So there are like a lot of sample preparation steps. Now granted, these sample preparation steps make it so it's like slightly, slightly less fully pathogen agnostic. So if there was some crazy bacteria that was spreading and it looked a lot like a regular bacteria, you wouldn't necessarily pick that up using this method. But that's something that the team is looking into, ways of making the method more generalizable to a full range of threats.
- [79:45]
B
Would have thought this is a hell of a bioinformatics problem because all of these things are evolving all the time. There'll be new shit showing up constantly. How do you pick out the one that's the super bacteria that's going to kill everyone? Everyone?
- [79:57]
A
Yeah. If you have a bioinformatics background, and this sounds like an interesting problem, you should maybe be working on this problem.
- [80:04]
B
I guess we are getting a lot better at, I guess telling, going from a sequence to figuring out what would the protein or the enzyme do. Trying to like, I think we kind of can do that to a reasonable extent. Now just say, well, here's a particular genetic sequence in an animal that we've never seen before. We can kind of guess what this protein is probably for. Does that help us?
- [80:21]
A
It might help us, it might also hurt us. So there's this interesting question about like the offense, defense, balance of detection. So imagine you had a perfect tool that would enable you to generate some protein function using a wide variety of different sequences. One thing you could potentially do is you could engineer a virus that has a very different sequence, but it basically is functionally the exact same as the virus you're trying to get. And if you're an attacker, this would be very useful first of all to get around DNA synthesis screening mechanisms, because when you're ordering that DNA, they might not recognize that as a dangerous pathogen that you're ordering. And then second, if it's spreading and they sequence it, it might not look like any other virus that people have seen before, even if the virus is in practice something that we kind of know what it is. So this might be a reason why really good tools like this could help the attacker. But I think there's this interesting argument which is like, if you're able to redesign a virus to that extent, the defender ought to be able to check and it ought to be in some sense cheap, cheaper to check than it should be to create. So kind of like a verification versus creation, like P versus NP type thing.
- [81:30]
B
Sure. I would think most of the time most genetic sequences are similar to other genetic sequences in other species because most of this stuff is conserved and reused. If you came up with a completely new protein from scratch using some AI driven tool that could figure out exactly how the protein would fold and what it would do, then it would stick out like a sore thumb, basically. It would never arise naturally.
- [81:48]
A
Possibly. Yeah. But I think that you'd have to design your bioengine informatics to detect stuff like that and have a really strong baseline and do other things. Yeah.
- [81:57]
B
Okay, so the Nucleic Acid Observatory, they are going around and swabbing people in Boston. Why are they doing it just in Boston? I think when I've heard this suggested before, it's always been, oh, you should do it in airports or you should be grabbing stuff off of airplanes.
- [82:07]
A
Yeah, sorry. They're looking at over 20 wastewatersheds across the US so yeah, it's much broader than just Boston. The swabbing is like an early pilot study that they're looking at. So they're, they're not sure whether or not to scale that up yet.
- [82:19]
B
Yeah, I guess ideally probably you'd be sampling from around the world to catch things earlier, but I suppose that's just like insanely more difficult to get permission to do all of that.
- [82:26]
A
Yeah, and airplane waste is interestingly, apparently really good because there's A, it's really cold and B, there's a lot of detergent in it. And so you know, when there's like human waste that goes in, it kind of just gets like preserved in the way it was, because a lot of times there's waste. There's like bacteria that grow and they, like, you know, explode in numbers. And so then the signal to noise ratio gets messed up. And so having like, like cold sterilized waste is, like, ideal for metagenomic sequencing. So, yeah, airplane wastewater underrated as how humanity gets saved. That's right, yeah.
- [83:00]
B
Is there much interest in doing this sort of work of detecting what pathogens are out there for just like, more mundane public health reasons? I was like, maybe you could bring in funding that's not focused particularly on this.
- [83:09]
A
Yeah, I think the metagenomic stuff is actually promising for a lot of different reasons. One interesting result that the Nucleic Acid Observatory found is there are a number of different types of flu. So there's like flu A and flu B. There's also flu C, which not many people have heard about. And that's because it's thought to be really rare and not even worth testing. But interestingly, because they were using this metagenomics approach that they were searching for basically all human viruses and all possible pathogens, they actually found that there was one city in Missouri that had flu sea level levels that were basically the same as the flu A level. So, yeah, I think we're learning interesting things that could be applicable more broadly for public health using these methods as well.
- [83:53]
B
So how early would we be able to detect things using this kind of method?
- [83:58]
A
Yeah, so I think it depends on how many areas you're sampling from and how deeply you're doing the sequencing right now. By the end of the year, they should be able to hit something like detecting something before it infects, say, 1% of a population, cumulatively, which is not very good, to be clear.
- [84:18]
B
Yeah, that's pretty late.
- [84:18]
A
Yeah, yeah, it's quite late. And so ideally, they could drive this down a few more orders of magnitude and get other possible signals, but I should say even the 1% still could be good in a kind of stealthy scenario where, yeah, that might be the difference between most people catching it versus stopping it. Still relatively 1 or 10%, I guess. Yeah. Which again, we need to improve that. And I think there's a lot of important work that needs to be done. But, yeah, I think they're on track.
- [84:49]
B
The challenge with this detection stuff, I guess, has always been that to catch it when 0.1% of people have it, rather than 1% of people, you kind of need to be scanning 10 times as much stuff to get to have the same probability of doing that. And so it's like 10 times more expensive. So each doubling time that you want to do it earlier, the costs escalate pretty massively.
- [85:06]
A
Yep. Yeah, I do think this is a big problem and, yeah, I think it'd be good to supplement this with other detection methods. Yeah. I mean, I do think this is like a weakness of this approach.
- [85:17]
B
And is that still the main reason why people think we can't. It's going to be very hard to use these sort of techniques to discover a new disease so, so early that you could just contain it and cordon enough and ensure that it doesn't spread.
- [85:26]
A
Yeah, I don't. I don't think the metagenomic strategy is going to be like the way you contain an outbreak. I think the more traditional, like approach is probably going to be like, better there.
- [85:34]
B
Scanning people coming into hospitals.
- [85:36]
A
Yeah, yeah, something like that. Again, that's like relying on the thing not being particularly having like a long.
- [85:42]
B
Leading period or I guess like symptoms that are masked because they look exactly.
- [85:45]
A
Like some other things. Looks like the common cold or something. That would be bad.
- [85:48]
B
Cool.
- [85:48]
A
Yeah.
- [85:49]
B
Is there much more to say on detection?
- [85:51]
A
No, just that again, there's a very wide number. There's a broad community of people working on disease surveillance and I think they're doing really important work there. I think the number of people working specifically thinking about these long latent things is very small.
- [86:10]
B
So there is one more thing we should definitely talk about. How do you detect mirror bacteria? Because they're completely different.
- [86:15]
A
Yeah. So, unfortunately, I don't think there's going to be any detection needed. I think in a mere bacteria scenario, the forests are going to be getting destroyed.
- [86:24]
B
They're going to be, or I guess.
- [86:25]
A
Lots of people getting animals, cities getting destroyed. I mean, it's not going to be subtle.
- [86:33]
B
I guess. What would it look like at the earlier stage when it started infecting people? I suppose you're saying you'd basically get set sepsis because it would get into your blood potentially and stop.
- [86:41]
A
I think it's actually really unclear. So, yeah, you might die of stroke because the bacteria accumulate. You might die of sepsis because it grows kind of out of control and then your immune system finally might catch some glimpse of certain things. It's actually not clear how you would die, but I think generally speaking, having an uncontrolled growth of something in your bloodstream generally leads to death. So that's. Yeah, cheery topic. I laugh about it all the time, too.
- [87:14]
B
All right, moving on from detection.
- [87:16]
A
Yeah, great.
- [87:17]
B
Pillar four. We got to get out of this. So we're imagining a scenario, I guess, in the worst case where you've got everyone wearing masks in order to go to work and keep society functioning. Everyone else is like, hiding in their homes, occasionally seeing their loved ones. I guess the detection phases is well and truly over. How are we going to get back to some sort of normality where civilization can resume?
- [87:34]
A
Yeah, so I think the way we dig ourself out of this hole is going to have to be through some sort of medical countermeasure eventually. And so that's basically what we did with COVID There were lockdowns, eventually we had the vaccine. The vaccine allowed us to get back to normal. So I think that's going to be similar for even more catastrophic events. Now, on the medical countermeasures thing, I think it gets a little bit more complicated when you're thinking about an intentional adversary that might be designing things to especially bypass your medical countermeasures. So, yeah, maybe I'll talk a little bit about the weaknesses of medical countermeasures before I talk about like, why I think they are. So I think the reason they're good is like, obvious, like in some sense, right? Like we all. Yeah, yeah, yeah. So like, you know, I guess people.
- [88:16]
B
Often have the reaction these days that pandemics aren't going to be such a severe problem because we'll just do an MRNA vaccine and that will basically solve it. Like why, why isn't that a reliable strategy?
- [88:25]
A
So I mean, my, my hope is that that will be a reliable strategy for like, a lot of different threats. And I think, like, the work that, you know, CEPI and like, other groups are doing is just like, absolutely, like essential. But I do worry that in the scenarios where there are adversaries that are intentionally engineering things that might not necessarily be sustainable or ideal. And then there's the more common kind of critique, which is medical countermeasures take a long time to produce. So with COVID it took almost a year. The 100 day mission is a thing that people are excited about and I think is really exciting. But that's still 100 days to get from having to have a vaccine. And if you compare the speed at which, say, like Omicron variant went through China, it infected something like 80% of the Chinese population within six weeks. And so 100 days in some sense is still far too slow. And that's why I think we need the protective equipment and the other things in order to slow down the spread and make sure, sure that we can keep things Running while we're doing the medical countermeasures.
- [89:30]
B
But do we think that MRNA vaccines, you can make one of those against most of these different threats?
- [89:35]
A
You might be able to make an MRNA vaccine against a lot of different things, but I don't necessarily think you could make an MRNA vaccine against anything. I think mure bacteria is a good example. You need to have a special conjugate vaccine against MYR bacteria, and I don't think MRNA would be suitable for that. In fact, bacterial pathogens in general are much harder to vaccinate against. So, like an anthrax vaccine, you need to take, like, five different doses. And, you know, generally speaking, the antibiotics are much more efficient. So, like, depending on the biological threat, the vaccines might be, like, more or less effective. And I think that gets into the second point, which is it's not obvious that you're going to be able to make a medical countermeasure against, you know, any possible threat. So, you know, there are several examples here. One would be, there were a number of researchers in Australia, and they were studying mousepox, which is like the mouse version of smallpox, and they inserted an immunoregulatory gene into the virus because they were trying to sterilize the mice for some experiment, and that ended up killing even the vaccinated mice. It was, like, highly lethal even in the vaccinated mice. And so this was just, like, one gene that they stuck into this virus. And so, again, if you look at the Soviet program, they had, like, thousands of scientists figuring out how to make their bioweapons just, like, overcome vaccination, medical countermeasures, again, plague that was resistant to 16 different kinds of antibiotics. And so, yeah, I don't think that I'm that optimistic about finding a medical countermeasure in advance of a threat, because there's so many different options that an attacker could pick. And so you're probably gonna have to do the medical countermeasure in a reactive sense, like, once you know what it is that's spreading that you need to counter. And then even then, it's not obvious that a medical countermeasure is gonna work quickly in any given situation. I think hiv, good example of this, HIV is a virus that infects the immune cells that are needed to mount a vaccination response in the first place. And this is why it's been so difficult. We've been working for 40 years to try and get an HIV vaccine, and it's been very, very difficult. And I think hopefully there are some promising signs recently, but the human immune system is in some sense kind of a fixed target. And so as technology gets better and better, it's less clear that you couldn't find vulnerabilities that break the immune system in pretty fundamental ways.
- [91:56]
B
Yeah. You're saying if you're actively malicious and trying to maximize the damage that your design disease does, basically it's very likely that you would choose a virus or bacteria that immediately goes and just damages the immune system as much as possible so that then an immune response is not possible.
- [92:09]
A
Right, yeah.
- [92:10]
B
And so that breaks the MRNA approach. I guess you could still do the antivirals like the specific chemicals or the antibiotics that target without going through the immune system.
- [92:18]
A
That would be the hope. Those are current, currently quite slow. There are not many people working to design new ones. And I think there needs to be a lot more money going into antiviral and antibiotic development. And open philanthropies funded some good work on new methods of making antivirals.
- [92:35]
B
So if we assume that kind of helping the immune system with a vaccine or MRNA or something like that is off the table, then it sounds like we're in a pretty difficult situation because we don't have an example of quickly, in 100 days or even a year turning around a new antibiotic or a new antiviral that really effectively hits some new arbitrary disease that we just discovered. I think nothing like that. And we'll be doing this in a very difficult time under massive duress.
- [92:57]
A
Yeah, yeah. So I don't think supplementing the human immune system should be off the table, but I just don't think it's necessarily a sure thing. And so I think we need to have backup options. But I think there is at least maybe some theoretical reasons to think that medical countermeasures pressures might end up being kind of defense biased in the longer run. This is still a somewhat speculative hypothesis, but might be interesting to talk about. The hypothesis is called the wrench hypothesis. And this kind of came about because we were thinking, or I was thinking about nanotechnology and the gray goo scenario where we have this little nanobot that's spreading and eating stuff and killing people. And the question is, could you develop some sort of countermeasure that would stop the nanobot? And I think the answer should be yes. I think probably there should be things that you could do there.
- [93:51]
B
Basically, you have to find a molecule that fits into some part of the nanobot and jams it up, but doesn't do that for humans.
- [94:00]
A
Yeah, exactly. Yeah, That's Precisely what the strategy would.
- [94:04]
B
There's got to be a shape like that and then a molecule that fits that shape.
- [94:07]
A
Yeah, yeah. And so, yeah, we call this the wrench hypothesis because it's like a wrench in the gears of a machine. And for those of you who are not familiar, this is exactly how like antibiotics work. This is just like basic, you know, so you can think of a bacteria as being a little machine that's like made of lots of little tiny machines inside of the bacteria. And the way antibiotics work is they go in and they're a molecule that just sticks to the machine and like gums up the gears and it just like prevents that machine from, from working. And if you break enough of the little machines in the bacteria, the bacteria itself ends up dying. And so I think there's this question of could you ever have some self replicating machine where it was impossible to find a little wrench that broke it? And I think the answer is no. Basically, if you're self replicating, you have to be taking in nutrients, you have to be pulling in molecules. They're going to be delicate things that are responsible for that. And kind of in the limit, you can imagine a nutrient with another thing attached to it that just ends up breaking it. So it's like hard for the thing to discriminate between malicious molecules and the ones that it needs to grow and replicate. This is also true of viruses, I should say as well. So viruses, they use a lot of your machinery to self replicate, but all the viruses have at least one or two little machines that they themselves make that are different from you. And that's again, what most antivirals are targeting. They're blocking and breaking the. Those little machines.
- [95:33]
B
Yeah. Okay. So in principle, we think that if our technology was sufficiently advanced, the defender would win here. I guess one reason is that they get to go move second. So whoever, someone's got to design the pathogen, they've got to design the nanobot. And then it's not able to adapt all that much, not all that quickly. And then you just get to choose whatever you think is the weakness.
- [95:51]
A
And maybe you could choose 10. Right. Like there's no reason to stop at one if you get really good at designing them.
- [95:55]
B
Okay, yeah. So what should we do in order to take advantage of this fact? Is there much we can do to potentially have medical countermeasures that are much more general? General are much faster?
- [96:04]
A
Yeah. So I think this is still early days, so this is still kind of like a sci fi technology, but you could imagine in the future, if models get really good, imagining basically being able to produce antibiotics and antivirals very quickly by you can imagine this kind of sci fi technology where you get a new sequence, you put it into your alphafold, that shows you what all the machines are doing. And then you can design a giant library of models, molecules that stick to those machines and break them. And then you run that through a toxicity screen and you're pretty confident that it doesn't break any of the human machines or break down into some other harmful molecule. And then you also might filter for things that you can manufacture at scale really quickly and then produce a lot of it and test it. I think this sounds good in theory. In practice there are actually a lot of bottlenecks. And the thing I'm to trying describing is very, very difficult. And this is not the type of thing that I think we could do in two and a half years. But I also think we're in the early stages of thinking this through and we have one researcher who's kind of like looking at the different bottlenecks. But I think there should be more people thinking about this and working on it.
- [97:15]
B
I guess there's probably huge broader interest across all biomedical research. I mean, there's been a huge effort to figure out how to solve the protein folding problem. Not just because we want to stop biological catastrophes, because it's incredibly useful.
- [97:25]
A
There's a very wide community of people working on things like this and thinking through it.
- [97:30]
B
Are there any particular missing pieces that you think OpenPhill could fund to speed this stuff up?
- [97:34]
A
I think we're in the early days of looking into this and I think that yeah, ideally we would hire someone who could figure out whether or not there's a way that funding could actually accelerate this kind of future.
- [97:44]
B
So I guess as you were saying, well, you can just find arbitrary stuff to throw at any nanobot or any bacteria or virus. It's also occurring to me, well, couldn't you just simulate the human body and then find arbitrary numbers of specific poisons that happen to break all of our machinery? I guess, why doesn't that put us in a pretty dim situation?
- [98:01]
A
Yeah, so I think you could do that and basically what you'd be doing is you'd be generating a big library of chemical weapons. I think the thing is the chemical weapons don't replicate. And so fundamentally it's hard to distribute them to everyone. Yeah, you might make a really good poison, but the reason biological weapons are scary is because they self replicate and they get everywhere, whereas Know, designing there are tons of molecules that already kill people. Like finding new molecules that kill people is.
- [98:27]
B
I found a poison you won't believe. Yeah, yeah, we already had those.
- [98:31]
A
So I think that's like the main argument for why this biases the defender.
- [98:36]
B
Okay, let's zoom out and consider the four pillar program as a whole. I guess there's a lot to like about it. I think even someone who was skeptical would probably think some parts of this might work. But in order to really be secure cure, we need kind of these four different things to be all working somewhat in order. We need to detect the thing early enough. Then we need to ensure that we roll out the PPE to tens or hundreds of millions of people, I guess ideally billions of people around the world. Then we need to be I guess bio hardening the offices of the things that spread and that things are getting to the house. And then I guess we've got to as everything is maybe falling apart and people are struggling to survive, we've got to do the best biomedical research as quickly as possible and then manufacture enough to a millions of people. It's quite, quite an effort. Do you think that all of these things would be able to work together or would probably one of them break and that would be the thing that would wreck us?
- [99:25]
A
Yeah. So I don't necessarily think you need all four of the pillars. I think it depends on what kind of threats you're facing and which ones you would need. If you really had the amazing sci fi medical countermeasures thing where you could produce 10 new antibiotics against a thing instantly, then you wouldn't actually, actually need like anything else other than protection.
- [99:45]
B
Just need masks for a little bit.
- [99:46]
A
Yeah, yeah. Or something like that. But again like I don't want to be banking on that kind of sci fi tech, which is like why I think just the like really robust like physical defenses, physical sterilization, you know, simple cheap masks is just like the way to go to buy time for the other things.
- [100:01]
B
Okay, so the four pillars is like meant to be in aggregate. It could defend us against like the worst case basic story. But like many other, many things will fall short of that and then like maybe you know, muddling through would be enough to at least prevent if extinction.
- [100:12]
A
Right. Yeah. And I think all of the four pillars basically benefit because I don't think future technologies break them. As technology gets better and better and better, all four of the pillars ought to bias the defender and get better and better faster than the attacker is getting good. So that's part of the idea behind them.
- [100:31]
B
So what is the biggest weakness of the plan if it didn't work, what's the reason?
- [100:35]
A
Yeah, I think there are two arguments. One would just be lots of human error. So we have yet to actually test a lot of this stuff, like it's working on paper, but we need people to actually run this stuff to ground and figure out how well do the masks fit after eight hours or 10 hours. I've worn them for a while, but I have not worn them for a full work week. And probably I should do that at some point to really get a taste of my own medicine if I'm telling other people to do this. So, yeah, actually we have a team week planned. We're going to do some of this. I think another big weakness is if you're worried about an AI scenario where an AI is deploying biological weapons, it might not be just the biological weapons. Maybe you also have to be worried about cyber attacks or drones that are picking off people, and that might make the defenses substantially more complicated. The other argument is, again, we're kind of imagining just a biological catastrophe. Normally, I think when you think about catastrophes, it makes sense to not think about correlations between them. The probability that we have a pandemic and an asteroid at the same time is obviously like low. But you know, with biological weapons, like the Soviet Union, like, the plan was that they would first nuke and then they would use the bioweapons like afterwards to like mop people up. Yeah. And so, yeah, I think in some scenarios you can imagine there being a tight correlation between things like nuclear weapons or, you know, other infrastructure destroying things combined with the biological weapons. And the plan is not robust to that.
- [102:11]
B
Trying to pull all of this off while we're also. I've just been through the nuclear apocalypse, I guess. Yeah.
- [102:16]
A
There's no way you can make that seems really rough. I mean, some of the stuff like PPE still should work, you know, you.
- [102:21]
B
Stop at home already. Yeah.
- [102:24]
A
You know, and it also would help you against the fallout. So, you know, but yeah, that's. That's obviously like a much more rough situation. And similarly with like the mere bacteria scenario, like you're not just having to protect humans, like your agriculture might be getting destroyed at the same time. And so that like compounds the problem substantially. So, yeah, I think there's just something about having a layered defense and making sure that these systems are robust enough that they can be done even in a really stressful bad environment.
- [102:54]
B
So it sounds like you're saying you don't have a plan for how we can protect against all of these things happening simultaneously. But I guess other people should try to make sure that there's not a nuclear war. Other people need to do their jobs too.
- [103:03]
A
Something like that.
- [103:04]
B
If this is such a good, and I mean, I guess with the benefit of hindsight, somewhat obvious plan in some ways, why hasn't anyone else proposed it? And when you've gone and shopped this, this around being like, presumably you've spoken to people in government or people in the broader pandemic control area, you'd be like, why aren't we already doing this? What's their reaction been?
- [103:23]
A
Yeah, I mean, I think the short answer is actually we haven't been talking about the plan much and this is like the first time we're talking about it publicly. So, yeah, we're going to do more writing. We're going to do more to get it out there. Yeah. I mean, on some of the specifics. Yeah. I mean, I think the elastomerics are like, very well received. I think. I think smart people in government do look at that and they think, oh, yeah, that actually just makes a lot of sense.
- [103:45]
B
Is there any plan for the government to buy up a whole lot of those or is anyone really buying them?
- [103:49]
A
Not that I know of. The U.S. department of Defense put in an order for a number, and I think that makes a lot of sense and that's good. But I don't know of any other big.
- [103:57]
B
So what fraction of the US population do you think would survive here? I mean, it sounds like in principle it could be like almost everyone, if like all of them, people don't make.
- [104:04]
A
Mistakes or something like that. I mean, there's going to be. Yeah.
- [104:07]
B
Would there be a big gain going from having stockpiled 50 million of these masks to protect the essential workers who have to go out to having 300 million for the US and I guess like 10 billion or something for everyone?
- [104:18]
A
I think so, yeah. I mean, one is like, the allocation is never going to be perfect. It's going to be really rough if you're going to have to be triaging a situation like that. And ideally you want to be operating from a place of abundance where you have more than enough for everyone and everyone can get a mask. So that would be the ideal situation. So, yeah, I think there are still returns to, to getting more and more people covered. I mean, you could also imagine like in the crazier scenarios where you're like, also trying to fight some AI takeover, like maybe a lot More people need to be going to work than you thought. Like people like, you know, dealing with the cyber stuff and like people going to shut down data centers or whatever. Like, you know, there might be. There might be a lot more that needs to be done there.
- [104:59]
B
Yeah.
- [104:59]
A
So, yeah, ideally, like, you could kind of fight a war against an AI in your sleep, you know, like without the bioweapons kind of, you know, like ruining everything.
- [105:08]
B
So let's say that you did manage to get a bunch of money into this plan. You managed to get the 50 million masks stockpiled somewhere. Presumably this isn't going to happen everywhere else in the world. How big a problem is it that there might be significantly more fatalities outside of the United States or at least outside of rich countries? And supply chains across the world are like, I think the economy is just enormously contracting at this time. Seems like that could make it just harder to get a supply of all kinds of other materials. You need to keep things running to have the scientific basis. You need to do the best ever. Medical countermeasures.
- [105:38]
A
Yeah, I do think this could be a big concern. If you look at some of the COVID vaccines, I think they had supply chains and I think it was over 200 components from a lot of different countries. And so, yeah, you might need complicated supply chains. That might mean you need to protect a very, very large number of people, even setting aside the humanitarian reason, to obviously protect as many people as possible.
- [106:00]
B
Yeah, so it sounds like, I guess you're focusing on the U.S. that's where you are. That's something that is perhaps within budget, roughly. But having, I suppose, convinced the US to have enough.
- [106:11]
A
I don't think we're necessarily focused exclusively on the US I think there are some reasons to think the US Is a good initial place. It's relatively autarkic. It has enough food to basically cover everyone. Energy independence. There are a lot of things that make the US relatively robust to catastrophes. But yeah, I don't think we should be stopping there. And I think we should be doing more research on other countries that we'd want to be covering. And generally we want to be saving as many lives as possible. So I think getting this is widespread.
- [106:44]
B
So if you wanted to get a mask for every single person on Earth, it costs about $50 billion. That's like a lot of money.
- [106:49]
A
Assuming we can actually drive the cost down to $5, which is not a shock. Sure thing.
- [106:52]
B
You know.
- [106:53]
A
Okay, so let's conservatively say 10, 10, 100, 100 billion. Yeah.
- [106:57]
B
Call it a Hundred billion dollars, I guess. Yeah. In terms of. So that's like what fraction of global GDP? Like a 0.1% of global GDP. Like one off, basically, or every 10 years.
- [107:06]
A
So it's every 20 years.
- [107:07]
B
Every 20 years. Okay. So it's like 0.05.
- [107:09]
A
Yeah. You could amateur his autumn.
- [107:11]
B
Is it 0.005% of GDP on an ongoing basis? I guess to have everyone have a mask like that. Yeah, I guess it seems very doable.
- [107:21]
A
Seems maybe like we should do that.
- [107:23]
B
I don't know how much we ice cream globally, but it's going to be a significant amount more, potentially.
- [107:27]
A
Totally.
- [107:28]
B
Okay. Is there any plan? I mean, do you want to hire people to potentially be pushing this overseas?
- [107:33]
A
Yes, definitely. So, yeah, we have a new nonprofit that we're setting up. It has an interim CEO. We want to find a great team of people. We just got it started. Yeah, we want to find a great team of people that's excited about the personal protective equipment problem. We need people who are manufacturing experts. We need people who are logistics experts, global health experts, experts. We need just a really big team of people to be moving this idea forward, manufacturing it. Thinking through whether or not we should be doing a philanthropic strategy and fundraising from a collection of donors, or whether or not we should be getting governments to do this, or both.
- [108:10]
B
Yeah, I guess earlier I was talking about, well, if there's other countries that are not covered, that would be damaging to supply chains. Could also be destabilizing. If people can see that some countries are protected and some countries are not, that they could potentially turn to border violence or potentially. Or if they just think that the writings on the wall, they're all going to die.
- [108:27]
A
Yeah, I mean, you might be worried about that. Interestingly, I do think in catastrophes, people actually tend to be quite cooperative. This is slightly different than the situation you're talking about where one full country versus another and that I could see turning violent. But I think the scenarios where people are killing their neighbors and stuff is actually quite unrealistic. You see an actual disaster, disasters, people are actually really altruistic and you know.
- [108:54]
B
Yeah, I guess inasmuch as there's a hostile attacker that is creating these diseases, are we assuming that at some point they've been killed or they've been stopped from producing anymore? Because otherwise they just keep coming, like every month a new one, then that's like a pretty bad situation.
- [109:09]
A
Yeah, I am assuming that.
- [109:10]
B
And yeah. Is that realistic?
- [109:12]
A
I think that's like. I mean, it depends on like what the threat you're imagining is like, if it's a state bioweapons program and you're like, fighting a lot war, then like, you know, that's. I'm. I'm assuming that the more powerful countries are going to like, step. Step on that country or, you know.
- [109:26]
B
I guess it would become a very clear target.
- [109:28]
A
Yeah. I think in the instance of like a terrorist who's using this, like, you know, the whole world's resources would be focused on this and, you know, even if they could hide, it's going to be hard to like, you know, go around. Yeah. Continue the work. You know, if you're worried about some AI system like that, that might be scarier because, like, maybe the AI system is hidden across like a lot of different servers or something. And it's like telling people that they're making countermeasures and ordering them around and turns out they're making the next generation of weapons or something. So, yeah, you can imagine kind of scary scenarios like that. But overall, I think it's a reasonably safe assumption that you can stop the attacks once they start.
- [110:06]
B
Let's talk a little bit more about the interaction between these catastrophic bio threats and AI in particular. I've heard people make the argument before that it's a bit silly to be working on bio. If you have a picture where AGI is going to come soon, it's going to be enormously powerful because if we produce an AGI and it's really aligned with human interests, then it's going to be able to come up with technologies in a better plan than what we've got here in order to protect us. If we come up with a misaligned AGI that wants to kill us all, then it's going to be able to come up with something that would. It'll be doing the thing where it releases 10 of these diseases all very quickly, and this is not going to be really sufficient to protect us or we won't be able to dig our way out. Yeah. What do you think of that? Is kind of perhaps too extreme in either direction. There's. Maybe there's a middle ground.
- [110:48]
A
Yeah. So on the AI making defenses, I think one weakness of this argument is that a lot of the defenses might be physical manufacturing. Like maybe you just need to physically create a lot of masks. You need to physically create a lot of air filters and chemicals and stuff. And yeah, AI benefits a lot of different things, but it kind of is biased towards things that are information heavy versus physical manufacturing, especially early on. Exactly. And, and so, yeah, if you have a number of different AIs, some of which are trying to protect you and some of which are trying to hurt you, I worry that the AIs that are trying to hurt you might be able to generate lots of biological weapons before the AIs that are trying to help you can generate physical stuff that can actually protect you. Just because it's faster to generate a small snippet of biological code than it is to generate mass produce protective equipment and protective buildings and structures and stuff like that. So I think that's one argument for wanting why this still could be scary. And interestingly, I think this could still be scary even if you have smarter systems that are on your side, even against slightly dumber systems. Because it might be that the slightly dumber system is still able to make arsenals of biological weapons, whereas the smarter system has a harder task of physically manufacturing lots of things. Although if we're right about the four pillars, then maybe we only need human level technology to manufacture enough defenses and it's just a matter of like getting those in place really quickly. And I think that's like why I'm excited about doing it on like such a short timeline.
- [112:16]
B
Yeah, I'm kind of imagining a super intelligent aligned AI. We go like, what is going to be our plan for protecting ourselves from diseases? And it's like, well, obviously you should wear masks. I really can't do much better than that for you guys. We do the medical stuff later.
- [112:29]
A
That's when you made the masks. Yeah, like, oh, that was bad. Yeah. And then, and then like in the opposite direction. I think you could, could argue that in a lot of the scenarios, the superintelligence or whatever is going to have plenty of options. But I don't know. Imagine you're a misaligned AI and you've managed to escape from the laboratory, but you're not wildly super intelligent and you have all these humans that are doing their thing, and you also have other AIs and other labs that are getting developed that are going to be more powerful than you, and then you near future. And so you might be willing to take a lot of risky gambles to try and gain power or otherwise do things. And so what you might want to try and do is create some sort of way of surviving, even if most of the humans have been killed, and then release a lot of biological weapons in order to knock down all of humanity and stop the other AI labs from doing it, even if it has a relatively small chance of success. This might be the thing that you might be incentivized to do so. Yeah, I don't think we should be assuming it's all or nothing. I think we should be working on the margin where there could be AIs that are in positions where they'd want to do this. I also think. Yeah. How much probability do you put on a multipolar world where there are lots of different AIs that have lots of weird motivations or lots of people with lots of motivations that are using AIs?
- [113:50]
B
Well, for that matter, also just AIs that aren't necessarily super intelligent, but just super erratic. That's one thing that we've seen with a lot of models lately is that they just aren't doing necessarily the thing that their operators want them to do in all kinds of crazy ways. And they can be given random instructions and the open weighted ones. Any rando can potentially give them a random goal, alter them to have a different mission than the original one, and then give them a bunch of compute and see what they can do. It could be much more random.
- [114:17]
A
Yeah, what's like Chaos GPT? People are like, oh, what's the probability that there'd be an AI that's intentionally trying to kill humans? Just make that. People just make that for fun. Like it's crazy.
- [114:28]
B
Yeah. So we should expect, I guess, people who don't recall chaos GPT. I guess early in the early days back with GPT4, someone immediately, I think, made a model whose goal was to cause human extinction. And it was like a bit comedic on some level because it wasn't able to do it. But at some point, if people might do that with an opal, made a model that was actually in a better position to do some real harm. Yeah, okay, yeah, I interrupted.
- [114:49]
A
No, we need to defend against. That's crazy. And again, the number of people that are working to prevent these were worst case scenarios is tiny. It's like fewer than 100 people.
- [114:56]
B
Yeah, you've talked about this window of vulnerability, which I guess is the idea that as technology advances and I guess AI and I guess all kinds of medical technology advances, there's a window between when it's possible to create an incredibly dangerous biological weapon, which perhaps is already open, that window of vulnerability is opened. But we haven't yet reached the point where the defensive technology has been created and scaled up such that basically it's no longer possible for those really to succeed. And we're basically just trying to bring forward the point at which the defensive technology closes that window. Window of vulnerability.
- [115:27]
A
Exactly. Yeah, that's the plan. We want to close the window. We want to close the window quickly. And the way we're going to close the window is these four pillars. That's the hope.
- [115:36]
B
It sounds like you could potentially do quite a lot of these different pillars without necessarily having to have the government do it, more or less. I mean, you can distribute the masks at a private. It's not so expensive. You can disseminate information about how to harden your home. You can do the Nikolaika Assets Observatory doesn't require the government to operate it.
- [115:53]
A
I think that is one of the advantages of the four pillars plan. I think if the governments are doing it, that's the best world to be in. And I think governments ought to want to do things like this. But yeah, I think a group of philanthropists could basically do this and potentially do it well.
- [116:09]
B
Okay, so I guess up until now we've been thinking basically exclusively about biological catastrophes that kill human beings. I guess sometimes they spread from human to human, sometimes they spread from the environment to human beings. But there are, I guess we could imagine other ways that humans could go extinct from biological catastrophes. Like if you had something like myrobacteria that killed all of the crops or some other super disease that destroyed agriculture, more or less such that we just couldn't feed ourselves and things progressively fell apart. And I guess, I think people have theorized about you could have some sort of bacteria or virus that killed some natural environmental process that we relied on to survive, like the creation, I guess, not having enough photosynthesis to create enough oxygen for us to breathe. Yeah. Why are you not spending very much effort on those potential threats?
- [116:56]
A
Yeah. So the good news is I think these two risks are substantially lower than the others. So yeah, as you mentioned, I think basically you can divide all biological risks into one of three categories. Either things that target the environment, broadly speaking, things that target agriculture, or things that target, you know, human bodies. And interestingly, when I, when I first started at Open Phil, I was quite worried that, you know, maybe we would put all of our investment into things like good protective equipment and better vaccines and stuff like that. And then it turns out that like, this whole time we should have been worried about an agricultural threat or something like that. And so we had a researcher look into this and the task that we gave them was imagine a worst case scenario. So, so one way you could approach this research question is you could try and generate a list of like all of the horrible things that you could do to agriculture. And then like, look at the list and think like, oh, is this like that scary or not? But that generates a lot of information hazards. So you don't necessarily want to be doing something like that. And instead there could be another way of approaching the problem which is like, just go ahead and assume a worst case scenario and then ask like, how many people could we save in that worst case scenario? And the worst case scenario that we gave them was imagine that all crops die instantly at the worst possible time in the harvest cycle and that you can never grow crops ever again. And we initially gave this research prompt as kind of like, okay, that's the most extreme example. Obviously there's no way we could survive that. And so then we're going to titrate and like make the scenario slightly easier each time to then figure out like what the threshold is. And then it turns out that even in that worst case scenario where all the crops die instantly and you can never grow crops ever, ever again, it turns out that you can feed the entire US population for guess how long.
- [118:47]
B
Like I said, I know the answer, but it's more than you would think.
- [118:51]
A
It's more than you would think. And so people typically say, well, like, how much food do we have stockpiled? And we have about 18 months of food stockpiled, but the actual answer is 500 years. And the way you do this is you have basically bacteria that eat natural gas. And using only about 15% of U.S. natural gas production, about 6% of U.S. electricity production, and about $200 billion worth of infrastructure, we could feed every single person in the US for 500 years. And obviously you could increase the capacity of that if you wanted to feed the rest of the world as well. And that's the worst case scenario where all of the crops instantly die. No time to adjust. You only have the food in your stockpile as an adjustment period. And. Yeah.
- [119:37]
B
What does this actually look like, though?
- [119:38]
A
I mean, it's not a pleasant future. You know, you're going to be eating like bacterial sludge.
- [119:42]
B
I played this computer game. So what you're doing, you're producing bacterial sludge, basically.
- [119:48]
A
That's right, yeah.
- [119:49]
B
So all of the plants have died, so we're not going for walks outside.
- [119:53]
A
That's not a great future.
- [119:55]
B
But what we're doing is we're getting natural gas out of the ground and we're bubbling it through these tanks with bacteria living in it. I guess this requires some electricity. The bacteria, there's these specific bacteria that eat natural gas and then we filter out the bacteria from the Water. And that's what everyone is eating. Is this a complete diet?
- [120:13]
A
It is, yeah. Yeah. And it's interesting because. Or I think there might be like some minerals and like vitamins you then supplement with. But yeah, basically this is, has all your carbohydrate, all your macros, you know, totally counted for. And interestingly, this isn't some hypothetical technology. They already use this to, to feed fish. So they're like natural gas plants that then like the excess natural gas they like shunted off and then it's like used to produce fish food. So this would just be scaling up an existing technology.
- [120:41]
B
Do fish like it?
- [120:43]
A
I don't know. It's like nobody ever asked factory farmed fish. It's like not, not pretty. But yeah, so this would be like scaling up like basically an existing technology. I think there are some things that like researchers or philanthropists could do here. Like you could engineer strains to be nutritionally complete for humans rather than nutritionally complete for fish. I think there would be some adjustment period and maybe doing a bit of that work ahead of time would be slightly better. But also keep in mind this is the worst of the worst case scenarios.
- [121:12]
B
Normally we have more of a transition.
- [121:14]
A
Yeah. If you think about all the stuff that all fed is looking into, you can turn trees into sugar and do other interesting things to get you stop gaps. And we were assuming the worst case scenarios.
- [121:26]
B
Yeah. So the common factor between all of these plans to feed people without agriculture is that humans actually consume like shockingly little energy, raw energy in some sense. You're saying we could feed everyone using 15% of the natural gas. So it's like almost all of the raw energy that we're burning is not going into human bodies, it's going into cars and factories and so on and producing electricity. So there's an awful lot of chemical energy lying around somewhere basically that we could repurpose for feeding humans if we're savvy enough. But I mean, if I imagine this actually happening, I don't think that we would feed everyone very quickly. Quickly, because I don't think that we would plausibly repurpose all of these materials to produce this bacterial sludge quite quickly enough. I mean, do you actually think that that conceivably could happen or is it more like. Well, half of the.
- [122:11]
A
There is a lot of food that's in the supply chain and a lot of that food is going to feed animals. And so in a catastrophe.
- [122:18]
B
So how much animal feed is there?
- [122:20]
A
I mean, I forget. But if you add it all up, it's something like 18 months of food in the US and that assumes that the catastrophe happens at the worst possible time in the harvest. If it happens after the harvest, you get more. More like two years.
- [122:30]
B
Is this something specific to the US That I guess it's a real agricultural powerhouse. Yeah.
- [122:34]
A
Unfortunately, the US is a bit of an outlier on how much food we have stockpiled. China has also stockpiled a large amount of food. Many European countries, it's like closer to like six months or something like that. And like the developing countries, they don't.
- [122:46]
B
Have as much natural gas either. So the US is like in a pretty beneficial situation. Yeah, but can you just take animal feed? That's. Why do we have 18 months of animal feed? Oh, I guess so. It's not.
- [122:57]
A
It's like basically in the supply chain and. Yeah, of course, like, because the animal consume a lot more and then you consume a lot less of that. It's just corn, soy, wheat, and humans.
- [123:09]
B
I thought that there was different kinds of corn that animals eat and the humans. But I suppose we would just cook it and we'll find a way to digest it one way or another. Okay, so you just think that the agricultural thing is not a problem because not everyone is.
- [123:22]
A
I don't want to say not a problem. I think in a mere bacteria scenario where you have to do this and you have to protect the people, that's like a pretty grim situation to be in. But I do think that, you know, to be clear, I also think there are a lot of other like, arguments for why agricultural biological threats are like going to be less severe than the ones that are targeting humans. Like, you know, you can pivot, you know which crops you develop. You can like genetically engineer crops to like be resistant, whereas you like can't genetically engineer new humans, like quickly or whatever. Yeah, you're stuck with the humans you have. So I think there are a lot of other haircuts against the agricultural catastrophe argument. But yeah, I think this is an interesting example where if you think about a risk window for agriculture, we might already have exited the worst case scenario risk window of agriculture. And maybe we're resilient to even the worst case scenarios.
- [124:11]
B
Yeah, this is a case where I write a little bit more about the. If one country has lots of food and everyone else is kind of starving, I think a case of a disaster situation where people usually do turn to violence is actually like long term sieges where people start starving and then they really do start basically turning on one another in order to get as much food as they can. So if like the US had lots of food, but like everyone else literally was dying of starvation, I think people, I mean, I guess the US Is hard to attack, but I think you could see, like, international relations fraying.
- [124:39]
A
Yeah, certainly, certainly. And I mean, I think, I think that's like, why I think all fed is like, doing interesting work here where they are actually just trying to feed, you know, everyone.
- [124:46]
B
Everyone, basically.
- [124:46]
A
Yeah, yeah. There are just like a lot of things you can do on top of the natural gas as like, like stop gaps. And that was just like, with 15%. So the US could produce a lot more to feed. Feed the rest of the world as well.
- [124:56]
B
Yeah. So I guess all fed has a. You mentioned they have a whole bunch of other ideas about other sources of calories that we could potentially use so we wouldn't be putting all of our eggs into the. Into the fish feed basket. Okay, what about the environmental. To be honest, I actually don't even know what are the environmental disturbances that people are envisaging that could cause humans.
- [125:15]
A
So, yeah, you could hypothetically imagine something that like, somehow shut off photosynthesis or like, you could imagine, like, you know, people talked about, like, mere cyanobacteria, like sucking out the carbon and like, creating an ice age that like, you know, makes agriculture really hard and stuff like that.
- [125:29]
B
Explain that. Like, I'm five.
- [125:31]
A
Yeah. So, you know, if you have a mere bacteria, the viruses are not going to be attacking it. Right. So if it's hanging out in the ocean, like, it's going to be drawing down carbon, like maybe at a fast, faster rate than like, other, other organisms, because it's not getting digested, basically.
- [125:50]
B
Okay, so background information here is that there's cyanobacteria in the sea kind of everywhere. They're like a massive driver of photosynthesis. So they're doing a lot of work.
- [125:58]
A
Yeah, so they do a lot of.
- [125:59]
B
Work to draw carbon dioxide out of the air, turn it into, into oxygen.
- [126:02]
A
Yeah.
- [126:03]
B
But one, their population levels are regulated by the existence of viruses that attack things that eat them. Okay, but if you had mirror cyanobacteria, then they would have no natural predators. All of the viruses that have evolved to control populations or that happen to control populations of normal cyanobacteria don't exist. They would just proliferate to an extraordinary degree, and you would have, like, they.
- [126:25]
A
Would suck all of the carbon, and the carbon that they draw down would not be digested and then spat back out. So it would like, sink to the bottom of the ocean because it would.
- [126:35]
B
Be in sugars that have the opposite handedness and no one could, no one could digest.
- [126:38]
A
Turns out I'm not. I mean it could be a concern, but I don't think it's like a real existential concern. Basically. All these scenarios take way too long to. Yeah, like just saying even if you.
- [126:50]
B
Had the mirror cyanobacteria throughout the oceans, it would just take decades. Centuries.
- [126:56]
A
Yeah, it would take many centuries. And they're like pretty obvious countermeasures. You could just like make a mirror of a mirror. You could like do. Yeah, you could do other things. You could do like basic geoengineering stuff. So you'd have hundreds of years to deal with it. And they're like pretty obvious, obvious countermeasures. That's what I want to. And, and, and just like more generally, like it takes a really, really long time to mess with like the earth's environment. So like, even, even if you had a magical button that you could hit to like stop all photosynthesis, which is like, you know, the worst possible thing imaginable, you'd still have like 1000 years of oxygen just like hanging out, you know, for us to like figure things out. So even in like worst of the worst case scenarios, I think these scenarios are like quite unrealistic.
- [127:35]
B
All right. I guess we've gone in pretty deep on the plan and, and various different I guess, objections that people could raise to it, but I think I'm reasonably sold. I'd really like to see a bunch of this happen and I imagine many listeners feel the same way. I guess as we flagged you're like hiring hand over fist or trying to get some really talented people I guess to lead on each of the different pillars. I guess. Yeah. Maybe we should go through all of the main roles that. All of the most important roles that you're hiring for at the moment.
- [127:59]
A
Sure. So yeah, most important roles right now include grant makers at open philanthropy. So I'm growing my team team trying to figure out how to get a bigger team of people to basically deploy funding. So if you are interested in a grant making role to deploy tens of millions of dollars to and notably not just the four pillars plan, but also across a wide range of different biosecurity issues. Yeah, I would want to hear from you. Fill out the Google form.
- [128:27]
B
Yeah. What sort of person is a good fit for a grant maker role?
- [128:30]
A
I think people that are entrepreneurial who want to talk to as many people as possible, collect information, I think there's a common kind of misconception that grant making roles involve just reviewing lots of applications that come in and then just giving the thumbs up or the thumbs down. And that's not at all what the role is like. And that's especially not what the role is like in an area where the field is so small and you kind of have to create the things that you want to see. And so I would describe most of the grant making roles at open philanthropy as being more similar to venture capital or headhunting, where you want to go out and find people and get them working on the most important problems.
- [129:10]
B
Do people need any particular bio background or any particular domain expertise?
- [129:14]
A
Yeah, so this is a great question and I think this is a really common misconception that people need a really strong biological technical background to contribute in biosecurity. It can be very helpful. Half of the people on my team have biology backgrounds, you know, PhDs, but the other half don't. I majored in economics and a lot of the people that got into the field did physics were just entrepreneurs, like some of the highest impact people. One of them was like a software engineer at Amazon and then ran a startup. So a lot of people from a lot of different backgrounds can contribute. You don't necessarily have to have a biology background. Yeah, one way of kind of organizing arguing this is like, it's not like you need a physics background in order to reduce the probability of a nuclear war or something like that. So I think there are a lot of things that people can do even if you kind of abstract away the biological details.
- [130:09]
B
Okay, so that's being a grant maker on your team. What's the next most important role you're trying to fill?
- [130:16]
A
Yeah, so the personal protective equipment plan. We're still in the stages of figuring out whether or not this is something we want to go big on and thinking through how that could possibly work. I still think there need to be a bigger team of people working on running all those details to ground and figuring out like, you know, should the mask design be a certain way and how low can we get those costs. We have an interim CEO and she's doing a great job, but I think we need a more permanent CEO for that. And we also need people who work, you know, in manufacturing, product design, you know, communications. There's a lot of work on the PPE project right now. That project has maybe three full time people on it and this might be one of the most important projects for humanity. So, yeah, if you're interested in that. I think we'd love to hear from you. You should fill out the Google form.
- [131:06]
B
Yeah. We don't often have so many roles for people who are interested in manufacturing or logistics, all that kind of thing. So if you've been listening to the show and thinking I wish that there was a role for me, I think this is the point place for it.
- [131:16]
A
Yeah, totally. The person, Emma actually did her engineering degree, mechanical engineering, and then ended up doing AI safety. She ran meter briefly and then now she's working on the PPE thing.
- [131:28]
B
So I guess there's improving the design of the mask, making it a whole lot cheaper, figuring out how you can manufacture it at a bigger scale and I guess how to deliver it and then also figuring out how you would deliver it in the worst case scenario. How would you get it out there everywhere?
- [131:42]
A
Yeah, this is a very concrete problem and maybe one thing I'll say more generally is that biosecurity really lends itself to people that want to take a very concrete physical problem and make progress on it. And I do think this is maybe in contrast with a lot of the AI work where a lot of it is quite hard to reason about and it's not clear. I mean a lot of it's not even clear whether or not you're net positive or something. And I think biosecurity, the problem is just in some sense a very simple problem. It's like there could be a thing that's spreading and you want to stop that spread and you want to erect fiscal barriers and fiscal sterilization. So it's in some sense like a very straightforward strategy and so it's more easy to measure your progress and figure out like are we actually cutting these risks?
- [132:26]
B
Yeah, where can people learn more about that role?
- [132:29]
A
Again, fill out the Google form. All of this will just be in the Google form.
- [132:33]
B
Okay, what's the next one?
- [132:34]
A
Yeah, so as I mentioned, glycols could be a really interesting strategy. You know how many full time people are working thinking about the supply chain of that and how to distribute it, Correct? Yeah, there are some part time people looking at this and that's how we've run these initial numbers. But I think this deserves one full time person who's going to be thinking about this and working on it. Similarly for the kind of air filtration, thinking about ways of improvising that and thinking that through. That's another example. There are zero full time people working on that. We need to actually validate that, think it through, figure out if this actually works. And those are Roles where eventually they could be leading teams if they're doing a good job.
- [133:17]
B
Okay, and what's the next one?
- [133:19]
A
Yeah, so I think on the medical countermeasures strategy, we have a researcher at Open Philanthropy, she's been working on this a little bit, but I think we need more intellectual effort here. We need more people thinking about what the medical countermeasure strategies should be. So also researchers thinking about that would be good. I think like maybe I will just like zoom out and say in general, I think there are a lot of roles in biosecurity. We also have scholarships, we have fellowships. If people want to get involved in the field, that's like a really good way to start. We offer career transition grants for people that want to get into the field and don't quite know where to start. And you know, those have been very successful. Some of the top people in the field like came in through, through that pathway. Yeah. So fill out the Google form because.
- [134:02]
B
It sounds like The Medical Countermeasures 1 is perhaps a role where you would benefit from having some domain expertise because you're often dealing with quite technical biomedical questions. What's viable and what's not.
- [134:11]
A
Yes, absolutely.
- [134:12]
B
And I guess on the bio hardening, I guess that sounds like maybe a role for someone who's more like hands on engineering side of things.
- [134:19]
A
Hands on engineering would be good. Yeah, absolutely. And just like experiencing managing teams, managing organizations, pulling projects together.
- [134:27]
B
Because it sounds like in general you want someone who has a lot of initiative and is going to be willing to go where no one has gone before on some of this stuff.
- [134:34]
A
Yeah, absolutely. Entrepreneurial people. Yeah.
- [134:37]
B
Is there anything, I guess if someone didn't feel like they were suitable for those roles, are there any other things you could point them towards?
- [134:43]
A
Yeah, so I think there are a number of organizations doing really important work. The early detection system, again, the one that I was describing, it still has maybe 10 to 12 people working on it. I think they'll be hiring for more roles soon. So more people with bioinformatics experience, wet lab experience, even I think just logistical or government affairs experience. I think generally speaking, policy kind of cuts across all of these different areas and is really important.
- [135:12]
B
So I guess a lot of people who would be up to doing these roles might also be considering going into AI policy, AI technical work, or I don't know, some other AI related project which is such a topical issue at the moment, I guess. Do you think that this work is competitive or maybe even more impactful than AI related work?
- [135:29]
A
Yeah, I think There's a strong argument that it could be more impactful. That argument is simply. It's more neglected. There are far fewer people working in this. In most of the sub areas, it's like three to four people. It's also very tractable. I think we have a basic plan that I think could cut the risk substantially, whereas in AI, I think it's a lot less clear how successful the intervention dimensions are going to be. And then finally, I don't think it's wildly less important. I think if you think there is a 1 to 3% chance of catastrophe causing an existential risk in bio versus AI, it's probably within an order of magnitude. And so the neglectedness and tractability arguments can easily mean that on the margin people are better off working in biosecurity.
- [136:14]
B
And I guess personal fit as well. Someone who's like, if their main focus is logistics, this might be a better fit.
- [136:19]
A
Yeah, absolutely.
- [136:20]
B
Let's talk about a bunch of tactical career stuff, opinions that you formed over the years. One you wrote in your notes is that you think people too often do work just expecting that someone is going to be later in the pipeline who's going to be able to make use of it. And often this is just completely delusional.
- [136:34]
A
Totally.
- [136:35]
B
Tell us about that.
- [136:36]
A
Yeah, I mean, this actually was kind of. So I think there are two things here. One is like, actually, if I think about my own early career, I came across some of these kind of existential risk arguments and I was thinking, oh my gosh, that's really not important. I should focus my career on that. And then I ended up doing some very silly things. I was donating to asteroid deflection charities, which was, in retrospect, not very effective. And I was doing my master's thesis on the Great Filter and the probability of finding alien life and accounting for different things, because maybe that could possibly help. And I think I just had this mindset of, oh, there are a ton of other people focused on existential risk that are way smarter than me and they're going to go off and solve the problem and I'm just going to have this little tiny drop of knowledge that I'll put into the ocean of humanity's knowledge to solve these problems and that will be my contribution. And I don't think I fully realized exactly how outrageously neglected these problems were. And how if I just put in a bit of effort, you could end up in a very important position with a lot of responsibility, which in some sense is terrifying. But I think the other side of that is yeah, I think people can have a lot of impact if they really take ownership of a problem in terms of like passing work off. I see this a lot in people who like think that they're going to influence policy by like writing a report that no one reads or that they're going to like, you know, do research on a problem, like with the hope that people are going to like read the research and like use the research. And I think one interesting thing that I found is I actually think the quality of the research that me and my team is doing is actually a lot better because we're making decisions about like, like how to allocate money. And so we have these very high stakes decisions. And so the research that we do is directly informing that decision. And so I think what you generally need is a really tight feedback loop between the decision that needs to get made and the research that's informing that decision. And I think if that feedback loop is broken, it's very easy for people to do research that's quite disconnected from important decisions or decisions that people are acting actually making in the world.
- [138:49]
B
So why do you think it is that people find it so natural, the idea of I'll do some precursor work and then expect that someone else is going to pick it up and make use of it, even when they have no idea who those people are? They haven't even gone to check whether they exist?
- [139:03]
A
Yeah, I'm not sure. One hypothesis is that a lot of them have not been in decision making roles and so they don't have a good model of what's needed to make those decisions. I also think academia can sometimes instill this habit where to do well in academia you have to be sort of working on the trendy topic and making a contribution there. And you can make a contribution in an increasingly sub, sub, sub specialized field where you are just kind of adding a little drop of knowledge into a growing ocean. And I think on certain topics it's very important. But I think if you're really trying to do good in the world, you want to be finding things that are extremely nice, neglected, where in fact there might not be any good work on it yet. And so, yeah, I think that's another big difference.
- [139:50]
B
Yeah, I think among sort of the Silicon Valley entrepreneur crowd, the conventional wisdom is that there's always far more promising, interesting opportunities for new businesses than there are entrepreneurs who can give them a real go. I'm not sure what the underlying reason for that is. I guess it's possible the conventional wisdom is wrong. But I think it's is probably right. And I mean the world is big, but there just actually aren't that many people who are trying to start new businesses actually making a product that has never existed before. That is kind of an abnormal pursuit and many people in the world are just not in a position to do that. So you shrink the people a lot. And then I guess technology is always changing. The frontier of what things we could have a crack at is always potentially quite wide and it's hard to tell ahead of time what is going to work and what is not going to work. So it could just be the case that there's almost always more ideas. It's much easier to come up with ideas than it is to come up with an entrepreneur who's got to actually give it a sufficiently solid go to tell whether it will work or not.
- [140:42]
A
Yeah. And this has been my experience trying to recruit for a lot of these roles. I think typically I will tell someone like I think you should take this role. I think you're a really good fit. And they say, well obviously there's the second person and the counterfactual because who's the replacement? And often there's no replacement. There's one person who I think can do the job. And people tend to be surprised. Surprised by this.
- [141:06]
B
Yeah, I guess. Do people tend to assume that the government is doing more than it is? That there's so many people in the government on netsec? Wouldn't they just be on top of all of this?
- [141:14]
A
Yeah, certainly. I mean I believed this before COVID I assumed that the government would just be really competent at handling Covid and the CDC would just have it sorted. And I was pretty shocked. Yeah.
- [141:28]
B
What do you think is going? I guess there maybe isn't the budget. I guess these groups aren't always stress tested to tell whether they can really do the thing. Especially if you're dealing with a risk that only occurs rarely. Maybe it will never occur over someone's lifetime. It's really hard to know whether you're on top of it or not.
- [141:43]
A
Yeah, totally. Yeah. I think it's much easier to deal with problems that are more chronic and ongoing where you can get feedback loops. I think if you look at a lot of the public health agencies, they were focused on things like tobacco use and HIV and stuff like that where I think they were doing good jobs. But it's just very different than a fast moving pandemic where you have to make decisions under uncertainty. Uncertainty very quickly.
- [142:04]
B
Yeah. You think another mistake that people make or another place where people struggle to make the right decisions is choosing something that is ambitious enough that it has a big impact, but isn't so ambitious that it's beyond what they can actually accomplish. Can you explain that?
- [142:18]
A
Yeah. So I think sometimes people make the mistake of just taking a sub problem of a sub problem of a sub problem and then just kind of assume that if they do that, there'll be someone to hand the baton to and that will be implemented impactful. And I think people can end up wasting their time there. But I also think sometimes people in the EA community will kind of dream up these, oh, we should study the grand strategy of how countries interact, and then that can help inform how we should think about AI. And I think that can be sometimes interesting, but usually I've been quite disappointed at actual concrete results that come out of thinking like that. And so I think the more promising thing, at least the strategy we've pursued in biosecurity, is try and carve off problems like subcategories of things. Let's just limit our focus to human. To human transmission. And can we solve that subcategory? What do you need to solve that subcategory and then have an ambitious plan to solve that subcategory or things like that. And so, yeah, I think this is a general tactic that maybe more people should be considering.
- [143:21]
B
And how do you strike that balance? Well, I guess it's something that you think if I built a team, we could kind of handle this, or we could come up with an answ. We could come up with a project that would solve it basically without having to necessarily rely on other people to do all that work for us.
- [143:36]
A
Yeah, I think you can tell if you're in the sweet spot between being too narrow and too broad, if you're actually making progress, like if the research is actually tractable, if that makes sense. Maybe my hot take is that kind of effective altruism really pushes people to think about what's important. And I think think that's really good. But interestingly, people just kind of forget the tractability. And so people spend a lot of time just being like, well, AI is the most important thing. I'm going to do my career in AI. And then they end up just kind of forgetting about tractability. And a lot of people end up in careers that might not be necessarily having that much impact.
- [144:16]
B
Do you think that's still the case in AI today? It feels like 10 years ago it was harder to find direction, but I'm not sure.
- [144:23]
A
I don't have a ton of visibility on this, but definitely that would be my hypothesis.
- [144:29]
B
Yeah, I guess I would have thought that technical AI safety is kind of clearer what the projects are.
- [144:34]
A
Yeah, I think it's more tractable now than it was 10 years ago, something like that.
- [144:38]
B
But I guess even if you're doing a sensible project in AI policy or AI technical work, it still is overwhelmingly likely that it won't matter at the end of the day that it will end up being irrelevant.
- [144:48]
A
Sure.
- [144:49]
B
Because you got unlucky.
- [144:50]
A
To be clear, I think there is.
- [144:51]
B
Hopefully, hopefully none of the stuff that.
- [144:53]
A
What I'm doing is going to matter because there's not going to be a biological catastrophe. So I think you're always operating under uncertainty and I think I also sometimes get the feeling that because effective altruism is focused on the most important things that everyone kind of herds towards the most important thing. It's maybe cynically a bit like 5 year olds hurting to a soccer ball. It's like that's the most important thing and then everyone goes to it and there's then a of lot, lot less thought to where should you be positioned, should there be other important, but maybe slightly less important things to be focused on and should people be spread out a bit more? And my guess is yeah, that is the case.
- [145:32]
B
Yeah. People were biased towards crowding or herding into the most popular thing, I suppose, yeah. What would be the dynamic? I mean, of course it's the social dynamic that maybe you want to be seen as doing the most popular part of it.
- [145:42]
A
It's just easier to have a path that other people are helping on.
- [145:48]
B
I guess there's no coordination mechanism, I guess to ensure that the ratios across all of the people who care about existential risk are sensible. That it's like you can imagine everyone kind of chooses the thing that at that point looks most important, but then they don't have any sense of like, well, is anyone going to do the second and third and fourth most important thing?
- [146:02]
A
Totally. Yeah. Yeah. So this is my job here. I am go for spread out, figure things out.
- [146:10]
B
Is it possible that you're a little bit biased towards wanting people to go into your area, take good jobs?
- [146:13]
A
Yeah. So yeah, even I think climate change, I think the conventional wisdom within effective altruism is like the whole world is focused on climate change. And so on the margin, one additional career focused on climate change is not going to have that much impact. And I think at a high level that's correct. It's better to focus on these more neglected Risks. But Hannah Ritchie wrote this great book looking at climate change through a more effective altruist lens, thinking, okay, what's actually effective? What's not actually effective? And it's like, why did it take 20 years for someone to write that book? Maybe an EA could have written that book a lot earlier. Because if you apply the kind of importance neglected tractability framework on other problems, you can make a lot of progress. And so yeah, I actually think more people should be doing weirder, more unique things.
- [147:05]
B
Yeah. I had an interview with Johannes Akva who was working on I guess effective altruist, my minded climate change grant making and I think their team, as soon as they started looking at that question, were coming up with completely different proposals to what was being funded by everyone else. They're saying from our point of view, we don't want to focus on solar and wind at all because if that does solve the problem, then basically it's already handled. So we need to be basically concentrating on the specific scenarios where that does not pan out, which is actually reasonably plausible, and then thinking, well, what would work in that case? Which was like, I guess they thought it was a question that was basically not being addressed by any other group almost anywhere.
- [147:38]
A
Yeah. How many, many full time people did they think were working on this?
- [147:41]
B
Well, I guess on that.
- [147:42]
A
Two or something.
- [147:43]
B
Yeah, I guess I wouldn't want to put words in their mouth, but I think, yeah, it was very small or that was like very niche issue at best.
- [147:51]
A
And I think this is the case with many things in the world where more things are outrageously neglected than people think. On most of these given projects there's a big broadcast community of people, but the number of full time people that are really ruthlessly focused on the problem, it's usually between two to five people. So if you're interested in any of these problems. Yeah, you could fill out the form. Yeah, fill out the form. Yeah. Desperately need you.
- [148:22]
B
All right, so I guess a final question is one that came in from the audience. People were curious to know, maybe it could be multiple questions. From a just selfish point of view, what should people do in order to protect themselves or give themselves a greater chance of surviving a really bad event like this? I mean to really solve, we kind of need the societal response because ultimately if like the rest of society falls apart, then you're probably toast. But is there any useful stuff that you do personally to prepare?
- [148:46]
A
Yeah. So I would recommend getting a good elastomeric respirator. I like the three M1s. I also have the EM Pro. And then I would also get enough food so that you can socially distance in a pandemic without needing to go outside for, you know, say, three months, something like that.
- [149:03]
B
And any advice on what food?
- [149:05]
A
Yeah, dried food. You know, the shelf stable stuff that lasts for a long time. Seems like a pretty safe bet.
- [149:11]
B
And how would you. Do you stockpile water or. I guess it seems I actually don't.
- [149:17]
A
Yeah, I actually do surprisingly little prepping myself. So. Yeah, if. If the catastrophe is bad enough that the water gets shut off or the power gets shut off. Yeah, I would get on a bicycle and get out of town. Yeah.
- [149:31]
B
I mean, should people have a plan for getting out of major cities if that's where they live?
- [149:35]
A
Seems reasonable, but I wouldn't necessarily trust my prepping advice.
- [149:39]
B
You're trying to save everyone.
- [149:40]
A
Not individual people.
- [149:42]
B
I guess. If we were at the beginning of a biological catastrophe that was like, some very nasty bioweapon that was released, I guess. How do you think you'd be spending your time?
- [149:49]
A
Probably on the phone trying to get people to care. And I was doing this during the early days of COVID I mean, in February, I was calling up these government lobbyists, trying to get people to pay attention. And, Yeah. I mean, people were like, oh, you know, you don't want to be crying wolf. And I mean, it was just crazy. It was crazy to me. Like how. In February. January.
- [150:08]
B
Yeah.
- [150:09]
A
Like, how was going on? Yeah.
- [150:11]
B
Did people ever apologize? I guess I don't know whether you called them back to request an apology, but. No, no, no. What do you think you would be trying to push them to do? I guess it's just like, wake up and take the thing seriously. I mean, if the bioweapon was bad enough, maybe people would react very differently. Like you were saying, kind of COVID in some senses, was in this sweet spot where it wasn't quite bad enough that people really had to react.
- [150:31]
A
Yeah, I mean, I think there are tons of different things, and there are different pandemic playbooks and different things the government should be doing. So, yeah, crash program on medical countermeasures, making sure the hospitals are prepared. There's tons of stuff that's just like.
- [150:49]
B
Implementing all of the kind of plan.
- [150:50]
A
That we're talking about. There's a lot more stuff that would need to be done other than the four pillars, but I think the four pillars are the kind of, like, basic building blocks that other governments could build on or things like that.
- [151:02]
B
Well, hopefully the response is swifter next time.
- [151:04]
A
Yeah.
- [151:04]
B
My guest today has been Andrew Snyder Beatty. Thanks so much for coming on the 80,000 Hours podcast, Andrew.
- [151:09]
A
Thanks for having me. It's been fun.