Loading summary
Transcript102 lines
- [00:01]
Eduardo Albrecht
Welcome to the new books network.
- [00:14]
Eli Karetney
Welcome to International Horizons, a podcast of
- [00:17]
Podcast Host
the Ralph Bunche Institute for International Studies that brings scholarly and diplomatic expertise to bear on our understanding of a wide
- [00:24]
Eli Karetney
range of international issues. My name is Eli Karetney. I'm the Acting Director of the Ralph Bunche Institute at the Graduate center of
- [00:31]
Podcast Host
the City University of New York.
- [00:33]
Eli Karetney
This year with John Torpy on sabbatical, it has been my great honor to host this podcast. In the fall, John will be returning to the Institute as Director and as host of International Horizons. I'm grateful to John for the opportunity to host the podcast while he was on leave and for the chance to develop new initiatives at the Institute like the Project on UAP Studies and International Security. As the Director of the UAP Project, I'll be hosting periodic public events and podcasts with prominent academics, researchers and government officials doing work on the UAP subject. I was fortunate to attend a conference in Washington, D.C. this past week and met with representatives of the Congressional UAP Caucus as well as scientists like Harvard's Avi Loeb, the astrophysicist who is leading the White House's new UAP Scientific Advisory Council. I also met Lou Elizondo, the Pentagon whistleblower who revealed the existence of secret government UAP programs and has played a central role in disclosure efforts. Mr. Elizondo will join me on our UAP podcast later this summer to discuss the Administration's approach to UAP and Congressional efforts to pass the UAP Disclosure act, which has just been formally filed in both chambers of Congress. We will also have renowned international relations scholar Alexander Wendt on the podcast to discuss his forthcoming book on the national security implications of UAP and the possible social impact of discovering the existence of non human intelligence. But today we're here to discuss a different kind of non human intelligence AI and its impact on governments and policymaking. We'll talk about how AI is transforming the relationship between the state, private enterprises and the lives of individual citizens. Are we at the precipice of a new technocratic system of total control, or are we entering a new age of tech assisted human flourishing? Somehow these seemingly contradictory realities seem to be converging. Here with me today is Eduardo Albrecht. Eduardo is a political anthropologist with a history of work across government, nonprofit, academic and multilateral organizations. His research focuses on the uses of artificial intelligence in state and international organization decision making processes. His new book, Political An Introduction to AI in Government and Its Impact on Citizens, published by Oxford University Press in 2025, investigates uses of AI to produce public policy decisions in a range of geographic contexts and seeks to better theorize the changing role of citizens in the act of policy production. The book is the first to utilize an ethnographic framework to compare the impact of AI in government on changing citizen state relations in the west, east, and Southern Hemisphere. Eduardo was a Senior Fellow at the United Nations University's center for Policy Research, has taught at Puqiang National University in South Korea, John Cabot University in Rome, and is now a professor and director of the Program on International Relations and Diplomacy at Mercy University in New York City. Welcome, Eduardo. Thank you for joining us on International Horizons.
- [03:55]
Eduardo Albrecht
Thank you. Great to be here.
- [03:57]
Eli Karetney
We had Jacob Siegel on the podcast this year to discuss the emergence of what he calls the information state. He argues in his book by that name, the Information State, that a new form of political regime has emerged, which is the culmination of a path that began with the Enlightenment, developed according to the logic of what the French philosopher Jacques Ellul calls technique, and leads through cybernetics and public private surveillance programs to a new system of control which operates through codes, protocols, and algorithms that determine what we pay attention to, how we form opinions, and even how we relate to each other. You've indicated that you believe this is actually the right diagnostic frame, and yet for you it's not all doom and gloom. How so? Where does your analysis dovetail and where does it depart from Siegel's?
- [04:50]
Eduardo Albrecht
Well, first I wanted to say I enjoyed your conversation with Siegel and I enjoyed listening to that episode. The we agree that the information society is really upon us in a in a way that is changing power relations. In fact, in my own book, I start with an analysis of how the body is turned into information. That's really the first, the first layer and that has a long history. As Siegel points out. It goes all the way back to fingerprinting and other practices of the modern state. But with new technologies, especially with data technologies like the cloud and AI, it's not just the body that is digitized, but also other aspects of our human existence. For example, our thoughts are also digitized. This is where perhaps we start to, to to differ a little bit, though I'm sure there's a lot of convergence in our, our approaches. So when the state can start to collect so much information on our preferences, on our ideals, on our values, and digitize that as well, it starts to become a little bit more than just the momentized body and the inflammation society, but it's entering into something that is more intimate, something that is more human, and something that we thought was protected. Now it's above the thoughts and the emotions that are digitized and informatized. Also our relationships are informatized and digitalized. So that's even one step further. How we connect with others, how we feel about others, how we feel about political ideologies, how we feel about political events. And then above that, I argue that there's yet another more ethereal line layer, which is the environment around us is also digitized. So it's not just the body, it's not just our thoughts and emotions, it's not just our relations, but it's also the temperature of the air around us. It's also the information environment around us. There's also the political and economic factors that influence us. It's also the relations between organizations and states. So think of the the broader environment is also digitized in any way possible. Now again, this goes from climate to economics to political indicators, but there's so many other things out there that can be collected through things like earth observation satellites. And all that information becomes yet another layer that digitizes our existence. And these four layers come together to create what I call in the book a kind of digital version of citizens. And the state has a relationship with that digital version of citizens as opposed to the real human beings.
- [07:36]
Eli Karetney
That's great, Eduardo, thank you for that explanation.
- [07:40]
Podcast Host
Thinking about kind of digitized citizens, you talk about in your book how governments increasingly rely on AI driven, quote unquote political machines.
- [07:52]
Eli Karetney
I'd love you to explain more about your concept of political machines which, which
- [07:57]
Podcast Host
collect massive personal data and also automate decisions about citizens. So maybe you can explain more about what kinds of decisions are being made by AI, what the risks involved in
- [08:12]
Eli Karetney
this are, and also how this might
- [08:15]
Podcast Host
create opportunities for enhancing democratic representation and for even improving policy outcomes.
- [08:23]
Eduardo Albrecht
Yeah, the, the political machines are the way that the government interacts with all of that data. So they are in a sense a consequence of the information society, informatized body and so forth. The fact that all this data is available means that government's authority administrative units need to interact with it somehow. And they obviously cannot do it through human administrators because it's way too much information. So political machines are a response to the availability of so much data about us, about our relations, about our emotions, about our environment. Now one thing that I'd like to clarify is that these political machines are actually quite mundane. They, they're, they're in a sense a little boring. They're not these like, you know, headline grabbing, Terminator, Skynet type type phenomena. It's not. It's more like a. A very localized. Different units use it in different ways. It's very dispersed across many different types of governance. And think of it as like a. Like a bubbling up from the bottom of just solving small problems that government has using all of this data and therefore deploying these new political machines as opposed to, again, human bureaucrats or human administrators. And I argue in the book that this quite mundane administrative layer of all these political machines bubbling up from different use cases across governance is increasing like a rising tide. And in some ways it's been going on for decades. And just, just now we're starting, starting to realize it, but it's starting to take over more and more functions that were previously done by human bureaucrats. And as AI rises the cognitive ladder, more and more complex functions are being taken over to the point in which in many cases, there's almost no human in the loop. Or in other cases, the human is sort of on the sidelines co thinking with these political machines in making decisions that in fact impact populations, can impact populations on a wider scale, can impact individual citizens, but our lives are being changed. So that's where I make the argument in the book that we need to understand this anthropologically, because the relationship between citizens and political power has changed completely due to the rise of these political machines.
- [10:54]
Podcast Host
So it sounds like much of the alarmism about the rise of AI and
- [11:01]
Eli Karetney
the political impact
- [11:03]
Podcast Host
is overstated. And in fact, these kind of mundane political machines create a lot of opportunities,
- [11:13]
Eli Karetney
if I'm understanding you correctly.
- [11:16]
Podcast Host
And yet you make the argument that our existing political structures need to adapt, that new institutional forms may be needed to adapt to this new reality. And you emphasize as part of that the need for greater citizen involvement in calibrating AI ethics and oversight. What does that actually look like?
- [11:42]
Eli Karetney
I'd love to hear more about your proposal for a third house and the
- [11:46]
Podcast Host
new legal frameworks that could actually let
- [11:49]
Eli Karetney
citizens see, contest, and set parameters for algorithmic political decisions.
- [11:57]
Eduardo Albrecht
Yeah, well, first let me. Let me clarify that I'm quite neutral as to whether these political machines are positive or negative in their net impact. Certainly in many cases, there's issues around surveillance, there's issues around human judgment. There's a lot of issues that need to be thought out from an AI governance perspective, including from a regulatory and legal perspective. And we are, as a society, starting to ask these questions. My interest as a political anthropologist, again is where are we going to? Where are we headed as institutions evolve. Now, the institutions of governance have evolved, but the democratic institutions have Not. And the way that we can, we can picture this is if in fact a larger and larger portion of government is administered by machines, does it make sense for us to have representative government, democratic government as we have it now, where an individual will vote for a representative and a representative will go to a Capitol and that representative will deliberate, deliberate with other representatives and make decisions based on the authority provided by the citizen that then bind government in this. And that way, does that, does that formula still make sense given the rise of these political machines? That's, that's my, my fundamental question now. I'm, I'm afraid that there might be a mismatch with the way that representative democratic government works and the way that these political machines are working. So that's where in the conclusion to my book, I come to this idea of maybe we need a third house, an additional chamber of government of democratic governance. We have, we have a Senate, we have a House of Representative. And if we look at the evolution of these bodies, they're in response to technological and economic changes in society. So may, should we start considering perhaps a continual development of those types of democratic institutions into a new layer, a third layer, where in addition to democratic accountability, we also include some of these technologies that are being used by government, but now we use them also for democratic representation. So where can digital technologies, AI come in to leverage democratic voices and interact in a efficient way with this new AI government of the political machines? And I have some ideas around that,
- [14:36]
Podcast Host
so I'd love to hear more about these ideas. And you're not only studying these things
- [14:41]
Eli Karetney
from an academic point of view as
- [14:42]
Podcast Host
a political anthropologist, but as a builder, as a technologist, you're helping create the tools that can be used in the third house. Can you tell us about the company you're involved in and the kind of technology that, that you're building that can be used by citizens on the democratic side to enable greater human agency, human empowerment?
- [15:13]
Eduardo Albrecht
Sure. So in addition to being a scholar, looking at this from an academic point of view, as you mentioned, I'm also an entrepreneur and a builder. And we have this company called Dubler D U B L R. You can check it out. Dubler AI. A quick shout out to my co founders, partners and associates on that project. And what we're doing with Dubler AI is we're trying to build the first iteration of that infrastructure of the third house. Now what Dubler does is it creates synthetic samples of stakeholders and populations utilizing all this data that we discussed as present and now used by governments. But this time using it for democratic purposes. So we take all of this, this data and this information is out there. And obviously we are very careful to make sure that we use an ethical framework in how we collect data about populations. But there's a lot of this data that is out there and it's legal to use and it's ethical to use. So we, we collect this, this data and it can be everything from academic sources to demographic sources to polling surveys. There's so much information about people and populations out there. And of course, as an anthropologist, this interests me because, you know, how can we extend the reach of, of our anthropological understanding by utilizing these, these, these new technologies? So we collect all this information and, and we create synthetic samples of populations. Now we have a validation methodology to make sure that these synthetic samples are as representative and as accurate as possible. So it's never going to be a one to one because people are emergent and they're always going to have new ideas that we cannot represent with data, definitely not with synthetic samples. But there are techniques that we use to make sure that these synthetic populations are as close as possible and representing the preferences of the real populations. And then we make this information available to different institutions involved in governance. Again from community organizers to nonprofits, to humanitarian organizations, and also local and state governments. Now these synthetic populations can help inform the decisions that are made by those institutions that are already using all that data to make decisions towards the populations. Where the democratic accountability portion comes in is we have to make sure that there is an institutional conduit between the real populations and the synthetic populations that represent them. So we're also building out in parallel to the synthetic samples and their use for governments. We're also building out the connections between representative of communities and real populations and these synthetic samples so that it's not done in a vacuum, but it's done in consultation with representatives and of real communities. Now that portion is what interests me particularly because that is building out that potential third house because that is the infrastructure of a new form of democratic accountability where populations can connect with these synthetic samples that then connect with the governance. Of course it's not going to be easy. There's a, there's a lot of ethical and regulatory issues to think through. But the way I look at this is that this is a long term project. And in fact I, I am happy to see that more and more people are starting to be curious about this, this, this idea and more and more people are experimenting with it to make sure that it's done correctly.
- [18:59]
Eli Karetney
I got to say, Eduardo, this mind
- [19:01]
Podcast Host
blowing stuff, it's absolutely fascinating for a non technologist, maybe help me understand better how this actually works in terms of the synthetic populations and in terms of how the synthetic population interacts with its kind of real world double and then how the synthetic population interacts with the state. Maybe just kind of take me through that a little bit.
- [19:31]
Eduardo Albrecht
First of all,
- [19:34]
Podcast Host
how does the synthetic population form its views, its concept of representation? Is it the result of constituent elements within the synthetic population or is it already formed as a kind of group digital being? And then how does that group entity interact with its real world bubble and how does it interact with the state?
- [20:03]
Eduardo Albrecht
All right, those are tough questions, but I did my homework and I have some answers. Although I want to caveat that these answers are not complete because the conversation is evolving a little bit like with the conversation around disclosure and UAP phenomena. So we're not there, but we're going in a direction and I appreciate that we're going together. Now there's three huge ethical challenges to make this even plausible from a democratic perspective and they connect with the technical challenges. So I'll frame these three challenges as the first is, if you create a synthetic sample of a population that infers preferences based on all that data that is generated by us humans, does that mean that that synthetic population also carries authority? And the answer to that question is a hard no. We have to be very clear, clear that the authority comes from the person. Because that's one of the main dangers I see on the horizon. The governments just pick this up and say, well, we consulted your synthetic sample, you agreed, so here's your policy, Enjoy. And that would be a tyrannical outcome that we want to avoid at all costs. So we have to understand the difference between inferring preferences and imbuing authority. Because representing a population doesn't mean that you're also caring the, the authority to, to act. And that's a very important democratic theory, foundational principle that we have to hold on to. Where does this authority lie? Where does it come from? And I'll come to some solutions for that in just a second. But first I want to talk about the second problem. If in fact these synthetic samples can go out and represent us to government, what happens when they start talking to each other? Because we know that the technology is there for not only synthetic populations to represent real populations, but also to converse amongst each other. So these digital citizens that I talk about, they're in a sense alive, not in the same way we are. It's a Different form of intelligence, but they can in fact talk to each other. And there's different experiments that are being done about that that are quite interesting. But the ethical and technical conundrum for our purposes is what if these synthetic samples start talking to each other and they deliberate in this potential third house that I was talking about earlier, and that deliberation gets further and further away from the actual preferences and needs of the real population. So that's a second ethical and technical problem that we have to address. And again, I have some ideas there, but I'll come to them in a minute. The third challenge is what happens if we get lazy and we are happy to delegate political deliberation to our synthetic samples and get, in a sense, dumber and dumber on politics because we say, oh, I've got my synthetic sample out there deliberating for me, I don't need to do any thinking on my own. And that's a broader problem that AI is starting to surface across all different types of knowledge production. It's, it's also in academia, it's, it's elsewhere. The idea that, you know, we're going to delegate to these thinking machines a lot of our thinking, and so that makes us dumber. So these three, these three challenges have to be thought through properly and neither are easy. Which one you want me to start with, or do you have any questions in the meantime?
- [23:42]
Podcast Host
You mentioned some of the experiments are ongoing. I forget the name of one of the experiments where the, the individual AIs were in conversation with each other and, and, and there was no human involvement. Humans could only observe. And some pretty wild conversations were going on. Are you, are you familiar with the experiment I'm talking about?
- [24:00]
Eduardo Albrecht
Yeah, of course, the Mult book. And there we go.
- [24:02]
Podcast Host
Can you say just to our audience, maybe a word about that and what lessons you and your company are drawing
- [24:11]
Eli Karetney
from, from that experiment?
- [24:12]
Podcast Host
Yeah, yeah.
- [24:13]
Eduardo Albrecht
Well, you know, there's different, there's new ones as well. There's another one called Habermolt, based on Habermas theories. The, the idea, I guess, you know, whatever you find MALT in, in the name, you'll. It's experiments in this vein. But to backtrack a little bit, because to understand these experiments, you have to understand the broader field of AI deliberation and what's been going on in the past years with. There's a lot that, a lot of, of experimentation going out with AI and democracy. And it's, it's, it's a fantastic world out there. In fact, I'm Working on a second book that will review all these different initiatives that are out there and try to extract like some trend lines. But to backtrack to get to the AI deliberation issue, these experiments are using AI to gather information from real human respondents. So say you have like a lot of surveys that, I don't know, an organization asked a lot of people what they think about subject X and all these people say all these ideas and they're all over the place. So the first iterations of AI deliberation would then collect all these different responses and sort of comb through them with, with essentially LLMs and extract points of confluence, points of divergence. And this is a really a fascinating field and it's been used for negotiations for conflict resolution. There's a lot there. Now what we're seeing is they're taking it to the next step and they're starting to cut out the human survey respondents so that you don't actually ask people what they think, but you take all this, this wealth of data about people and you create an agent that represents that person. Then you have those, those agents deliberate and these are these different pockets of innovations that are happening. So the, the lessons that we're learning as a company is that that's a problematic approach in it's non trivial and we have to take it very seriously in terms of where the humans do eventually plug in. Because the danger, going back to challenge number one is that you, you lose the connection to the real human and the authority that the citizen gives and you provide governments an excuse to be able to make policy decisions based on these technologies that are actually further and further away from citizens real needs and preferences. So the non trivial portion of the question is how do we assure that citizens plug back in to these experimentations, these technologies, these AI deliberation tools in a way that actually respond to real citizens needs. And I have some ideas there, maybe
- [26:53]
Podcast Host
just say a word about trying to get my head around this, the establishment of some kind of a protective wall between the state and these mechanisms of AI deliberation. And on the other hand, to have kind of linkages between the process of AI deliberation and the kind of the real world source of human authority. So how do you maintain a kind of permeability between the AI deliberation and the people? On one hand, on the other hand, create a protective wall between the state and these processes of deliberation.
- [27:32]
Eduardo Albrecht
Exactly. Yeah. The word I use in the book is how do we tether real humans to these systems? And I think that tethering is important because we have to. That's the authority of the third house, that tethering, connecting it to real humans. One way to look at it in terms of a useful metaphor is the way that representative government works is that you, you choose a representative. So in New York, we just had a bunch of elections, right? Or primaries at least. And the you're voting for a person to represent you, then that person in a way goes off and has those deliberations with other representatives to make policy. So we already have a framework in which we delegate authority to a human representative. Now what's happening here and what I find interesting from a anthropological perspective in terms of human development and societal development is that like it or not, that human representative will eventually be out of a job. And it might not happen now or in the next decades, but eventually the technology is going to be there and government is going to be operating at such a fast pace with all these technologies that the human representative no longer makes sense. Too slow, too fallacious, too incorrect. And that human representative will be replaced by these, these new AI deliberation technologies. These, an amalgam of all these technologies, synthetic samples, these deliberations. Where it becomes uncomfortable is that as a society we have to now realize that we're no longer delegating authority and our political preferences to a human individual, but to an artificial intelligence.
- [29:24]
Podcast Host
So thanks for the pause there for a second. I want to kind of, while you're inside the answer, I have a personal example of something that I've experienced, maybe
- [29:37]
Eli Karetney
we all have, that feels like it has echoes of what you're talking about.
- [29:42]
Podcast Host
So maybe let me mention this example
- [29:44]
Eli Karetney
and then you kind of respond to
- [29:46]
Podcast Host
it in the context of your vision
- [29:50]
Eli Karetney
of how things are developing.
- [29:52]
Podcast Host
Social media, you know, there's a sense that my interaction with whatever social media
- [29:59]
Eli Karetney
platform I'm engaging with, let's say X,
- [30:03]
Podcast Host
Twitter, formerly Twitter, is my feed, my news feed. The content that is being presented to me is an outcome of choices and habits and preferences that I have fed into the algorithm. So the hope was because of the people that I'm reading, the subjects that I'm interested in, the people who I follow and who follow me, that ecosystem then produces content for me that is kind of tailor made to my, as I said, preferences, choices, the content that I'm interested in. But that's a kind of an ever evolving ecosystem. And the algorithm is determined not by me. The algorithm is actually controlled by some mysterious programmer hired by X to tweak the algorithm for their own interests, for a Profit driven interest. And I find that as we all have found that as their decisions about how to change the algorithm have evolved over time, the result is my daily news feed. The content that's being presented to me is evolving as a result. So how do we take that lesson, those experiences, and guard against a situation where our digital representatives, our synthetic samples, our doubles, are not being controlled by a mysterious programmer tweaking the algorithm for purposes of state interests and control or for some other purpose other than know, democratic representation?
- [32:00]
Eduardo Albrecht
Yeah, absolutely. And I love the example of social media because it's, it's the same kind of co thinking that's happening between you and, and algorithms to tweak your experience in ways sometimes that you don't even know. Let me say that there's a, A, a very important first approach that is regulatory and there's a lot going on there. There is, there's a lot of pushes, especially from Europe and elsewhere around AI transparency, AI governance, audits, red teaming. There's all sorts of interesting developments happening in the regulatory and legal space. And I'm completely for all of those. They're important conversations and they're changing the laws. So that's the first kind of line of defense that we have. But there's a longer term game here that I'm, I'm interested in again as a, as a social scientist, which is okay, let's say that those legal battles all happen and, and that's great. We still have not addressed the more fundamental problem of people becoming dumber, which was my, my third challenge. People are becoming dumber because they're not deliberating themselves, they're not doing a bunch of the stuff that we were doing earlier. So your, your social media feed is curated by an algorithm that was, is now doing something that you used to do when you would curate your own news sources. So in a way this, this co thinking that is occurring is making you dumber in certain aspects. Now, as an optimist, I think it's also making us smarter in other aspects. So on balance, I'm, I'm hoping it'll make, you know, make us. On balance, it'll be a good thing. But we have to be intentional and deliberate about how we understand this phenomena occurring, especially this co thinking phenomena when it comes to political issues. Because the social media thing, that's essentially the public conversation, but transplant that to the policymaking conversation, it becomes a little bit more impactful on you as a person because there's going to be laws that come out of that that mean that you could, you know, go to jail or not go to jail, that you could make money or not make money, that you can have certain freedoms and not have other freedoms. So when we transplant that same CO thinking phenomena from social media to policymaking, it becomes even more important and even more impactful and more necessary that we understand the difference between our thinking as humans and the machine's thinking and where there's overlap and how we manage that overlap. Now managing that overlap is going to be important specifically because those that don't manage it will find themselves on the receiving end of a lot of political machines decision making without having any participation. And this is what really worries me, that those that do not manage the CO thinking in a way that augments their own thinking might in fact find themselves further and further away from our participation resources in a way that might create a new class division.
- [35:04]
Podcast Host
That's really interesting.
- [35:05]
Eli Karetney
Wow.
- [35:07]
Podcast Host
This gives kind of new meaning to, and it may be a new risks associated with the idea of implied consent, which has always been a problem in democratic systems. You know, to what extent does silence or non participation, non involvement create a kind of, you know, imply consent on the part of decision makers? So then as you say, you have a kind of two tiered class system where those who are informed, involved, engaging in the process may be empowered and those who are, not to put a fine point on it, are getting dumber and losing significant political power as well. So I completely see the risks there. Look, so let's, I want to kind of shift in two directions. One set of questions about the proper kind of regulatory response, which you've already mentioned, you're completely open to. I'm kind of thinking through what that looks like. Regulations at what level? Local, state, regional, cultural, global. But maybe let's end there. Let's first address another topic which you brought up, and maybe let's kind of pause here for a moment, is human bodies and data. And maybe take us through some of
- [36:40]
Eli Karetney
the history and where we are now
- [36:43]
Podcast Host
in terms of how state practices, historical
- [36:47]
Eli Karetney
state practices like fingerprinting, eugenics, sterilization, you've discussed, all these things in your book were the result of efforts on the
- [36:57]
Podcast Host
part of the state to impose ever greater control. What do those historical practices and the
- [37:05]
Eli Karetney
kind of contemporary practices tell us about the risks of new technologies, new forms of biometrics, facial recognition?
- [37:17]
Podcast Host
You talk even about emotion detection.
- [37:19]
Eli Karetney
Not sure what that means. Maybe you could say a few words about that. What does that do in terms of
- [37:25]
Podcast Host
the states and their corporate partners potentially creating new systems of control?
- [37:33]
Eduardo Albrecht
Well, that's a lot. And I'll take the regulations question first and then I'll touch on the longer term, the trends. So in terms of regulations, in addition to the many important efforts that are happening globally around AI governance, there's two fundamental rights that are going to have to emerge as kind of the pillars of any real AI governance. The first is access to data. Right now it's the wild west of data. In the last few decades, there's so much data produced about us that we hardly know. Most, most citizens have no idea how much data is produced about them. So the right to know in a, in a simplified way kind of think about it as a kind of freedom of information act for the individual and the data that is collected by government and government partners about, about them. The second, which is even more complex is the freedom of thought. Now the problem with these, with these AI machines in government is that they're often thinking for you or they're, especially when it comes to, and you ask the questions about sentiment detection. They extrapolate an emotion based on say, your facial expression or you know, the things you say on social media that is ascribing to an individual a thought. And then based on that thought that has been ascribed, certain decisions are made that might impact that person's livelihood. That's problematic. And I think that the freedom of thought to be able to say that's not what I'm thinking or you, or there's more nuance in what I'm thinking or I don't fit into this or that bucket of thinking is, is important and has to be the second pillar alongside access to data. Now to answer your second question about the broader historical trends going all the way back to the first question about the informatized body and the information society and how this is a real societal transformation that is occurring over many decades, if not centuries. The danger, I believe, is that governments without this type of third house and without the type of systems that we're trying to create at Dubler will eventually have a relationship only with the data. And the, the real human bodies will be kind of a, like an, an addendum to the, to the, to the real formula of power. You know, think of your, your, your human body, your, your, your, your flesh as sort of being on the outskirts while these political machines are having these conversations with all this data about populations and making decisions that then impact your flesh that is out there in the outskirts without, without you really as a human physical presence having any relationship with that new structure of power. And this is A danger that I see us going towards because as these governments interact with our digital doubles, it's incredibly efficient. You can make so many decisions so quickly. And that efficiency, as we know, is going to, it's going to invite more and more of that type of usage. So governments will have the illusion of efficiency, while in reality the fundamentals of accountability and of transparency and of democratic participation will continue to erode. Now, this phenomenon has occurred throughout history, and one of my favorite authors on this is James C. Scott, who wrote a great book called Seeing Like a State, A state wants efficiency. It's, it's a giant machine that just wants to be efficient. So as these, these huge bureaucratic organizations increase their efficiency and, you know, they pat themselves on the back for, for being more rational, for being more efficient, for doing, you know, better warfare, better management of populations, the actual human component becomes sidelined to the point in which this, this, this illusion arrives at a tipping point and crashes. And in fact, we've seen that with, for example, totalitarian or a lot of socialist experiments throughout history or that, that dream, that illusion that the state has of efficiency without the human component eventually becomes empty, vacuous, and collapses on itself to, you know, huge catastrophic consequences. So having that, that, that democratic component reinserted into the structure of power in a way will make sure that that catastrophe doesn't happen. Because you, you're taking those human bodies from the outskirts where they're being pushed and you're bringing them back into the formula of, of power, the structure of power, in a way that assures that power doesn't eventually derail and catastrophes ensue.
- [42:42]
Eli Karetney
A question I wasn't preparing to ask, but, but I, but I have to now ask. And again, this feels very personal. Despite the abstraction, the really kind of,
- [42:51]
Podcast Host
and the, the, the sophistication of the
- [42:54]
Eli Karetney
ideas and the models that you're presenting,
- [42:57]
Podcast Host
this, this hits home in a very personal way.
- [42:59]
Eli Karetney
And I'll explain how.
- [43:01]
Podcast Host
I mean, here you describe a situation where in the best case, there's a
- [43:11]
Eli Karetney
tether, as you said, between our real world selves, our physical bodies, our actual person and our digital representatives. And that tether ensures kind of democratic reliability or kind of true representation.
- [43:29]
Podcast Host
And then, and that allows, then we can kind of transfer authority presumably to our digital representatives who can, who can
- [43:38]
Eli Karetney
kind of defend us and make sure
- [43:41]
Podcast Host
the state then is aware of our
- [43:43]
Eli Karetney
preferences, choices, policy, policy decisions. And that could in principle kind of improve the democratic quality of policymaking and governance.
- [43:56]
Podcast Host
But here's something I'm thinking about because in that model, then there's a tether. But we could also be. Our physical bodies can be left alone. I can be playing basketball with my kid or fishing with my friends and going for hikes, and my digital twins are kind of doing my political work for me as long as that tether
- [44:21]
Eli Karetney
has been kind of established.
- [44:23]
Podcast Host
But what I'm thinking is this, at this stage in the development of these
- [44:29]
Eli Karetney
technologies, I have something, you know, I've heard Elon Musk say, we're already cyborgs. You know, the phone is already an extension of my physical body. And Meta's got their glasses, they're all working on glasses. And of course, Mr. Musk is working on implants as well, brain implants.
- [44:51]
Podcast Host
So that creates a situation where my
- [44:53]
Eli Karetney
physical body is not left alone from its digital companions.
- [44:59]
Podcast Host
And I feel that day to day,
- [45:01]
Eli Karetney
and I know, and I look all around and my kids feel it and my friends feel it.
- [45:05]
Podcast Host
I'm reaching for my phone all the
- [45:07]
Eli Karetney
time against my better judgment. I can't help it.
- [45:11]
Podcast Host
My newsfeed is calling me. My notifications are buzzing in my pocket.
- [45:16]
Eli Karetney
And I pick up the phone and my attention is trapped. And I.
- [45:20]
Podcast Host
And I'm ashamed to admit this, but I want to be hiking, I want to be outside, I want to have feet in the grass. And instead I'm stuck inside the screen
- [45:31]
Eli Karetney
for moments and I have to force
- [45:34]
Podcast Host
myself to put it down in that situation. I mean, in that situation, the tether feels like it's happening in another way. The tether is coming from other sources of power that are tethering into my habits, my attention, my opinion formation, my thoughts. I mean, this gets to pretty creepy stuff. It feels like we're just a hop, skip and a jump away from, from a mind control situation. My mind, my thoughts and opinions are being formed from outside of my body rather than being, rather than my will tethering into my digital companion. I feel like I'm. I'm being tethered into, from outside of myself. So, yeah, there's a question in there somewhere. How do we, how do we regain. How can I personally regain agency, regain my attention and sever the link? You know, choose when I'm tethering in and also be able to kind of sever that link and, you know, be have some peace from the screen?
- [46:47]
Eduardo Albrecht
Well, let me just say I'm so happy you went there because I think we get to the crux of the issue. Is the tether just another chain? Is it, is it in fact leading to human flourishing or taking away from human Flourishing. I think that's the foundational question that we always have to ask. And I like how you described us kind of being sucked into the screens. I've got kids too, and 90% of parenting is basically making sure that they have activities outside of the screens. So one worrisome trend that I identified when researching the book is that data collection technologies are actually getting closer and closer to our physical body and are on the cusp of, if not already have crossed the line in terms of penetrating the body. So that is quite literally no longer a tether, but a chain, right? If, if now you can't pull it out of your brain, you can't pull it out of your, you know, under your skin. And this is, this is, this is all empirical and there's plenty of, of evidence out there how that, that data cloud that is around us is getting closer and closer to our, to our bodies and is now penetrating it. Think of the way that wearables are starting to collect information about this. Think of the way that, you know, you mentioned Elon Musk. He's got a company called Neuralink. We all know what, what his intentions are there. So there, there's going to be this moment in which it actually penetrates the body and it's going to be even harder to, to, to get away from, than the screen because ironically, the screen, you can't put it in a box. And, but once it's, once it's under the screen, under the skin and connected to our neurological activity, and it's going to be harder to turn it on and off. And also the, the direction is going to be inverted. So it's no longer going to be information that we create and is uploaded, but it's going to be information that's then downloaded to us. And that becomes a huge risk from the perspective of our judgment, autonomy and rights. Now, where can we see a silver lining in this development is that if we find ways that we tether, but with the control being in our hands, and that's what a two rights that I talked about earlier are fundamental. So first of all, the tether has to be conscious. So we need to know in a very simplified way, and I'm talking very simplified, like I don't want a giant Excel sheet with all the data you have about me, but think of like, just like three simple visualizations that tell me what, what kind of, you know, information and how much you have about, oh, all of a sudden now it's conscious, right? It's no longer this, this surreptitious system happening beyond my consciousness. Now I, I have some, some control because I know I have knowledge. And the second is the freedom of thought. Because once those, those technologies penetrate the skin and they start to go into our brains, it's going to be very difficult to be able to, to differentiate what is actually our thought and what is the representation of our thought or what is a thought that is being implanted into us. So those two rights hopefully represent the silver lining to push against this despotic trend that we see happening, unfortunately.
- [50:07]
Podcast Host
Wow.
- [50:07]
Eli Karetney
Absolutely fascinating. Really. I so appreciate that thoughtful response.
- [50:14]
Podcast Host
As a last question, I promised I keep it under an hour and we're approaching the hour mark. So as a last question, I'm thinking a lot about, and I heard you mention it also in a talk you
- [50:27]
Eli Karetney
gave at the Carnegie Council on Ethics,
- [50:30]
Podcast Host
this idea of kind of freedom of thought. And it really, it stuck with me when you said it then in your book and as you say it now. And I'm thinking about what protective mechanisms, you know, we have in place to guard our freedom of thought. And it isn't such an easy thing to even know when thoughts are coming into our minds from the outside. And sometimes we, we think they're our own thoughts and they might not actually be. You know, I mean, this happens at a simple level of advertising promotions. You know, I want to buy, my kids want to buy something because they just saw a billboard, right? This can be very kind of low tech stuff. But nowadays we're being bombarded by information and images and fashions and preferences that condition our thinking, help form our opinions. So it's hard to know when our thoughts are actually our thoughts. And you've discussed this before, but I guess as a final question, at what level of protections or regulations should we be thinking about laws and regulations that can guard us from the dangers that we've been talking about? Are these happening at the local level, state level? Is there a national element to this? Are nation states the ones responsible? Ultimately, are we still in a world of nation states here or must this go beyond the level of the nation? Are we talking about global regulatory framework?
- [52:08]
Eduardo Albrecht
Okay, before we get to the global versus local, I wanted to talk a little bit about the, the importance of the freedom of thought in terms of the way that freedom of speech evolved. Now in, in a few days it's going to be Fourth of July and we're going to celebrate 250 years of, of, of this country. The, the, one of the foundations of the political system that we enjoy here today in the United States is freedom of speech the reason that freedom of speech was important back then, it is now because without freedom of speech you can't have meaningful deliberation. And without meaningful deliberation, you can't have policy that responds to population's real needs. So it's a very simple equation. Freedom of speech allows for deliberation, allows for policy that is accountable to populations, and that is a safeguard against despotic government. Very simple equation. I think we can all be on board with that, that now we're entering this new phase in the development of the state where the state is now capable of things that it wasn't capable of before. So it can be despotic in ways that it couldn't be before through these surveillance technologies and political machines. So that that freedom of speech, while important, might not be sufficient as a safeguard against that despotic tendency that states have. So where freedom of thought comes in as an evolution of the freedom of speech principle, as a counterweight to the state's authority is if in fact a lot of the political machine's decision making is made based on what it thinks that you're thinking, then a lot of that, that thinking is happening without you ever speaking. The so called deliberation is happening at a subverbal level. It's happening in the representation of thoughts that populations have. So the AI deliberate, the, the deliberation that is occurring is occurring in the AI world. It's no longer occurring in the physical, verbal world. Deliberation has, has shifted its, its locus to the machines, and it's based on what it thinks that you're thinking. Therefore, it's no longer sufficient to have freedom of speech in order to influence that, that AI deliberation, you have to have freedom of thought to say, wait a second, that's not what I'm thinking. You cannot base policy based on what, what you think I'm thinking. So that's a logic where I think freedom of thought is important. Now to answer your question about global versus local governance, that's a very interesting one. And it touches upon the nature of these technologies. They can in fact govern globally. And in the book, I look at it at different levels of governance. So I look at local government, I look at state government, national government, regional government like European Union, and then I look at multilateral global governments like the United Nations. So all of these different layers, they used to be separated by time and space. But what we see is that these technologies actually allow for a much quicker convergence into the different layers of governance. So the dichotomy between global and local is starting to fray because these technologies have the capacity to govern at so many different levels simultaneously. Well, excellent, Eduardo.
- [55:36]
Podcast Host
Thank you so much.
- [55:38]
Eli Karetney
Thank you for your time, your thoughtful responses. Good luck to you with the book, with your research, with your various projects, with your company.
- [55:50]
Podcast Host
You're operating in an important space in a fascinating space, and you're doing really great work.
- [55:56]
Eli Karetney
So I really thank you for your
- [55:57]
Eduardo Albrecht
time and I thank you for this conversation. This was, this is definitely a good one. Appreciate it.