Loading summary
A
Your film is now ready to be shown.
B
Good morning, I'm Justin Hendricks, editor of Tech Policy Press. We publish news, analysis and perspectives on issues at the intersection of tech and democracy. Is the future something to be calculated and controlled, or something we shape together through democratic struggles? How should we read the convergence of Silicon Valley's dark enlightenment thinkers with a resurgent authoritarian right? And is Europe truly reckoning with what has shifted the United States? What's driving the continent's anti regulatory mood? And what counts as evidence sufficient to legislate a fast moving technology? I address these questions and more with today's guest, one of the authors of a new book that argues that three decades of underregulation have produced concentrations of wealth and power we now must confront. And that the survival of democracy in the digital age would depend on citizens, civil society and a new generation willing to treat their work as carrying responsibility not just for safety, but for fundamental rights and self government.
A
I'm Paul Niemet. I'm a visiting professor at the College of Europe in Bruges, where I teach a postgrad seminar on AI Law and Data Protection and Privacy Law. That's my retirement activity. I have been for 30 years an official of the European Commission and the Director for Fundamental Rights and Citizens Rights. And recently I've published a book together with my old study friend Matthias Pfeffer, who is a journalist and philosopher, and Jurgen Pfeffer, not related to Matthias Pfeffer. Jurgen is a professor for informatics at the Technical University of Munich. He previously was at Carnegie Mellon and the title of the book is the Open Future and Its Enemies How We Can Protect Free Society from AI Dictatorship.
C
I'm excited to talk to you about this book today, but before we do, I want to hear just a little more about that career, a little more about what brought you to the point you're at and to being able to put together a work like this. Just give us the brief history of your career, where you got started, how you ended up at the Commission.
A
I studied law at the University of Hamburg. I actually also was twice in the United States, so I know America quite well. Once was 16 as an exchange student for half a year in Quincy, Illinois, on the Mississippi, close to Hannibal and St. Louis. I. I love that time there. That was 1978. And then again as a Fulbright scholar, I did a Master of Comparative Law at George Washington University in Washington. And so I'm a career official of the European Commission. And since I'm also a political person, at a certain point in my career, I let's say made it above the glass ceiling. Namely I became a political civil servant and appointee directly by. By the commissioners and the college. I was a member of a cabinet of a Danish Commissioner for development aid. I was for many years a trial lawyer for the European Commission in the Court of Justice of the European Union. And in the last 17 years I held positions in the area of justice and consumer protection and in the justice field. For example, I was the lead director for getting on the books the general Data Protection Regulation, famous gdpr but also a number of codes of conduct which predate the Digital Service act. For example the code of conduct on hate speech and incitement to violence on Internet which we concluded with the big Internet companies. And I was also final thing the Snowden investigator of the European Commission together with, with others. But, but I was a co lead of the delegation which investigated the impact of the Snowden revelations of the on the European Union and its citizens, namely mass surveillance and so on. So let's say in the last 15 years all my work was in the triangle of democracy, law and technology.
C
Well, that certainly gives my listeners reason to sit up, pay attention as we talk through this book which is called the Open Future and Its Enemies How We Can Protect Free Society from AI Dictatorship. I guess I'll just start because that's a. Is a sharp subtitle there. Folks might think just on the face of it, this is a kind of doomer book. Just looking at the COVID and looking at the way that things are phrased. Is this a doomer book? Is this a book, you know, that is negative in its, in its general orientation? How do you kind of think about the mood of this book, as it were?
A
Well, I think every new idea is born out of a certain criticism and discontent with the present. That's, you know, every, every technological improvement, every technological innovation is basically coming from an idea of, you know, what could be done better. And that's what this book is also about. It's absolutely not a Duma book. On the contrary, it says the future is open. We are shaping it all together. Meaning that it makes sense to engage in democratic processes in technological innovation, but also in societal debates, in civil society, in politics, because we together shape the future. It is not predetermined, it is not inevitable. It is certainly not calculatable in mathematical terms or predictable totally by AI. So the message is guys, get engaged. And it is a particular call not only, but in particular to the technical intelligentsia, those people who work on technology in the many different ways people work with and on technology to re engage with democratic process in a constructive way. Because we at least believe that innovation has not only a technological side. Also, democracy is a highly innovative way of governing ourselves. And this way of governing ourselves we want to maintain for the future. We want to see artificial intelligence and democracy thriving in the future.
C
I want to get into this notion of the future a little more. I mean, you spend a bit of time talking about the idea of the future is fundamentally unpredictable. You bring in Heisenberg, Turing, Godel, others. You say early on that today the future is contested by futuristic technology. The ability to calculate and thus control the future is central to the fantasies undergirding AI development. I underlined that in the introduction of the book because that sort of feels like something that, that hits. There's a kind of fundamental anxiety going on among all of the creators of artificial intelligence. They're at once sort of telling us that this technology is going to make the future unpredictable and unknowable, and that this technology is the way that we can, you know, shape and control and harness that future. So that feels very true to me.
A
Well, I would say our argument is that those who claim that the future will be alone shaped and determined by technological innovation, in particular by AI and general AI, they have not understood two basic facts of life. One in physics, in quantum physics in particular, there is the famous mistake of Einstein, who thought that everything in the world is rule based. And from quantum physics we have learned that that's not the case. We cannot predict how the quantum will behave in the next moment. So in physics we have this element of unpredictability and uncertainty. And in the human mind we have the ability to imagine, to imagine things which don't exist. And this imagination is very open. We cannot predict what humans will imagine tomorrow. And these two openings, the open future, are a call on us to shape the future together, not to leave the shaping only to, as Lawrence Lessig once said, the 200 engineers who determine how we live in the future maybe, but to shape it in democratic process. Meaning that we all participate and yes, that we make rules for technology where this is necessary. You know, at least in Europe, we have a very good history in regulating technology. It starts after the Second World War, actually, together with America on the use of atomic power. Atomic power is often quoted as something which is, you know, like artificial intelligence. I remember, and I think we have quoted here Elon Musk and Bill Gates, who said AI is like atomic power. Well, the civilian use of atomic power after the Second World War was accompanied on the global level with the creation of a legal framework and an institution, namely the International Atomic Energy Agency, which had very wide inspection rights in countries, inside countries, so in international organizations which can go into countries and check what's happening on atomic energy. And it was accompanied. And actually the putting into business into operation of civilian atomic energy installations was predated by adoptions of laws which regulated the risk of this technology. And I would say that was a good experience, even though atomic energy later had catastrophic impact. And some countries, like the one I know best, namely Germany, have said goodbye to atomic energy for that reason. But what we want to show in this book is that the drive against laws which govern AI, which come from the Trump administration, who even want to forbid American states to make law, which come from demagogues like Peter Thiel who say that those who want laws and those who want to strengthen the United nations are the Antichrist, which, of course, for me, living in Rome, near the Vatican, and I'm not, I'm not a Catholic, but of course, it's a formulation which makes me. Listen up. We want to say that they are on the wrong way. This is really very ideological thinking, very egoistic thinking. These guys, they only think about enriching themselves. Our analysis is that their worldview is that very few rule, namely a few autocrats in politics, a few oligarchs in technology, together run the world. And our vision is, no, the open future is a future which has to be shaped in democracy by all of us. And this requires that we all engage in democracy. Of course, if we all lean back and do nothing, well, you know, then others take over. And that's not a good thing.
C
You spend a bit of time on this ideology, the dark enlightenment characters that we've talked about on this podcast in different ways. Curtis Yarvin, Peter Thiel, Mark Andreessen, the lineage to people like J.D. vance, et cetera, you know, in your view, maybe from the perspective of Europe, which feels like the dog, which feels like the tail here, you know, is it the far right kind of wagging tech, or is tech somehow just a vehicle for the far right?
A
Well, Kara Swisher, I liked her work and her book, the Burn book. You know, she starts the book by saying, well, it's capitalism, after all. And she describes how, you know, tech started in a very idealistic way. Also, she was a very idealistic reporter on tech, accompanying the developments. And then already in the first Trump administration, there was a third lineup and the second Trump administration, even more a basic coming together of Trumpism also even Trump as a person and the leaders of big tech. And so I think our message is it is naive to only talk about the technology. We always have to talk about the technology, plus the ideology of those who propagate, who develop and who make money with this technology. This has to be seen together because in the end, AI is a power technology, and it is used to gain power over people and power over political systems. I think that's one of our main messages. Let's not continue this naive debate about the great potentialities and so on. Let's look at the realism of who drives this forward and what their agendas are, and let's read their books and articles and understand the intentions and let's take that serious. And only if we look in a holistic way at this ideology together, of course, with economic realities and technological realities, we will be able to grasp what politically is necessary to give a framework to this technology, but also to the business models which go with this technology. A framework which makes this technology a technology which supports democracy, which supports fundamental rights, which supports people in their freedoms, rather than doing the opposite, namely taking away freedom by manipulating and taking away democracy by basically supporting autocrats and authoritarian forms of thinking and authoritarian forms of government.
C
You say in the book, the pace of authoritarian upheaval in the US Is so breathtaking that it is necessary to look at the fundamentals of this transformation. Sometimes when I talk to folks in Europe, I get the sense that there's a little bit of an avoidance of this fact or a sort of desire not to focus too much on it. Maybe it's sort of unnecessary to focus on, maybe the truth of it. So painful. But do you think it has sunk in, in Europe, really the extent of what's happened here in the United States? Do you think that you're, you're, for instance, your former colleagues in the commission or others in European politics have really accepted what has clearly happened here in the United States?
A
Well, I think for all of us in Europe who are political people, it's, you know, we all grow up, grew up with an America leading on democracy in the world. I mean, you know, after all, it was America, together with Britain, who brought democracy back to Germany after fascism. You know, I was a Fulbright Scholar, so, you know, with. With American scholarship, going to America to learn a little bit more about the democratic American way of life, about the rule of law, studying law at George Washington. So for all of us who grew up in this transatlantic atmosphere, looking up to the history of democracy in the United States in particular and the great achievements of democracy in America. This development, of course, is a shock. It's a complete loss of orientation and the good culture of, of working together. And I think it is sinking in more and more. But you know, my feeling is the worst is already behind us. We have now seen that the interventions of, the very aggressive interventions of Trump. And by the way, also already in the first mandate, you know, Steve Bannon, he came to Italy and brought money with him and you know, appeared here with, with the right wing Prime Minister Meloni also Elon Musk did this. They intervened in the election. J.D. vance went to Hungary. All this was not very successful. We now have two governments in Europe which have come back to democratic mainstream. In Poland and in Hungary, namely and in the United States. We see that Trump has very, very bad opinion polls now below 30%. And I must say, the recent paper of open AI on the industrial policy for the intelligence age, which mentions the word democracy quite often in my view is a signal that a certain repositioning is taking place in the tech scene, namely they are preparing already for the life after Trump. I think the predictions in the tech scene are that this was the bad opinion polls with the war and so on. You know, this is not good for Americans. They don't like this. So this regime will not continue. And of course, if there is a return to normal rules of democratic government, you will see something in America which is a little bit alike to the awakening and the adjustment of the whole political system which Germany, for example, saw after the Nazi time. You know, there was a lot of thinking in, you know, changing how politics is run, for example, how the media are organized and so on. And my bet is that this will happen in America in the post Trump time and people are start to think about it. I think it's good that they are starting to think about it. And I would say it's worth the investment because certainly neither, hopefully the American majority of Americans, but certainly neither others in the rest of the world and certainly many here in Europe. We wouldn't want to see America sinking again so deep as it has been sinking with Trump.
C
I kind of want to come back to that point of conversation at some point a little later. In particular, whether you feel the impetus towards, you know, European tech sovereignty and some of the other movements towards protection from American tech or the dominance of American tech will persist after the Trump administration. If in fact you're right, that the kind of authoritarian slide that we're witnessing here is interrupted somehow. But I want to maybe just spend A moment or two on your general critique of artificial intelligence. I mean, you say that AI itself is a extremely problematic technology. Just fundamentally, how do you arrive at that conclusion? What are the basics for why you state that?
A
Well, I think what we are saying is that claiming that this technology has the human ability of intelligence and can replace humans in the form of general artificial intelligence in everything, namely also in good judgment. This is a claim which goes too far because after all, this technology is a technology of mathematics, it's a technology of algorithms. It's not a technology which is in any way able to even emulate what makes humans and what makes the good judgment of humans, the broader political understanding, the willingness and ability to change positions and to compromise all our emotions, all our bodily functions, which are of course linked to how we are thinking, both our motivations, you know, where do they come from? They come from dreams about the better, they come from discontent. All this AI doesn't have. So, you know, in the same way that mathematics doesn't explain which question the professor of mathematics is asking himself or herself for the next research project. These machines can't give the orientation in the world which humans need beyond rewriting what already exists. So I think in part our book recalls, or is an effort to recall that our abilities go far beyond just assembling all readings and all facts of this world and then writing them up nicely. And we believe it is important, especially for humans as participants in, as citizens, participants in democratic processes, that we maintain the self confidence that it is legitimate, that we decide, we decide individually, we decide together. We are not failing creatures, we are not failing machines of information treatment, you know, compared to AI. But we have both legitimacy and abilities which go beyond that of AI and everything which anthropomorphizes AI or claims that AI will be able to take over from humans in all respects. We believe that's, that's misguided. It will be a tool and we must keep it that way. And it is good to keep it this way. Any claims which say, for example, you know, Eric Schmidt. Yeah, we have to get used to the fact that AI is unexplainable and we have to live with it because if we ask for explanations, then the performance of AI will go down and people will die. I think that's what, what he said about AI in health. Well, our answer there is asking for reasons and asking for explanations was really enlightenment. It was ending the, the, the. The government of, by the church and by religion. It was a calling by Kant and other philosophers on humans to think themselves and to leave this state of just accepting things. And we don't want to see humans basically being dominated by a new religion of AI where we have have to accept all the results AI produces. I personally, I'm also very critical of this whole ideology of trust in AI. I would say rather the opposite. I would say in the same way that we have been teaching and we have been taught and we have been teaching our children to be critical towards power, to be critical towards media, we should teach them to be critical towards AI because AI is power. And in part AI is taking over functions which media before had, namely information preparation and distribution. And so we have to be extremely critical and we have to learn new methods of understanding how to make sense of what AI gives us to make judgments whether it's useful or not. Is it really true? Is it balanced? Is it one sided politically, one sided in other respects, one sided, Is it maybe discriminating between people of different color or different sex? You know, all of these questions must remain very high on the agenda. So I would say it is healthy to say people need to be critical towards power. They need to be critical towards, towards technologies. That's what we have always said in democratic states and we should continue saying this also towards AI.
C
One of the other arguments this book is that the concentration of power we see in the tech industry is a result of what you call the systematic subsidy of underregulation over the last 30 years. You say that overregulation or even deregulation I suppose is on everyone's lips these days. Used to disparage law and justice, which are pillars of democracy. And I'm talking to you on a day where we've, you know, just seen the Commission put off its, the AI regulation. A lot of headlines here suggesting, well that's got to be in part because of pressure from the United States. We've got the omnibus proposal and conversation kind of going on as well, you know, as, as a person who of course helped build the gdpr. He's been studying the insides and I'm sure moving commas and phrases in a lot of this documents over the last few years. What do you make of this moment, particularly in Europe, with regard to the orientation to regulation?
A
Well, you know, people get the policy they voted for. The new European Parliament is much more right wing than the previous parliaments were in the past. When, when we adopted gdpr, for example, the leadership in the European Commission was with Vivian Reading, Vice President. She was from Luxembourg. She was a conservative politician, you know, of the same type of Mrs. Angela Merkel, the German Chancellor, same party, namely the European People's Party. In Germany, the cdu, this would be Republicans. In America, let's say the Pre MAGA and Trump Republicans. And she pushed forward the gdpr. So the idea that the GDPR is a creation of, you know, Green or left wing or something is a complete misunderstanding. And it was pushed forward by a conservative vice president and it was adopted with a very big majority, including the Conservatives, the EPP in the European Parliament, together with the Liberals, the Social Democrats and the Greens. Now the Parliament is much more right wing, I would say. You know, there are too many in European politics which have been impressed by Elon Musk's, you know, loud talk and the chainsaw in American administrations. The proposals which have been made are largely not based on empirics. They are not scientifically underpinned. But as also some high officials of the Commission have said, they are catering to a feeling, namely a feeling of overregulation. And of course a feeling which is supported by the ideology which comes from MAGA and from American companies and which has always come from American companies. You know, that's nothing new. And of course also business people in Europe of a certain type of. And have this on their lips constantly. In fact, I studied law in Hamburg, which is a business town, and in my first semester I learned whining is the greeting of the businessman. And the whining which was meant at the time was, oh, you know, the taxes are too high and the labor laws are too protective. So, you know, this view, this neoliberal view of how the world should be has always been around, but it has got some, let's say, push in the last European elections. It is pushed by America. And that has an influence on what the Commission is trying to do now in Europe. But will they succeed with this? Will this bring growth? Will they actually get all this through the European Parliament? We will have to see the function of the EPP as the centralist. The right centralist party is key here because they must choose whether they stick with the Democratic parties, namely the Social Democrats, the Greens and the Liberals, and make the compromises which are necessary to have such a majority. Or do they fall prey to the temptation to go with the extreme right wing? And this will be a test for this party and it will be also a test for, for the Commission whether it goes along with this. My hope is that the EPP has the responsibility and understands its responsibility for democracy and doesn't fall prey to this temptation. And so, although These regulations are, they seem to be very technical, but that is the crux of the digital age, that much of the regulation which sounds very technical is actually very, very important for fundamental rights of people, for the freedom of people and for how democracy functions. And I can just hope that, you know, in this very ideological drive of, in catering to feelings of businessmen, European politics doesn't go too far to the right and is not too destructive of, I would say, the good things which we have built in the past and which created legal certainty and which are also part of the European identity.
C
So I want to just drill in on one small thing which you address in this same part of the book that focuses on these questions of regulation and law and power. You talk about the idea of, you know, evidence based legislation. You write that the, the path to underregulation has already been set with this demand for evidence based legislation. You know, I think about European digital regulation, I do think about this kind of notion that seems built into certain frameworks like the Digital Services act, that, you know, we're going to get the evidence, we're going to get the science, we're going to see if this rule is working or if it isn't, we're going to be able to empirically judge the impact of these policies. Why is that problematic in your view?
A
Well, it depends on what you consider evidence. I had recently a very good discussion about this with a number of very important scientists. And there are two ways one can read evidence. One is to say best available knowledge, which includes theory, which includes scientific theory. The other reading is more narrow. Evidence means empirical evidence, things which can already observe. My argument is only if we make in a fast moving technological area the rulemaking dependent on empirical evidence. The law regularly comes to late and we have to wait for the catastrophe. So my argument is we should not have a narrow reading of evidence based in the sense that must be empirical evidence. But we must allow ourselves to employ the human ability which AI doesn't have, namely to think ahead and to imagine what a technology could lead to, what the potentials of the technology could mean for democracy, for fundamental rights and freedoms of people. And this thinking must be enough of if there's a good reasoning to justify legislation, even if there is no empirical evidence yet that these things are happening. But if they are potential, if the potential is serious and can be described derived with scientific arguments, I think that should be enough to make legislation which looks forward and which avoids the great catastrophe, rather than having to wait first for the catastrophe and then regulate after and the Catastrophe which we are worried about is that democracy will be done away with because at this stage of technology and of this stage of total surveillance potentiality, once democracy is done away with, it's very difficult to come back. And I just hope that America still will make it. As I said, you know, we were all very, very happy to see Poland and Hungary come back. Let's hope America will be able also to come back and for example, you know, make Congress work again and make it possible again that Congress adopts laws on these very important and high risk technologies like AI. It would be much better for the world, also for Europe, but also for the transatlantic relationship if America would take charge of regulating the huge economic and technological power it has rather than leaving it to, to Europeans.
C
I wonder if we are capable of that if we don't now sort of see these technology firms, their size, their concentration as vehicles for empire.
A
Well, I must say I'm in many contexts still very connected to America, both with my mind and my heart. And I remember, for example, you know, Al Gore saying, you know, we don't want the stalker economy. That was about all the privacy of people being invaded by Facebook. I have also seen much scientific work in America from the academic world and from civil society, which is very critical of what's going in America. For me still the best source of criticism of American companies are American civil society organizations and American journalists, including also the specialized press, but also the broadsheets. So the mass surveillance issue, the snooping on people, that is something Americans also genuinely don't like in the opinion polls of people, you know, what, what do you think about the government or big corporates sucking up all your private data and making profiles about your life. The American people are as critical as Europeans are. So I don't think that there is, let's say at the basis, a different attitude, the same on economic power. America has a great history of antitrust and it has just got to remember this history and bring it back. And so, you know, I think we will watch and I have hopes for the self healing power of American democracy, that it will overcome these dark times and then use again the, the, the, the instruments, you know, of, of an independent ftc, not under the thump of the president of a Justice Ministry which can really do its job, of a Security and Exchange Commission which is independent and so on. So I think the potential in America to rebalance this concentration of power, which is also not good for, for, for American business and American people is there. And well, it's your job to get Democracy moving and to get enough people to engage not only in the next elections, but I would say also on a little bit more continuous basis to make American democracy thrive again.
C
So perhaps consistent with the optimism you just expressed, you, you write late in the book that resistance is stirring in many places that you think that, you know, movements against authoritarian and intolerant politics and against manipulation, expropriation and disenfranchisement by the big tech companies, their business models and technologies is growing. You end this book with a call to action. What, what do you think tech policy press listeners should go and do? What do we all have to do to help constrain the power of this industry and to help usher in a future, you know, where artificial intelligence isn't such a threat to democracy?
A
Well, we are trying to say to people that, that there are three really three levels of, on which they can engage. So first is the micro level, your personal choices. You know, which technology do you use for what purposes, and you know, which technology do you allow your children to use and so on. And then there's a second level which is your closer environment. You know, what technology does your school use and what technology is used in your company? What technology do you participate in as a manager or a worker? And you know, are there choices possible and is a debate possible? You know, can you convince people in your company, you know, to use a different type of technology, you know, move to open source, move to the feediverse, move to more transparent business models? And then of course, third level is the political level, you know, well, people can of course engage with political movements, political parties, they can engage in civil society. And this is some civil society in America is extremely strong. And I would wish that this, this engagement, especially of those who know about this technology and who work with it, the people who have studied informatics, the people who have studied mathematics, physics, the engineers that they understand that we are in a new age in the sense that they now carry a responsibility for fundamental rights of people and for democracy, which previously they didn't carry. They always carried a responsibility for the safety in terms of avoiding physical damage, in terms of avoiding an electric shock to a person. But now their responsibility goes much further. And I think, you know, this is something which really people need to take to heart and when they work in a technological context, that this technology now touches on much more than just, you know, previously, the, the physics of things. It touches on how our society works. And therefore we need also the universities to take charge and to start educating engineers, which we call engineers. For democracy we need people who are educated, on the one hand to understand the technological potentiality of what's coming, what can this technology do, what can AI do? And on the other hand, they must have an understanding of society, of rights of people, of how democracy works, to understand the impact of the technology and its potential impact on society and to be able to do impact assessments of technology so that we can together then orient technology towards sustaining democracy rather than undermining it, and sustaining fundamental rights and freedoms and liberties of people rather than taking them away from through, you know, dirty engineering for persuasion which makes people dependent and allows to manipulate people rather than thriving, making people thrive in their freedoms and their conscious choices.
C
Almost at the very end of this book you say the future of democracy, like the future of freedom and self determination, will be decided in the digital arena. It is by no means certain that democracy will survive AI, but perhaps it can also emerge stronger from this transformation. The choice will be made through digital policy. Sounds like a call to action for tech policy press listeners, no doubt on this Thursday morning that I'm speaking to you. Are you optimistic, are you optimistic that we'll be able to run this course?
A
Yes, I would say alone the fact that tech policy exists and is so successful, it's already a cause for optimism. I'm also the European Ambassador, Honorary Ambassador of the Boston Global Forum. I have very good discussions there, for example with people from the MIT and, and Harvard University. I've really met many people in America, all of which, you know, I call, you know, the good America. I think it is very strong. I think it is still there and there's ample opportunity to engage and to come back to the recent paper of OpenAI but also for example to mention Woodrow Hartzock and her and his colleague at Boston University, the paper on AI and democracy written by lawyers and many, I've seen many other papers by professors of informatics and so on. You know, if you now search for a democracy, there is a positive work trend, a positive trend in America to work on these subjects with optimism and with rigor for the future. So I, as I said initially, I read this paper of OpenAI on the industrial policy for the intelligence age as a repositioning a little bit like Mark Zuckerberg tried in 2019 with his article in the Washington Post where he called for regulation, seemingly giving up the drive against regulation and democratic regulation. So I would hope that now, this time we will not see again a falling back into the anti legislation drive which we saw with Facebook and Mark Zuckerberg. I would hope that this new work which is taking place in America and Europe on AI for democracy, on engineers for democracy, on AI, which supports people's choices as free people and not as manipulated people, that this is becoming really stronger. And I think it is already a world trend and many Americans play an important role in this. And I would say, you know, not only congratulations and I pull off my head, but I think you have a huge potential to come back in the age of artificial intelligence as lead nation on democracy in the age of artificial intelligence. And that's what I would wish for the future, the post Trump future of America.
C
Well, let's make a plan to come back and revisit this and check in and see how it's going at some interval in the future. Paul, I appreciate you speaking to me today. This book's called the Open Future and Its Enemies How We Can Protect from Free Society From AI Dictatorship. Paul Nimitz, Matthias Pfeffer and Jurgen Pfeffer. My best to them as well. Thank you so much for joining me.
A
Thank you. Justin.
C
That's it for this episode. I hope you'll send your feedback.
B
You can write, write to me at Justin at TechPolicy Press. Thanks to my guest, thanks to my co founder Brian Jones and thank you for listening.
A
Tech policy press.
Host: Justin Hendricks
Guest: Paul Nemitz, Visiting Professor at College of Europe, former European Commission Director for Fundamental Rights
Date: May 10, 2026
In this episode, Justin Hendricks interviews Paul Nemitz, co-author of the book The Open Future and Its Enemies: How We Can Protect Free Society from AI Dictatorship. Nemitz draws on his decades in EU policy, especially on GDPR and digital rights, to dissect the dangers posed by unregulated AI. The conversation covers the intersection of technology, democracy, ideology, and the urgent need for collective responsibility to prevent the consolidation of authoritarian power through AI.
"It is absolutely not a Duma book. On the contrary, it says the future is open. We are shaping it all together." (05:22–05:39, Nemitz)
Contrast Between Tech Fantasies and Reality (06:44–11:54):
Quote:
"Those who claim that the future will be alone shaped and determined by technological innovation...have not understood two basic facts of life." (07:39–07:46, Nemitz)
Linking Tech Power to Political Authoritarianism (11:54–14:48):
Quote:
"AI is a power technology, and it is used to gain power over people and power over political systems." (13:17–13:27, Nemitz)
Transatlantic Shock and European Response (14:48–19:10):
Quote:
"For all of us who grew up in this transatlantic atmosphere, looking up to the history of democracy in the United States...this development, of course, is a shock." (15:39–15:58, Nemitz)
Why AI is a Problematic Technology (19:10–24:55):
Memorable Quote:
"We should teach [children] to be critical towards AI because AI is power. And in part AI is taking over functions which media before had..." (24:19–24:34, Nemitz)
Roots of Tech Power: The Subsidy of Underregulation (24:55–30:29):
Quote:
"...the crux of the digital age: much of the regulation which sounds very technical is actually very, very important for fundamental rights of people, for the freedom of people and for how democracy functions." (29:37–29:50, Nemitz)
Risks of Overreliance on Empiricism (30:29–34:15):
Quote:
"...if we make in a fast moving technological area the rulemaking dependent on empirical evidence, the law regularly comes too late and we have to wait for the catastrophe." (31:35–31:47, Nemitz)
Multilevel Engagement and Democratic Responsibility (37:34–41:04):
Quote:
"We need...engineers for democracy... educated to understand the technological potentiality... [and] to understand the impact of the technology and its potential impact on society..." (40:00–40:39, Nemitz)
Concluding Outlook (41:04–44:10):
Key Closing Quote:
"The future of democracy, like the future of freedom and self-determination, will be decided in the digital arena. It is by no means certain that democracy will survive AI, but perhaps it can also emerge stronger from this transformation. The choice will be made through digital policy." (41:04–41:37, Nemitz as quoted from the book)
On AI as Power:
"AI is a power technology, and it is used to gain power over people and power over political systems." (13:17)
On Regulation and Evidence:
"...if we make in a fast moving technological area the rulemaking dependent on empirical evidence, the law regularly comes too late and we have to wait for the catastrophe." (31:35)
On Democratic Responsibility:
"...technologists now carry a responsibility for fundamental rights of people and for democracy, which previously they didn't." (39:04)
On Optimism and Civic Engagement:
"Alone the fact that tech policy exists and is so successful, it's already a cause for optimism." (41:37)