Loading summary
Transcript78 lines
- [00:04]
Tom Uren
Hello everyone, this is Tom Uren. I'm here with the Gruck for another between two Nerds discussion. How are you, Grok?
- [00:10]
Grok
Fine. And yourself, Tom. How are you?
- [00:12]
Tom Uren
I'm well. This week's episode is brought to you by Island. I had a discussion with Michael Leland from Ireland all about info stealers and protecting credentials and how enterprise browsers can help with that. So catch that on the podcast channel this week. So, Grukuya, I discussed this briefly with Patrick today, but I came across this report from Anthropic and came out in April, so I missed it at the time. But it talks about an influence as a service platform where they used Anthropic's Claude, their LLM model, to basically run, well, not just one campaign, but multiple campaigns at the same time. And in some respects I thought this was quite impressive and I showed it to you and your immediate response was, that's so pedestrian and rudimentary for our listeners.
- [01:09]
Grok
I am rolling my eyes so hard right now.
- [01:13]
Tom Uren
And now, to be fair to myself, I'm thinking about the sort of technical aspects of rolling up an AI to manage a whole campaign. And you're coming at it from the perspective of, I guess, information warfare, like winning or demoralizing hearts and minds and.
- [01:35]
Grok
All that kind of stuff being effective as opposed to being impressive.
- [01:39]
Tom Uren
Yeah, yeah. And I think there's some interesting things here that we're going to pull out. So let me just read out some of the things that this report, which is from Anthropic, talking about the malicious use of its tools. So it says that this actor, whoever's behind this influence as a service platform, used Claude basically to do it. Seems like almost all the decision making in the campaigns. So they detected at least four separate campaigns. So Claude maintained detailed political alignment guidelines for each Persona. So it doesn't say what social media this was on. Some of it looks like threads based on screenshots. I'm guessing X as well, maybe Instagram, I don't know. Whatever. Two Evaluated whether drafted content aligned with each Persona's political viewpoints, decided how to react to content posted by other users according to the Persona's legend, generated appropriate responses in the Persona's voice and native language. You're laughing. Creative prompts.
- [02:48]
Grok
It's like somewhat like it's, you know, used a fork. Use the fork to eat a steak. Use the fork to eat broccoli. Use the fork. Like, yeah, look, we get it. You can, you can manipulate a utensil. Like, that's not impressive. That's literally table steaks. Like you have to do that. Just. Sorry, keep going.
- [03:10]
Tom Uren
So basically it got the Personas to behave appropriately according to their legends, which were manufactured as well. And it also incorporated image generation tools. It would create images and decide whether they matched what they wanted, whether they were aligned with instructions or not. So I thought this was like super interesting in that they've gone, whoever's running this network, they've gone all in on getting AI to work. And for them, it seems like this is maybe it's a labor saving tool. And it also seems like it could be a marketing tool in the sense that you can go to a prospective client and say, we've got this amazing AI powered influence network.
- [03:57]
Grok
So to me, it's more of a labor saving device in that some sleazy salesman can scam people into buying their influence agentic thing because it's AI for influence. And it means that you don't have to hire a bunch of people, you can just offload it all to Claude. It strikes me as it's probably a very good sales tool for selling whatever this is.
- [04:21]
Tom Uren
Yes.
- [04:21]
Grok
Whereas this is not a good sales tool for selling influence.
- [04:25]
Tom Uren
Right. It doesn't actually work. Yeah, yeah, so that's exactly what I thought as well. And later on in Anthropic's report, they say, they're almost apologetic. They say it has limited viral impact, but this wasn't really what the operation was designed for. So it's designed for, they claim persistence over virality, relationship building over content spread. And I'm like, I don't think anyone wants just to hang around achieving nothing, do they?
- [05:00]
Grok
I mean, if you're trying to sell something based on number of hours, like, yeah, of course. Like, it certainly doesn't seem to me like it would be effective. Like it might be good at selling itself, but it doesn't seem like it's good at selling themes or influence. And I think part of that is a profound misunderstanding of how actual information operations work.
- [05:22]
Tom Uren
Yeah, yeah. So I showed this to you and you immediately sent me a document about World War II. Would you call it an operation?
- [05:30]
Grok
Yeah. So this, at the time, less sophisticated time, this was called black propaganda. So the idea with black propaganda was that you would pretend to be someone else, and then the propaganda that you were providing would be underneath the guise of this other entity. So in this case it was black propaganda and that they were pretending to be Nazi Germans, but it was aimed at Bulgarians. And because of the desire to discredit the Nazis, it was meant to be a German radio badly Imitating, like doing a bad black propaganda version of a Bulgarian radio.
- [06:17]
Tom Uren
So it was. Was it the British, really?
- [06:19]
Grok
So, yes, so this was the British.
- [06:21]
Tom Uren
So it was the British pretending to be the Germans pretending to be Bulgarians.
- [06:26]
Grok
Badly. So they had. They had Germans who spoke Bulgarian with a heavy German accent doing, you know, like, we Bulgarians must come together to support Zafura. You know, like just these heavy accents to sound so bad that it was obvious to anyone listening that it was not actually real Bulgarians. And then they did a whole bunch of other tricks because there was a legitimate German black propaganda station. So it was made to be very, very similar so that you easily get them confused. So they're on a similar radio frequency, they're just a bit off from each other. And then like when the. The German one would have, you know, a John Smith, the British one would have a John Doe or be, you know, a James Smith.
- [07:17]
Tom Uren
Right.
- [07:18]
Grok
But what's actually brilliant is the way that they did their propaganda. So this document is in sort of two parts. The first is this. It describes the overall plan and how it was supposed to work, the strategy from when it started to when it ended. It's a sort of after action report about the whole thing.
- [07:38]
Tom Uren
Right.
- [07:38]
Grok
But then I think that the true genius is that they include as appendix A, a transcript of a typical broadcast.
- [07:46]
Tom Uren
So what was the overall goal? It was to. Was it to demoralize Bulgarians who were being forced to fight on the German side?
- [07:54]
Grok
They were trying to get the Bulgarians to not support German war aims in general, just to turn against the Germans, sort of like undermine faith in Germany, to create an anti German trend, to discredit any Bulgarian regimes that did collaborate with Germany to play up how big the opposition was, like how strong the Allies were. Like, if there was real indigenous defeatism, like if people were sort of pessimistic about the war, they wanted to bring that up and emphasize that. And they also wanted to sort of attack the will to fight. So it was. It was all very political. Right. Like the goals here, it's to undermine the will of actually fighting against the Allies. That was the point. And it was. It was influencing people to not like the Germans and to just want the war to be over with. Very, very straightforward. The way that they went about doing it is where it's just so, like, so amazingly brilliant. One of the things that helped them a lot with this was that at the start of the war, the British captured a wire service device from the Germans. So they had the official press releases that were issued by the German propaganda office. So they were able to take the official news items and then reword them before they were put out. And that allowed them to shape the way that the sentiment was going across. I think we should get into some of these because they're just so clever. But more broadly, I think that the key difference here is that the overall objective of this station was very, very clear. They were going to decrease the will to fight. They had a clear target and they knew how to do it. They were going to take these things and they were going to transform them into things that would depress people, that make them not excited about the war.
- [09:47]
Tom Uren
Yeah.
- [09:48]
Grok
And so because they had that sort of strategic objective in mind, they knew how to adjust these things. I think that that's one of the things that's missing from this other thing. Right.
- [10:01]
Tom Uren
It's just, well, so anthropic rights, you know, four distinct campaigns. The one I like most is, number two, promoting the United Arab Emirates as the superior business environment while criticizing EU regulatory frameworks. Now that's a topic near and dear.
- [10:18]
Grok
You say that to my face.
- [10:21]
Tom Uren
Near and dear to everyone's hearts. That's really going to get people fired up. And whereas you're talking about a world war. Right, right.
- [10:32]
Grok
Slightly different emotional impacts there, I think, but I think it's more than that in that, like, that objective is kind of like, it's kind of vague, like attacking the will to fight. It seems more nebulous, but it's actually much easier to go after than something like, you know, making people think that the regulatory framework of the EU is too onerous. That's very, very clear. But like, what is the emotional impact of that? Like.
- [10:58]
Tom Uren
Like, I mean, I guess you just hang around waiting for an EU Commission press release and then make part of it. I don't know. Whereas in the example you're talking about, there's the Nazi propaganda that's coming out constantly, that you've got this constant feed of relevant material that is. Is grist for the mill.
- [11:20]
Grok
Right. You've got something to work with.
- [11:22]
Tom Uren
Yeah. Daily you get something that people care about as well.
- [11:25]
Grok
Right, right. I mean, it's news about a war that involves you. Like that's a high valence topic, you know.
- [11:34]
Tom Uren
Yes.
- [11:35]
Grok
But the thing is, the real genius about this is the way that they were able to take these, like these little news slugs from the Nazi news feed and then convert them into basically the same thing, only much more pessimistic or emphasizing the parts that they don't like for Example, they start out with item two. It goes. Here is today's communique of the German High Command from the Fuhrer's headquarters in Normandy. After especially hard fighting, the enemy succeeded in penetrating our lines. On the western sector in Brittany, fierce fighting against bandits and parachutists continues.
- [12:15]
Tom Uren
Right, so if I'm reading this correctly, the original was that in Normandy the enemy yesterday attacked on the western wing, employing strong forces. On the whole, he was repulsed and after hard fighting succeeded only at some certain point in penetrating our lines. In fighting in Brittany, 20 terrorists and 58 enemy parachutists in civilian clothing were annihilated. So it's subtly different. Right, so you know, the enemy succeeded in penetrating our lines, whereas the original.
- [12:47]
Grok
He was repulsed almost everywhere except at some minor points where he did penetrate. But, you know.
- [12:53]
Tom Uren
Yeah, so it's not a huge difference.
- [12:54]
Grok
Right, right, exactly. I think that's sort of part of the trick to it is you don't want to be. You don't want to be obvious. It's not substantially different. It's the exact same key points. It's just presented without the same caveats and framed to emphasize sort of the negatives that gives a flavor of what they do. But I think this one example coming up next is the truly brilliant way that they approach this. So this was, again, it's based on a real news item, but it's just been expanded a little bit so it goes like this. So this is going to be item number four. Berlin. This morning's Press devotes long articles to the Normandy front describing the heroic deeds of the panzer division made up by the Hitler jungen. When these 17 and 18 year old boys learned that they were to fight the Anglo Americans, their joy and enthusiasm almost became hysterical. The words of the Reichsjudenfuhrer Axemann are of interest in this connection. He writes, when one sees the fierce expression on the faces of these lads moving up in their tanks to fill up the thinning ranks of their older comrades, comrades at arms, one's heart is filled with joy and with gratitude towards the Fuhrer who has succeeded in converting the one time spoiled youth of Germany into an efficient war machine. Up till now, the fields of Normandy have been strewn with the sacred bones of 7,386 Hitler youths who laid down their life on the altar of National Socialism. Their example is worthy of emulation. And I am more than certain that the Bulgarian state youth organization Branik will not fall short of its German comrades in arms.
- [14:43]
Tom Uren
That one's. I think that one's quite powerful, right, because it's, it's, it's basically saying your sons are going to die for Hitler.
- [14:52]
Grok
And far away, your sons are going to die in France for Hitler. You know, like all of the bad.
- [14:58]
Tom Uren
Things and I suppose all the other sort of fairly subtle but straight stuff is in a way is building a track record of actually appearing to be a genuine German propaganda thing. So you're playing it straight. And so that when it comes to this piece, which is, I think, quite impactful, you've got a reason to believe that that's the real story, right?
- [15:25]
Grok
Like if a press release item from the Germans was minor, they would just give it straight. Like if it wasn't something that they were trying to twist in particular and it wasn't against their aims, then they just let it go directly, which helped to build credibility by saying, we have the exact same news items as other people. The thing is, like even that Hitler Youth thing was based on a real item from German radio. It's funny, it's like it's almost the same thing up until there's this sort of, this extra bit. Vin Sassen reports on the activities of a tank division made up of members of the Hitler youth. When these 17 and 18 year old boys had been told that their fighting in Normandy had been mentioned in a recent High Command communique, their joy knew no bounds. Champagne could not have achieved such exhilaration. I mean, part of what helps us is that there were actually Hitler Youth fighting, so they weren't making it up. What they were doing was emphasizing that Hitler Youth had to go and fight because there weren't enough men left. And then they connected it to Bulgaria and of course they added that detail of the number of Hitler Youth had already died, which, I don't know the real number, but I don't think they did either.
- [16:43]
Tom Uren
Like we've mentioned how the stakes are just so much higher for this kind.
- [16:49]
Grok
Of thing than the, the death of a child versus the regulatory framework of the EU business environment are. I mean, they're similar depending on how sociopathic you are.
- [17:02]
Tom Uren
So to me it feels like the modern covert influence stuff, like no one cares is basically the problem with them. And so particularly in this case, you've got the influence service provider and you've got anthropic and both of them have incentives to make Claude seem amazing. So for the influence service provider, you can go to a potential customer and say, I've got this amazing AI magic dust everywhere on my influence campaign. It does all these amazing things. It'll guarantee consistency. Looks like people.
- [17:40]
Grok
Yeah. You can have as many themes as you want. We can expand infinitely.
- [17:45]
Tom Uren
Yep.
- [17:46]
Grok
All of the.
- [17:46]
Tom Uren
And likewise, anthropic is also. We want to make Claude seem amazing.
- [17:51]
Grok
It reminds me of many, many, many years ago. There's a. There's a press release with an FSB general who basically said, it's unfortunate. Like, it's a sad case that cybercrime is such a problem. And it is unfortunate that Russia has the best hackers in the world. Yeah, it's bad. It's too bad. Ours are the best at being so bad.
- [18:21]
Tom Uren
Yeah. And so I guess it's that no one really cares if it doesn't work. Provider wants to sell a product, they don't really care if it works. As long as they can sell it, it doesn't matter. It's not like people's lives are at stake.
- [18:36]
Grok
Yeah, well, I mean, it's a fly fishing joke, right? Where the guy buying lures for fly fishing asks the salesman, like, do these work? The guy's like, I don't know. I don't sell to trout. Like, the point is to sell them to fishermen. Like, who cares what they do after that? Yeah, it's. I think it's very much that. It's the, like, the incentive structure is. Is entirely built around selling this product. Whether you're anthropic selling how, you know, unfortunately we have the best AI in the world, or whether you're the influence people selling it, and it's probably even the people buying it. I mean, if, if you're trying to promote the UAE as a better business environment, you don't really care about whether it's effective or not. Right. Like, how do you even mention that sentiment? You're just going to say, like, yeah, we had this much engagement, so you can see, like, that's impressive. Like, here's some metrics. It seems like you shouldn't even bother with that sort of influence campaign. Like, what's the point?
- [19:39]
Tom Uren
I think the other ones are equally like, riveting, like promoting development initiatives and political figures in Kenya, supporting Albanian figures and criticizing opposition figures in a European country. Now, I was also thinking about the process behind it. So in the anthropics example or case study, well, someone's paid for it, but once they've paid for it, it's the magic machine decide stuff, it seems.
- [20:16]
Grok
Whereas it's primitive as well. I think that's the thing, is that it doesn't have this sort of like, deeper subtle machinations going on. It's very much you believe that the EU has a restrictive regulatory environment for businesses. Now, whenever you write that's your guiding motive. It's very, very boring. And it's very surface level.
- [20:39]
Tom Uren
Right. I mean, in the report, it's not clear that Claude is making the sort of top level decision it says it makes, tactical engagement decisions.
- [20:50]
Grok
Right. What I'm getting at is that the directive given to Claude is to create people whose belief system is this and then to just run those people like it's, it's not even pedestrian, it's, it's boring and it won't work. I mean, it's not as effective as human creativity.
- [21:12]
Tom Uren
Yeah. So that's what I was sort of thinking about is the, can you replace the human creativity with AI? And I'm like, my guess is no. And I was thinking about what the process behind your example, your World War II example would be. It seemed to me like it could be something like a newsroom where you've got a group of people sitting down, looking at the press releases coming in, trying to decide which ones are worthy of, you know, subtle manipulation, which ones should we let go through and how do we do that and what's the. It felt like there was a process behind it where there was a bunch of clever people sitting around in and making those decisions.
- [21:50]
Grok
Yeah, I mean, that's how it happened. It was like it was group stuff where they would discuss and like they'd have themes that they wanted to get across and they would sort of bounce ideas around and develop these things. And I mean, in his autobiography, Sefton Delmer says, you know, like, there's a lot of scripts that were written at 2am after a lot of wine.
- [22:14]
Tom Uren
So is that where this document comes from?
- [22:17]
Grok
It was from his agency. So the Political Warfare Executive was the British Ministerial group or the, the British agency in charge of doing information warfare. And he was put in charge of it and ran the whole thing. And he sort of built up these divisions, like these different groups that were doing their own thing. So he was recruiting people and all that. He was training them up into sort of how to, like how to do this work, how to basically approach these things. I mean, the thing that I think is missing is that this political ideology is just the flavor of how you communicate. Whereas the World War II examples, there was these overall objectives of what they were trying to achieve, and the flavor of how they communicated was just the way that they communicated. But the things that they were saying had deeper purposes. There was more intentionality behind what they were doing than just Promoting a particular ideology like that was the surface text and the subtext was much, much deeper. And I think that that's missing.
- [23:25]
Tom Uren
Yeah. So I guess you're saying that it had a degree of intentionality and purpose that at least doesn't come across in this report.
- [23:33]
Grok
Right.
- [23:33]
Tom Uren
So I mean, it says the operation utilized Claude to make tactical engagement decisions, determining whether Personas should like share, comment on or ignore specific posts created by other people based on political objectives. So I mean, there is the possibility that at a strategic level someone in the campaign is brainstorming ideas of. It's not evident from this report, brainstorming ideas of what to promote or how to subtly change or alter messages. But it's not clear in this report.
- [24:05]
Grok
I guess I think what Claude is doing is not far off from what the Russian IRA was doing in the 2000 and tens because they did not have the subtlety of the British stuff either. They were just doing, here's your political ideology. Go with that. Here's your key phrases for the day. Write at least three posts that have these phrases. They also had things like, they called them plays. So these little staged interactions. So there'd be like three people. So in a comment section, one person would be like, they promote an idea, someone would discredit it and then someone else would come in on the side of the idea. And these two people who were promoting it would convince the second person.
- [24:55]
Tom Uren
Right.
- [24:56]
Grok
You know, through a discussion. They would convince them through, you know, facts, logic, reason. And he convinced, like you have convinced me. Indeed, the EU is a bad regulatory environment for business. So I'm reminded of this, I think it was a Medusa article from like 2018 where they interviewed someone who had worked at the Russian ira. And the title of the article was I was young and needed the money. Yeah. So I mean, it was not very sophisticated stuff that they were doing.
- [25:36]
Tom Uren
So I mean, is that the sort of take home message here that mostly these covert influence campaigns don't work because we're not at war and no one cares and it's not. Yeah, you know, someone on the Internet is wrong.
- [25:53]
Grok
Yeah, pretty much influence campaigns that work are the ones that get influential people and have them promote things like influencer marketing. And even that doesn't work so well these days because it's being exploited to death and so people don't trust it as much.
- [26:11]
Tom Uren
Right. I mean, I guess word of mouth works when you know the person behind the mouth. And yeah, if it's just someone random online who could be a bot, then it's I guess not going to be that engaging or effective.
- [26:27]
Grok
Exactly. I mean, you need to be part of a community. And I don't think that that's the objective that they started out with. I don't think that these guys are part of a community. Like during the 2016 campaign, the Russian bots that were being effective were part of communities. They ingratiated themselves into the political discourse and then they used their position within that community to spread their narratives, which were mostly indigenous narratives that they were just promoting instead. But it's not subtle and clever stuff. It's finding people who have influence and making them say things that will resonate with the group that they're talking to. It's advertising, it's boring, it's not clever and it's not fun.
- [27:18]
Tom Uren
So we get the Grux stamp of Dis endorsement table flip.
- [27:25]
Grok
Like there's been this sort of real flip flop on disinformation where at one point everyone was like, disinformation is the biggest threat we've ever faced. And now it's like, disinformation is not a thing you need to worry about at all. Except in some places there's basically being this sort of pushback. And I'm reminded of how at the end of World War I, the Allied forces were dropping propaganda leaflets on the Germans. And so like the first plane that was dropping leaflets and that crashed behind the German lines, they shot the pilot for being a spy. Basically, the propaganda was viewed as essentially spying sort of stuff. It's so beyond the pale that you're going to get executed in the field. Like, it's not a thing that you can do during war legitimately. It's outside of bounds. And after the war there were people writing papers and articles and things saying, the next war will be fought entirely with propaganda. It'll be an information war using the newspapers and leaflets and the radio and. And like, you know, they fully believed that propaganda was so powerful that it won World War I. Whereas these days we don't even think that propaganda was a part of World War I. Like, that's. It's not even ever mentioned. We always just think of the, you know, Uncle Sam wants you, or, you know, Stop the Hun, that sort of thing. But at the time that they had this firm belief that this was a new type of warfare that had been invented and it was going to be dangerous and the biggest thing ever. And like, we've now learned how to use it during wartime and we have a better appreciation of where it fits overall. And I think that that's the sort of same thing that's happening is it's been rediscovered for the Internet, like, oh, my God, propaganda. Like, the next war will be fought entirely with information warfare on the Internet. And it's whatever. And then it turns out that the next war isn't fought entirely with propaganda over the Internet. So, obviously, propaganda over the Internet is a complete red herring. It doesn't exist. It's nothing to worry about. I think we've overcorrected, but we'll go back towards in 50 years, if we're still around, we'll be able to say, okay, that's the position of disinformation. This is what it does. Here's how to think about it. It'll have its place.
- [30:03]
Tom Uren
Yeah, I doubt we'll be podcasting then.
- [30:08]
Grok
BTN 50,000, thanks to.