← Mobile Dev Memo Podcast
Loading summary
Transcript34 lines
- [00:02]
A
You know those channels your colleagues keep bragging about? The ones getting all the credit? Yeah. They might be doing squat. Attribution makes every channel look like a hero, even when it's a zero Incremental tells you who's actually doing the work. It's like a lie detector for your marketing budget. Start using incremental today. Get your demo@ incrementl.com that's incrmnt.com. mention that you came through the Mobile Dev Memo podcast for a special 15% discount for the first six months.
- [00:35]
B
The problem is that the distinction needs to be drawn between the competence of the economists and the correctness of their analysis.
- [00:46]
C
Welcome to the Mobile Dev Memo podcast. I'm your host, Eric Suefert, and I'm joined again by Peter Stewart. Peter, welcome back.
- [00:53]
B
Thanks for having me.
- [00:54]
C
Eric, I had you on six months ago in December. We talked about a topic that I find very fascinating, which is the application of AI to the news and media publishing space. You gave us a really good overview of what was happening in that space. My sense is a lot has changed in that six months. I think when we first spoke, you had just launched your publishing platform, Ballora. You had maybe encountered a sort of like a strand of negative sentiment about the application of AI to publishing. Please introduce yourself to the audience or those who didn't hear the first podcast and then walk me through what's changed in the world of publishing in terms of the application of AI and the perceived acceptability of using AI in the publishing workflow since we last spoke.
- [01:41]
B
Yeah, no, happy to chat about that. Once again, thanks for having me on. So I'm Peter, I'm the co founder of Alora along with my technical co founder Danny Bellion, and I come from a background of having been a cycling journalist since I was 21 years old, before that as a general journalist and before that a student journalist with Eric. And yeah, I kind of went through heydays and magazines and then latterly my last role was as the editor of Cycling News, which is the world's biggest cycling website. About a year ago I came up with a kind of loose idea of developing something that was going to be more AI native as a concept from publishing and sort of in conversation with Danny, who I met at a barbecue. Having not spoken to him for like six years or something, we got chatting and we decided to give it a go simultaneously quit our jobs suicide pact and go for launching this platform. As you mentioned, it was originally a cycling website and it is still a cycling website, but our focus was always on developing a platform behind it that would ultimately take the lift out of modern journalism and modern publishing. A large part of that has actually just been the decomposition of the writing process and the publishing process. So Danny, having not been in the publishing background, looked at what I did day to day and sort of looked at where he thought an agent or an AI tool could do the job better or at least augment the job substantially. So where we started with the cycling publication, we quickly kind of pivoted to producing this as like a public facing platform that other publishers could use. And yeah, since we chatted, we have about a dozen different publishers using the platform on a kind of licensed basis across totally different sectors. But it's trying to help more publishers kind of be able to use effectively what is a suite of different AI tools in a very kind of tight orchestration to take the heavy lift out of their publishing workflow. Sometimes that's just basic stuff like tracking news, news visibility. Sometimes that's kind of more involved in terms of CMS uploading, finding images, sometimes it is extending through like AI drafting articles and sort of like doing some sort of more automated content. But key to it has always been like the kind of like visibility element, the research element and the verification element and the CMS kind of manual uploading agentic process, which have been a help for anyone in publishing, regardless of their view on AI writing or AI drafting.
- [03:48]
C
And so what's happened in the kind of news and media publishing space with respect to the adoption of AI in the six months since we spoke last year?
- [03:56]
B
Yes, I think when we last spoke it was probably when there'd been quite a lot of stuff published about the quite slow uptake of AI in journalism across the uk. Certainly I think the Reuters had a stat that about 20% of journalists had never used an AI tool in any journalistic task at all and only sort of about a quarter were using it on a daily basis. Which large part of that was just AI transcription tools like OSHA and things. Since then there's been sort of like some more public kind of embracing of AI tools. Like you have a large, quite high profile partnership between News UK and Symbolic, which is an AI, AI kind of editorial suite. And outside of the uk we've certainly had other countries that have had more kind of fundamental AI infrastructure become part of their workflow with multiple agent generated news, with sort of almost effectively totally automated local news processes or news production. But in the uk, I think probably as with the us, a lot of the kind of development's been slightly Hamstrung by a lot of sort of quite high profile, kind of embarrassing cases of AI being misused or causing editorial errors. And that's where most of the news has hopped. So I think most AI use is still quite discreet and speaking to trainers in the AI space. While I think more companies are looking at ways to use AI, most companies are still quite early in their process of currently still getting to terms with a Claude kind of subscription and Claude Cowork Tools and Gemini kind of basic chatbots. We haven't seen the kind of full agentic process go into many publications and certainly not in a public way.
- [05:25]
C
I think for the most part any major news publication now has an LLM generated summary at the top. And you know, I don't think readers, consumers, you know, object to that. I think, in fact I think people probably find it really helpful. I started including that in the posts I send by email just because I think it's important for people to get a sense of like what this thousand words is about before they kind of commit to it. Right. And so if you just kind of read a two sentence summary that I think is pretty obviously generated by LLM, maybe it gives you a sense of whether this is worth your time or not. And I think people find that helpful. But where's that boundary, right? Like where does it go from? Like this is a helpful perk that you're providing to me versus like oh, you've deceived me and you're making me read slop. Like where do you draw the line? Because the summary, I think if we just maybe just kind of accept that everyone views that as a useful tool and that's like an acceptable use case to like writing the full article with an LLM probably wouldn't be seen in the same light. Where's the line? If I had whole paragraphs that were LLM generated, where do you think you would draw a line there just on the output side of things.
- [06:36]
B
I think what you see with lots of publications is they have sort of developed language that will state maybe if an AI is used to create a substantial part of the article, we will make it transparent. And that's kind of the language you might see with say the Guardian or the BBC. I think the BBC might specifically say that writing is not allowed or research is not allowed with AI tools basically to be even more clear cut. But I think there's quite like a furry line when you say AI is used for any part of the writing process versus AI is maybe generating articles quasi autonomously. To my Mind, it gets super cynical and kind of a bit distasteful when something is positioned as, say, commentary or opinion and it's clearly AI generated. Because I think that's when you feel as a reader, well, have you really articulated this yourself or is it just the result of two prompts? Or even one prompt be like, write me a hot take. I think it's really important to consider how that's different. To say something that might be like a financial write up of a quarterly report. And at that point you're like, well, really all you're trying to do is you might have a huge document that's emailed to you in press release form or an SEC filing and you're just trying to delineate the key information for your audience. And at that point I'd say, well, if you have a robust pipeline in place, and this is my view, when you have verification in place and the information is correct, it doesn't really make a difference in my perspective if that's AI written or human written from scratch. Because the core thing is it's an informational journey for the reader and if they've sort of found that that has been fulfilled appropriately, then I think AI use or no, AI use isn't really a problem. I think generally the amount of AI written content on the Internet is actually far, far more than most people appreciate. Even people like Guardian, prestigious publications, they get caught out with pangam type screenshots saying your sports writers have been using 100% AI generated content for the last six columns they've published and they haven't often been able to rebuff that in a very efficient or kind of robust way, because most publications don't have control over what every single writer is doing all the time. And there's always the argument, oh, the AI detector is not 100% reliable, or what have you. So it's not like a fair jury. In terms of the substrate that sits below kind of substrate is probably the wrong word. In terms of the layer that sits below, like the nationals and the prestigious titles, I'd say AI generated copy is already effectively ubiquitous. And you're thinking about all the trade magazines, all the B2P publications, all the kind of individual site owners that exist there. A lot of these people are financially sensible and responsible for their output and their income and they're naturally going to look at what is the most efficient way to get content out to a level which that audience expects. There's also, I think I never expected working in this field, but the amount of Offshoring that's kind of a combination of offshoring and AI use is also super, super common across lots and lots of sectors. You might have or like showbiz or sporting titles that have teams in the Philippines or India or somewhere that are using kind of chatgpt to mass produce content. A lot of this gets penalized by Google from time to time, but lots of it doesn't if the domain is established and has lots of authority. So yeah, the actual broad ecosystem of how AI is used, I think has changed substantially. My expectation now with most publications, when I see content, definitely if it has regular output, quite commodity content, is that AI has played a part in the creation of lots of the copy. Whether that's obviously detectable is probably more of a technical question than it is ethical. Because people that use AI a lot are probably getting better and better at disguising the kind of obvious AI isms. And that kind of becomes to me the most insulting bit when someone's not even tried to cover up the fact that's not weakness, it's fragility or something. And it's like, come on, this is freaking obviously chatgpt talking. Get rid of that. So you kind of end up more resentful of the person's technical gap and not thinking, why didn't you cover this up? All that say now I'm not necessarily. This is a good outcome. A machine written, kind of like ecosystem of publishing is kind of scary in a way, but we have to be honest that people aren't going to make life difficult for themselves. If you have basically the world's most efficient intern sitting right next to you, willing to do all your writing for free. You're not going to labor through yourself and do it if you've got a mortgage to pay and you're conscious of hits and time and whatnot. So I think in the broadest term, that has changed quite a lot across the whole ecosystem of publishing. Where that sits for big publishers, I'm not sure, but truth be told, while I was saying before that they've got AI tools in place and are trying to work on that, what I hear from people that are trainers in the space is that a lot of them, because they've had such hostility and such fear amongst the staff, they still are maybe like a year, two years behind a lot of what other sectors might be at. So they're now currently getting into using Claude cowork or setting up an agent or something, but they're not quite where other industries may be. You have even Say in marketing, I think a lot of marketing teams have got quite efficient content automation pipelines in place already. It's kind of maybe a bit of a given. And increasingly even from the SEO teams that used to be very anti AI, they'll accept the assumption that AI has played quite a big part in writing because it's just naturally efficient.
- [11:29]
C
Yeah, I mean, I think it's a great point. Where does the audience delineate? Acceptable use is probably what ultimately is going to define what acceptable use is and not just these kind of legacy media organizations trying to maintain like the status quo. Because it's just like you said, I mean it's just not going to be economical. But like, I think because I was writing to this company called Bending Spoons, filed as I filed for an IPO yesterday. And so I was writing up the F1, which is like the foreign equivalent of an S1. It's their, their filing document. And like, you know, the, historically I would have just read that whole thing and it'd be like, that's 100 pages. And like, okay, that's gonna take two hours. But now I just feed it to ChatGPT and I have a, I have a list of questions and you know, tell me where I can find this information and, but you know, it also, if it's giving me just some, call it a two sentence paragraph that summarizes the growth. There's no discernible difference between what I would have written and what it wrote. It's just a bunch of numbers concatenated with like ands and commas. Like it's, it's really, there's, there's no, you wouldn't be able to tell. And like I said for that purpose it's like, okay, if I copy, I don't remember if I did. I try, I try not to have any, anything make it into my articles that is purely machine written. But like if I copied and pasted that, I don't think it's a, it's not a violation of trust on the part of the reader. Now that's not analysis, that's just reporting numbers.
- [12:50]
B
Right?
- [12:50]
C
Reporting metrics. Right, but what have you heard? Like what's, what is the range of attitudes across big media organizations? Does it tend to map by size? Like BBC says? Absolutely not. And then some small blog says we just don't care. Like what, what is, what's, what is the range of, of attitudes and how does that map to the size or the prestige or the age of the organization?
- [13:13]
B
I think really big organizations haven't changed particularly quickly. And that's probably not to do with ethics as much as to do with just the inertia of a very large business and the way it operates. BBC has thousands and thousands of journalists, the Times has hundreds. Those people have their own workflows on their own desks. And telling them to switch to something else is going to be really hard at the best of times. And I think quite commonly these big organizations might roll out a Gemini license and then uptake's absolutely tiny. I'm actually headed to a conference next week from publishing across Europe and it'd be interesting to see there more direct on the ground information. But I think you have a scene where in the Nordic countries or Scandinavia or even kind of Belgium, you have much more embracing of AI tools and AI agentic processes. Like Mediahoos, which is a big publisher that owns 100 different publications, including nationals. They have a whole agentic news workflow that they're very open about. I don't know what the bylines look like on that. I don't know whether it is subsequently attached to someone's LANIM or it's transparently machine written. I think generally places in the UK that have started to go down the AI drafting route are probably not super vocal about it. And I think that problem with that is that I think trust kind of falls through the floor when someone knows something is AI written. I think. I think the assumption is that much like I said at the beginning, that if someone's using AI, it's total laziness play. It's not to do with actually trying to improve the content or be more efficient. And there's lots of studies in this that trust of AI media is super low, but worth qualifying that stuff. That trust of media generally is also actually super low. So people are naturally not going to trust AI if they don't trust media anyway. So I think there's a big range of full kind of manual written kind of policies that will be very carefully adhered to and then people that are experimenting a bit more boldly. A lot of it's going to be on a very basic level, a kind of economic cost per article thesis. If you have the Times, the Sunday Times, the Economist, they don't publish a huge amount of articles for the staff they have. They've got a very expensive subscription base. Their readers probably expect a large investment. A writer has always spent one or two days, maybe more, writing an article. By comparison, you might have digital writers at lots of publications that might have to do 10 plus stories a day. And then at that point you're like, well, if the story was going to commodity content anyway, it doesn't really change anything if it's machine written. And so it's not surprising that a lot of those titles will flip to a machine written kind of model. But I'd expect that the more prestigious titles, the fts, the Times, the BBC, will continue to have a very rigorous copy as human written. Whether that actually changes anything or it actually makes the experience better for the reader, I kind of question. And some of these people like the ft, certainly have strict policies. They have actually offered totally AI generated content as a service that's like newscast. That was done a few years ago using a third party AI kind of newswire service called Noah Wire. Yeah, I don't think that's ever made it to consumer readers, but it was envisaged as a B2B service. Again, a lot of that stuff probably has been experimented with quite early when the models were still not really quite up to it. But I think there's definitely been a sea change when you have Opus 4.6, probably the moment I'd say that stuff jumped forward quite dramatically in terms of capability with writing and 4.8. The actual models now are super capable. And now it's not a question of quality. I don't think on the writing front it's really just about kind of ethics and accountability.
- [16:38]
A
Plenty of mobile growth strategies are still built on broken journeys and incomplete data and it's costing brands big time. Branch fixes that it's your AI powered MMP that unifies your links, measurement and user experience across every channel to successfully acquire, retain and engage your app users without wasting spend. Learn more@ Branch IO and while you're there, check out Branch's AI Search and Discovery report covering insights from more than 300 marketing, growth and digital leaders on how they're responding to the rise of AI search.
- [17:13]
C
I want to move on from the output part of the discussion because I think it's the more superficial topic within this conversation. But how do you view that like accountability piece? Because my sense is like you could view the brand or just the association of the publication with, you know, rigorous journalism, exhaustive fact checking, unbiasedness. You could view that, that kind of brand association as a license to embrace this. Because it's like you, you sort of give them the benefit of the doubt that they'll embrace it responsibly. Right. And so it's actually interesting that you said that the northern European countries are embracing the use of AI in News because they tend to be, I think, a little bit more restrictive with anything that has to do with labor and workers rights and stuff. But if you have this reputation, if you're the Walter Cronkite sort of halo effect, then you'd think that your readers might give them more leeway to introduce this because it's a. Well, I trust that they're using this, but they're also validating it, verifying it. They're using it in ways that are aligned with their sort of brand image. And so I'm going to accept it from them, but from another publication I might not. Just because they don't have that same level of trust. Do you think that's roughly right, a good way to look at it?
- [18:38]
B
Yeah. To be honest, I think my view on this stuff is that I think the level of trust is very high with those brands. Also the level of skepticism. Now, I don't know if you've noticed, but if there's a commonly shared LinkedIn post about some headline in a newspaper being all wrong or it's always like, look, AI messed up again. It's like, well, probably not actually if someone miswrote a headline, I don't think AI would do that. It's quite good at doing that sort of stuff. Probably just some sub editor that wasn't paying attention. You get the odd end of a newspaper. It's a newspaper indie that had. For the next question, let me know if I can help out by putting this in an email. Yeah, and that stuff happens quite a lot. And then you have more high profile mess ups like you had the kind of New York Times when they had an AI generated thing kind of summary was attributed as a quote. Or you had a book review plagiarized from the Guardian and you had Ars Technica had that huge issue with AI generated quotes as well. There was a whole list of issues. And I think when what ultimately in my view happens is titles don't have a very clear or sensible AI policy. Some individuals in that business use AI tools in a very untechnical, unwise way. They have a big scandal and then they totally freeze up. And we don't use AI at all. It's all human stuff. And it's kind of like that recoiling back just because that fear of trust is so strong. My view of this stuff has always been that I think if you have super robust AI policies that involve actual AI use and really clear traceability, and that's what I don't think there is a lot of processes in place where you have, okay, here's our research agent, here's where that's checked by a human. Here's where a draft might be checked by a human. Here's where an edit is checked by a human. Here's where an edit is completed by a human. Here's an n gram comparison of the edit versus the draft and you can totally see how much has been changed by the person. I think that would be a state where you think, okay, well this seems like it's a robust, auditable kind of process and I think, I don't know where they're at with it, but the EU has a transparency act for AI generated content I think will come into effect in August. I haven't checked on it recently, but for that they actually offer an exemption of news publishers that have very clear auditing in place. So if they're saying we clearly see who set the software up through written, which research tool was used, you don't have to disclose and say this is AI generated content. How that law is going to work in effect, I have no idea because how are you going to enforce that across the whole of news media? So yeah, I think in answer to your question, I think there's probably a bit of a technical gap where titles haven't begun aggressively working on an AI kind of infrastructure that makes sense for them. And also they're faced with so much audience hostility, people just hunting down some sort of a scandal to set them up to be like, hey look, the media got it wrong and they're using AI for lazy purposes. And alongside that you have some super kind of ill advised projects where I think this one publisher in the US had sort of said that they have the right to publish articles that AI generated with writers bylines with or without their consent. And that's just kind of crazy management decision where it's totally obviously going to have a big backlash. I think generally trust is a complex issue and I think trust of AI being used well will be super low for brands where there's already a massive public skepticism anyway, which typically is the bigger brands that are more public.
- [21:38]
C
Right? Yeah, I think about the use of Grok on Twitter for instance. So it feels like a very effective and let me know if you see this differently, but it feels like a very effective way of just combating misinformation because you just, okay, well I'm going to use the LLM which is also as kind of tool act access and can do searches in real time and just validate what people are saying. And like, yeah, there's a lot more AI slop on. On Twitter now. But like you. But you can also use the integrated LLM, which is grok, to. To validate what people are saying. And if they're just spouting nonsense and like, that feels like a, you know, so, so right there you can kind of see the, the pros and the cons. Like, well, okay, there's a lot of AI slop. Someone just went to chat GBT and asked for whatever, 300 words on some topic. And it, it's just, it's. It's just poorly written and not engaging prose. Right. But at the same time, like, the benefit of applying LLMs in this environment is I could just fact check stuff almost in real time and call people out and utterly neutralize what they're trying to do, which is spread information. Because you could just see the. You see the GROK response and then someone attaches a community note. And I think that actually is like a really ingenious technique for slowing the spread of just false information. So right there it's visible on Twitter of the dual use. Right. Or the sort of dual impact.
- [22:58]
B
Yeah, I think that's really. I actually think about that all the time. No joke. It's funny you say that because I was actually talking to my partner about it today. Yeah. A really interesting specific example of that in action in real time was about two weeks ago, there was an article published across the international media which is about these Thai police officers that had apparently gone in drag to bust the drug rink. And it was back from this picture of them all in drag at the police station, having busted loads of seized loads of drugs. And it was probably three or four days of it running until eventually it came out. The image they are generated and everyone had to issue corrections. The thing that stunned me about that story is that I looked at it and it wasn't actually picked up in the UK press at all that this was the case. But I took the Telegraph's picture that was on their actual website, ran it through SynthID and Synth ID, GroupGemini's inbuilt AI detection tool. SynthID flagged that it was an AI generated image based on the Synth ID pattern. And you're like, well, that's totally bewildering that nobody throughout this whole news cycle thought to just put it into Gemini to be like, hey, this is AI. That tool is free. Anyone can do that. And lots of AI generated content won't have the Synth ID because it might be done from other AI image generation tools. But the Most common one is Gemini or ChatGPT. And ChatGPT also now has the synth ID in it. You could easily use AI to actually combat misinformation, but there's not necessarily the processes in place to do that. That's one element I think is really interesting that people sort of think AI disinformation, but actually AI also is a solution to that. Even that a I kind of disseminated disinformation. And then I think the point about AI tools I think being used to combat disinformation is super interesting because while we one hand talk about the lack of trust in AI generated content. Yeah. I constantly see as you say, grok is true on Twitter and I think people go to ChatGPT, check everything people are going to pile on. But I check medical stuff on ChatGPT and I feel confident to be like, well I'm an intelligent human on top of this. So if it says go to hospital, I probably need to use that information in relevance to what I care about the world to be like, okay, that's fair advice versus on Google, typically medical stuff is just kind of terrifying because you're like, well here's an info page about having some awful terminal disease that might have the symptom. So Yeah, I think AIs are super effective actually providing information and being grounded and also weirdly trusted by seemingly all sides because everyone from all spectrums seems to be like, I don't think they that the Guardian, someone who's like a Republican would be like fat Guardian. Is this story true? They've just innately distrusted. But in a way that I don't understand, everyone seems to just align themselves to believing what an AI tool will say. And my best guess is that there's no implicit agenda within an AI tool that's just existing to kind of react to the questions you provide it so you don't feel like it's trying to imprint on you. And it probably might be. I'm sure there's lots of studies into kind of like right or left wing traces with enhances and stuff, but ultimately people seem to just be happy that it's responding to them the way they address it, which probably will mold to whatever kind of political ideology you have or maybe. Typically I think it does tend to mediate against any extreme. If you're a very left wing person, chat to be may push you more center and same with right wing, but generally I think it engages you on your level, which I think is why people seem to have this trust that they can just Engage in dialogue in a way that doesn't work with the wider news media where I think you constantly really have to align yourself to one political way of thinking to agree with one publications output. And that's probably just to do with the fact that ultimately, and this is me getting a bit philosophical, so excuse me a second. But ultimately the news is ambient and huge. It's all happening all the time. And definitely now more than ever, newspapers report on what Donald Trump said on Truth Social and that will be the main component of a story. They don't need to spend as much time being at a press conference to get a quote. Stuff is just happening all around us. So newspapers increasingly have to choose their segment of reality along a certain polemic of the way they want to communicate that. And I think in the world of AI increasingly I think firstly platforms are already distributing that information naturally in a way that is suited to its audience. Like TikTok or Instagram may serve reels that provide news to people in a way that's very different to historic news production. But I think AI tools for a layer on top of that, where they can totally interpret everything that's going on and produce a really useful answer and explain it to somebody that's quite reassuring in real time. So I think those things just make an AI tool quite, quite trustworthy for the general population. I think that's been more and more the case. But the problem is when you think about it on a sort of theoretical level, if people are using an AI tool to validate a journalist perspective, the kind of game is totally inverted because the person that's trusted to be the kind of source of information is now being corrected by the AI tool, which is kind of madness. So yeah, that's kind of my take on that. A long answer. Apologies.
- [27:40]
C
Well, yeah, I guess my point was more just this just allows us to have a better sense of whether we can trust the underlying kind of call it factual scaffolding of things that are reported. Right. Like to your point, like I, I, you know, you've got to, you got to trust that the, the tool itself is unbiased. Right. So like you know, no one's gonna, at Guardian or like you know, at Teen Vogue, is it true that. I don't know, take your pick. But like if it was trained on Teen Vogue's corpus, I, I don't know that I would consider that to be an unbiased fact checker. But, but assuming you do trust it, like if I'm seeing right, you know, I, I could, it's much easier to do fact checking now. I don't have to go to like a team of people that are going to comb over each sentence and like what? And go search where. Because a lot of cases like there might not be a, a discrete source of like to validate some piece of information. But if I'm seeing something that's not. No, I'm not talking about opinion pieces but if I'm just seeing something that's written as news, I'm assuming that if I can assume now that they used an LLM based fact checker to validate what was written, it's probably trustworthy or at least it hues to the preponderance of information that exists on that topic. And so that should make me maybe give me more trust in the news that I read. And if I'm assuming that these tools are applied to the output, not generated the output, but it just applied to it as some sort of filter and there's just more accessibility of fact checking then maybe I have a better sense of trust for everything that I read. Going from the biggest publication down to someone's substack.
- [29:17]
B
Yeah, that's actually, I think I was thinking about that today and I think there's a sense that you've got AI not trustworthy, human trustworthy, but obviously people are not trustworthy. You have to assume that an editor and my time as an editor, the amount of copy that came across my desk was huge. I didn't have time to check. Okay, let me go and Google that this city actually is in Italy or whatever. I'd just be like cool, well that looks fine. Journalists presumably knew what they're talking about, so no spelling mistakes, let's go. Don't quote me on that. Obviously you check the stuff, that's high stakes, but you don't have a kind of constant rolling fact checker. But as technology changes, I think in most sectors you have an expectation that if a tool is available and it's capable, there begins to become a dynamic where not using it becomes the negligent thing. If it refused to use a CT scanner or didn't do X rays, I just don't believe in that. I'm an old school orthopaedic surgeon. They'd be like, no, what? Use the frickin technology. Otherwise if something goes wrong then that's suddenly a culpability issue for you. And I feel like it's very feasible that if this is not already the case, that could become the case with journalistic publishing work because you've got A grounded search on Google Gemini Pro or something, just giving examples. One that we use for one step of our verification. It is incredibly capable at returning information and not just information that is based on training data. It will find search based validated information in real time and be like, okay, this 100% is true based on this source or this actual primary evidence based on what someone said on X or something. And there are gaps there and you sometimes have hallucinations kind of cascade through news media but even then there's points where you could be like, well then develop an anti hallucination validator to go find the original source. And there's steps that you're like, you can actually kind of introduce tiers of validation and verification that would be totally impossible for any sort of news team unless they had a crazy budget of hundreds and hundreds of heads. So. So in that sense I feel that yeah, AI can and will ultimately make stuff like a higher bar of verification and truth when it's used in a sort of intelligent and well deployed way.
- [31:17]
C
Right. And to your point, you might even want to disclose the fact that you didn't use it. Right? And again, not the Teen Vogue fact checker LLM, they're coming to you.
- [31:27]
B
That legal team is super litigious.
- [31:29]
C
How dare you defend ZBO or the Breitbart at Breitbart LLM fact checker. But like if you didn't use any sort of LLM based fact checking, you might want to disclose that like no LLM was used to fact check this article, then I would probably be less, I would be more skeptical of what was written there. Like that it wasn't at least validated against some common terminology or common reference metrics. If it was using some source that was seen to be biased in some way and that wasn't flagged like that to me, I would want to know that because I just assume now know and maybe it's. It's an invalid assumption, right? I assume that anything I'm reading on the Internet has at least gone through a pass, right? Like, like I, I do it like check, give me spell check and grammar check at the. At a minimum. But like just to. And so like again I, I just do think that the widespread availability of these tools should fortify a lot of the stuff that we read. And you know what I do is I've got one of my prompts that I'll take an article through before I. Especially if it's like a more controversial topic. I re. I read this adversarial. Tell me the top five criticisms that someone would have against this if they were very hostile to the ideas that I'm, I'm articulating here. And like that's really helpful because it like it helps to maybe soften some of the edges or like take to sort of like deflate some of the more tendentious, you know, things and like just make it less likely that someone's going to explode when they read it. And I would imagine that you just see that effect. And so like I don't think that that's highlighted as a benefit like the widespread availability because again people focus on the output. But if you focus on, focus on just these tools as a layer that the human written output passes through to improve them, I don't think anybody could argue that that is a benefit.
- [33:12]
B
Yeah, actually it's funny because we, the Vaura platform, we have a whole pipeline of from article idea to verified final CMS ready content. But we have actually developed a whole separate pathway which is manual writing. And in that pathway you just write into our platform. But as you go you can select text and you can say challenge this or find a new angle or fact check it. And it does against research document that exists against the topic. And that has been super interesting because I'm like, I don't think at any point you can say you have an objection to that in terms of AI interfering with the process because you can take or leave the advice. You can be like, cool, that was your perspective. I'm going to ignore that. But typically it's super effective at being like you only have your own perspective in writing. It is effectively someone else like a live real time editor. Saying that doesn't really make sense, does it? Don't you think that people are going to question whether that's a valid argument in light of this? And I find that's really super hard helpful. As I'm writing stuff, like more analysis type stuff to be like, okay, highlight this, challenge this, angle this. And I feel like this could become the standard way to ultimately see writing reach a new level of quality. And I think that people might be squirming and hearing that, but I think we think about AI tools like helping coding or whatever, like CLAUDE code or coding agents that really accelerate and in a way that most people no longer have an objection to. No one's saying, oh, Claude code, just create slop. It's like most of the Internet now is being built with CLAUDE code. And then, and I think writing could be the same. I think as the technology develops, I think we'll see tools that actually augment writing in a really positive way and actually lead to writing being better. Because you're giving people that might not have had the experience of having like an editor or like someone else's feedback suddenly having an LLM tool to say, this could be a bit better. Have you thought about that? And you know, most seasoned writers have had that in human form. They've had someone go through and say, like, you need to tighten up your writing, we need to knock the edges off this. And that's like a really key part, I think, of maturing as a writer. And I don't know many people, people that wouldn't have had that at some point in their careers that have suddenly gone on to be very, very accomplished or technically strong writers. So I think that's why it's a real positive. Obviously the flip side is that when one editor, and this is the getting of it, when one editor is very kind of edit heavy at a title, everyone's content tends to conform to that person's writing. And I think the same thing can happen with ChatGPT. If the whole world is using ChatGPT as their de facto editor, then everyone's writing will naturally Trend towards what ChatGPT inherently thinks. Good. But I think there are flaws there and there are risks, but I think there's still a lot of scope for originality and for just treating an LLM as a really healthy adversarial or supporting writing assistant or co writer.
- [35:43]
C
Yeah, right. I'm not talking about give me advice on the pros. I'm talking about fact check this and give me a sense of what the holes are in it.
- [35:53]
A
Mobile game developers no longer need to pay up to 30% in major app store fees. With Xsolo, you can create a direct storefront, cut fees down to as low as 5% and keep players engaged with bundles, rewards and analytics. Start today@xsola.com that's x s o l l a.com or use the link in the episode show notes.
- [36:20]
C
I want to talk about the Valora platform because I think that to me like what you've built is I think like if you just, if you just kind of like had a one liner, like a, an elevator pitch for the, for the platform, like kind of layman would probably say, oh, you're going to help people, you're going to contribute to more AI output on the Internet. And like that's, that's absolutely not what the velour platform is. I mean, you know, you gave me access And I've used it. I want to talk about that because I think what I'm impressed by with Velora is the capabilities with respect to, you know, finding, processing, classifying, sorting, ranking, news and ideas as a tool. Right. And that to me is what an LLM is truly good at. LLM does next token prediction. And it, it generates generative, it generates text. But I think a lot of the sort of a misunderstood value is just doing all those things, classifying, you know, ranking, sorting, like, you know, summarizing that kind of stuff is the real value to me. And talk to me about how, how the Velora platform is used as a publishing tool, because I think that's the piece that I think people don't appreciate about the power of AI in publishing. And we talked about one piece just now is like fact checking. Right. But talk to me about the whole suite, like the whole sort of like end to end use case of this stuff that has nothing to do with generating text.
- [37:32]
B
Yeah, definitely. Well, I think you have to accept that ultimately whenever you think about AI tools, you can take objections to them and say, I don't like the idea of this, I don't think it's trustworthy for writing copy. Which is all totally fine, but there's some things you can't debate an AI tool is better at. And one of those things is certainly taking a huge amount of information and quickly processing it and understanding it in a very top line way. So you know, news discovery in terms of like signals and understanding of what is interesting to your audience. You can ingest like a thousand sources every hour. And an AI, typically a person might scroll their competitors and their beat might look at social post, but they might have, you know, 30 signals or sources. An AI can look at thousands and it will sort in an intelligent way. This is what is relevant to you at this particular moment. These are the most kind of high interest topics for you to pursue. As a starting point point in my mind there's just no competition there. There's a value in someone looking through the whole kind of news of the picture, but you can't possibly do it as well as an AI tool. Now another element you can't do as well or perhaps you need to do in sort of symphony is kind of research. And I think definitely there's kind of like there's value in doing your own research. But as you said, you can't look through dozens and dozens of sources, hundreds of pages of PDFs in, in five minutes. It's not something that a human being is able to do. If you had enough time and resource, maybe. But ultimately there's a limitation that you simply cannot scope this much information. You cannot draw so much research together. You can't understand the picture widely like that. So that is something that I think an AI tool again is fundamentally better at. In symphony of the person then at the other end, the pure manual processes so well. Yeah, finding an image that's relevant, that's like a long personal like train time thing. Populating fields in a cms that is a very laborious kind of menial task that people have historically done. CMSs have been built because people have to do them, have to populate these fields. They have to say, here's a title field, here's a subhead field. When you really think about it, that doesn't really need to have any human interface. If an agent or an AI tool can understand that, that's where that copy needs to go. We only created a complex infrastructure to enable someone to put it there, because there's no other way of taking copy from here and putting it there. Like when an AI can do it, you just need to abstract that step entirely away from the person looking at it or doing it. Something like SEO or alt text. It's all just like kind of menial work that people end up having to do. And that final level, just like creating a validation there that's really, really robust on top of human validation and then actually creating distributions. So say like creating publishing a piece but then creating social assets that accompany it, creating suggestions for social copies, stuff like that. This is all stuff that takes a long time. But very few of those steps really add anything at all to the creative process that people actually value in journalism or piece of writing. It's just a lot of unnecessary surrounding labor and noise within that. In the central part of the platform, we do have drafting, but that's totally down to an editor or team's discretion whether they decide to do something manually by using an AI augmented writing tool that will challenge or make suggest angles, or they just ask for a first run draft. Within that first run draft, we also have a kind of grill me phase. So it'll be like the AI tool will come back and be like, what are you thinking for this story? What's your actual angle? What are you trying to focus on? There's this bit that was in the interview, there's this bit that's in the PDF document, what's actually important to you. And it will go through seven or eight steps of that and then it will then kind of go away to form a more cohesive brief that you can look through and then it will go through to a draft. And at that point, some of that content that I've gone through has been kind of heartbreaking because I've done that for a feature. I was like, okay, I'll try this grill me process and I'll use all this information. And this is the final version. I'm like, wow, I don't really know what I can do here because it's just like it took a long time to prompt it, maybe 20, 30 minutes of putting information. But when it came out, it was exactly what I'd expect, but in a way that was super neatly stitched together. And then it's like, I can change a bit of wording here or there, but I'm not really adding anything, it's just customizing. But when you've added that level of information, the draft you'll get back is super high quality. And that's probably what gets lost in the kind of one shot slop kind of AI writing world, where you've got someone just being like, Boom, here's a LinkedIn post that says nothing and means nothing versus here's a lot of information, here's a lot of context. Then there's an iterative writing process. So we kind of do all that and we do that for multiple types of content like news or features. And the idea is that you kind of give time back to the journalist to do something interesting. And that's kind of the key core principle. What is interesting is not whether you spent 25 minutes writing a quarterly financial report or two hours in the morning looking through a thousand sources to find out what was the most interesting bit of information in the morning in the kind of like the Canadian fishing beat. But I think there's that stuff that doesn't really add, but that's interesting is going out into the world speaking to people, actually looking at whole topic and being like, I think the most interesting angle here is this from my lived experience and actually adding something human to it. And I think we often think, okay, well the human bits in the writing. But if it's totally formulaic, programmatic writing anyway, someone isn't really adding their perspective, it's just a drain on time. So we have to be realistic about is that something someone should be doing. And I think ultimately that stuff people will be doing less in the future. Whether that's because of AI or whether because that content just becomes less valuable because it didn't really Take any effort to create. It's not scarce in any way. That's the kind of interesting question that I think is going to surface in the next few years. But I think journalism will have to get to a point where people do more interesting things with their time. They actually find interesting stories, they have interesting conversations, they form interesting perspectives. And without that I think there's no real moat against someone just churning out absolutely tons of content at scale using an AI tool without any sort of, of high benchmark of verification, interest or journalistic ethics in it.
- [43:12]
C
Yeah, and I think that's something that maybe just people that have never kind of written as a professional activity just miss how much reading is involved in publishing something. There's a lot of times just now I'm writing about Apple's new foundation models framework and I had to read two technical papers just now, this morning. Instead this blog post will be 800 words. But like it took me, you know, it takes 45 minutes to and to not read a technical paper. Takes a very long time to read but like just, just kind of scan it and get the big picture idea. It's very, very, very time consuming. And you know, with a technical paper I don't really trust just dropping in a chatgpt. But like there's a lot of stuff like, like an earnings report. Like I, I, yeah, I could spend 45 minutes reading this whole thing word for word. I could drop it in chat GPT and say hey, I'm looking for these pieces of information, tell me where to find them and give me three anchor words that I could search for to go exactly to that point and then skip all the questions that I don't care about and it's just so much more efficient and maybe I get to write two earnings analysis in a night instead of just one because I didn't have to read both earnings call transcripts word for word. And that's not AI generated output. There's nothing to do with actually generating the words. It's more about out inform me and give me the latest information as fast as possible so that then I can ingest that and then write my analysis right like that, that, that true and then you know, find me new information that could add more context to my, my understanding of this topic. Like that that is really underappreciated because like how do I like is this unknown unknown? It's like where do I even go to look? And if there's a tool that can, that's just been, you know, knows it's got a huge bank of, of resources that I can scan basically every 10 minutes to find stuff that's relevant to me. That's just helpful, that's all. It doesn't generate the output. It makes the writing process more efficient. That really, I think, is the true advantage of using these tools. Right?
- [45:10]
B
Yeah. And that's it. And to like, kind of visualize that a bit more clearly for us, that's like we have in the platform, we have sort of an RSS and a web kind of scraping section, we have a social media section, we have a YouTube section. And that stuff's all this poll kind of super frequently. Some stuff might originate from international press or a different press in a different sector might originate from a competitor. But a lot of it's just social media that's happening, like YouTube transcripts, finding a very specific part of a long podcast that's actually really newsworthy or an interesting starting point. And yeah, that stuff that is just generally impossible to do in a personal level. But what's cool about the platform is that the journalist decides what the sources are. It's not just saying, hey, find this interesting story. You're in charge of saying, I think these international outlets are really important. I'm going to over index them. I think these social accounts, the ones I really want to follow, these are the YouTube channels I want you to look at every single day and tell me what's coming up there. And that's where you get a bit more like originality and curation kind of put back into the process compared to like, you know, just the sort of a general sort of sweep of saying, Claude, tell me what's interesting.
- [46:09]
C
All right, I know we're at time. Take this moment to promote the Velora platform. How can people test it out? How can they, how can they add it to their writing workflow?
- [46:16]
B
Yeah, so we're just like Velora V E L O R A build and I'm Peter Stuart on LinkedIn. S t u A R T and you can just come and add me on LinkedIn. Ask to use the platform. We let people use it. We're still kind of in the early phase. We want people to use it and feedback and tell us what they like or hate or thinks. Really good. So, yeah, just reach out and I'm happy to do demos and we're also happy to like, work with people on their specific use cases because we're just super fascinated in how this scales out to someone working in, say, fishing or someone working in bird watching or someone working in super specific insurance financial stuff. And so far we found that it's actually able to help people in almost all cases because some of these tools just are kind of totally title sector agnostic, which is really fun. So, yeah, reach out to me and please don't hesitate. I'm friendly and I'm always happy to have a chat. And even if you don't want to use the platform to have a chat about AI, my favorite topic, AI and journalism. So yeah, don't be a stranger.
- [47:06]
C
Thank you, Peter.