Loading summary

Transcript67 lines

[00:00]
A
The CMO Confidential Podcast is a proud member of the I Hear Everything Podcast network. Looking to launch or scale your podcast, I Hear Everything delivers podcast production, growth and monetization solutions that transform your words into profit. Ready to give your brand a voice then visit iheareverything.com welcome to CMO Confidential, the podcast that takes you inside the drama, decisions and choices that go with
[00:30]
B
being the head of marketing.
[00:32]
A
Hosted by five time CMO Mike Linton. Welcome marketers, advertisers and those who love them to Chief Marketing Officer, Confidential. CMO Confidential is a program that takes you inside the drama, the decisions and the politics that go with being the head of marketing at any company in what is one of the most scrutinized jobs in the executive suite. I'm Mike Linton, the former CMO of Best Buy, ebay, Farmers Insurance and Ancestry.com here today with my guest Neil Mann. Today's topic, AI Agents as Teammates. Are you really ready to work together now? Neil is the CEO of Known, that's N o a n. A company that creates an AI native platform that helps businesses organize their data into a fact layer and an API for both humans and agentic agents. Previously was head of Global transformation at Anomaly and an editor at the News Corp. Of Australia, the Wall Street Journal and Sky News. He's been an editor of all kinds of new stuff. Welcome to the show, Neil.
[01:44]
B
Thanks for having me, Mike. Looking forward to chatting.
[01:46]
A
All right, let's set the stage for our listeners. What exactly is an AI agent? And how do I I know the difference between an agent and just a series of prompts?
[02:00]
B
Yeah, it's a great question. And you'll find in the market right now, people are confusing language, even the likes of Microsoft using the word agent, when it's not necessarily an agent. A good way to think about it is to break it down into two. Two areas, really. One is assistants, which respond to you. So you make a request and it comes back and responds to you. And then an agent actually takes actions on its own. And you may have set it a predetermined goal, but it will actually take actions. Use what's called tooling to execute things for you and then come back in a feedback loop and work out whether that worked or not and then do it again. And that's a good way to kind of separate them in your mind and they're best used together. You know, you might use an assistant to actually set up and launch your agents, but you need to think of them as kind of two different things.
[02:48]
A
Two different things all right, so tell us, when you watch teams build and use agents, how do they decide what is going to be the agent? And then how do you put the agent into a workflow if it's not just like Neil's agent or Mike's agent, it's an agent for a group of people.
[03:10]
B
I think the first thing to think about is actually to take a step back because the majority of people jump to execution first. They're like, we want something to do this. And as a result they'll try and jump to a short term solution that might be a little bolt on. Whereas actually if you take a step back and go, what are the most important processes that we could try and automate? And then how do we close the gap and do that as easily as possible? Because one of the things that happens with AI when you've got the right structure is you can really rethink things from a first principles perspective. What are we actually trying to achieve? What's the shortest pathway to get there? And then build your agents on top of that. And typically, you know, in marketing you're going to see people creating things that create content quickly. Emails, maybe an agent that goes and finds their dashboard and gives them their data. But if you take a first step back and say, hey, what are all the processes we do? And then clearly think through the ones that you could really streamline and then build your agent for that.
[04:11]
A
Hey, Neil, can you give us a good example of a company that has, or even a hypothetical if you have to blot out the company where, you know, they have built agents and they've incorporated it into the workflow?
[04:27]
B
Yeah. I think on LinkedIn, a really good person to look for is Jason Lemkin. Right. So Jason Lemkin is an investor, he runs sasta. He's constantly out there talking about the agents that they've built. And, and there's a couple of interesting lessons for me around how he's done it. So they have one agent they call 10K, which is their marketing agent. It pulls all their marketing data daily. It comes up with ideas for them, and he's very open about talking about how he's using that. Now the one thing you see though is he also talks about the challenges he's seeing. And one of those is that they've built multiple agents and as a result they're constantly updating each one. And that right there is the problem that we solve for because a lot of people right now, they're making ad hoc agents all over the place. And actually what they really need to do is rethink their business so that they have a single source of truth that every agent runs from. And that's actually, for me, it's really interesting watching him do it openly. And if you haven't seen it, we'll put the LinkedIn in the notes. But it's well worth looking at how he's openly talking about building it and then also seeing some of the holes in the way that they're thinking about it.
[05:30]
A
So I think this is really good. Plus, you keep reading stories about how agents sell all your stocks when you're not looking or erase all your code because they're like, this code sucks. So I'm going to erase all the code and start again. So when you're a leader, if you're out there, you want to do this, you got a team that by definition is probably in various stages of. I love AI. I'm nervous about it. How you even set up the collective team to work with agents and, you know, design outcomes, measures and collaboration. You can't just say, all right, we're going to build a bunch of agents and you guys figure it out. I guess you probably could. And some companies have done it, but that's probably wrong. Tell us what's right.
[06:15]
B
That's going to be messy. The most important thing, if you're a marketer, there's two key things. Actually, the most important one is understanding structure. So marketing in particular is often a very scattered part of the business. Things are being made all over, all over the place. There's lots of creativity happening. The challenge with that is that when you're working or want to work with AI, you actually need real structure for AI to reference.
[06:41]
A
Hey, Neil. And when you say structure, there's data structure, IT structure, like the organizational design structure, decision making structure. Do you mean all of that or just pieces of it?
[06:55]
B
Let's pick off the one that most marketeers understand, which I would say is like semantic structures. They destruction of the language and the things that you use. If you think about marketing in particular, everybody knows and understands the mission and vision. It's a concept that's clear to everybody. Everybody knows and understands brand positioning or brand experience principles. These are concepts that are clear to team members. But the problem in most marketing departments, and I guarantee this is the case for pretty much everyone listening to, is that if you quickly searched within the Google Drive, there'd be 30 different versions of your mission and vision, There might be 40 different versions of your brand positioning. You can't guarantee that, Dave, in the Email department knows which one we're actually working from. Now, the really important thing to understand here is that those concepts are things that AI also understands. It understands what mission and vision is. It understands the impact a mission vision can have on content or things that are delivered in the marketplace. So as a result, the first thing you need to do as a marketer is get a clear structure and a single source of truth around all the key concepts to your area of the business. And the second part that for me is really exciting if you're actually in the marketing space, is that traditionally, you know, I worked in transformation for the best part of nearly 15 years, and it would always start in product and engineering.
[08:12]
A
Right.
[08:13]
B
Whereas the really interesting thing about this transformation is that it's driven by language, because who, if you own the messaging, you can then implement AI within the business. So marketers all of a sudden are actually now in the driving seat. It's one of the reasons you see a lot of the uptake. Yes, you see it in coding, but you also see it in content creation, marketing services of AI. And so those two things are what you need to understand. One, get your context in order, the areas of the business that you own so that it's referencing just one of each of the core things, and then to like, actually realize that you're in the driving seat when it comes to implementing it within the business.
[08:49]
A
And this says I have to collaborate with all my, all my partners across all the business to make sure this is pretty clear, right? Because this is, you know, when you think about big companies where this stuff could be all over the place in multiple databases, this collaboration thing is a big deal, right?
[09:08]
B
It's everything. That's literally everything. That's what it comes down to is you need, you know, we call it a business brain. You need a business brain built on a single source of truth that are like LEGO bricks. Each of those is only one. There can only be one brand positioning, one mission and vision, one pricing structure. You need that organization so that when you're then working with AI, it's actually referencing the right pieces of information and all of the team, when they're using it, are referencing the right piece of information. The biggest complaint I've heard, I've been talking to a lot of Fortune 500s. The biggest complaint I've heard is team members making random stuff using ChatGPT or Claude directly. That's off brand. That's referencing something old. Now this gets worse when you consider that marketing often has to work with sales in a B2B and sales is
[09:53]
A
making all kinds of stuff up, all
[09:54]
B
kind of stuff up from old PDFs from five years ago, you know, and, and that right there is the problem you need to solve for. And if you're a CMO right now, you need to say, hey, where is our structured source of truth? And let's organize it. Now your version one for that is making sure you've got one Google Drive with the right elements in it. Version two, people use markdown files, which you're starting to see. And then we've obviously built a platform where we just extract that and it's all on an API that everybody can build from. And that's your kind of holy grail that you want to move towards.
[10:27]
A
So in some companies, this is like looking at Mount Everest when you think of all the different databases. And you just wrote an article, I think, for Harvard Business Review where referenced AI slop.
[10:40]
B
Oh, that's not me. I sent that to you. So.
[10:43]
A
All right, well, somebody wrote this article and you sent it to me. So I take it back that you wrote it. But you actually, I want you to talk about AI slop. And, and because I think it goes back to what we're just talking about in the infrastructure and what you get if you don't do this right.
[11:03]
B
So the, the problem if you don't do this right is LLMs by their nature are going to hallucinate because of what they are. They're ultimately probabilistic maths machines. They're going to hallucinate and make things up. So as a result, the problem you have is if you're at a business where, let's say you've got a team of five people and two of them are working from a single source of truth, they're going to both get pretty similar output and it will be accurate if it's a real source of truth. If you then have three people who aren't, they're going to start to create things that are drifting or off brand that are different now add another 100 people into that and then a thousand. If you're like a, a global company and you've got ultimately like slop being made everywhere. And then the problem is a lot of people aren't necessarily reading it, checking it, that finds its way back into your, your single source of truth, your reference, and then it gets worse and it just goes from there and before you know it, you're a complete mess.
[11:59]
A
So the plural of slop is super slop. So the bigger you are, the more slop you can have, hey, so how do I know if I have slop? Like, because I think I've. If you're doing the right thing, you're training people on AI before you give it to them, you're having use cases, then you're rolling those out through the company. You've done all of this, right? But when you look at things, how do you know it's slop or it's not sloppy?
[12:26]
B
So I think the challenge here is if you've actually done it right and if you've really approached it in the same way. We think about it where every single one of those LEGO bricks is on an API that's referenceable by AI and humans. As a result, you can then check against it. So you can run checks, check it against my brand facts to make sure that it's on brand. You should have built a system that allows you to cross check and reference to make sure that something is right and accurate. We've put a lot of time into building that internally. There's ways to kind of hack it together within your departments, but you want to have consistency checks so that you're not letting things get out there that are sloppy ultimately.
[13:04]
A
And do you need a little audit team to do this, or, you know, a QA team? You know, what do you, what do you put on this?
[13:13]
B
Well, so this is the.
[13:14]
A
Has to be objective, because if I'm judging my own thing, I'm going to call it not slop. I'm going to call it beautiful.
[13:22]
B
Well, in the, in the AI first world. So for a company like ours or companies that use our platform, you can automate that and run it, because all those LEGO bricks are there to check it. Now, this is the kind of job that you could build an agent around. You could say, I'm going to build an agent that checks output every single time against these facts to make sure that it's accurate and not sloppy. And then you may want a check yourself to check it. And I always do. I think humans in the loop is, is a really important thing when, when working with AI. But if you're working at a big company and you don't have that kind of structure, number one, it's going to be hard. But number two, you need to think about it like an editor, right? Like, I came from that space originally in my career, editors were there to check. And I think what you're seeing a lot now is content creation has just become so easy for people who couldn't traditionally do it. And so as a result, everybody across the organization is starting to create content.
[14:13]
A
If you have some air traffic control,
[14:15]
B
everybody is doing thing, everybody's doing everything. Multiply that by everyone's got a LinkedIn profile that they want to post on. All of a sudden it gets out of control and you need some sort of editorial checks against that. You may have to actually, if you've not built an agentic business yet, you may have to put those in place in a more human way. But it's really worth it because I think one of the things we're seeing now is that particularly as CMOs like we talked about, they own the language, you own the output, and your brand is represented. And people are very good at spotting AI slop these days. And that can look pretty badly on your brand.
[14:50]
A
Hey, so one of the things we talked about pre show was the documentation required to actually set this up. You know, that leads to clarity, that leads to all this other stuff. Tell us about the difference between good documentation and crappy documentation.
[15:11]
B
So I think a really important lesson for everybody is the structure is everything. People have been told that AI is good with unstructured data. You know, you can throw it at a document to make sense of it, which it can do, but it thrives on structure. So what you want to think through is, is imagine your business like the brand positioning, mission and vision, all of these different elements, they're just building blocks that AI can then put together and leverage. So you want to actually think through how you, how you structure that in an almost file like way, which for a lot of people I think is a big difference from the way that they traditionally had decks and PDFs. You know, coming from the marketing space, how much of your life is, is spent making decks. And now you need to think about breaking that out. So it can take time. We, we have a model that extracts those facts for you and then you just verify them and it's dead easy and you can get set up in minutes. But what's fascinating actually is when we look at the usage data, the amount of time people spend managing those and tweaking the language. And I think this is really interesting as a marketer, because back in the day you couldn't guarantee that your brand positioning would come to life in your content. For example, right? The massive disconnect did Dave in the email department or the blog.
[16:26]
A
You like this Dave guy, don't you?
[16:27]
B
Dave. Everyone likes Dave, right? And then could he actually execute? Whereas now the AI will reference that and imbue it in the content. So for the first time you can actually guarantee that the language actually manifests in the world in the right way. So people spend a lot of time managing and tweaking those facts. And so you don't want to rush it, but you don't need to spend, you know, months getting every fact right to start with. You need the core things that you can execute. And then I always recommend to people to think through it, like transformation, because it's corporate transformation. Ultimately, it's the biggest transformation your business will go on. Start in marketing, then maybe bring in sales, then hr, and then all of a sudden you can start to spread across the business.
[17:12]
A
Okay, so now let's say I have these agents, my team, I have the structure, things are working. How do I think about pay plans and communicating with a team that includes agents? And how do I know if I have a large team? How do I really know if my team is collaborating well with the agentic agents in the structure?
[17:38]
B
So I think a really important thing to consider is to not let everybody just bang away on AI themselves. Right? Right now you've got people all over the place making things across businesses, often doing it. There was a report out this week that people were doing it behind the scenes, didn't want to openly acknowledge they were using AI. The first thing is you need to be transparent with people that this is our approach, this is our strategy to leveraging agents in the workplace. The second is then if you've built your company as an API or you have that single source of truth, you can start to measure what's actually happening and look at roi. So on our platform, for example, we have a new metric called knowledge work replaced, where I know when you do a certain task or you create a certain asset, how long that would take you in the real world. So you can then, as a result, show people how much time they saved. And obviously there's a monetary value that can be attached to that. So you want to make sure that you're tracking it. Because, you know, one of the things that I think a lot of marketers are probably going to get a shout from the CEO about is, hey, we want to make this amount of savings, for example, or make us X amount
[18:40]
A
we have to return. You know, a bunch of our guests have said the reckoning of ROI is coming. You can't just spend all this money. You're going to have to produce results. While a lot of that may be cost savings, some of that hopefully is growth. And I didn't mean to interrupt you, but yeah, go on with this whole story about all Right. How do you check on it?
[19:01]
B
Yeah, so I've heard that multiple times I've heard people being, you know, CMOs have said to me they've been told they got to save 20 million, 50 million to your point, drive growth with AI. So what you want to make sure is everything is trackable. You need to know what was deployed, when it was deployed. Think of it like a system, not one off integrations. That's the way you need to think about your business, rethink what your business is ultimately. Because if you think about it as a brain that everything runs from, then look at how do we track what assets are being created, how do we track how many times people were using it? You want to actually be able to track that in a, in a large scale company because otherwise you're not going to measure any roi.
[19:38]
A
Can you give me any example of how you might like, it could be a hypothetical company or a real client or anything, how you might set this up and say, okay, here is my center brain and how it works.
[19:53]
B
So for us on our platform, the first thing we do is we run an extraction model that takes the facts from your documents and starts to set you up. Then you can start to manage those yourself and add them. And what you're doing there is you're actually coming out of the document space and you're building those LEGO bricks in your brain and then they're ultimately on the API.
[20:10]
A
So give me pick any hypothetical like a SNAP CAC company or a bank or a credit card company and say, all right, I'm gonna go get all these documents.
[20:20]
B
Yeah, so if you took like let me think, you know, we're working with a Fortune 500 that I'm off actually off to after this. We're working with a Fortune 500 in the kind of legal space. First thing you want to do is get the centralized documents together around, let's say with brand, right? Because your business is ultimately broken up into key, key sectors, key sections. So let's get the brand documents together. Let's then extract all those key elements of that and the key facts from it. Let's verify the ones that are true today because there's ultimately going to be things in there that are actually old. And then, then let's get those set up on the API. Now as soon as you've done that, you can then deploy that API and run agents from it and you can start doing content creation, et cetera, in, in minutes. But the first thing you need to do is just get that structure. The second Thing is also just making sure that those facts themselves are well structured for AI. And so I always tend to start with brand. And then you want to go into how the product works, product features, for example, these are key things that once you've got them stored, AI can then reference to do things like customer support. But you need that actually within a platform that's actually referenceable by AI and humans.
[21:32]
A
Got it. So let's say I look at all my stuff and I have to fire an AI agent. Someone's probably really attached to that agent. Give us some tips on firing the agent and managing the humans around it.
[21:49]
B
It's a really interesting point because people will get attached to them and they will think that something is doing a good job for them.
[21:54]
A
Well, and if they built it, they're going to love that thing, even if it's no good.
[21:59]
B
And, and the reality is like, agents can be expensive, right? So we're seeing this right now. We saw Uber pull back on its coding, AI coding. So I think the most important thing, and it's funny because it's almost like humans, right? The most important thing is what's the ROI to the business of this agent? In the same way you think about an employee, are they delivering for the company? The second thing you're then going to go is, what's the cost of this agent? Just in the same way with humans, you would look at like that cost benefit analysis. And again, this is why tracking is so important. Otherwise it's just arbitrary that you're deciding something's not worth using and then you just, you can nuke it. And the beauty is you can easily turn it on and off. You know, that is one of the great parts of it. But I think the one thing we need to be aware of is if you do not, and this is such an important conversation to have right now, the majority of token usage. So AI agents, assistants, whether you're using ChatGPT or you're using agent platforms like Paperclip, etc. They're running on tokens. And the reality is the majority of token usage right now is wastage. And so a good way to understand this is if you think about you've had a long conversation with ChatGPT and then you find yourself scrolling back through trying to find the one piece of information. Everything you had in that conversation was wasted tokens. So it's really important to think through, like, if you're going to deploy AI agents in your business, the fact layer is the most important part because they won't waste Tokens because they're going to use the right facts at the right time. So as part of that ROI conversation, you need to look at like how have we structured this and set this up and have we set it up for the success of an agent or have we actually set it up for them to fail? In the same way with humans, if somebody joins the company and you brain dump on them a terrible load of documents that are a mess and expect them to do their job well, they'll fail.
[23:46]
A
Yeah. If the job spec is whatever you think it should be, it's probably not a good job specific. So you can't give that to an agent. So tell me. So we, we have all our listeners out there, some who probably are just getting started in this other than hiring your company, give them tips for how they should get started with their first agent and onboarding the agent and everything else. And you know, if I'm, if I'm the first step for my whole company, give me some tips.
[24:20]
B
So again, it's funny, you know, as we're talking about it, the more I think this is so much, it's so close to just onboarding a human. Right. Like the first thing is think about how you're going to onboard the agent that you want to use.
[24:31]
A
Never ask for extra vacation time though.
[24:34]
B
No. And what you're going to find is they're probably going to overwhelm you with responses. Right. So like that's part of, it's like how much do I get back? I was talking to somebody yesterday who said that their agents are constantly firing, firing messages at them in Slack and it was just overwhelming and they don't, they don't need it. So I think the first thing is actually think of them a bit like the way you would think of a human. Right. And you'd say, okay, well I would, I need to give them solid information about what I want them to do. I need to give them solid information about the company that's well structured. And then you need to go through, thinking through your use cases. What am I actually going to try and use this for? And I think for most people one thing to understand is that because an agent can take multiple actions, it can gather context in different ways. It can be multi step. Now those can be overwhelming to start. So you might want to just do
[25:25]
A
something that's just an example of a multi step.
[25:28]
B
So I for example built one that would search when it found a company. We would, if we were finding a company we wanted to work for, it would work with. Sorry One, if we had a company that we wanted to work with, it would search the Internet, look at the company's house records in the UK or the company records in the us, try and find who the core founders of that company were, then go to LinkedIn, look through LinkedIn, find those founders name, match them, then find their email, then draft them an email, then send them that email. So it's about like probably 15 steps with multiple steps broken out and then it could just run. And so it can be quite a complex workflow. And the challenge is if your instructions aren't good, it can break down. So one of the things to be aware of is as you think through working in this space is again, just like with humans, clarity of instruction is the most important thing.
[26:25]
A
And this, but these will never push back on you, they will just do more work, right? So you have to, you have to really like probably manage them differently that way.
[26:36]
B
It's, it's been really interesting. So I've had users say to me, you know, people who've been in business for 20 years, they've said, wow, I've realized that my communication is terrible because I'm vague. I use wishy washy language when I'm trying to instruct an assistant or an agent. And as a result the response isn't great. And the default for people is to blame the agent or the assistant in the same way. In the office space, we've all seen it, somebody gives terrible instructions, everyone leaves the room and is like, did the CMO say that? Like what we supposed to do? So it's very similar, but a lot of people said, wow, this has made me be clear with my instructions to both humans and agents. So it's really important that you think through the approach and a good thing to do is to ask an assistant and this is how you can use assistance in this. How should I best structure this instruction for an agent? And it will break it out into the relevant step. So use AI as your partner in that space.
[27:28]
A
Super interesting. Tell me what this means for like all the added ad and media and other PR agencies out there. Is this, how is this a threat to them? What is this?
[27:42]
B
It's a really interesting question. I think it's, you know, if we, if we look back and just to rewind, if we look back, there was a fundamental shift maybe 12 to 15 years ago in the marketing space where content marketing started to become a big thing, right? And all of a sudden people started to see that, you know, they needed to create content. And what you saw is the agency started to create content marketing arms and they would create content. And then all of a sudden people started taking that in house and you got the big brands bringing the content creation in house, which is where it sits for many now, in the same way, I think what you need to consider here is that what's your role as an agency and how is AI helping you when it's working with clients, maybe streamlining your operations? You know, we've got agencies using our platform, for example, because there's a fully integrated network, so they don't need a cr. They're using our platform to run all of their business development. And then they have every client in the platform and all of the strategy of the clients in the platform. And to give you an idea of how powerful this can be, you know, just, just before this call, I integrated the known API into Lovable, the website and app builder, and asked it to build a website for a brand that we've got in our platform, a demo brand, and with one request, build me a website based on this API. And it built the website in five minutes. Now, why that's so exciting for agencies is that you could be doing a pitch and you could have all of the team working from the same source of truth as you're pitching and as you evolve the strategy, which happens during pitches, right. You're changing and tweaking. You could bring to life 20 different apps, 20 different websites in minutes, all from the same limitless creative, too. Limitless creative, all from the same fact layer. And when I, when I saw that happen, I thought, wow, this is, this for agencies is potentially huge. So I think it's about using AI in the right way and working with your clients in the right way with it. Use it for your business development and use it for your, your creative, your pitching, your ideas and getting things to market faster.
[29:44]
A
Excellent. Thank you for that. Hey, so before we get to our traditional last question, I have to ask you, most common mistakes you see in the marketplace now, you've already best practices, but let's go over the mistakes.
[30:00]
B
The most common mistake I see, honestly, it's the same mistake that you see from cmos when you're in the room with them. They're too focused on the execution. They are, you know, a lot of people, I think, in, in the marketing space focus on the execution. And so, you know, you're pitching and it's, it's very early and you do a pitch and they jump on one idea and they don't like it. They're focused on the execution. You see the same thing in the marketplace. People are too focused on trying to look at AI execution and output rather than focusing on the issues within the business that AI is going to reference. That's number one. Number two is using some of these app based like advertising creative that just spews out lots of ads. I had one the other day that was from notion that my wife actually sent me when she saw it and it was a garbled mess and it had obviously been automated and, and pushed on meta. And I think just one thing that people need to be aware of is their brand is a system. Right, we know this. Your brand is an ecosystem. The challenge with AI is that it can now become just so expansive so quickly that you as a marketer can completely lose control of it. And so one of the reasons why we're building our platform as we are is that if I change one fact, everything that runs from that API will update instantly. And so what you want to know is that if you're putting out more creative into the world, more ideas, more websites, whatever you're building, you need to have control over that brand because it's always one touch point can turn off consumers and in the age of AI it can get sloppy.
[31:39]
A
Excellent. Yeah, we just had a discussion with Michael Tref on this of code and theory about how you got to have control over all these planes or they will fly everywhere and you need an air traffic control system. So I think that's. That really is a good way to bring us to our traditional last question. Funniest story you can share on the air and or practical advice we haven't talked about yet. You can pick one or both, but you must pick at least one.
[32:11]
B
Oh God. Can I take a minute to think about this? I should have come through.
[32:14]
A
No, you can't have a whole minute.
[32:16]
B
Funniest story I can share on the
[32:18]
A
air or practical advice we haven't talked about yet.
[32:23]
B
I'm gonna go with the practical advice one because I, I think one of the important things that people need to take away is that AI is not a bolt on. This is the greatest transformation that your business will ever go on in our lifetime. My company is just an API. I do not use documents. That's it. It's a business brain that I can talk to. That is one of the greatest transformations in business that's ever been. Now the important thing as a marketer is you're sat at the center of this transformation as I mentioned. As a result, you really need to understand how transformation within Companies works and how to actually help execute that. And the best way in my mind to do that is to watch the Mindhunter Netflix series about serial killers. I kid you, it is the greatest corporate transformation story you will ever watch. And when you watch it with that mindset, you'll understand why. Because it was two guys in the FBI who realized that serial killers existed and that they realized they had to find a new way to track these guys down. And they went through everything you go through in corporate transformation. They pushed out in front, they trailblazed. They then realized that they had to get C suite sponsorship, which you're going to need. They then realized they had to productize this. Then what happened is what's called the frozen middle in transformation. Everybody who was just at the FBI was doing their day job. They were like, that's not how we do it. We've always done it a different way. And they had to galvanize the workforce to bring them on to this new approach to hunting serial killers. And it honestly, I've given, I've told so many of my former clients to watch that series with that mindset because when you implement AI in your business, it's not a bolt on. It can't be a bolt on or your company will die. You have to make it central. And you're going to go through transformation to do that. And Mindhunter is the way to know how to do it.
[34:13]
A
Never before have serial killers been referenced on CMO Confidential. So perhaps Netflix will send us some swag. So I think that is a great way to end the show. And thank you, Neil, and thanks to everyone for listening to CMO Confidential. If you're enjoying the show, please, like, share and subscribe. You can find all of our more than 170 episodes on Apple, YouTube and Spotify, which include an update from the front lines of AI, A Spock on the bridge perspective. Colonel Mustard in the study with the job spec, how poor design shortens CMO lifespans. Your customers aren't as loyal as you think they are. And managing the geopolitical landscape. Hey, all you marketers, stay safe out there. This is Mike Linton signing off for CMO Confidential.

Summary7 min read

Podcast Summary: CMO Confidential

Episode: Neal Mann | AI Agents as Teammates – Are You Really Ready to Work Together?

Host: Mike Linton
Guest: Neal Mann, CEO of Known
Release Date: August 4, 2026

Overview of the Episode

This episode dives deep into the transformation AI agents are driving in marketing organizations and the challenges leaders face in integrating these agents effectively into team workflows. Mike Linton and Neal Mann explore not only the technological aspects but also the structural, cultural, and leadership shifts needed to make AI agents productive teammates rather than rogue appendages generating "AI slop." The discussion ranges from practical workflow tips to the imminent reckoning on AI ROI and even includes an offbeat analogy to Netflix’s Mindhunter as a blueprint for corporate transformation.

Key Discussion Points and Insights

1. Defining AI Agents vs. AI Assistants

Main Point:
- AI assistants respond to user requests (think: advanced prompts).
- AI agents take predetermined goals and act autonomously, using tools and feedback loops, often without direct human intervention.
Quote:
- “A good way to think about it is… assistants respond to you. An agent actually takes action on its own... You may have set it a goal, but it’ll use tooling to execute things.” — Neal Mann (02:00)

2. Integrating Agents into Team Workflows

Don’t leap straight to bot creation. First, understand which business processes can and should be automated.
Use a first-principles approach: What are you truly trying to achieve? Build agents for the highest-impact tasks.
Quote:
- “...the majority of people jump to execution first… Actually, if you take a step back and go, what are the most important processes we could try and automate?... Build your agents on top of that.” — Neal Mann (03:10)

3. Challenges of Multiple, Ad-Hoc Agents

Cautionary tale: Organizations, like those led by Jason Lemkin, often end up with multiple fragmented agents, leading to endless maintenance and inconsistencies.
Key Recommendation:
- Centralize knowledge and data — strive for a single source of truth so all agents “speak the same language.”
Quote:
- “A lot of people right now… are making ad hoc agents all over… What they really need is a single source of truth that every agent runs from.” — Neal Mann (04:26)

4. Necessity of Structure for Effective AI

Marketing teams are inherently messy; AI requires clarity and structure (semantic, organizational, data).
Marketers must centralize core assets — mission, vision, brand positioning — and ensure accessible, up-to-date versions exist.
Quotes:
- “When you’re working or want to work with AI, you actually need real structure for AI to reference.” — Neal Mann (06:14)
- “Everybody knows brand positioning… But there might be 40 different versions… if you quickly searched your Google Drive… You can’t guarantee Dave in the email department knows which one we’re actually working from.” — Neal Mann (06:54)

5. Collaboration Across Departments is Everything

The single source of truth isn’t just for marketing; sales, product, and other teams must use the same foundation to avoid “AI slop.”
Quote:
- “That’s everything. That’s what it comes down to... You need a business brain built on a single source of truth... like LEGO bricks.” — Neal Mann (09:07)

6. The “AI Slop” Problem

“AI slop” refers to fragmented, off-brand, or hallucinated content created when AI accesses inconsistent or outdated data.
As company size grows, so does the potential for “super slop” if controls aren’t in place.
Quote:
- “LLMs by their nature are going to hallucinate… If you have three people who aren’t [using a single source], they’re going to create things off-brand, and add another hundred people, a thousand… you’ve got slop everywhere.” — Neal Mann (11:02)

7. Audit and Quality Assurance for AI Output

Build mechanisms (automated and human) to cross-check AI-generated content against your brand facts.
Editorial oversight still matters, especially for content representing your brand externally.
Quote:
- “This is the kind of job that you could build an agent around… An agent that checks output every single time against these facts.” — Neal Mann (13:21)

8. Documentation and Structuring Knowledge for AI

AI thrives on structured, modular knowledge (think: files/LEGO bricks), not buried in slides or static PDFs.
Continuous updating and verification matter more than perfection from Day One.
Quote:
- “Structure is everything… People have been told that AI is good with unstructured data… But it thrives on structure.” — Neal Mann (15:10)

9. Onboarding Agents (and Humans) Thoughtfully

Treat AI onboarding like human onboarding; provide clear, structured instructions and business context.
Over-automation can become a distraction (e.g., “agents firing messages at them in Slack and it was just overwhelming”).
Quote:
- “Think of them a bit like the way you would think of a human... You need to give them solid information about what you want them to do… and about the company that’s well structured.” — Neal Mann (24:19)

10. Measuring Collaboration and ROI

Transparency in approach and measurement is essential. Leaders need systems to know what’s deployed, usage, and savings (e.g., “knowledge work replaced” metrics).
Quote:
- “The reckoning of ROI is coming. You can't just spend all this money. You're going to have to produce results.” — Mike Linton (18:39)
If an agent isn’t delivering, be ready to “fire” it — just as with a human hire — but base the decision on data, costs, and impact.
Quote:
- “The most important thing is what’s the ROI… The second thing is what’s the cost of this agent… That’s why tracking is so important.” — Neal Mann (21:58)

11. Implications for Agencies

Agencies must rethink their role. AI’s speed and scalability mean that any asset (websites, pitches, apps) can be created near-instantly from structured brand data.
Agencies gain when they use the same AI systems for rapid pitch and campaign development but must stay vigilant on control and consistency.
Quote:
- “For agencies, this is potentially huge. You could have the whole team working from the same source of truth as you’re pitching… bring to life 20 websites in minutes…” — Neal Mann (29:13)

12. Most Common Mistakes in Market

Focusing on execution before organizational readiness and system structure.
Using low-quality, automated ad-creating tools that generate off-brand, inconsistent, or even nonsensical creative.
Quote:
- “People are too focused on trying to look at AI execution and output rather than focusing on the issues within the business that AI is going to reference.” — Neal Mann (30:00)

Notable Quotes & Memorable Moments

Analogy of AI agents to humans:
- “The more I think this is so much—it’s so close to just onboarding a human…” — Neal Mann (24:19)
On "AI slop":
- “If you don't do this right… before you know it, you're a complete mess.” — Neal Mann (11:02)
- “The plural of slop is super slop.” — Mike Linton (11:59)
Transformation advice via Netflix’s Mindhunter:
- “The best way in my mind to [understand transformation] is to watch the Mindhunter Netflix series about serial killers. I kid you not—it is the greatest corporate transformation story… When you watch with that mindset, you’ll understand why.” — Neal Mann (32:22)
On instructing agents vs. humans:
- “I've realized that my communication is terrible because I'm vague... I use wishy-washy language… As a result, the response isn't great. The default for people is to blame the agent...” — Neal Mann (26:36)

Important Timestamps

Defining agents vs. assistants: 02:00 – 02:47
Example of Jason Lemkin’s approach and limitations: 04:26 – 05:29
Necessity of data structure: 06:14 – 08:12
“Business brain” and single source of truth: 09:07 – 09:54
“AI slop” explained: 11:02 – 11:59
On measuring and controlling output: 13:12 – 14:49
Documentation best practices: 15:10 – 17:12
How to fire an agent: 21:49 – 23:45
Onboarding first agent (practical tips): 24:19 – 27:28
Agency implications: 27:41 – 29:44
Common mistakes in market: 30:00 – 31:38
Mindhunter analogy for transformation: 32:22 – 34:13

Practical Advice & Recommendations

Start with structure:
- Before deploying agents, centralize core knowledge (brand, mission, features).
- Ensure single, up-to-date sources for key business assets.
Treat agents as real teammates:
- Provide clear instructions and context.
- Monitor collaboration and outcomes as you would with human staff.
Avoid “AI slop”:
- Cross-reference and audit agent outputs against authoritative facts.
- Automate checks where feasible but don’t skip human editorial oversight.
Measure and justify ROI:
- Track tasks, time, and impact before scaling agent deployment.
- Be ready to retire agents that are costly or deliver little value.
Transformation is core, not an add-on:
- Think of AI as central to business transformation, not a bolt-on solution.
- Expect resistance and plan for change management across teams.

Final Takeaway

AI agents are not mere productivity boosters or digital elves; integrating them successfully requires rethinking structure, culture, and leadership. Marketers are uniquely positioned to drive this transformation, but only if they claim control, build systems of truth, and focus as much on organizational design as technical deployment.

“AI is not a bolt on. This is the greatest transformation your business will ever go on in our lifetime... If you implement AI in your business, it can't be a bolt-on or your company will die. You have to make it central, and you're going to go through transformation to do that.” — Neal Mann (32:22)

Loading summary

Transcript67 lines

[00:00]
A
The CMO Confidential Podcast is a proud member of the I Hear Everything Podcast network. Looking to launch or scale your podcast, I Hear Everything delivers podcast production, growth and monetization solutions that transform your words into profit. Ready to give your brand a voice then visit iheareverything.com welcome to CMO Confidential, the podcast that takes you inside the drama, decisions and choices that go with
[00:30]
B
being the head of marketing.
[00:32]
A
Hosted by five time CMO Mike Linton. Welcome marketers, advertisers and those who love them to Chief Marketing Officer, Confidential. CMO Confidential is a program that takes you inside the drama, the decisions and the politics that go with being the head of marketing at any company in what is one of the most scrutinized jobs in the executive suite. I'm Mike Linton, the former CMO of Best Buy, ebay, Farmers Insurance and Ancestry.com here today with my guest Neil Mann. Today's topic, AI Agents as Teammates. Are you really ready to work together now? Neil is the CEO of Known, that's N o a n. A company that creates an AI native platform that helps businesses organize their data into a fact layer and an API for both humans and agentic agents. Previously was head of Global transformation at Anomaly and an editor at the News Corp. Of Australia, the Wall Street Journal and Sky News. He's been an editor of all kinds of new stuff. Welcome to the show, Neil.
[01:44]
B
Thanks for having me, Mike. Looking forward to chatting.
[01:46]
A
All right, let's set the stage for our listeners. What exactly is an AI agent? And how do I I know the difference between an agent and just a series of prompts?
[02:00]
B
Yeah, it's a great question. And you'll find in the market right now, people are confusing language, even the likes of Microsoft using the word agent, when it's not necessarily an agent. A good way to think about it is to break it down into two. Two areas, really. One is assistants, which respond to you. So you make a request and it comes back and responds to you. And then an agent actually takes actions on its own. And you may have set it a predetermined goal, but it will actually take actions. Use what's called tooling to execute things for you and then come back in a feedback loop and work out whether that worked or not and then do it again. And that's a good way to kind of separate them in your mind and they're best used together. You know, you might use an assistant to actually set up and launch your agents, but you need to think of them as kind of two different things.
[02:48]
A
Two different things all right, so tell us, when you watch teams build and use agents, how do they decide what is going to be the agent? And then how do you put the agent into a workflow if it's not just like Neil's agent or Mike's agent, it's an agent for a group of people.
[03:10]
B
I think the first thing to think about is actually to take a step back because the majority of people jump to execution first. They're like, we want something to do this. And as a result they'll try and jump to a short term solution that might be a little bolt on. Whereas actually if you take a step back and go, what are the most important processes that we could try and automate? And then how do we close the gap and do that as easily as possible? Because one of the things that happens with AI when you've got the right structure is you can really rethink things from a first principles perspective. What are we actually trying to achieve? What's the shortest pathway to get there? And then build your agents on top of that. And typically, you know, in marketing you're going to see people creating things that create content quickly. Emails, maybe an agent that goes and finds their dashboard and gives them their data. But if you take a first step back and say, hey, what are all the processes we do? And then clearly think through the ones that you could really streamline and then build your agent for that.
[04:11]
A
Hey, Neil, can you give us a good example of a company that has, or even a hypothetical if you have to blot out the company where, you know, they have built agents and they've incorporated it into the workflow?
[04:27]
B
Yeah. I think on LinkedIn, a really good person to look for is Jason Lemkin. Right. So Jason Lemkin is an investor, he runs sasta. He's constantly out there talking about the agents that they've built. And, and there's a couple of interesting lessons for me around how he's done it. So they have one agent they call 10K, which is their marketing agent. It pulls all their marketing data daily. It comes up with ideas for them, and he's very open about talking about how he's using that. Now the one thing you see though is he also talks about the challenges he's seeing. And one of those is that they've built multiple agents and as a result they're constantly updating each one. And that right there is the problem that we solve for because a lot of people right now, they're making ad hoc agents all over the place. And actually what they really need to do is rethink their business so that they have a single source of truth that every agent runs from. And that's actually, for me, it's really interesting watching him do it openly. And if you haven't seen it, we'll put the LinkedIn in the notes. But it's well worth looking at how he's openly talking about building it and then also seeing some of the holes in the way that they're thinking about it.
[05:30]
A
So I think this is really good. Plus, you keep reading stories about how agents sell all your stocks when you're not looking or erase all your code because they're like, this code sucks. So I'm going to erase all the code and start again. So when you're a leader, if you're out there, you want to do this, you got a team that by definition is probably in various stages of. I love AI. I'm nervous about it. How you even set up the collective team to work with agents and, you know, design outcomes, measures and collaboration. You can't just say, all right, we're going to build a bunch of agents and you guys figure it out. I guess you probably could. And some companies have done it, but that's probably wrong. Tell us what's right.
[06:15]
B
That's going to be messy. The most important thing, if you're a marketer, there's two key things. Actually, the most important one is understanding structure. So marketing in particular is often a very scattered part of the business. Things are being made all over, all over the place. There's lots of creativity happening. The challenge with that is that when you're working or want to work with AI, you actually need real structure for AI to reference.
[06:41]
A
Hey, Neil. And when you say structure, there's data structure, IT structure, like the organizational design structure, decision making structure. Do you mean all of that or just pieces of it?
[06:55]
B
Let's pick off the one that most marketeers understand, which I would say is like semantic structures. They destruction of the language and the things that you use. If you think about marketing in particular, everybody knows and understands the mission and vision. It's a concept that's clear to everybody. Everybody knows and understands brand positioning or brand experience principles. These are concepts that are clear to team members. But the problem in most marketing departments, and I guarantee this is the case for pretty much everyone listening to, is that if you quickly searched within the Google Drive, there'd be 30 different versions of your mission and vision, There might be 40 different versions of your brand positioning. You can't guarantee that, Dave, in the Email department knows which one we're actually working from. Now, the really important thing to understand here is that those concepts are things that AI also understands. It understands what mission and vision is. It understands the impact a mission vision can have on content or things that are delivered in the marketplace. So as a result, the first thing you need to do as a marketer is get a clear structure and a single source of truth around all the key concepts to your area of the business. And the second part that for me is really exciting if you're actually in the marketing space, is that traditionally, you know, I worked in transformation for the best part of nearly 15 years, and it would always start in product and engineering.
[08:12]
A
Right.
[08:13]
B
Whereas the really interesting thing about this transformation is that it's driven by language, because who, if you own the messaging, you can then implement AI within the business. So marketers all of a sudden are actually now in the driving seat. It's one of the reasons you see a lot of the uptake. Yes, you see it in coding, but you also see it in content creation, marketing services of AI. And so those two things are what you need to understand. One, get your context in order, the areas of the business that you own so that it's referencing just one of each of the core things, and then to like, actually realize that you're in the driving seat when it comes to implementing it within the business.
[08:49]
A
And this says I have to collaborate with all my, all my partners across all the business to make sure this is pretty clear, right? Because this is, you know, when you think about big companies where this stuff could be all over the place in multiple databases, this collaboration thing is a big deal, right?
[09:08]
B
It's everything. That's literally everything. That's what it comes down to is you need, you know, we call it a business brain. You need a business brain built on a single source of truth that are like LEGO bricks. Each of those is only one. There can only be one brand positioning, one mission and vision, one pricing structure. You need that organization so that when you're then working with AI, it's actually referencing the right pieces of information and all of the team, when they're using it, are referencing the right piece of information. The biggest complaint I've heard, I've been talking to a lot of Fortune 500s. The biggest complaint I've heard is team members making random stuff using ChatGPT or Claude directly. That's off brand. That's referencing something old. Now this gets worse when you consider that marketing often has to work with sales in a B2B and sales is
[09:53]
A
making all kinds of stuff up, all
[09:54]
B
kind of stuff up from old PDFs from five years ago, you know, and, and that right there is the problem you need to solve for. And if you're a CMO right now, you need to say, hey, where is our structured source of truth? And let's organize it. Now your version one for that is making sure you've got one Google Drive with the right elements in it. Version two, people use markdown files, which you're starting to see. And then we've obviously built a platform where we just extract that and it's all on an API that everybody can build from. And that's your kind of holy grail that you want to move towards.
[10:27]
A
So in some companies, this is like looking at Mount Everest when you think of all the different databases. And you just wrote an article, I think, for Harvard Business Review where referenced AI slop.
[10:40]
B
Oh, that's not me. I sent that to you. So.
[10:43]
A
All right, well, somebody wrote this article and you sent it to me. So I take it back that you wrote it. But you actually, I want you to talk about AI slop. And, and because I think it goes back to what we're just talking about in the infrastructure and what you get if you don't do this right.
[11:03]
B
So the, the problem if you don't do this right is LLMs by their nature are going to hallucinate because of what they are. They're ultimately probabilistic maths machines. They're going to hallucinate and make things up. So as a result, the problem you have is if you're at a business where, let's say you've got a team of five people and two of them are working from a single source of truth, they're going to both get pretty similar output and it will be accurate if it's a real source of truth. If you then have three people who aren't, they're going to start to create things that are drifting or off brand that are different now add another 100 people into that and then a thousand. If you're like a, a global company and you've got ultimately like slop being made everywhere. And then the problem is a lot of people aren't necessarily reading it, checking it, that finds its way back into your, your single source of truth, your reference, and then it gets worse and it just goes from there and before you know it, you're a complete mess.
[11:59]
A
So the plural of slop is super slop. So the bigger you are, the more slop you can have, hey, so how do I know if I have slop? Like, because I think I've. If you're doing the right thing, you're training people on AI before you give it to them, you're having use cases, then you're rolling those out through the company. You've done all of this, right? But when you look at things, how do you know it's slop or it's not sloppy?
[12:26]
B
So I think the challenge here is if you've actually done it right and if you've really approached it in the same way. We think about it where every single one of those LEGO bricks is on an API that's referenceable by AI and humans. As a result, you can then check against it. So you can run checks, check it against my brand facts to make sure that it's on brand. You should have built a system that allows you to cross check and reference to make sure that something is right and accurate. We've put a lot of time into building that internally. There's ways to kind of hack it together within your departments, but you want to have consistency checks so that you're not letting things get out there that are sloppy ultimately.
[13:04]
A
And do you need a little audit team to do this, or, you know, a QA team? You know, what do you, what do you put on this?
[13:13]
B
Well, so this is the.
[13:14]
A
Has to be objective, because if I'm judging my own thing, I'm going to call it not slop. I'm going to call it beautiful.
[13:22]
B
Well, in the, in the AI first world. So for a company like ours or companies that use our platform, you can automate that and run it, because all those LEGO bricks are there to check it. Now, this is the kind of job that you could build an agent around. You could say, I'm going to build an agent that checks output every single time against these facts to make sure that it's accurate and not sloppy. And then you may want a check yourself to check it. And I always do. I think humans in the loop is, is a really important thing when, when working with AI. But if you're working at a big company and you don't have that kind of structure, number one, it's going to be hard. But number two, you need to think about it like an editor, right? Like, I came from that space originally in my career, editors were there to check. And I think what you're seeing a lot now is content creation has just become so easy for people who couldn't traditionally do it. And so as a result, everybody across the organization is starting to create content.
[14:13]
A
If you have some air traffic control,
[14:15]
B
everybody is doing thing, everybody's doing everything. Multiply that by everyone's got a LinkedIn profile that they want to post on. All of a sudden it gets out of control and you need some sort of editorial checks against that. You may have to actually, if you've not built an agentic business yet, you may have to put those in place in a more human way. But it's really worth it because I think one of the things we're seeing now is that particularly as CMOs like we talked about, they own the language, you own the output, and your brand is represented. And people are very good at spotting AI slop these days. And that can look pretty badly on your brand.
[14:50]
A
Hey, so one of the things we talked about pre show was the documentation required to actually set this up. You know, that leads to clarity, that leads to all this other stuff. Tell us about the difference between good documentation and crappy documentation.
[15:11]
B
So I think a really important lesson for everybody is the structure is everything. People have been told that AI is good with unstructured data. You know, you can throw it at a document to make sense of it, which it can do, but it thrives on structure. So what you want to think through is, is imagine your business like the brand positioning, mission and vision, all of these different elements, they're just building blocks that AI can then put together and leverage. So you want to actually think through how you, how you structure that in an almost file like way, which for a lot of people I think is a big difference from the way that they traditionally had decks and PDFs. You know, coming from the marketing space, how much of your life is, is spent making decks. And now you need to think about breaking that out. So it can take time. We, we have a model that extracts those facts for you and then you just verify them and it's dead easy and you can get set up in minutes. But what's fascinating actually is when we look at the usage data, the amount of time people spend managing those and tweaking the language. And I think this is really interesting as a marketer, because back in the day you couldn't guarantee that your brand positioning would come to life in your content. For example, right? The massive disconnect did Dave in the email department or the blog.
[16:26]
A
You like this Dave guy, don't you?
[16:27]
B
Dave. Everyone likes Dave, right? And then could he actually execute? Whereas now the AI will reference that and imbue it in the content. So for the first time you can actually guarantee that the language actually manifests in the world in the right way. So people spend a lot of time managing and tweaking those facts. And so you don't want to rush it, but you don't need to spend, you know, months getting every fact right to start with. You need the core things that you can execute. And then I always recommend to people to think through it, like transformation, because it's corporate transformation. Ultimately, it's the biggest transformation your business will go on. Start in marketing, then maybe bring in sales, then hr, and then all of a sudden you can start to spread across the business.
[17:12]
A
Okay, so now let's say I have these agents, my team, I have the structure, things are working. How do I think about pay plans and communicating with a team that includes agents? And how do I know if I have a large team? How do I really know if my team is collaborating well with the agentic agents in the structure?
[17:38]
B
So I think a really important thing to consider is to not let everybody just bang away on AI themselves. Right? Right now you've got people all over the place making things across businesses, often doing it. There was a report out this week that people were doing it behind the scenes, didn't want to openly acknowledge they were using AI. The first thing is you need to be transparent with people that this is our approach, this is our strategy to leveraging agents in the workplace. The second is then if you've built your company as an API or you have that single source of truth, you can start to measure what's actually happening and look at roi. So on our platform, for example, we have a new metric called knowledge work replaced, where I know when you do a certain task or you create a certain asset, how long that would take you in the real world. So you can then, as a result, show people how much time they saved. And obviously there's a monetary value that can be attached to that. So you want to make sure that you're tracking it. Because, you know, one of the things that I think a lot of marketers are probably going to get a shout from the CEO about is, hey, we want to make this amount of savings, for example, or make us X amount
[18:40]
A
we have to return. You know, a bunch of our guests have said the reckoning of ROI is coming. You can't just spend all this money. You're going to have to produce results. While a lot of that may be cost savings, some of that hopefully is growth. And I didn't mean to interrupt you, but yeah, go on with this whole story about all Right. How do you check on it?
[19:01]
B
Yeah, so I've heard that multiple times I've heard people being, you know, CMOs have said to me they've been told they got to save 20 million, 50 million to your point, drive growth with AI. So what you want to make sure is everything is trackable. You need to know what was deployed, when it was deployed. Think of it like a system, not one off integrations. That's the way you need to think about your business, rethink what your business is ultimately. Because if you think about it as a brain that everything runs from, then look at how do we track what assets are being created, how do we track how many times people were using it? You want to actually be able to track that in a, in a large scale company because otherwise you're not going to measure any roi.
[19:38]
A
Can you give me any example of how you might like, it could be a hypothetical company or a real client or anything, how you might set this up and say, okay, here is my center brain and how it works.
[19:53]
B
So for us on our platform, the first thing we do is we run an extraction model that takes the facts from your documents and starts to set you up. Then you can start to manage those yourself and add them. And what you're doing there is you're actually coming out of the document space and you're building those LEGO bricks in your brain and then they're ultimately on the API.
[20:10]
A
So give me pick any hypothetical like a SNAP CAC company or a bank or a credit card company and say, all right, I'm gonna go get all these documents.
[20:20]
B
Yeah, so if you took like let me think, you know, we're working with a Fortune 500 that I'm off actually off to after this. We're working with a Fortune 500 in the kind of legal space. First thing you want to do is get the centralized documents together around, let's say with brand, right? Because your business is ultimately broken up into key, key sectors, key sections. So let's get the brand documents together. Let's then extract all those key elements of that and the key facts from it. Let's verify the ones that are true today because there's ultimately going to be things in there that are actually old. And then, then let's get those set up on the API. Now as soon as you've done that, you can then deploy that API and run agents from it and you can start doing content creation, et cetera, in, in minutes. But the first thing you need to do is just get that structure. The second Thing is also just making sure that those facts themselves are well structured for AI. And so I always tend to start with brand. And then you want to go into how the product works, product features, for example, these are key things that once you've got them stored, AI can then reference to do things like customer support. But you need that actually within a platform that's actually referenceable by AI and humans.
[21:32]
A
Got it. So let's say I look at all my stuff and I have to fire an AI agent. Someone's probably really attached to that agent. Give us some tips on firing the agent and managing the humans around it.
[21:49]
B
It's a really interesting point because people will get attached to them and they will think that something is doing a good job for them.
[21:54]
A
Well, and if they built it, they're going to love that thing, even if it's no good.
[21:59]
B
And, and the reality is like, agents can be expensive, right? So we're seeing this right now. We saw Uber pull back on its coding, AI coding. So I think the most important thing, and it's funny because it's almost like humans, right? The most important thing is what's the ROI to the business of this agent? In the same way you think about an employee, are they delivering for the company? The second thing you're then going to go is, what's the cost of this agent? Just in the same way with humans, you would look at like that cost benefit analysis. And again, this is why tracking is so important. Otherwise it's just arbitrary that you're deciding something's not worth using and then you just, you can nuke it. And the beauty is you can easily turn it on and off. You know, that is one of the great parts of it. But I think the one thing we need to be aware of is if you do not, and this is such an important conversation to have right now, the majority of token usage. So AI agents, assistants, whether you're using ChatGPT or you're using agent platforms like Paperclip, etc. They're running on tokens. And the reality is the majority of token usage right now is wastage. And so a good way to understand this is if you think about you've had a long conversation with ChatGPT and then you find yourself scrolling back through trying to find the one piece of information. Everything you had in that conversation was wasted tokens. So it's really important to think through, like, if you're going to deploy AI agents in your business, the fact layer is the most important part because they won't waste Tokens because they're going to use the right facts at the right time. So as part of that ROI conversation, you need to look at like how have we structured this and set this up and have we set it up for the success of an agent or have we actually set it up for them to fail? In the same way with humans, if somebody joins the company and you brain dump on them a terrible load of documents that are a mess and expect them to do their job well, they'll fail.
[23:46]
A
Yeah. If the job spec is whatever you think it should be, it's probably not a good job specific. So you can't give that to an agent. So tell me. So we, we have all our listeners out there, some who probably are just getting started in this other than hiring your company, give them tips for how they should get started with their first agent and onboarding the agent and everything else. And you know, if I'm, if I'm the first step for my whole company, give me some tips.
[24:20]
B
So again, it's funny, you know, as we're talking about it, the more I think this is so much, it's so close to just onboarding a human. Right. Like the first thing is think about how you're going to onboard the agent that you want to use.
[24:31]
A
Never ask for extra vacation time though.
[24:34]
B
No. And what you're going to find is they're probably going to overwhelm you with responses. Right. So like that's part of, it's like how much do I get back? I was talking to somebody yesterday who said that their agents are constantly firing, firing messages at them in Slack and it was just overwhelming and they don't, they don't need it. So I think the first thing is actually think of them a bit like the way you would think of a human. Right. And you'd say, okay, well I would, I need to give them solid information about what I want them to do. I need to give them solid information about the company that's well structured. And then you need to go through, thinking through your use cases. What am I actually going to try and use this for? And I think for most people one thing to understand is that because an agent can take multiple actions, it can gather context in different ways. It can be multi step. Now those can be overwhelming to start. So you might want to just do
[25:25]
A
something that's just an example of a multi step.
[25:28]
B
So I for example built one that would search when it found a company. We would, if we were finding a company we wanted to work for, it would work with. Sorry One, if we had a company that we wanted to work with, it would search the Internet, look at the company's house records in the UK or the company records in the us, try and find who the core founders of that company were, then go to LinkedIn, look through LinkedIn, find those founders name, match them, then find their email, then draft them an email, then send them that email. So it's about like probably 15 steps with multiple steps broken out and then it could just run. And so it can be quite a complex workflow. And the challenge is if your instructions aren't good, it can break down. So one of the things to be aware of is as you think through working in this space is again, just like with humans, clarity of instruction is the most important thing.
[26:25]
A
And this, but these will never push back on you, they will just do more work, right? So you have to, you have to really like probably manage them differently that way.
[26:36]
B
It's, it's been really interesting. So I've had users say to me, you know, people who've been in business for 20 years, they've said, wow, I've realized that my communication is terrible because I'm vague. I use wishy washy language when I'm trying to instruct an assistant or an agent. And as a result the response isn't great. And the default for people is to blame the agent or the assistant in the same way. In the office space, we've all seen it, somebody gives terrible instructions, everyone leaves the room and is like, did the CMO say that? Like what we supposed to do? So it's very similar, but a lot of people said, wow, this has made me be clear with my instructions to both humans and agents. So it's really important that you think through the approach and a good thing to do is to ask an assistant and this is how you can use assistance in this. How should I best structure this instruction for an agent? And it will break it out into the relevant step. So use AI as your partner in that space.
[27:28]
A
Super interesting. Tell me what this means for like all the added ad and media and other PR agencies out there. Is this, how is this a threat to them? What is this?
[27:42]
B
It's a really interesting question. I think it's, you know, if we, if we look back and just to rewind, if we look back, there was a fundamental shift maybe 12 to 15 years ago in the marketing space where content marketing started to become a big thing, right? And all of a sudden people started to see that, you know, they needed to create content. And what you saw is the agency started to create content marketing arms and they would create content. And then all of a sudden people started taking that in house and you got the big brands bringing the content creation in house, which is where it sits for many now, in the same way, I think what you need to consider here is that what's your role as an agency and how is AI helping you when it's working with clients, maybe streamlining your operations? You know, we've got agencies using our platform, for example, because there's a fully integrated network, so they don't need a cr. They're using our platform to run all of their business development. And then they have every client in the platform and all of the strategy of the clients in the platform. And to give you an idea of how powerful this can be, you know, just, just before this call, I integrated the known API into Lovable, the website and app builder, and asked it to build a website for a brand that we've got in our platform, a demo brand, and with one request, build me a website based on this API. And it built the website in five minutes. Now, why that's so exciting for agencies is that you could be doing a pitch and you could have all of the team working from the same source of truth as you're pitching and as you evolve the strategy, which happens during pitches, right. You're changing and tweaking. You could bring to life 20 different apps, 20 different websites in minutes, all from the same limitless creative, too. Limitless creative, all from the same fact layer. And when I, when I saw that happen, I thought, wow, this is, this for agencies is potentially huge. So I think it's about using AI in the right way and working with your clients in the right way with it. Use it for your business development and use it for your, your creative, your pitching, your ideas and getting things to market faster.
[29:44]
A
Excellent. Thank you for that. Hey, so before we get to our traditional last question, I have to ask you, most common mistakes you see in the marketplace now, you've already best practices, but let's go over the mistakes.
[30:00]
B
The most common mistake I see, honestly, it's the same mistake that you see from cmos when you're in the room with them. They're too focused on the execution. They are, you know, a lot of people, I think, in, in the marketing space focus on the execution. And so, you know, you're pitching and it's, it's very early and you do a pitch and they jump on one idea and they don't like it. They're focused on the execution. You see the same thing in the marketplace. People are too focused on trying to look at AI execution and output rather than focusing on the issues within the business that AI is going to reference. That's number one. Number two is using some of these app based like advertising creative that just spews out lots of ads. I had one the other day that was from notion that my wife actually sent me when she saw it and it was a garbled mess and it had obviously been automated and, and pushed on meta. And I think just one thing that people need to be aware of is their brand is a system. Right, we know this. Your brand is an ecosystem. The challenge with AI is that it can now become just so expansive so quickly that you as a marketer can completely lose control of it. And so one of the reasons why we're building our platform as we are is that if I change one fact, everything that runs from that API will update instantly. And so what you want to know is that if you're putting out more creative into the world, more ideas, more websites, whatever you're building, you need to have control over that brand because it's always one touch point can turn off consumers and in the age of AI it can get sloppy.
[31:39]
A
Excellent. Yeah, we just had a discussion with Michael Tref on this of code and theory about how you got to have control over all these planes or they will fly everywhere and you need an air traffic control system. So I think that's. That really is a good way to bring us to our traditional last question. Funniest story you can share on the air and or practical advice we haven't talked about yet. You can pick one or both, but you must pick at least one.
[32:11]
B
Oh God. Can I take a minute to think about this? I should have come through.
[32:14]
A
No, you can't have a whole minute.
[32:16]
B
Funniest story I can share on the
[32:18]
A
air or practical advice we haven't talked about yet.
[32:23]
B
I'm gonna go with the practical advice one because I, I think one of the important things that people need to take away is that AI is not a bolt on. This is the greatest transformation that your business will ever go on in our lifetime. My company is just an API. I do not use documents. That's it. It's a business brain that I can talk to. That is one of the greatest transformations in business that's ever been. Now the important thing as a marketer is you're sat at the center of this transformation as I mentioned. As a result, you really need to understand how transformation within Companies works and how to actually help execute that. And the best way in my mind to do that is to watch the Mindhunter Netflix series about serial killers. I kid you, it is the greatest corporate transformation story you will ever watch. And when you watch it with that mindset, you'll understand why. Because it was two guys in the FBI who realized that serial killers existed and that they realized they had to find a new way to track these guys down. And they went through everything you go through in corporate transformation. They pushed out in front, they trailblazed. They then realized that they had to get C suite sponsorship, which you're going to need. They then realized they had to productize this. Then what happened is what's called the frozen middle in transformation. Everybody who was just at the FBI was doing their day job. They were like, that's not how we do it. We've always done it a different way. And they had to galvanize the workforce to bring them on to this new approach to hunting serial killers. And it honestly, I've given, I've told so many of my former clients to watch that series with that mindset because when you implement AI in your business, it's not a bolt on. It can't be a bolt on or your company will die. You have to make it central. And you're going to go through transformation to do that. And Mindhunter is the way to know how to do it.
[34:13]
A
Never before have serial killers been referenced on CMO Confidential. So perhaps Netflix will send us some swag. So I think that is a great way to end the show. And thank you, Neil, and thanks to everyone for listening to CMO Confidential. If you're enjoying the show, please, like, share and subscribe. You can find all of our more than 170 episodes on Apple, YouTube and Spotify, which include an update from the front lines of AI, A Spock on the bridge perspective. Colonel Mustard in the study with the job spec, how poor design shortens CMO lifespans. Your customers aren't as loyal as you think they are. And managing the geopolitical landscape. Hey, all you marketers, stay safe out there. This is Mike Linton signing off for CMO Confidential.

Podcast Summary: CMO Confidential

Episode: Neal Mann | AI Agents as Teammates – Are You Really Ready to Work Together?

Host: Mike Linton Guest: Neal Mann, CEO of Known Release Date: August 4, 2026

Overview of the Episode

Key Discussion Points and Insights

1. Defining AI Agents vs. AI Assistants

2. Integrating Agents into Team Workflows

3. Challenges of Multiple, Ad-Hoc Agents

4. Necessity of Structure for Effective AI

5. Collaboration Across Departments is Everything

6. The “AI Slop” Problem

7. Audit and Quality Assurance for AI Output

8. Documentation and Structuring Knowledge for AI

9. Onboarding Agents (and Humans) Thoughtfully

10. Measuring Collaboration and ROI

11. Implications for Agencies

12. Most Common Mistakes in Market

Notable Quotes & Memorable Moments

Important Timestamps

Practical Advice & Recommendations

Final Takeaway

Podcast Summary: CMO Confidential

Episode: Neal Mann | AI Agents as Teammates – Are You Really Ready to Work Together?

Host: Mike Linton Guest: Neal Mann, CEO of Known Release Date: August 4, 2026

Overview of the Episode

Key Discussion Points and Insights

1. Defining AI Agents vs. AI Assistants

2. Integrating Agents into Team Workflows

3. Challenges of Multiple, Ad-Hoc Agents

4. Necessity of Structure for Effective AI

5. Collaboration Across Departments is Everything

6. The “AI Slop” Problem

7. Audit and Quality Assurance for AI Output

8. Documentation and Structuring Knowledge for AI

9. Onboarding Agents (and Humans) Thoughtfully

10. Measuring Collaboration and ROI

11. Implications for Agencies

12. Most Common Mistakes in Market

Notable Quotes & Memorable Moments

Important Timestamps

Practical Advice & Recommendations

Final Takeaway

Host: Mike Linton
Guest: Neal Mann, CEO of Known
Release Date: August 4, 2026

Host: Mike Linton
Guest: Neal Mann, CEO of Known
Release Date: August 4, 2026