← The Artificial Intelligence Show
The Artificial Intelligence Show
#199: AI Answers - Do Custom GPTs Still Matter? AI Output Validation, 2026 Job Disruption, Preventing Burnout, and Build vs. Buy
01:00:53
Loading summary
Transcript77 lines
- [00:00]
A
There is no shortcut and nor do I think there should be a shortcut for verification. I think that we should slow down and publish good quality stuff that we have verified and that humans have signed off on. And I think that's good. Like, otherwise we would all just pump out AI slop. Welcome to AI Answers, a special Q and A series from the Artificial Intelligence Show. I'm Paul Raitzer, founder and CEO of SmartRx and marketing AI institute. Every time we host our live virtual events and online classes, we get dozens of great questions from business leaders and practitioners who are navigating this fast moving world of AI. But we never have enough time to get to all of them. So we created the AI Answers series to address more of these questions and share real time insights into the topics and challenges professionals like you are facing. Whether you're just starting your AI journey or already putting it to work in your organization, these are the practical insights, use cases and strategies you need to grow smarter. Let's explore AI together. Welcome to episode 199 of the Artificial Intelligence Show. I'm your host, Paul Raitzer along with my co host today, Kathy McPhillips, Chief Marketing Officer at SmartRx. Hello, Kathy.
- [01:19]
B
Hello, Paul.
- [01:19]
A
I feel like we haven't seen each other in a while. Like, did I see you yet this week? You're.
- [01:24]
B
You were vacationing?
- [01:25]
A
Well, I was sort of. I mean I was vacationing for two of the days and then I was in Miami. I wouldn't say that was a vacation like, but I haven't been in the office in. I was here yesterday, I think losing track. So we're recording this on Tuesday, February 24th. It'll, it'll. This will drop on Thursday, February 26th. This is episode 199. You're not in the wrong place. Like if you're used to seeing or hearing Mike with me. Mike and I continue to do the weekly. This is a special edition of the Artificial Intelligence Show. We are in our 14th episode of this series, Kathy AI teachers. Yeah, so this is a series we do in partnership with Google Cloud. So it's presented by Google Cloud. This is a series based on questions from our monthly Intro to AI and Scaling AI classes. So if you've never been to one of those. Every month I teach a free intro class and then I teach a free scaling AI class and Kathy is my co host with both of those. We have been doing the intro class since 2021. So we are on, we're coming up on. The next one will be 56. Kathy does that sound right? Yeah. So we've had over 50,000 people go through the Intro to AI class and then scaling AI, we are going on to our four 14th. Ish.
- [02:42]
B
15th, May. 15.
- [02:42]
A
Yep, 15. Okay. So we've been doing that one for about a year and a half. And so these are our, you know, part of our way to sort of accelerate AI literacy and make it as accessible as possible to everyone. And so when we have these classes, the intro in particular will get, you know, somewhere between 12 and 1500 registrants each time. And so we will get dozens of questions. And so a while back we thought, well, let's start taking some of those questions that we don't get to because we try and allow like 20 minutes or so at the end of each class for Q and A. We'll normally get to somewhere between seven and 10 questions in that setting, but then there's dozens of unanswered questions. And so the goal of this series is to answer as many of those questions as we can. So this one in particular is a follow up to our Intro to AI class that we held on February 10th. Because I've been traveling, we've been trying to nail down a date to do this one. So we are doing follow up questions from our Intro to AI class. So that's just helpful context. When you hear the types of questions we're getting, it helps frame where they're coming from. So again, special thanks to Google Cloud for sponsoring this series as part of our AI Literacy project. In addition to sponsoring this AI Answers podcast series, Google Cloud is also our partner for the monthly Intro to AI and five Essential Steps to Scaling AI Classes, a collection of AI blueprints which we launched three of them this week. So you're listening to this. On Thursday, we will be wrapping up our AI for Departments week with Google Cloud. So we're doing AI for Marketing came out on Tuesday. AI for Sales. No. Yeah. AI for Sales came out on Wednesday. And then AI for Customer Success drops on Thursday when this episode drops. So lots of good stuff going on. There's both webinar and then a free ungated access to AI blueprints. And so you can learn more about google cloud@cloud.google.com and then this AI Answers episode is also brought to us by AI Academy by Smarter X AI Academy helps individuals and businesses accelerate their AI literacy and transformation through personalized learning journeys and an AI powered learning platform. Right now we have 12 professional certificate course series available on demand, with more being added each month. We just released our newest course series, AI4 Financial Services, taught by our director of research, Taylor Rady. It covers real world applications of AI across banking, insurance, wealth management and more. So you can start applying AI strategically in your organization today. So you can go check it out. We have course series and certificates by industry, by department. We have a foundation series that has piloting AI and scaling AI and AI fundamentals. So there's tons to learn. Plus Genai reviews dropping every Friday and then AI Academy Live sessions each month. So go to Academy SmartRx AI to learn more. There are individual and business account memberships available. So there's Academy SmartRx AI. And now I'm going to turn it over to Kathy. She can give us a little background on how these AI answer sessions work and then we'll just dive right into the questions and as you know, kind of explain like, I actually haven't seen these questions. So we do this just like we do in the live class. I don't see the questions in advance. Kathy gives me a doc. I don't ever look at it. No offense, Kathy, but I usually don't have time to look at this doc before we get going. So these 15 questions are raw to me, which mean I might not have great answers to some of them. And I'll be honest with you if I, if I don't have a great answer and hopefully I'll give you some guidance of maybe how to go find a great answer for it.
- [06:09]
B
Okay, so we take the questions from our intro or scaling classes. Sometimes you take them from some of our virtual summits or other things and we will synthesize them. Claire on our team helps. She's built an amazing GPT because the more we do these, the more complicated it gets because we do get repeat questions. So we don't want to make these podcast episodes redundant to questions that you've already asked. So there's a whole process we go through that we make. You know, we synthesize the questions, we put them in an order where it flows, we run them through to see if the questions have been asked before. And if they have, but they are really good questions, we'll either ask again because maybe your answer is going to be different if it's 12 months ago, or we tweak it a little two
- [06:47]
A
weeks ago, sometimes based on how these models are advancing 12 months.
- [06:52]
B
And then sometimes we'll just tweak it so it just hits a different audience or does something different. So Here are the 15 questions that we landed on. So we're going to get Started. All right, number one, I am a beginner in AI. When using structured prompting, do you need to do it every time you interact with an AI platform or can the system carry that structure forward from prior conversations?
- [07:12]
A
So I don't know that there's a perfect answer to this question. I think that probably prompting to me is like this ongoing experimentation. And so I think if you, this is why you would build a GPT or a Google GEM or you know, a CLAUDE project, if you wanted to follow the same instructions each time, then you write those system instructions once and you train that specific GPT or GEM or project to behave and output in a specific format referencing a specific knowledge base. So if you want that level of performance and consistency, that's an argument to create a GPT or a GEM or a project. For me personally, with prompting, I do experiment quite a bit where sometimes when I'm working on a new project, I will give it very detailed prompt with context and examples. And here's the role I want you to fill and I want you to first ask me questions like I'll give very detailed prompt and then a lot of times I will actually give a very basic prompt just to see because sometimes what you find is if we put too many guardrails on the AI, it actually isn't as creative and strategic. And so what I will do a lot of times is I will start with like, especially I'm working on a high profile strategy project, like a high value strategy project. I will actually experiment with different models. And so let's say like Gemini, I might give a very detailed comprehensive prompt to ChatGPT, I might give like the same prompt. And then Claude, I will give a very basic prompt. And what I'll do is then I'll actually look at how the different models behave. And so I don't think that there's any one right answer to how to exactly prompt in every case. Moving forward, I think that it's constantly evolving as the models get smarter, more generally capable. And what I'm finding is context is always helpful, but more and more it's kind of good to experiment with these models and just see what they are capable of without you telling them too much detail about what you want from them. Oftentimes you get surprised. So I'll often go to a project and say, listen, here's what I'm trying to do. What do you think I should include in it? So before I go tell it what to do, I'll ask it for its guidance and then I'll actually build from there. So I would think about it as like an iterative process experiment with both know comprehensive prompts and simple prompts and just keep experimenting. Don't assume once you lock in like all the models are going to work the same way.
- [09:37]
B
Yeah. And are prompt libraries even a thing anymore? Like do people really, if you put something in there and a week later, is it going to be something different? Or is there value in still keeping a, a library of things that have worked really well for you?
- [09:49]
A
I, I do think that there's value and mainly it's because a lot of people have like the blank page page syndrome when they go to work with these things. So for people who aren't used to having it integrated to their daily workflows, they're not, you know, constantly thinking before they start something. Can AI help me with this thing? The, the sample prompts to me are, are very helpful as a starting point for people. It helps them actually realize the different ways they could be using it. So yeah, depending on what the prompt library is being used for, it's good go to and then even, you know, for personal use, just to go back and remember some of the prompts you've used before. I keep a journal when I'm. Whenever I' working with like higher value projects and I'll actually go back and reference the prompt I used because what I like to do is take that same prompt and put it into new generations of the models. So I have one project in particular I've been working on for like six months. And so I have in my journal, which is just in a Google Doc, here was 2.5 pro from Gemini. This was the output I got with that prompt. Well, when 3 comes out or 3.1 pro, I'll take that same prompt and I'll actually drop it back in and I'll start the project again or like see if anything has really changed with the model based on that.
- [10:58]
B
Sure. Okay. Number two, you kind of alluded to this a little bit, but I'm trying to understand the practical value of custom GPTs. If ChatGPT can actually, can already answer questions directly, what problem does a custom GPT actually solve?
- [11:11]
A
Yeah, it's consistency of the output. So if you're doing something over and over again, then you don't have to provide that context every time. So I don't know. Like an example would be, you know, I shared how when we were creating AI Academy, like the new version of AI Academy that launched in 2025, I had to create 20 courses. So three certificate series but totaling 20 different courses. And so I built an AI learning assistant I called Aila, which, which actually was a Google gem which functions like a custom GPT does. And it was trained on our instructional design principles for Academy. It was trained on our roadmap for Academy. I explicitly told like, here are the things I want you to function as. Like, I don't want you to create the courses, I want you to actually help assess them. I want you to help me build outlines, I want you to write abstracts for the courses once I've completed them, like I'm going to give the PowerPoint or in our case Apple Keynote. And then I want you to actually write an abstract based on that. And so I told it to do this specific thing and then over like a two month period while I was creating all these courses, every time I needed something, I didn't have to go back in and say, here's what AI Academy is, here's what I'm trying to do. It was already pre trained on that and so that's a use case. Another example would be Jobs GPT, which is a custom GPT that I built, I don't know, like a year and a half, two years ago that helps assess the impact of these AI models on Jobs at a task level. And then it actually provides a rationale as to how you could be using AI to do these different tasks differently. So you could put your job title in and it'll actually assess your job title and the tasks you do based on its pre training, basically with what I, the knowledge base I gave it. So the argument for creating a GPT there is because I shared that and made it free for anybody to use. And so now I don't know. Last time I looked over 30,000 conversations that happened in jobs GPT and that was probably six months ago because I just created this thing that was specifically designed to do a very specific task and then I made it available for everybody. So I could just take a prompt and say like, okay, here's a prompt, go do it. But it's way harder to do that. So we created a thing, like productized it, put it on our site, like, hey, go click here. You can go play with this thing and, and see what it does for you. So those are a couple of cases, consistency of projects when you're going to want to use the same kind of prompt over and over with the same context. And then when you want to create something that you can share out that helps people do something I also do is like, I just ran a workshop yesterday where I did Problems GPT which helps people identify problem statements and value statements where I can potentially help solve something. And so I actually use Problems GPT in workshops. So to help people brainstorm faster, they can just go in and say, all right, here's what my job, here's what I'm struggling with. It'll help them write a problem statement, it'll create a value statement, and then it'll actually build a strategic brief that I trained it on to help them solve that problem. And so like none of that, it's really hard to do with just like prompt libraries basically.
- [14:05]
B
Sure. Another use case is that I have my PAL GPT that I use every day and I also have a Kathy GPT that I have shared both of them with members of our team. So a lot of that knowledge base, they, you know, they have access to that GPT and they're using it in their day to day. So sharing it among the team is a, is a great reason to have custom GPTs. Number three. With the constant evolution of AI models, are SaaS providers becoming more model agnostic, including bring, bring your own model offerings?
- [14:36]
A
So the, the way this, the software providers work, just I think to provide a little context that people aren't clear exactly how this works. There's like five companies in the, well in the United States, there's other companies internationally, but in the US there's like five companies that are actually building the models. So anthropic OpenAI meta xai Microsoft to a degree. Did I miss somebody? Google Microsoft OpenAI anthropic xai meta yeah, okay, so maybe six. They build the models, they're the ones spending the billions this year, hundreds of billions to build data centers, build energy infrastructure, train the models, and then make those models available to other companies to use. So your company might contract directly with one of the model companies and get direct access to it. Like maybe you use Gemini, you know, out of the box. Like you just go to the Gemini app and you do what you do or you build it into workspace. The alternative is software companies like HubSpot, Workday, Box, Salesforce, they don't build their own models. They pay those labs, the AI labs, the AI model companies, to access the models. Almost like a utility. They're just basically paying for tokens to, to use the technology. And so if you go into like a HubSpot and you're interacting with an AI within HubSpot, it's helping you build a landing page or it's giving you insights into your data that is not HubSpot's model. They are likely. Well they were originally, I think using OpenAI. They probably use a symphony of models. Now they're probably trying to use the lowest cost version of a model. So maybe they have a model selector that's like, okay, someone wants to write an email, let's just use the cheapest version of Claude because we can charge the customer more money and we can make a bigger margin on it. So what's happening is the software companies build these models into their software. They don't actually build the model and then when you use it within their software, you're actually using a third party model, but you're just paying, you know, the markup fee from the software company. So each software provider handles this differently. Some probably have exclusives with these model companies. But I think more and more the software companies are realizing it's probably smart to diversify and so they likely have relationships with multiple model companies to be able to serve up the intelligence that you then use to do your work within those software platforms.
- [17:08]
B
Yeah. Okay. Number four, I've built a number of custom GPTs for work processes since the switch to the 5.2 model. I found the voice and tone aren't as reliable as in the previous versions. Is that something you're hearing from others and do you have any advice?
- [17:23]
A
Yeah, this is a problem. So the again, I'll just always try and like provide a little context here in case the question isn't clear for everyone. So like let's take Jobs GPT as an example. So I built JobGPT. I think it was originally built on GPT4, like when I first built it. And so it's trained to do a specific thing. And then when each new model comes out, so you have 4.5, then you get GPT 5. As the creator of that GPT, I have to go in and be like, but does it still function the way I intended it to function? So this is a very low level version. But the software companies have to deal with the same thing. The individual users have to deal with the same thing when the model changes. These models are trained in different ways. They're trained to have different, as weird as it sounds like personalities and tone. They're trained to be good at some things. Like so for example, Sam Altman admitted that when they came out with 5.2, they put less resources into making it good at writing. And so GPT4 and even GPT5 were very good at writing because they were trained to be good at writing. They they prioritized agentic capabilities on the later models. And so it actually, they spent less time and money getting it to be good at writing. And so you might actually notice that when you go in and use these later models that the voice and tone starts to change and that's not on you. Like they just changed the model. So this is a real problem and it's something people have to be aware of. And it's why when new models come out, if you have go to use cases or if you have custom GPTs or gems, you need to go in and experiment with them and make sure they're still doing what they're supposed to do and they're still sounding like what they're supposed to sound, because you can go in and change the system instructions. So if you've built the GPT, and again, if you've never built a GPT, it's like giving a project brief to a human. You just go in and say, hey, here's what I want you to do. Use natural language to do this. You explain what you want it to function as and you give it whatever context you want and then you just save it and publish it. So you can go back into those system instructions or the project brief and say, listen, I want your tone to be more like GPT4. I want you to be more comforting. I want you to be more friendly, like, whatever. And you can try and coax that tone out of it by telling it that's what you want. But this was a huge issue for OpenAI because people got very attached to the tone of the 4 Series models. And then when they moved over to the 5, especially 5.2, there was a lot of backlash from users that it didn't feel like the same model to them. It didn't feel like that. Like people had gotten used to talking to it, they had developed relationship almost as weird as that sounds with these models. And so it's very odd to have them switch. So, yes, it is a common thing. It is not going to stop. Like, this is going to be a ongoing issue as new models come out and they get infused into the software you use or the GPTs or gems you've built. You're going to have to constantly deal with this. There's really no, I don't know a way around it. I don't know how the labs would solve for that. They determine the model's behavior each time they publish it. So sometimes it's going to change.
- [20:35]
B
All right, number five, do you have an AI output validation process you can share I've seen colleagues take AI generated output and publish it directly or send it to a second LLM for verification. This can mislead decisions and create risk. What's a better approach?
- [20:51]
A
Better approach is the human has to be responsible for the output. There's no easy way around this. So we have these responsible AI principles that I published a couple years ago, I guess three years ago now, it was January of 23. And one of the first principles what the human has to remain in charge. The human is responsible for the output of the AI. And so as much as we would all love to just run five AI research projects a day and publish, you know, these 30 page reports and get all the SEO credit and you know, be able to drive awareness and audience growth with all this stuff, that that's just not how this works. These things make mistakes, they hallucinate, they have errors in them. And just because we can take shortcuts on the creation of it doesn't mean we can take shortcuts on the verification and the critical thinking of like the output itself. And sometimes you just gotta trash something because it was based. Like what we've seen with the deep research in particular is it can read amazing. Like if you just read this report, it sounds incredible and it picks these great headlines like 60% of people aren't getting any ROI from AI or whatever. And then you dig into it and you realize that the source it used, the primary source for the main headline, was from a source you would never use. Like it's like a third party platform that got its data from somewhere else and you gotta dig five layers deep to figure out where they even got the 60% from. And then you find out it came from a Wikipedia, like you just don't know. So there is no shortcut and nor do I think there should be a shortcut for verification. I think that we should slow down and publish good quality stuff that we have verified and that humans have signed off on. And I think that's good. Like, otherwise we would all just pump out AI slop. It's, it's not, it's like AI content farms from like 2010 range where you pay $0.02 a word and everybody started pumping out a bunch of crap. Like we don't want that, we don't want the Internet filled with junk. And that's basically what's happening. So I do use like, I will take something like let's say I do a research report and deep research from Google. I will take it and throw it into Claude and say, can you Assess this. I want to be critical of the citations. Make sure they're all like, that's fine, that's a good stepping stone. And you might save yourself 30%, 50% of the time to do that first level of verification. It might find a bunch of stuff and flag stuff for you, but you can't get the human out of the loop on the verification process.
- [23:16]
B
Definitely. Okay, number six, can you suggest tools for building AI agents? We hear so much about their potential, but where should someone start if they want to build something like an AI sales outreach agent or a content flywheel generator?
- [23:29]
A
I would start with the existing platforms in your tech stack and see if they enable these kinds of things. So if you're a Microsoft shop, you can go build agents and you know, Microsoft Copilot, if you're, you know, Google Gemini user, you can go build them in AI Studio. You can, you know, create agents to do different things. They're largely rules based agents, but they can help. If you're in Salesforce, you can build agents like right within the CRM. So like your platform may enable the building of agents. Claude, you know, Claude code, you kind of use Claude code as an agent to build stuff. So like they're, they're starting to just get embedded and sometimes you're not even going to know that that's what you're doing. It just depends on the task you're trying to perform. So I, on the podcast recently, I shared the example of how I've been using Lovable to. Lovable is basically an AI agent that, that does code to build apps and I have no coding ability. So I just go in and like, hey, I want to build an org chart app. I want to make it interactive. I want to be able to envision like future iterations of how our organizational structure could look like. Help me, help me build that. And it just does it and it asks me questions and you build out the thing. So I would say like the popular building of agent ones are Claude code Codex. You know, Gemini has coding capabilities, lovable. But you know, that space is moving so fast. Agent AI is another one. Dharmesh Shah, co founder of HubSpot, created that platform. And it's in essence like you can go, it's like a marketplace for agents basically. You can go in and build your own or you can, you know, sign on to use an agent. So agents are an interesting emerging space. They're in software design in particular. They're becoming more autonomous and reliable and that's starting to trickle out into other areas of Knowledge work. But most of the AI agents that are actually approaching autonomy, high levels of autonomy is largely happening in encoding and software. It's not like marketers and salespeople and customer success and executives. For the most part the AI agents you would be using today are still pretty rudimentary and a lot of rules based stuff, which is not the full autonomy people envision when they hear the term AI agents.
- [25:42]
B
Right. But Matt Sear at our AI for Agency summit, his presentation was like, okay, I actually can do a couple of these things and get started. So they're, you know, look, look for some of those use cases and some of those people that are doing some cool things and you know.
- [25:55]
A
Yep, they're starting to become more real like by the day. Like it's, tons of innovation is happening and they're, they're making coding accessible to everybody. So anybody's gonna be able to go build whatever they want. And I, I think like 2026 is gonna become very real for a lot of people.
- [26:09]
B
Yep. Number seven AI labs are developing new tools like Claude coworkers and Excel or PowerPoint add ons. Do you believe knowledge workers will soon face the same disruptions developers and SaaS companies are exper and if so, how should they prepare?
- [26:25]
A
I think the technology is going to be there to cause that level of disruption. I think human friction and the friction of slow moving enterprises is going to prevent it from being massively disruptive in a very short period of time. So I think the software coding agents, you know, like we've talked about like cloud code as an example, it's disrupting, you know, stocks daily. Like we hear every day, like Claude releases, you know, or anthropic releases, some new Claude plugin and some new category of stocks crashes like 10 to 20% overnight. And it's all just based on the uncertainty of what does this all mean to all these legacy software companies. So I, I do think that in the next 12 to 18 months most of knowledge work will be solved. Like in theory, Claude and others will be able to do the vast majority of what knowledge workers do. So I would say there's not going to be too many technical limitations to looking at industry and saying, well let's just use Claude and Gemini and ChatGPT and let's just like automate that industry. Having spent a lot of time with large enterprises, especially in the last few weeks, that could be true today that like you could just go get Claude and automate like every job. If that was true today, it would take five years for these Enterprises to do something about it. Like it's. The enterprises are moving so slow with not just like adoption and integration and like transformation, they're moving slow to even provide the most basic AI assistance to all of their employees. So I just feel like if you'd asked me this two years ago, I would have said, yeah, by 2026 we will probably be like right within the midst of the disruption. I don't know that that's true for most industries. I think it's going to be a really long run. And I've said this before, like we could have AGI today, we could agree as a society that we have reached artificial general intelligence. And I don't know that it would change anything within enterprises. Like it's again, like generative AI is three and a half years old. It was in November. 2022 is when ChatGPT emerged and gave access to the world. And I meet with enterprises every day who have not yet provided a license to it to their workers. So we're three years plus in and to the most basic thing, which is give them an AI assistant for 20 bucks a month. And we're sitting here, you know, in many instances not doing anything. So I, I just am not a believer that the disruption is going to happen that fast. Now I would not, I wouldn't say that you shouldn't have a sense of urgency about it though, because it, it, it will start to impact jobs. I'm just saying it won't completely change
- [29:14]
B
industries and it might start to impact jobs before the companies are ready to use AI to, to replace those jobs.
- [29:20]
A
Yeah, like jobs are going away. Like they're in, in mass starting kind of started last year. But like 2026 is not going to be pretty from a jobs perspective.
- [29:30]
B
But the companies aren't prepared to lose those people.
- [29:35]
A
Yeah, they're just not aware of how unprepared they are.
- [29:37]
B
Correct.
- [29:38]
A
I think that it's largely happening because they think AI is driving all these efficiencies and productivity and they don't need as many people, but they don't really have a plan for like what that looks like.
- [29:49]
B
Right. Okay, moving on number eight. As early AI first organizations report increased workloads during the initial AI transitions, what strategies can leaders implement to support employees and prevent burnout?
- [30:04]
A
I assume this question comes from some of the recent reports that are basically saying the organizations who are racing ahead and figuring out the ways to integrate AI across teams and employees and workflows are just increasing the productivity, but they're also just increasing workloads like they're not seeing like the byproduct of, you know, more time to work on pet projects and more time to like, not work. I think what's happening is the people who are fully aware of what's possible, like know how to use these tools to think, create, understand reason, assist in decision making, assist in problem solving. They now realize they can go build apps and software without having to talk to anybody in it. Like it's, it's like this insanely inspiring world of like, anything is possible. And I can just speak from my personal experience, like, I just want to build stuff, like I just want to solve things and so I don't feel burnout personally. Like, and I, again I, I know I'm probably unique and like I'm the CEO of a company that does this stuff. So like I, I probably have a little bit of different perspective than, than you know, say like a VP at a 30,000 person company. But like, I just want to do stuff and build stuff and like solve problems and I find it so exciting. But I am definitely creating more output than I used to create. I'm doing things faster, solving problems faster than I used to solve them. And I could see how you become so immersed in that that you just like overwork yourself. I could. More realistic though, to me is the enterprise example where you have, let's take it, say you have a marketing team of you know, 50 and there's five people who have actually figured all this out. They're the ones that are daily active users, they've built assistants that are personalized to them, they know how to build apps, they've got a permission to connect to some data. So like they're doing all the things and the other 45 people on their team aren't. And all of a sudden those five are doing the work of like two or three employees and they're realizing they're the only ones producing at this level and they're getting paid the same as their peers who aren't doing any of these things. And like, I think that's where a lot of the burnout and friction is going to come in is when you have these AI champions or like early adopters who are just outpacing their peers and they're doing more work than everybody. And everybody's kind of maybe starting to say like, hey, can you help with this? Can you help with that? Can you help me figure out how to do? I could see a scenario where burnout starts to happen there, but I think this is an organizational design thing. It's a change management thing. Like, companies are going to have to figure this out. Your employees are going to be able to do more once you enable them to do it. And you got to figure out how to make that happen without burning them out and, like, giving them the grace of, like, hey, you just yesterday afternoon, you created an app that's going to save us a half a million dollars next year. Like, hey, take Friday off. Like, there has to be some structure. Like, I've had this personally, I've shared a little bit on the podcast. Like, I've done a couple of things in the last couple weeks where, like, I'll work on something for, like, two days that would have. You used to take me two months. And it's probably going to be worth millions of dollars to our company over the next, like, one to two years. And I'll finish it and like, an hour later I'll feel guilty that I'm not working on something else. And it's like, dude, take the afternoon off. Like, you just did this thing. And so even for me, there's this mentality of sometimes you have to look at the value of the output, not the time you spent on it, and you have to be able to give yourself time back. But I think that, you know, intrinsically motivated people are going to just want to keep building the next thing. They're going to be motivated by, like, the game of this, of, like, I just, like, create something new. We can, like, you know, increase our score. Like, we do this thing. And I think companies are going to have to manage those people to make sure they don't burn out, to give them the grace of time off and help them. Like, all right, you did a huge thing. Like, the company really appreciates it. Like, take a week off, get some extra PTO or whatever it is. It's not going to be monetary. I don't think, like, monetary is good for a time period, but that's not, like, usually the thing that motivates those kind of people. Once you get to a certain threshold, like, once you make enough money, it's just like, all right, I want to build stuff, I want some time back, things like that. So these are hard things that HR departments and leaders are going to have to really start thinking about. I've not seen a company yet that I've heard of that has solved this.
- [34:41]
B
Yeah. And I've talked to you about this before, but I'm kind of down this rabbit hole of how we define productivity right now.
- [34:46]
A
Yeah.
- [34:46]
B
So. And if productivity is you taking the afternoon off because your brain is fried, but you're coming back the next morning fresh. Like, that's a productive use of your time.
- [34:55]
A
Yeah. And I'm like, micromanage. Like, I'll give you an example. So I, I try and do, I try and do the elliptical. So I get up at like 5, 35, 45 every morning and I try and do like 30 minutes on the elliptical. And then I usually try and get to the gym like three to five days a week. Normally I think of those times as times to listen to podcasts. It's like, okay, like I'm going to be 30 minutes. I, I can get through a one hour pod at 2x speed. And, and I get D. Like, oh yeah, that's 45 minutes I can get to. And so everything in my life is like, how do I consume the most amount of information so that I can synthesize that information for our own business and for the podcast? And every once in a while it's like, no, just listen to music for 30 minutes. And so I'm managing burnout at this very, very, like down to the 30 minute block level of like the grace of, you don't have to listen to a podcast for 30 minutes. But again, I actually, like, I find it invigorating. But every once in a while I can feel myself that burnout creeping in and it's like, all right, I'm actually not going to do any work on this Saturday. I'm just going to take the day off and hang out with the family. So. But again, people need, they need guidance to do that. Well, I think a lot of people who are already early adopters of this are probably the type of personality who are going to want to just build and they're going to want to keep working.
- [36:13]
B
Well, when you figure out that, how to solve that, please let me know.
- [36:16]
A
To shut the mind off. I know it's, it's genetic. I get it from my mom.
- [36:22]
B
Okay, number nine. What roles and skill sets are most at risk for obsolescents? And how can leaders help team team members in those roles successfully navigate the transition?
- [36:32]
A
Yeah, this is, this is a really hard one to answer. I'll try and answer first by what I think is safe. Ish. So I think emotional intelligence is really important. I think interpersonal communication is going to be absolutely critical because I think more and more that's what jobs are going to be. You know, if you're a wealth manager or a consultant or a journalist, like we talked about this week, like you're going to have more time to be with other people, to talk to other people, to engage at a human level. And so that ability to do that is going to be really important. So I think we're going to look for that. Critical thinking is essential, but being able to work with the AI, that's also very good at critical thinking. So I think I often boil it down to knowing what questions to ask and what to do with the answers and then knowing how to talk and work with and learn from the AI, like at a very high level that that's basically what all knowledge work is going to come down to. And so what, what do you need, what skills do you need to work well with AI? Well, you need to probably be a problem solver because you have to be able to identify problems to have it solve. And so you have to like have that context of what does a problem look like. And you know, what questions do you ask about something? If something's not working in your company or you're building a strategy or you want to produce an analyt it often comes down to what are the right questions we would be asking here. So if I'm thinking about our own company, I'm going to say like what are the AI generated reports I want to see every morning when I open up my computer? I would actually start with well, what are the questions I want answered every morning? Are we on track to achieve our goals? Which resources are providing the greatest return on investment? Where are we falling short? There's these standard questions I would want and, but to, to ask those questions is a skill like it's, it's understanding the context of business. So it's, I think like those are some of the things writing. I know that AI is really good at writing, but I actually think writing is like a fundamental skill of all workers, all knowledge workers, professionals and leaders. Because writing is thinking. And if you can form strong outlines and you can tell stories and you can be critically assess sources and information and facts, like that's critical. Even if the AI is assisting you, you still need that. So I would say writing is still a fundamentally important skill. And then the roles, the safety of different roles is largely dependent upon which companies get funded by venture capital firms. Because there's no role that's actually safe. Like it really is more a matter of when is someone going to justify going after the labor market for a specific role. And what I mean by that is like in the United states there's about 11 trillion in wages annually, somewhere between 4 and 6 trillion of that are knowledge workers, digital workers, people who use computers for a living. So in essence, what's happening is the AI labs and the software companies themselves are looking at that 4 to 6 trillion and saying, which portion of that do we go after? So if lawyers account for, I'm just going to make up a number, but let's say 400 billion in annual wages. That's a pretty nice total addressable market to go after with. If you're Harvey and you want to build an AI, yeah, it helps attorneys, but also maybe eliminates the need for some attorneys. You can do accounting, you can do consulting, you can do marketing, you can do sales, you can do customer service. And so that's, in essence what's happening. That's the future of the economy is over the next five to 10 years, it's just going to be picking off roles. Now, it can't really eliminate the need for humans, but it can reduce the number of humans needed. And I think that's the thing people have to come to grasp is like, we're not. These companies aren't trying to eliminate humans totally. Well, some of them actually are, but most of them are not trying to limit humans. But the byproduct of that effort is that fewer humans are needed to do the same work.
- [40:42]
B
So, yeah, and you've talked before about, you know, working with your accountant, working with your lawyer. My dad's my accountant. So like, I took in my tech stuff and I did a lot of it on my own. And he was like, hey, what are you trying to do? You trying to fire me? I was like, no, but I'm also very curious on this sort of stuff. Yeah, but I can go into him and say, like, I actually saved you a few hours.
- [41:00]
A
Yeah, I've done it with accountants, architects, like, doctors. I, I have a co x of all of them. Like, I, I want to come in educated to those conversations. I've equated it, Kathy, to like, how, you know, 10, 15 years ago, when you were buying a car, you were just very dependent upon the salesperson. You know, you just, like, you didn't, you didn't have a ton of information about the car buying process. And now if you go buy a car and you haven't done all the research, run all the numbers, know what the market fair market value of that car, like, you need to know all of that. Know what percentage you can get off on the thing by, like, if you don't walk in and buy a car with that information, you've done a disservice to yourself, for sure. And I kind of feel that way about anything. I hire an advisor for a professional for. I'm not trying to replace them. I just want to come from a more educated standpoint. I want to know what questions to ask them. And yes, that might reduce the billable hours, it might reduce the amount of time that we have to spend together, but the value creation is still tremendous. And I understand that's maybe disruptive to those fields, but I think that's just the reality of how it's all going to work in the future.
- [42:03]
B
Number 10 for business leaders modernizing analytics, should they continue investing in traditional BI platforms or shift toward AI first reporting systems? And what does the right architecture look like moving forward? And how should leaders decide?
- [42:18]
A
I'd love the answer to this one. It's actually something I'm thinking about myself, so I will tell you what I'm currently considering for us. So if you think about running our company, there's probably like four or five platforms that hold the vast majority of the analytics data I would want to have access to at any given time that I would want to be able to ask questions of at any given time. So an example would be like HubSpot is our CRM. And I may want to say like, like we just launched the I for Financial Services course series. So I might want to say how many customers in our database of like 150,000 people work in the financial services industry Historically, I would have to go in the CRM. I would have to build a list, I'd have to pick the properties. I would have to like, make sure all these things have to happen versus Can I just talk to it? Can I just ask this question in HubSpot? The answer right now is to my knowledge, no. Like, HubSpot has yet to enable that level of intelligence in the platform. If I'm wrong HubSpot, please reach out and tell me that it exists because I would love to use it. But I, you know, imagine the same thing for our accounting data, for our learning management system data. Like all these core things that are essential to running our business right now. I have to go to those platforms. You could have a single BI tool. Like, you could build it into like dashboards and you can visualize it and things like that. And we have that too. But in theory, I have to go to these different places, seek out this information, and then I have no intelligence. All I have is data. And so I'm thinking about saying, okay, back to this sort of first principles. What are the questions I want answered about our data every moment of every day. Like, what would I need to know? And then what is the best way to architect that? Is it to talk to our financial reporting system, our CRM system, our learning management system, and say, what AI are you building into your platform that we can come in and talk to our data and get insights and recommendations out of it, not just visuals and charts. So one option is to rely on each of those individual platforms to build intelligence into their platform. The alternative is to connect all of them to a Claude and then I just go into Claude and I talk to it or to, you know, in Gemini, let's say that they would enable this kind of thing. And then I could just schedule an automated report saying, hey, I want to know what's happening with our P and L, I want to know what's happening with our top customers. Like, whatever. Here's the questions I have. Go through Generation say, okay, this report's great. Can you actually run this every night at midnight and send it to me in an email? So imagine that level reporting. So now I never actually have to go into those platforms for the majority of the use cases. I'm just, it's showing up, it's surfacing itself to me in an almost ambient way. Like, here's what you need to know, here's the three, you know, key facts or highlights from yesterday's data report, and here's three actions you might want to take. We think you should reach out to Kathy about this. You should reach out to, you know, Jess about that and talk to Tracy about this. Like, that, to me, is the vision for, like, what it should look like. I'm not clear yet how exactly we get there.
- [45:22]
B
Okay, number 11, as AI capabilities expand rapidly, how should leaders think about what to build internally versus what to buy from vendors or partner on and what is the decision framework?
- [45:35]
A
So the, the general practice I grew up with in business was like, if it's core to your business, build it. If it's ancillary to your business, then you let somebody else build it and maintain it. So, like, what I mean by that is, like, if you're Amazon at a very, very high level, you're not going to build your business on Shopify like you're Amazon. Like, you're going to build your own E commerce engine. You're going to manage that E commerce engine because it's, it's absolutely, you know, fundamental to the business you're in. So I would say it's getting harder and harder to. When we think about gen AI to build your own things. Like if you're a software company trying to build your own models, even though they're fundamental to your business, is next to impossible. Like you can't compete with these bigger companies. And the general idea that like the next version of the smartest model is just going to obsolete whatever fine tuned model you created to do a specific thing. Like they just keep proving time and time again that the bigger the model, the more general it is, the better it's going to be at everything. And so I think that more and more you just need to assume you're not going to be the one building the intelligence that you're going to find the partners to build with. But then the complexity becomes like, what are they building on? So if you go find a software product that enables AI and marketing or sales or customer success, the challenge for me is do you go through the software vendor or you go direct? So like, like I'll use the, again I'll just share a personal example, like the AI agents on your website, like customer service, you know, do you use the one that's kind of built into your current tech stack or do you go work with a company that specializes in building on top of the best models to create the most advanced customer service agent with the fewest hallucinations and the highest satisfaction score rating and whatever. And so like those are the kinds of things we have to start thinking through is, you know, what is the, the thing we're trying to solve, what is the potential value of it, what are the risks related to it, what are the options we have and then what do we actually go and, and do and what do we build ourselves? So yeah, this is a tricky one and it, I think it's shifting. I think the traditional idea of build core build, you know, and buy everything else. I don't know how true that remains. And I'd have to actually probably sit down and think about individual examples within our own business. Like we didn't build our own learning management system, like that's core to our business, but we looked at a bunch of vendors and we found a vendor with an AI roadmap that we believed in, that we thought was going to give us the best chance to move quickly and start iterating and then that would enable us to improve that platform by customizing it. And so, you know, I could have stopped and spent a million dollars and tried to build our own learning management system, but it would have taken us three years and it probably wouldn't have been what we needed. So Yeah, I don't know. There's no, there's no real right answer for this, I don't think. It just depends on each unique situation.
- [48:46]
B
This will be a question we'll ask again in like two months or 12
- [48:49]
A
months or whatever change. If you ask me next week, number
- [48:53]
B
12 holding companies are building billion dollar AI operating systems and the big four are investing billions in AI consulting. Div. For independent AI forward agencies, what is the leading, the lasting competitive advantage? Is it trust, is it agility, the ability to actually build and ship while others are still writing decks?
- [49:12]
A
Yeah, it's not just the big four that are building the consulting practices. Claude and OpenAI and Google are going to come after that business too, which is weird because they have alliances with these companies and then they're building like massive competitors to the companies and a very weird dynamic right now. Very weird marketplace. The so the AI for agencies, you know, we just did our AI for agencies summit. Was that this month? Just like two weeks ago. Yep. So man, time moves fast. You know, it's, it's, it's a very difficult environment right now for agencies. There's pricing pressures. You know, I think brands are aware that a lot of what they used to rely on agencies to do the tactical work they can do themselves, you know, with a chatbot and you know, a tenth of the time and a tenth of the cost. So some of that gets commoditized, some of the creative work gets commoditized, some of the strategy work gets commoditized. I think that the agencies that stay on the leading edge of this, that are constantly, you know, evaluating it, rethinking their services, doing impact assessments, looking out, you know, 12 to 18 months, how their industries, they service are going to change, how the organizations they work with are going to change, how consumer behavior is going to change. I think you just have to stay out on that edge and see around the corner. I don't know any other way around this. I don't know there's any service I could guide an agency to offer today that I wouldn't have some level of confidence is going to be obsoleted in 18 months. Like it's, it's very hard. And then I also think it's hard to time the market right. And one example there would be change management. Like every enterprise in the world is going to need AI change management guidance, whether it comes from internal people or outside agents, agencies. And that means not just like, hey, we bought licenses to ChatGPT and we gave them to our 30,000 people. But what does that mean? How do we personalize use cases? How do we deal with the fact that 50% of our employees hate AI and don't want to use it? How do we make sure they're not just doing it as an answer engine and they're actually using it to do real high value strategic work? Like there's so much more that comes with true AI adoption and the scaling of AI. And if you're in a firm today that offers those services and you're experts in change management, the unfortunate reality is most companies don't realize yet they need that. So you're like, you're ahead of the curve. And so you gotta figure out a way to offer the services they new need today while they grow into the services they actually need and don't know it yet. So I don't know that there is an answer to what is a lasting competitive advantage. I don't know that such a thing exists at the moment other than staying on the edge and figuring out what the next competitive advantage is. I think there's going to be these like almost like S curves of competitive advantage. It's going to be like, oh, you're it, you're like the leader in X when it comes to this as an agency. And then it just like plummets, like falls off a cliff because like now everybody does it. Or ChatGPT just enabled the thing you were doing out of the box or like Claude introduces co work and now it just does the thing, whatever. Like it's going to be constantly this like, all right, well that gave us an edge for 12 months. Now what's the next edge we have? So it's a very difficult environment, which also means there's tremendous opportunities. A lot of agencies are going to struggle and fail in this environment. They're not designed by it. They don't have the right leaders in place who understand the moment. They don't have a situational awareness to how fast this is all moving. And if you're in an agency that does, you have a tremendous opportunity to get out ahead of everybody else and figure this stuff out as you're going.
- [52:42]
B
Absolutely. Number 13. Some slower AI adopters on my team struggle to discern what looks like AI and what doesn't. Any tips or resources to help or is this just something that improves over time with experience?
- [52:56]
A
I'm going to, I don't know if I'm going to interpret this one the right way, but the way I'm interpreting it, Kathy, maybe you have a different interpretation, is if they're using AI to help them with something and they output it and someone else looks at them like, well, you just used AI to do that. Like that's how I'm interpreting this. I've had certainly plenty of instances of this where you get something and you're like, this is literally just copied and pasted from ChatGPT. Like I can tell from the format I don't have to read this thing. Yep. And so then the question becomes, okay, like let's have a meeting and I want you to walk me through this. Like, I'm not reading this 25 page document that I know you didn't even read. Let's sit down and. And I'm going to have actually just grill you on questions about it and maybe I'm going to take your output and say, hey, one of my employees just gave me this 25 page creative brief. I'm not sure that they did any critical thinking about it. What? Give me 10 questions, I can ask them about it. That will prove to me they actually know what they created. Like something like that I could totally see happening. I could see that happening. In HR examples where you're like interviewing job candidates, you need to get to the point where you're saying like, did they actually think about this? Did they apply critical thinking to this? Did they verify the outputs of it and do they have confidence in the material that they've presented to me? And so you can usually figure that out pretty quickly. You don't have to take like 30 minutes to do this. But that's how I deal with it. It's like if you're handing something in or you're publishing something publicly, you better have the confidence in that stuff, that it is accurate, that you actually understand the topic and that if put on a stage and asked questions about it for 10 minutes that you could actually confidently answer those questions. So like, do the work. Like, there's no substitute for actually understanding a topic.
- [54:38]
B
Agreed. Number 14. I am new to AI and trying to use it more in my work. During the super bowl, which was just a few weeks ago, also AI had a big presence. Claude's commercial stood out, especially its message that ads are coming to AI but not to Claude. As ads enter AI platforms, how might that impact day to day workflows? What should we be paying attention to as this transition unfolds?
- [55:02]
A
I don't know that the ads are going to impact the day to day workflows much at all. If you have the paid versions of ChatGPT supposedly, well, the free and the go version, which is the $8 a month will have ads. The teams Enterprise Pro like those won't have ads. As of right now. Google says that Gemini won't have ads. Claude obviously is taking the position, you know, that they don't. That doesn't mean they won't eventually experiment with them. Ironically, on the way in today I was here goes that like if I have 15 minutes, I listen to a podcast, OpenAI podcast. That's actually what it's called, the OpenAI podcast. Episode 13 is the thinking behind ads and chatgpt. And it's actually with one of the executives that I think leads that team. One of the leads at on ads, Asad Awan. So you can go listen to that and it shares their perspective on ads. I don't think largely they're going to change much of anything for business users. I think they'll largely be excluded from what you do for a while. Maybe they eventually figure out a way to elegantly integrate them. But I think there's gonna be a lot of experimentation before any of that would ever happen.
- [56:11]
B
Yeah.
- [56:11]
A
And then the only thing to pay attention to would be like if your brand once, like right now you have to be a major brand to like be getting in the pilot for running these ads on chat GPT. But I think if it's proven that large language models, AI assistants is a viable place to run ads, I think the question more becomes on the brand marketing side, should you be there?
- [56:30]
B
Yeah, it's only an episode 13.
- [56:35]
A
They just launched that podcast last year. Yeah.
- [56:37]
B
Wow. We're hit. We're at 200 almost.
- [56:39]
A
I know. Yeah. A little out of the curve.
- [56:42]
B
Okay, number 15, if you could give every business leader one AI powered superpower starting tomorrow, what would it be and why?
- [56:49]
A
I think it would be the situational awareness thing. I mentioned earlier that way too many CEOs have no real concept of what's happening. And this is like at every size company I talk to publicly traded, venture backed, private equity owned small business, mid market. Like the, the true concept of how advanced these models already are, how, how many use cases there are across organizations that don't even require you to connect any data. Like so many times I hear there's a slowdown because they're waiting for it or concerns about data leakage and things like that. And I think if, if the leaders truly understood the moment and where we are and where we're going in the very near future and how disruptive it's all going to be, they would be moving way faster than they're moving. And so awareness and understanding of AI at a deep level is like the superpower I would give them, because I think everything else comes from there. If you know the what. What you know, some of the leaders know, you would be acting in a way more urgent way. You would be establishing a vision, sharing that vision with your team. You would be providing the right tools, personalized training to them. You would be standing up centers of excellence across departments. You would be empowering the leaders of each department to identify and personalize use cases. You'd be doing all of that. But it has. First, you have to understand the moment. And I, again, I can speak with a very, very high level of confidence because I'm in the room with these executives every week running workshops and doing speaking engagements with hundreds of these executives. And I'm very confident that what I'm saying is true. Like, there is a very low level of understanding of this technology and how far along it really is at the highest levels of almost every organization I talk to.
- [58:46]
B
Okay, thank you. And speaking of awareness and understanding, our State of AI for Business survey is in the field right now. And if you're a marketer and you take the survey, please give it to your other departments and your organization. We want a big holistic view of this. For this year's report. We have big goals on the number of people we need to take the survey. So please, it's SmartRx AI backslash survey. So if you've got a few minutes, take that and please share it with the rest of your company. We'd love to get a lot of responses on that.
- [59:18]
A
Yeah. Last year we had, what, 1800? I think so.
- [59:20]
B
Almost 1900 last year. We're trying to, like, almost triple it this year.
- [59:24]
A
Nice. I'm usually the one that's saying triple it. That's good.
- [59:27]
B
I say that out loud, those goals.
- [59:28]
A
Yeah, yeah. You're usually like, don't say it out loud. Don't say the goal. All right, well, thanks, Kathy. It was great. Thanks, everyone for, you know, attending the classes, for asking great questions. And just a reminder. Yeah. Mastery members, episode 200.
- [59:44]
B
Yes.
- [59:45]
A
Yeah. So March 2nd is. If you're. If you're an AI Academy mastery member, you are invited to attend a live recording of episode 200. So on March 2nd, we're going to do the live recording. Like, Mike and I are going to show up, do our usual thing, and then we're going to just take questions from the audience. After that, we'll turn off the recording and we'll just hang around and take questions. So it's a, it's an experiment. We've never done this, so 200 episodes in, we've never done one like this, so it should be fun.
- [60:11]
B
So the episode that drops on March 3rd, we're going to be recording it on March 2nd in the morning. So if you're an AI mastery member, you can log into the learning management system and sign up for that and we'll see you at 9am on Monday for the recording.
- [60:24]
A
Yeah, should be fun. All right, thanks, Kathy. Thanks everyone for joining us.
- [60:27]
B
Thanks, everyone.
- [60:29]
A
Thanks for listening to AI answers. To keep learning, visit SmarterX AI where you'll find on demand courses, upcoming classes and practical resources to guide your AI journey. And if you've got a question for a future episode, we'd love to hear it. That's it for now. Continue exploring and keep asking great questions about AI.