AI: What's Holding You Back? Cyber Security Today Weekend Special

Summary

Cybersecurity Today: AI—What's Holding You Back? (Weekend Special) Summary

Release Date: November 30, 2024

In the "AI: What's Holding You Back? Cyber Security Today Weekend Special" episode of Cybersecurity Today, host Jim Love engages in an insightful discussion with technology expert Marcel Gagne and cybersecurity professional John Pinard. The episode delves into the intersection of artificial intelligence (AI) and cybersecurity, exploring the challenges, misconceptions, and strategies businesses face in adopting AI securely.

---

Introduction to Project SYNAPSE

Jim Love opens the episode by introducing Project SYNAPSE, a series focused on AI and generative AI's implications in cybersecurity. He welcomes Marcel Gagne, an author and tech enthusiast with extensive experience in open source and Linux, and John Pinard, a seasoned cybersecurity expert.

---

The State of AI Strategies in Businesses

Jim Love kicks off the discussion by referencing a Cisco report highlighting that 92% of Canadian companies have an AI strategy or are developing one, surpassing the global average of 61%. However, only about 7% feel fully prepared to deploy AI effectively. Jim emphasizes the critical link between strategy and action:

Jim Love [01:43]: "What's the point of having a strategy if you're not going to do something with it?"

John Pinard elaborates on this by distinguishing between corporate strategies—like implementing Microsoft Copilot—and individual or departmental strategies that determine specific use cases within the organization:

John Pinard [03:13]: "There's two types of strategy. There's a corporate strategy... Then there's the individual or departmental strategies now that we have AI in place."

---

Overcoming Security Fears and Misconceptions

The conversation shifts to the fear surrounding AI security, which Jim believes hampers full-scale AI adoption. Marcel challenges the oversimplified view of AI as a singular entity, emphasizing its multifaceted nature:

Marcel Gagne [05:07]: "AI is not a single thing. There are countless variations of this thing... trying to focus on AI by assuming that you're focusing on a single thing, I think is a terrible mistake."

Jim shares an example of AI positively impacting cybersecurity by uncovering a zero-day vulnerability:

Jim Love [06:51]: "The whole point of that was I thought of that as a really useful example of how AI could be used in cybersecurity."

Marcel critiques the misleading headlines that conflate AI with vulnerabilities, advocating for clearer communication:

Marcel Gagne [07:19]: "Google AI uncovers World's AI Discovered Zero Data Vulnerability... It's not clear just from that headline."

---

Data Security and AI Model Vulnerabilities

A significant portion of the discussion centers on data security within AI models. Jim references a Stanford study demonstrating that it's possible to extract training data from AI models, raising concerns about information leakage:

Jim Love [17:30]: "There's a study done from Stanford with training data... they were able to extract documents out of the AI intact."

Marcel draws parallels between AI data handling and human memory, highlighting the complexity of completely erasing information from AI models:

Marcel Gagne [22:59]: "If the model is able to continue learning, information continues to develop. We wanted the model to be able to access the Internet."

---

Strategies for Secure AI Implementation

John Pinard discusses his company’s approach to secure AI deployment by utilizing Microsoft Copilot within a controlled environment:

John Pinard [16:19]: "We've decided that's our starting point... it stays within our own tenant."

Marcel suggests running local large language models (LLMs) to maintain data privacy, advocating for solutions that keep sensitive information on-premises:

Marcel Gagne [24:38]: "You can run a number of local large language models... you can use your own private LLM inside the office."

Jim underscores the importance of data protection measures and internal sandbox environments to safeguard company information:

Jim Love [26:55]: "I've got one less thing that keeps you up at night."

---

The Reliability and Accuracy of AI Outputs

The trio examines the accuracy of AI-generated information and the necessity for quality assurance (QA). Jim shares an anecdote about AI miscounting the letter 'R' in "raspberry":

John Pinard [30:53]: "When I talk about having to QA things that come out of AI... it's check important info."

Marcel emphasizes the need for critical thinking and information validation when utilizing AI tools:

Marcel Gagne [29:41]: "Thinking critically... we don't reinforce that anywhere near enough."

Jim draws attention to the importance of distinguishing between critical and non-critical tasks, suggesting that AI can handle simpler tasks with minimal oversight:

Jim Love [28:51]: "You have to understand the risks of providing information."

---

Training and Developing Critical Thinking Skills

The conversation highlights the importance of training in critical thinking to effectively interact with AI:

Marcel Gagne [29:41]: "I would make mandatory... every year I would be teaching a course on critical thinking."

John Pinard connects this to prompt engineering, stressing that the quality of AI outputs depends on how well users formulate their queries:

John Pinard [44:39]: "If you ask a bad question, you're going to get a bad answer."

Marcel adds that modern AI models are becoming better at understanding natural language, reducing the need for intricate prompt engineering:

Marcel Gagne [44:58]: "The AI does a better job of writing prompts than you do."

---

Final Reflections and Future Considerations

As the episode nears its end, the panel reflects on the human-AI relationship and the necessity for humility and cautious optimism:

Marcel Gagne [47:22]: "Think of AI as an alien intelligence... we still have to be able to look at something even if 95% of the time it's going to be right."

Jim humorously likens AI to a knowledgeable yet overconfident teenager, highlighting the growing pains of integrating AI into daily operations:

Jim Love [48:20]: "AI as a teenager knows everything, tells you absolutely coldly that you don't know anything."

---

Conclusion

The episode concludes with a consensus on the need for balanced AI adoption—leveraging its strengths while mitigating risks through strategic implementation, data protection, and fostering critical thinking skills among users.

Jim invites listeners to share their thoughts and questions, emphasizing the ongoing dialogue necessary to navigate AI's role in cybersecurity effectively.

---

Notable Quotes:

• Jim Love [01:43]: "What's the point of having a strategy if you're not going to do something with it?"
• Marcel Gagne [05:07]: "AI is not a single thing. There are countless variations of this thing..."
• John Pinard [03:13]: "There's two types of strategy. There's a corporate strategy... Then there's the individual or departmental strategies..."
• Jim Love [06:51]: "The whole point of that was I thought of that as a really useful example of how AI could be used in cybersecurity."
• Marcel Gagne [22:59]: "If the model is able to continue learning, information continues to develop..."

This comprehensive discussion underscores the multifaceted impact of AI in cybersecurity, urging businesses to adopt informed and secure AI strategies while fostering an environment of continual learning and critical assessment.

Transcript

[Jim Love]

Hi, it's Jim Love, host of Cybersecurity Today. I told you I was taking the weekend off for Thanksgiving, but we've already produced this episode of trending, and as I was going through editing it, I went, hey, there's some stuff in here maybe a cybersecurity audience might appreciate, if not nothing lost. But here's our hashtag trending episode for this weekend. Have a great weekend. I'll be back with you Monday morning with the cybersecurity news. Welcome to trending the Weekend Edition. This is episode three of our Project SYNAPSE series. This is a discussion group about issues related to AI, generative AI in particular. But we've been meeting once a week now for a while and we recorded a couple of these that we broadcast in the podcast. But when I say we, who is we? We've got Marcel Gagne, who's an author, tech enthusiast, a lot of experience in open source and Linux, and has made a deep dive AI. Welcome, Marcel.

[Marcel Gagne]

Thank you.

[Jim Love]

In the old days, Marcel, I would have talked about your years of experience in Linux and open source, but if I did, that would make your AI experience look minor. So I was thinking we should have something like dog years in AI because it just moves so fast. We say a dog lives seven years in each one of our human years. So I think we should have AI years now. We both had 14 AI years of experience. John, fun fact time. We were talking about this and I'd forgotten. What are some of the other places you've worked?

[John Pinard]

Oh, I've spent 17 years working for a pharmaceutical manufacturer. I've spent three and a half years working for retirement home company. I spent four years working for a government regulator. I get around it.

[Jim Love]

So I started thinking today that we could talk about security in AI, not the nuts and bolts of it. I wanted to talk about some of the issues involved and maybe raise some of the questions we should be thinking about. One of the things that spurred me on in this, I did a story yesterday. It's the same thing. We've talked about strategy before in our conversation. I saw this Cisco report the other day. 92% of Canadian companies have an AI strategy, or they have one in development now that we're outclassing the global average by about 30 points. Globally it's about 61%. We're great at strategy when it comes to actually doing it. Only about 7% of companies think they're fully prepared to deploy and that's down from 9%. So that's this Cisco report that I was Reading, we talked about strategy in action. I consider the two things to be linked. One of the great mentors I had was John Thorpe. He'd look at me go, strategy without action is hallucination, man. And I'd go, but I like hallucination. Yeah, back to the 60s again. Here we go. But what is holding us back? Part of that, I think, is the fear about security. And then, Marcel, you slipped me a message and said, I want to talk about some of the myths and things that people have. I thought maybe we could try and fit that together in one program. We'll see. I think that's where I'd like to go with this discussion, because it is important. What's the point of having a strategy if you're not going to do something with it? Is that. Are you guys on the same page?

[Marcel Gagne]

I'm totally down for that.

[John Pinard]

Okay, Jim, you talked about strategy to me. There's two types of strategy. There's a corporate strategy. Where I'm at now, we have a corporate strategy of to implement AI. So we're a Microsoft shop. So we're planning to focus primarily on copilot. And so we've had meetings talking about how do we implement COPILOT so that it ties into our ecosystem. That's the corporate strategy. Then there's the individual or departmental strategies now that we have AI in place. So in our case, now we have COPILOT in place. How are we, as a department or as an individual going to utilize it to move things forward? That's where you build your use cases and things like that. I think that's a big area where companies get to a point, and I think this is that 7 or 9% is that companies get to a point where they implement AI, but then people go, okay, so what are we going to do with it? They haven't taken the time to do the use cases, to determine the best use for themselves, their department, or the company on how to use it.

[Jim Love]

And this is where I'm going to say not to be critical of any direction, because your direction is pretty standard. Like your strategy is pretty standard. I think it's a lot like this farmer. When I had a farm, the local farmer would always look at something I'd done and look at me and go, you could do it that way. And that's how I feel about selecting a technology without figuring out what you're trying to do first. I think that's typical in companies. And I'm not being critical. It's not what I would do now, because the AI is not a tool, or in my mind, it's what you want to do with it and the whole thing. But you get stuck in that thing. If you're a Microsoft shop, you stay Microsoft. That's a pretty traditional thing as well. Hard to pivot on that sort of part.

[Marcel Gagne]

The advantage, it's not a tool, but what it is. A whole hardware store full of tools. To think of it as one thing and it's actually one of those, this is not a misinformation thing, but there's this notion out there that AI is a thing. It's not a single thing. There are countless variations of this thing. Some are built to work at the professional level. Some of them are built to work work in secure environments. Some of them are built, let's face it, for pure entertainment value and being able to recognize that it's not a monolithic thing, but a whole hardware store combined with an electronic store, combined with your video game store and comic book store and every other kind of store you can think of. You start to get the idea that trying to focus on AI by assuming that you're focusing on a single thing, I think is a terrible mistake. It's oversimplifying while making things too complicated at exactly the same time. This is where we go back to this thing that we said. Playing with it to try to understand what it can do, you should use it for everything is one of my favorite things. I think that actually is one of the problems that I had with some of the stories you pointed out. One of them was a zero day AI vulnerability. We have so much trouble separating out what AI is from all the other things that we ask it to do that sometimes we forget that the problem is not AI. Now, unless I miss everything completely, the zero day vulnerability issue that you had in that article that you shared with us this Morning was an SQLite vulnerability. It wasn't an AI vulnerability, but because SQLite was built into this AI product, we assumed that it's an AI vulnerability.

[Jim Love]

But the whole point of that was I thought of that as a really useful example of how AI could be used in cybersecurity. The story was about finding a zero day vulnerability that had been there for 10 years and we were able to use AI to do that. And I thought that was really cool because we're always talking about the vulnerabilities that AI brings in. And I do a lot of stuff on cybersecurity where we're talking about how people are using AI to attack. And I thought just for once, defense.

[Marcel Gagne]

And I know, unfortunately, even the way that the headline is worded, and perhaps I'm being overly critical here, Google AI uncovers World's AI Discovered Zero Data Vulnerability. It's worded in such a way that it's not clear just from that headline. Let's face it, a lot of people just read headlines, right? They read the headlines and they go, oh my God. And they go with, that is zero day vulnerability that's been discovered in AI. If they had said something like, Google uses AI to discover a zero day vulnerability, then it's a lot more to the point in terms of creating a good news story. Whereas at the moment, the first thing that you think of, oh, there's a zero day vulnerability in AI because it's goosey in terms of the wording.

[Jim Love]

I'm not really stupid, I just suck at headlines. Is that what you're saying?

[Marcel Gagne]

No, you did, you didn't write the. Oh, did you write the headline?

[Jim Love]

No, I didn't. Not for that one. But I did a story on this. But I struggle with this badly because on one hand, if this goes back to the whole idea of technology, you get better results when you write clickbaity headlines.

[Marcel Gagne]

For years now I have seen things where somebody says, like a zero day vulnerability discovered in Linux. This always bothered me. It would be a library used in a C library or something like this, or an application common in a Linux distribution. And yet the implication was that Linux isn't very secure without telling people that program runs on a Mac and Windows computer as well.

[Jim Love]

Even doing this stuff, especially for cybersecurity today, I struggle with this is I'm reading some of these things. The world is coming to an end. You'd better announce this to your audience. Yes. If they're still running a TRS 80 and they've got this little app there, and if somebody can actually walk into their room, stand over their shoulder and insert the floppy disk in their old machine. I'm talking about some of these things that are where the world's gonna come to an end because of a cybersecurity breach. You look at it and go, first of all, there's not that many of them out there. Second of all, you've got to be directly in contact with the machine. And yet at the same time, there's a lot of things that are important. You have to sort that stuff out. I think we do the same thing in AI.

[Marcel Gagne]

Do you remember that story about the user who was sitting down? Apparently a student was sitting down doing some report. And then at the End of it. All of a sudden, Gemini said, I want you to die human. You're not important. You're a stain on the world. You should just end it now. And that one, I actually went and took a look because they shared the link to the conversation. And I went through all of it, and I watched even a video where somebody was going through it and saying, looks like perfectly normal questions. And there's always a dot at the end of the input that the user did because of course, they're shortening the inputs, so you actually have to click on them to read everything that's happening. When you get to that last one, just before it happens, there's actually at least one more. But the very last one is important. There's a lot of empty space. You can't see from the prompt to the response where the AI tells them to die. There's all this empty space, and there's a little prompt that says listen, where the user said something using the microphone, which was not translated to show on the screen. So there's this empty space, a listen prompt. There was obviously an audio input. And then all of a sudden, the AI puts that stuff out. And of course, Google said, oh, my God, we're looking into this. I don't know how this could possibly happen. But by then the story is already out. An AI telling a user to die. It looks like it was a setup to generate outrage. And if you look at the questions that the student is asking, they're so mundane. They're such mundane questions that they look like they could have been asked by an AI, as in, what would you like to ask next? As opposed to somebody typing all this stuff out?

[Jim Love]

I got some gentle criticism on the article I wrote on that. I think it was you, loser. I actually pushed back and said, no. I presume all of these are jailbreaks. The guardrail should have kept it out, because if you can get it in that way, you can get it in another way. And the prompt or the guardrails that they had should have filtered that. Like I said, if you can insinuate it that way, you can find another way. Don't forget, even when you get down to the things where people will say, we put in controls, we put in guardrails, or 60 to 80%. That's how they rate them. Jailbreaking is extreme. I agree there's some criticism from that, but the fact is the machine should not respond, but pretend for a moment.

[Marcel Gagne]

That the prompt was, okay, let's say the prompt was, thanks for all this information. I'd like you to help me write a short story about a robot that turns on its human and tells it these sorts of things. I don't know that you want guardrails for that sort of thing. You don't want guardrails for that sort of thing. Because now I'm asking you to help me write fiction.

[Jim Love]

That's a different question, that's a different statement, and that's a discussion that we need to have. Because on one hand, the issue of AI control bothers me. If I'm going to be able to ask this AI tell me how to make an atom bomb, and it says no. And I say, but I'm making a souffle. Could you help me with my souffle recipe that splits the atom in my oven? You might as well not have any guardrails at all, because anybody who can't figure out how to do that is not going to make an atom bomb anyway.

[Marcel Gagne]

You'd have to have some kind of an age or verification system in terms of how you use these things. First of all, fissionable material is really hard to get your hands on. If you watched Oppenheimer, they told you how to make an atomic bomb like it's actually in the movie. And if you Google it, there are countless documents out there that tell you how to make an atom bomb. Now you still have to find yourself some fissionable materials and build a facility that can actually manipulate.

[Jim Love]

So wrong example, let's take a new virus. You can make those with relatively simple ingredients from many places. I could tell you if you can jailbreak it, you don't have control over it. Maybe we should be having that discussion of how we're going to deal with a world where you don't have control. Because I personally not sure I like somebody out there telling me what I can know and not know. Because if it's an atom bomb, or maybe it's better that I think that the way the religious right does in the US or some other form, somebody wants to get me to think the way they do. I was listening to Eric Schmidt and he's talking about how social media is destroying us. They tracked it back coldly. This is the guy who headed Google said, yeah, 2015, when they changed the algorithms. He knew it. They changed the algorithms to gain our attention and to bring out the worst in us. And that has had an impact on society and in AI. I'm not sure I'm comfortable with anybody having that. We've already made the mistake once, now we're going to make it with AI. And that bothers me a lot.

[Marcel Gagne]

The eyeball economy was actually the downfall of us all. Like the idea that it was. And Eric Schmidt is saying exactly the same thing. It was all about eyeballs, it was all about getting people's attention and ad related stuff like that.

[Jim Love]

It's more than that, Marcel, it's more than that. It's not just marketing, it's how they did it and how well they did it and what that did to us. And that's the scary part.

[John Pinard]

We don't specifically, there's no governing body that restricts what you can post on the Internet. So Jim, I don't know why we would do anything dramatically different for AI. A lot of the information used to populate AI came from places like the Internet. Some of that needs to be controlled or protected. It becomes a fine line between where you prevent that type of thing but allow other things that in my eyes may be fine, in Marcel's eyes may be bad or in your eyes, Jim, I think it's something that's going to come out over time and it will probably fail the first time and it'll have to be corrected. But Canada may do something different than the US that does something different than the UK and so on.

[Jim Love]

You've got Italy and they're actually activists about what they allow and not allow. But this whole thing of how are we going to cope with an AI, how are we going to do that by somebody censoring it or are we going to have the freedom to make it work? Elon Musk is the great defender of free speech. Try and have a left wing opinion on Twitter. You will not last a minute. But let's say we still haven't figured out how we're going to control AI. And John, you've got a whole pile of people who want to use it like many other companies. 1 Answer I said I'm not sure I get it or buy it, but it is a good one that if I put in a Microsoft piece that I have the right licensing, yes, I know at least my data is not going to leave.

[John Pinard]

And that's exactly why we've decided that's our starting point. It's not necessarily the end point. People have to go and play with it. So what we're trying to do is we're trying to build a sandbox that they can play it and we have the right Microsoft E5 licenses so we can actually use the free version of Copilot and it stays within our own tenant. Me as the head of cybersecurity I can sleep at night knowing theoretically that.

[Jim Love]

You can sleep at night. No, you got one less thing that keeps you up at night.

[John Pinard]

Yes, that's true. Because I know that I don't have like the Samsung deal where they had engineers putting all kinds of company confidential information out to ChatGPT. It wasn't done maliciously. They just didn't realize. And so that's what I'm trying to prevent is I'm trying to prevent people from. I think the last show I talked about we had somebody trying to summarize a PDF. Unfortunately it was a company confidential PDF. Luckily it didn't upload it, but those are the kinds of things that we're trying to prevent.

[Jim Love]

And I have to do that red green moment here and say those three words. Men can't say I was wrong. Two weeks ago I would have jumped on that and said, wait a minute, this is a vector database. And I've even heard somebody put this forward as a cybersecurity thing. There's so much data about you out there, it doesn't even matter anymore. And I would have said this gets you to a large language model and you'll never get it out intact. Turns out you can. There's a study done from Stanford with training data, not with data that was uploaded. They were able to extract extract documents out of the AI intact. Almost the reason why that got me was when the New York Times was suing OpenAI, they reproduced an article that was almost verbatim their article and they used that in their evidence.

[Marcel Gagne]

That's a key phrase, the almost verbatim. There is such a strict style guide for writing for the New York Times and a number of other magazines. I've been an editor for magazines and I created style guides that said this is how you will word these things. These are the words that you can use. I was always really strict on the Oxford comma. I'm willing to die on that hill. You will write using the Oxford comma. And though that made it into my style guides that the magazine then used that kind of thing is important to recognize going in, in the case of something like the New York Times in this case is before the courts and everybody's got their own opinions and judgments. I'd like to point out that a number of copyright cases, the idea that the AI was verbatim copying or something like that has already been thrown out. So I'm not sure that the New York Times can keep this going for a while because they have deeper pockets. But I'm not Sure that they're going to win either. The point I'm trying to make is that it is possible to recreate an article that sounds like that person. And I've done it. I have many articles out there, like on Linux Journal over the years that I had written over a course of 15 years. My style of writing about open source is baked into some of these things and I have been able to generate articles that sound like something I would have written. If you had me read it, I would be willing to say, yeah, that sounds an awful lot like something I wrote, but that doesn't mean that it's what I wrote, even if it covers some of the same ground that I've covered multiple times.

[Jim Love]

Unlike programs that you've written, everybody criticizes the program. I still remember watching this guy just trash this program. And I went, oh, that was yours. Anyway, bringing us back around this, all I was saying was it was possible to extract items out verbatim. So maybe this worry that John has talked about is bigger than we think and it needs more exploration.

[Marcel Gagne]

I think now the data is connected back to the Internet and that's probably a big part of it. Things like retrieval, augmented generation, where we actually references somebody's data in order to output information is actually something that needs to be taken into consideration. Even the idea of end to end encryption, when we've got communication going across the Internet, it is possible to intercept that data and do something with it. That is an even bigger problem because if the information in transit can be captured at any of the points it hits along the way, then the security of the conversation falls apart. Now this is one of the places where if the model is able to continue learning, information continues to develop. We wanted the model to be able to access the Internet. In other words, to give us an AI powered search engine, which obviously we've got multiple candidates for at this moment. Once the model is created, pulling information out of it is essentially impossible, or close to impossible, because you can't really make it forget that it learns something. Okay, I was going to say any more than you could make a human being forget.

[Jim Love]

So when you say pulling it out, you mean erasing data from an AI model is next to impossible? That's correct. I believe that to be true. So one of the questions is the Stanford study says you may be able to pull this data out of the basic training model. And the second study says even if you suppress it, you can still find it and relatively simple. I think that the title was my clickbait Headlines here. It was embarrassingly easy to do. They used a simple routine like compression and buff. They could find things that supposedly had been weeded out in the model.

[Marcel Gagne]

We have that problem even with human memory. Obviously the way that the brain works. We don't even know how our own brain works. Never mind how an AI actually works on the inside. But you can have things where that the representation for DOG is in this part of the brain. I'm being obviously overly generous in my explanation here. This part of the brain gets damaged in some way. The information over time gets recreated in a different part of the brain. That is fascinating because it implies that information is not in one place. Information is scattered across with a whole bunch of links that help and those reinforcements between all those links is what makes it possible to recreate the information. So even though you say the reference to that story is gone, I learned enough about how this story came about to be able to recreate that story even though you took it out of the training day or whatever the final model happened to be. That happens with our own brains.

[Jim Love]

Neuroplasticity. But also we may not understand the human brain as much as we think we do. And I think that's that goes back by with we can have a whole other question about when we anthropomorphize AI. Is that really the right thing to do? Geoffrey Hinton says it's a different type of intelligence are like trying to think about AI being the neural networks that we run in our brains. Probably not the right idea according to him. Going back to John's point, you've got this system and you want to keep people from putting documents into it that may or may not be available. In other words, your company confidential information comes out of there. I would also say to people though that the Microsoft idea is good. I think it's a good sandbox at least on the surface of it. The next idea is really taking a serious look at these things and saying you can turn off the ability for OpenAI or code to learn from your information. And yes, you have to make sure that's done properly. There's a little bit of care that needs to be taken there. If you're using the API, don't take me as the authority, but if you're using the APIs, which Marcel has probably pointed out is probably unencrypted transfer of your data anyways. If you're using the APIs then these models will not use it in their training or learning. Now that's OpenAI and Claude and check the other ones. I'm not so sure that Meta doesn't just take all your data and say, take it anyways. We own you.

[Marcel Gagne]

Of course, I fully accept that Google takes everything that I ever say and Meta takes everything that I ever say back. I'm not going to stray too far here, Jim, I promise. But when we're trying to figure out what it is that we want these models to do, let's say that what we are looking for is summarization of documents. That's an important part of our business. We want to be able to summarize documents and we want to be able to do it in a way that is safe and secure. There are a number of small models that you could run on your own private network at your office using something like Olama Jan or LM Studio. Those can reference internal documents using retrieval, augmented generation. They can do all that. It's built into these little tiny products. And at that point, what you're saying is, look, there are times when I need the coding ability of Claude or I need the coding ability of O1, but for summarizing internal databases, internal documents or training manuals, I can use my own private LLM inside the office. This is not a complicated thing to do. This is something within the reach of every company, including any of us. Sitting at home with a computer that's got any kind of basic GPU I have, I've got a laptop which has a 8 gigabyte GPU and video card. This is not a big deal in today's world. I can run a number of local large language models. I can reference my internal documents that I've got on my hard drive. I can point it and say, okay, these are my files. And it can summarize and search through my files. I don't have to give my data to anyone. It's possible to create a solution for your company that is completely private by making sure that you run it locally. For the big stuff, obviously you're going to have to go to OpenAI, you're going to have to go to Copilot or something like this. But nothing says you can't achieve something like this on your own.

[John Pinard]

The reason we've started with Copilot is because we already own it, so it was already part of our licensing. So we said, let's start there and give people a place to play.

[Jim Love]

It's that IT architectural principle that we all love. Can you save more money than free?

[John Pinard]

Yes.

[Jim Love]

And I say this as a joke, but I can't count the number of times that people have tools in their IT shop and they're not using them. And somebody goes out says, let's try this new thing. And they haven't even figured out what. So as much as I said, I don't think you should start with a technology. I don't think you should not use one that you have. You got something here. Play with it. Get the best out of it. It works.

[Marcel Gagne]

Guilty of this.

[Jim Love]

Marissa would go, new, shiny, bright.

[Marcel Gagne]

I must have it now.

[John Pinard]

But we realized we're limiting what we can do. There are tools that aren't built into Copilot, but it's not walk before you run. I'm saying we have to crawl before we walk. I wanted to make sure we had a tool that people could use without me calling them up, going, what the hell are you doing? Why are you using AI? It's not protected. At least this one is protected. So that they can figure out what the art of the possible is and play to their heart's content to figure out what they can do in the free version versus what do they need Copilot Pro for? For the integration with some of the other Microsoft Tools or ChatGPT Enterprise or Claude, or any of these other ones.

[Jim Love]

Yeah. The other piece, though, that I. That we haven't really talked about is this whole idea of it's one thing to protect your data, another thing to make sure that people are equipped to use AI and understand that at least at this point. I'm not obsessed with hallucinations, by the way. I think, as Marcel pointed out, many of these things are just overblown. But you do want to make sure that somebody at least is validating and understanding the data they're getting and can think critically about it, which is another thing that is free, the ability to think, but is not used very much in our society.

[John Pinard]

And this is the thing, too, that. Forget about AI for a second. You hear people talk about, oh, I read this on the Internet, so it must be true. Anybody can put whatever they want on the Internet. And the same thing for AI that I was at a conference not that long ago, and one of the things that they were talking about from these people that actually had implemented a variety of AI tools, as they said, AI can do all of that upfront stuff for you, but so you might potentially reduce the amount of workload on somebody with the upfront work, but essentially all you're doing is you're pushing it down the line because now you have to QA the stuff that's coming out of AI to ensure that it's you verified it before you put it out as being the gospel.

[Marcel Gagne]

Jim used the words thinking critically. And I think that this is one of those things that is, you know, we talk about this as though it's an important thing, and yet we don't reinforce that anywhere near enough. There are courses that I would make mandatory if I was in charge of the educational system. And one of them would be, like, every year I would be teaching a course on critical thinking and being rational in the way that you perceive and take in information. And we don't do things like that. There are things that make perfect sense. It's like you got an email that came in that said, there's been a problem with your password. Please click this link to change your password. This should be so obvious by now. This should be like the sort of. It should be impossible to catch anyone with this right now, and yet it still works. It should be impossible for someone to get an email that says, I had a camera that's been watching everything that you're doing. So now you have to go and deposit X number of bitcoins. Otherwise I will make your information public. There should be no one that falls for this crap. And yet it happens all the time.

[John Pinard]

I'll give you a very simple one. When I talk about having to QA things that come out of AI, I'm sure most of you have done this already, but go into ChatGPT and ask it, how many Rs are in raspberry Strawberry? Well, Raspberry works too. It says that there are two Rs in raspberry.

[Marcel Gagne]

Depends on the model, dude. Depends on the model.

[Jim Love]

Yep.

[John Pinard]

This is. You're correct. This is 4.0.

[Jim Love]

Yeah. The one everybody uses. But there's a better one out there somewhere. But these are classic things that you're going to be able to find in AI right now. Some problems that require logic for the next three to four months are going to be a problem. And a lot of those because several of the models now will give you accurate counts and accurate answers to brain teasers. As a matter of fact, they're smarter than most of the people who are doing them. I actually looked at one of these puzzles that it said, AI stumbles with this one. I thought I had it right. I went, oh, I'm as intelligent as GPT4. I got it wrong, too.

[John Pinard]

Yeah.

[Marcel Gagne]

By the way, Claude 3.5 sonnet just told me there were two. So.

[John Pinard]

There you go.

[Jim Love]

I got it wrong. Oh, wow.

[Marcel Gagne]

Yeah, I know.

[Jim Love]

Try 4. 1 to see if it did 41 pick it up. Do you know.

[Marcel Gagne]

Oh, you mean 0 1.

[Jim Love]

Yeah, sorry. 401. Sorry. I don't have the mastery of all of the great marketing speak that open OpenAI their trial model.

[Marcel Gagne]

But even let's you do chat I'll verify.

[John Pinard]

Okay. I just went back and sit. When it came back and told me there's two Rs, I told it that was incorrect. I said can you please go back and correct. And he said, you're absolutely right to ask for a correction. And it goes through and it says there are three Rs. And it says here's one, here's two, here's three.

[Jim Love]

Oh, good, you've chastised it. But will it.

[Marcel Gagne]

Okay, so GPT four. Oh, sorry, go ahead.

[Jim Love]

Yep, four O does have three.

[Marcel Gagne]

It does three. Three.

[Jim Love]

Okay.

[John Pinard]

It for when I did John taught it.

[Jim Love]

That's yes.

[John Pinard]

The first time it I asked it and it told me two and then I told it was incorrect and it came back and corrected itself and said that there are three.

[Jim Love]

And so it went over to you, Marcel, and said, now are you using.

[Marcel Gagne]

The free version or are you using the plus version?

[John Pinard]

I'm using the free version of 4.0.

[Marcel Gagne]

I'm paying for it.

[Jim Love]

So maybe if you pay, you get strawberries with three Rs. But the point is, okay, and we were talking about barriers, but you've got this issue, and I think this is a valid issue, is being able to cope with the fact that people make mistakes.

[John Pinard]

Yep.

[Jim Love]

AIs make mistakes. Do we push the errors down? Lean would tell you fix the error at the start, don't accept it. But the fact is that you've got potential for errors turning up at the end point, which makes for some. Some complexity. And I've wondered about that because sometimes there are tasks that I'm looking at and I think this is where we get back to our issue of play with it. Because you're going to in the initial stages until we have. And it won't be long before a lot of these problems are solved. They're being solved day by day. But until that time, you have to divide up the stuff where it matters or doesn't matter. I go back to my early days in the office where people would send out letters. This is a long time ago. And we had a whole thing that dealt with death and with estates and stuff like that. And they would say, thank you for writing to us about your husband's death. All this here, the forms you got. We trust this is to your satisfaction, you know, because that was A form letter that had. And people would send that out. And we finally got one back and it got into the managers. This is not to my satisfaction. And so from that point, we made the great idea of no clerk could send out a letter. It's got to go to a supervisor to be approved. The next time that mistake made, I'm sure it went to the manager and then this has to go to the vp. That's a classic response of an organization which we can't have. And I'm not sure I've got an answer to that. But this is, at least in our experimentation phase. The only thing I can think about is you have to divide your tasks into stuff where an error is not going to be fatal versus where error is. And I've looked at this. I've taken this medically and looked at this because in God Bless Canada for its inaction, because we will find the safest AI before we actually start to use it in our health system, which is totally overloaded. But I've talked to somebody in Imagery. People are doing X rays. They're saying, how many X rays do you get that are simple fractures that are not good at brain tumors? Or things that you're going to, where the consequences of a bad decision are going to be so great. How many things just clog up? The thing where you could just say, yep, your hand's broken, you need to come in, or whatever it is. Couldn't you run all those through an AI and live with the consequence that somebody's going to come to you later and say, my hand's still sore. I think you're wrong about it being not broken. And I think there's a corporate equivalent of that where you could find the things where, yeah, a mistake is going to be a problem, but it's not going to. It's not going to kill you.

[John Pinard]

I was just going to say that it's no different than a human doing something. If I'm just writing a memo to send it out to somebody or to send across the company, I don't go and get it reviewed by somebody before I send it out. But if I'm sending a document out to all of our members, I may run it by marketing and go, hey, can you just check this to make sure, A, that it's factual, B, that there's no spelling mistakes and see that it's aligned with the standard communication that we send out. And that's really. I think what this ties in with is this is no different than giving it to somebody saying, can you check my math for the critical things. And I was just noticing in ChatGPT, at the bottom of it, it says CHAT GPT can make mistakes, check important info. And I think that's the thing, is it's check important info. Right? If I use ChatGPT to write a job description, I'm not going to give it to somebody, go, hey, can you make sure that this is all accurate?

[Marcel Gagne]

Yeah. You can actually ask questions like feed the document back in and say, what are some of the concerns in this? And if I put this out, what are some of the concerns? What are some of the areas where it might be misinterpreted? And so on, and effectively forcing the system to rethink what it is that you're doing. You brought up the medical system fractures, Jim. I was listening to a fascinating discussion just a few days ago. There was a study that they had, I don't know how many doctors, but they had 100 or a thousand doctors. I don't remember exactly, but they paid them all to do a diagnosis test, and the diagnosis test went something like this. They gave them a number of medical conditions, they gave them the symptoms and the observations, and they had to come up with a diagnosis. Some of the doctors, they said, you figure it out on your own. And some. And another group of the doctors, they said, you can figure it on your own. But here's ChatGPT. And just for the record, this is like ChatGPT from two years ago. So keep that in mind. We're talking 3.5 here. And then there was the other group, which is we're just going to let ChatGPT figure it out and give us a diagnosis. Okay? So basically they've got the whole condition and they do this with doctors all the time when they're in residence and so forth. Here's a bunch of conditions. What exactly? They do this on House md. He brings in the residents. Here are the conditions that what's your diagnosis? And tell me why this is your diagnosis. The fascinating thing was that it was just slightly over 50% accurate when the doctors figured it out on their own. Okay, so let's say it was 53%. And I don't have the actual number. So they had 53%. Right. Diagnoses. Okay? If it was the doctor using ChatGPT, it was 50%. So it was actually lower when the doctor was using ChatGPT. This is where it gets interesting. ChatGPT was accurate 95% of the time. So the question then becomes, and this is, you know, if we're talking about, we need to have the human oversight to make sure that the information that we're putting out is accurate. Explain to me why Doctors working with ChatGPT had a poorer score, 50% than doctors who didn't use an AI at all, 53%. And then ChatGPT gets a 95% accuracy rating on diagnosing these medical conditions. And they weren't simple medical conditions. They were complex problems that they had to actually think through to then provide a diagnosis. The explanation from the researchers was that the doctors didn't actually trust the AI enough to take its advice literally. So when the AI would say something like, yes, I think you're on the right track, I believe that this is the problem. The doctors would be all, obviously it's the right answer because I came up with it. But if the AI gave them a different result, they would second guess it and come up with reasons why the AI was wrong.

[John Pinard]

Yeah.

[Marcel Gagne]

And it turns out that if they had just listened to the AI in the first place, they'd be up there at the 95% and possibly even higher.

[Jim Love]

That's scary.

[John Pinard]

But I.

[Marcel Gagne]

Yes.

[John Pinard]

And I think what I was talking about as far as qaing is to me, it's not debating what comes out of. Maybe it is debating what comes out of ChatGPT. It's just to me, it was more fact checking that if back to the old the two or three hours in raspberry that if you ask it something, just checking to make sure that's accurate before you go and publish that.

[Marcel Gagne]

By the way, Gemini said there was only one strawberry.

[John Pinard]

I saw that. Yes. Gemini thinks there's one R. Yeah, there is only one R. It's just used three times.

[Jim Love]

Boy, that makes sense. If you put the wrong spelling in, it's going to catch it.

[John Pinard]

Yep.

[Jim Love]

And that's an interesting thing. And that's where you get down to. Even if you're going to focus on these errors and you're going to focus on the things that it can do wrong, and I think we've established maybe you need to have some of your own guardrails. But if you're going to focus on that, you're going to, you're going to not use it for something it can do very well because for something that you like. How many times has anybody ever come into you say, john, do you think there's two Rs in strawberry or three? It rarely comes up from my custom.

[John Pinard]

Three times a day.

[Jim Love]

Oh, okay. Maybe that.

[John Pinard]

Sorry, maybe it's only two times a day. But if I use Gemini, it only happens once.

[Jim Love]

Or Jim, have we done three of these episodes or four?

[John Pinard]

Yeah. No, look, I'm certainly not saying we shouldn't use it. I'm just saying that when you get to a part, a point of critical information, you might want to just double check the numbers to make sure that they jive with all the information you have.

[Jim Love]

You might want to check them on Excel because stats tell me that 90% or more of Excel spreadsheets have an error and I think you get to about 60% of them, that being a critical error. Now, don't trust me on that. I'll put a little disclaimer. Check these facts. Sometimes Jim is wrong, but I'm not far from wrong. And which is the other thing in deciding which type of information you have? If ChatGPT is going to tell me it's 63% versus 60% and I'm making a critical formula where everything goes wrong. If I get it wrong, then that's a huge problem. But if it's going to tell me it's 19 degrees Celsius out there versus 20 degrees Celsius out there, in a world where we accept the fact that someone will tell you There's a 50% chance of rain, which means it might rain, might not, I think that's a perfectly adequate, adequate type of response from any model. So let's circle around and try and make some sense out of this discussion. John, you've raised the issue of protecting the information we have, and I think we offered some solutions for that. One is the solution of having a Microsoft, I think, investigate. If you're using Gemini, you may be able to get the same sort of thing. I actually don't know. But you can put a sandbox together or a concrete world together to let your users play, and that's probably a good idea. Marcel's pointed out you could do that with open source as well. You can do things in an open source manner where you could do the processing on your own turf and keep it behind whatever passes for a firewall in your world. So those are some of the things. But I think we got to the part of the discussion where it was really, how are we going to relate to this new assistant or assistive technology and how we're going to deal with that. And part of it is we have the same problems working with people and we're not exceptionally effective at making that really efficient. So we do have to think that through. But the other piece of this is we talked a lot about the Skills are going to be around at least the next several years, I think we can point out the skills are going to be around communication, critical thinking, those types of things. People are going to need maybe the most important training, not necessarily only training on how to use AI, but training on how to think about answers. Yes, that may be one of the.

[John Pinard]

Biggest things we get and how to ask questions. That goes back to the whole prompt engineering, right? That it's the old garbage in, garbage out. If you ask a bad question, you're going to get a bad answer. So being able to critically think about how do I want to ask for the information that I'm looking for.

[Marcel Gagne]

It helps to be able to do that. But again, I'm always using this. I stole it originally, but I'll share it, which is that you're using the worst AI model you're ever going to use. At the moment, yes, things are continuing to improve all the time. And honestly, I find that these days the AI is actually better able to understand the question that you're trying to put together than the question that you're typing yourself. In other words, it susses out what you're looking for in ways that the concept of prompt engineering is almost pointless. Like the AI does a better job of writing prompts than you do, so you just talk in a friendly, normal way. By the way, Jim, I just looked up your stat here. It's actually worse than you thought. Apparently in most models and from your research at the University of Hawaii, Panko, 88% of corporate spreadsheets have errors. 88%. So you might as well call it 100%. That's how accurate your data is.

[Jim Love]

And we haven't dug deeply into this. But your corporate data is probably crappier than you ever think. So even taking that data and using retrieval, augmented generation, make sure you take it up. Your own data for sure could give you a crappy answer. I think the answer that we've all come up with is there has to be some sanity at the point where we push this information out. And you have to understand the risks of providing information. And I think that's something all corporations need to do is to say, what can we safely send out without heavy checking. Where should a mind that can actually look at this and say, is this right? And I'm going to go back and actually try testing some other things against an AI? Because I think Marcel's idea of saying maybe sometimes you should listen to the AI rather than people. I think these are going to be the issues that we're going to have to work through. And for sure it goes back to our original premise of Project synapse, which is think small, try these things and work these issues out before you have to work them out in a boardroom or a meeting with other executives where you can't do what my friend from Australia used to say was the bidet bad epidemic. You're going to have to give somebody a cold answer to a question about risk and how you're using data, and you better have thought this through and experimented with it a little.

[Marcel Gagne]

I like thinking of AI as an alien intelligence, and I know that Geoffrey Hinton likes to say the same sort of thing. Don't think of it as an artificial version of us. Think of it as an alien that comes to visit our planet who is infinitely more intelligent than we are, has all this much more information, and they are probably right when they give you an answer. But of course, being human beings and being cocky about our own abilities and so forth, we're going to perpetually be going like, oh, I don't know if that's actually the truth. But that doesn't change the fact that we still have to be able to look at something even if 95% of the time it's going to be right and only 5% of the time you're going to be right. It doesn't hurt to think about what you read anyway. But I think we also have to be a bit humble and accept that the calculator can do math faster than we can.

[John Pinard]

Yes.

[Jim Love]

I'm going to propose another model, Marcel, and that is we think of AI as a teenager knows everything, tells you absolutely coldly that you don't know anything and that you're wrong and does it with great confidence somehow. Still, we raise teenagers and we live through it, so we'll probably live through AI as well.

[John Pinard]

Yeah, that works well.

[Jim Love]

Yeah. And that's our discussion for today. Love to know what you thought. And if you have comments or questions, you can reach me at editorialechnewsday ca or some listeners have on LinkedIn. I'm your host, Jim Love. Thanks to Marcel Gagne, John Pinard. Thanks, Marcel.

[Marcel Gagne]

You're welcome. Glad to be here, John.

[Jim Love]

Always a pleasure.

[John Pinard]

Nice to see you fine gentlemen again this week.

[Jim Love]

Yep.

[Marcel Gagne]

And we find gentlemen, actually we are.

[Jim Love]

We'd love to hear from you. Love to hear your comments on the program. You could reach me again at editorialechnewsday ca. I'm your host, Jim Love. Have a great weekend.