← The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
The Agile Brand with Greg Kihlström®: Expert Mode Marketing Technology, AI, & CX
#629: Why generative AI transparency is so important with Matt Van Itallie, Sema Software
00:28:39
Loading summary
Transcript48 lines
- [00:01]
Strayer University Announcer
The Jack Welch Management Institute at Strayer University helps you go from I know the way to I've arrived with our top 10 ranked online MBA. Gain skills you can learn today and apply tomorrow. Get ready to go from make it happen to made it happen and keep striving. Visit strayer.edu Jack Welchmba to learn more. Strayer University is certified to operate in Virginia by Chev and its many campuses, including at 2121 15th Street north in Arlington, Virginia.
- [00:31]
Carvana Customer
Hey sweetie, your mother showed me this Carvana thing for selling the car. I'm gonna give it a try. Wish me luck. Me again. I put in the license plate. It gave me an offer. Unbelievable. Okay, I accepted the offer. They're picking it up Tuesday from the driveway. I haven't even left my chair. It's done. The car is gone. I'm holding a check anyway. Carvana give it a whirl. Love ya.
- [00:54]
Strayer University Announcer
So good you'll want to leave a voicemail about it. Sell your car today on Carvana. Pick up fees. May apply Foreign.
- [01:13]
Greg Kilstrom
Welcome to Season seven of the Agile Brand where we discuss the trends and topics marketing leaders need to know. Stay curious, stay agile and join the top enterprise brands and Martech platforms as we explore marketing technology, AI, e commerce, and whatever's next for the Omnichannel customer experience. Together we'll discover what it takes to create an agile brand built for today and tomorrow and built for customers, employees and continued business growth. I'm your host, Greg Kilstrom, advising Fortune 1000 brands on martech, AI and marketing operations. The Agile Brand podcast is brought to you by Tech Systems, an industry leader in full stack technology services, talent services and real world application. For more information, go to teksystems.com to make sure you always get the latest episodes, please hit subscribe on the app you listen to podcasts on and leave us a rating so others can find us as well. Now onto the show.
- [02:12]
Greg Kilstrom (Interviewer)
Do you know who or what created the foundation of your software and can you trust it? Today we're joined by Matt Van Italy, CEO of SEMA Software, a leading code and software analytics platform that's analyzed over
- [02:25]
Greg Kilstrom
$1 trillion worth of software.
- [02:27]
Greg Kilstrom (Interviewer)
A lifelong tech expert, Matt has been instrumental in shaping the AI industry with innovations like the Gen AI Code Monitor and Generative AI Bill of Materials. He's here to discuss transparency and generative AI for software development and how AI is transforming coding practices while introducing new challenges in accountability, ownership and quality assurance. Welcome to the show, Matt.
- [02:51]
Matt Van Italy
Thanks so much for having me, Greg. I am so excited.
- [02:54]
Greg Kilstrom (Interviewer)
Yeah Looking forward to talking about this with you. We've talked about, as you can imagine, Gen AI and AI a lot but this is a topic that I think needs some more discussion. But before we dive into all that, why don't we start with you giving a little more background on yourself and what inspired you to found Sema software.
- [03:14]
Matt Van Italy
Yeah, for sure. So I'm the son of a math teacher and a computer programmer, so if you had told my earlier self, well you're going to end up use running a company that treats code as data. Probably, probably makes sense but, but I had a much a very circuitous path to get here. I worked in government reform, I worked in enterprise software companies and I founded Sema just based on a logical problem that I that an itch that I needed to scratch which was, you know, sitting in software company executive team meetings, the chief revenue officer and while we're at it, the chief marketing officer would talk about the state of the sales funnel using Salesforce or you know, the CRM of their choice. And of course a CRM like that is an executive dashboard explaining the state of sales and up funnel marketing under certain circumstances and the state of the sales team thanks to code. And then we'd get to the CTO amazing ctos I worked with and they would explain the state of code encoders by hand. Qualitative perspective. Well, I kind of think it's this, it's kind of that and like my logical mind just exploded. Why can't we use code to create an executive dashboard about code just like sales and marketing all these other teams have. And so I love solving logical problems that try to have some meaningful impact. And this one is just so fun because it's a really hard problem trying to get make code understandable given all the different languages and all the different components of what does it make to mean to have good code?
- [04:48]
Greg Kilstrom (Interviewer)
Yeah, yeah, definitely. Yeah. So let's, let's dive in here then and we're going to talk about a few things. But overall, you know, transparency in Gen AI for software development as I mentioned, you know, I've talked a lot about Gen AI for MarTech and customer data and other things like that on the show, but less about using it in software development. And yet it's being used plenty tools like GitHub's Copilot or and others are transforming that process. For those less familiar, again we have a lot of marketers listening to the show, maybe a little less familiar on the engineering side but for those less familiar, can you explain a little bit what do these tools do and what do you see as some of the most significant benefits that they offer offer to those software engineering teams?
- [05:37]
Matt Van Italy
Absolutely. So let's just describe it generally. First, there are LLMs that help give advice and draft code. Just like I hope every single one of your listeners is now using one or more LLMs to draft and write human language, this is about computer language instead. And just like my goodness, certainly from my own experience, how much geni tools can help with human writing, it is incredibly helpful to coders. Whether it's check my work or give me 100 different proposals and give me the best one, return the best one or autocomplete this or suggest it is great for developers. Not all situations under all circumstances, but many of them. And as a result it helps organizations be that much more productive. Again, if you can extrapolate listeners from how much more productive you are with human language gen tools, the same is true and maybe even more so in some circumstances. From coders using it to code. And so it's. The introduction of gen AI is absolutely one of the biggest productivity boosts of the last 25 years for coders.
- [06:48]
Greg Kilstrom (Interviewer)
Yeah, yeah. What are some of the, you know, those benefits aside, you know, what are some of the key challenges that organization. Organizations face when integrating some of these AI, you know, gen AI tools, particularly around accountability and security.
- [07:05]
Matt Van Italy
Yeah. So we do say that the biggest risk is not using it enough because it's so valuable and has such impact. But if we put aside that and think about the prevention of negative risks. There are, there are four in particular. Security risk being introduced. Any code can come with security risk regardless of whether it was an LLM or a human writing it. So you got to make sure that it's sufficiently safe. Intellectual property risk under certain circumstances. You don't own the what the output of of a gen tool and that includes for code. Third is maintainability and understandability for the future. You know, imagine if a junior member of the marketing team just produced something based on prompts and just turned it in and didn't read it. You'd wonder if it was correct. You'd wonder if the person knew what it was about, it would be unmaintainable, that document would be less maintainable. Very much true for code. And then finally it's something sort of, sort of out there, but it's already. Here is something we call exit risk, which is, you know, SEMA got started in technical due diligence, helping companies evaluate the health and the, the overall validity, let's say of, of Software organizations, they're purchasing. It's already happening that in the diligence process, folks are looking and saying, well this code was 90% written by gen. Maybe we don't need the company anymore, maybe we can just build it ourselves. So it's really changing M and A now. I don't want you to be scared. The flip side is you can build M and A or investment ready products faster because you can go faster. But for all four of these risks, security, intellectual property, risk, maintainability and exit, the solution, I jump ahead to a question is making sure that a human stays in the loop. You gotta make sure code coming out of LLMs is read by humans, is reviewed by humans. Code reviews, all of the code quality tools matter as much and arguably more, including the human, the human judgment to make sure that the right code is being introduced.
- [09:03]
Greg Kilstrom (Interviewer)
Yeah, I mean so you, you mentioned a good example. As you know, if you have a junior engineer very early on writing code, you're going to want to review it as well. Um, so you know, much like that, what you just, what you just were talking about, you definitely want a human to review it. Do you think there's danger in like, are, are people not thinking of that, that review process as much? Because they kind of take for granted. You know, again, I know in the content creation world it's like people kind of take for granted that ChatGPT will generate good content, but when you review it, it's, there's weird writing styles or whatever. Again, I know that world more than, more than the, the coding world to do. People have less likelihood from your perspective to feel like they need to review that stuff?
- [09:53]
Matt Van Italy
Yeah, it's a great question. I do, I'm worried about it a little in part because it comes out looking so polished and so right that sometimes people let their guard down. But to me that really is overwhelmed by just the craftsmanship nature of coding. You know, folks are not coding for, I mean some people are, but most people are not coding for a paycheck. They're coding because they love coding and so they want the opportunity to dig in and to make things better rather than figuring out how to most efficiently get stuff done. And that's, you know, so we, we are evaluating, we built software to look at how the ingredients list, if you will, of a code base, how much of it is gen AI and of that, how much was human modified. And the good news is almost every code base we've seen as well within the appropriate human level, we say blended gen code needs to be blended very Almost all code bases are meeting that blended standard, which they may be using gen a little or a lot, but definitely when they do use it, they're putting humans against it to make sure it's, it's appropriate, which is a great sign.
- [10:59]
Greg Kilstrom (Interviewer)
And to your point, it's not that AI generated is bad, it's just, it's the understanding and transparent documentation of it is important. Can you talk a little bit more? I know you touched on this, but talk a little bit more about why that transparency is so important.
- [11:16]
Matt Van Italy
Yeah, there is no back to my data days. If you don't have metrics, if you don't have data, you're not having the best form of the conversation that you could have. You can guess about whether or not the code is maintainable or understandable or is correct, but if you haven't measured it, you don't really know. You don't really know if the code's getting reviewed, which is super important. Literally by definition, if you haven't reviewed the code and modified it, you won't get some kinds of intellectual property protection. Now that's not true for all companies. I don't need to, not everyone needs to worry about that yet. But you know, copyright protection, your, you know, teams on this call are very familiar. You can't get copyright protection for stuff that comes out of an LLM because machines don't get copyright protection. And so if you're an organization who's seeking copyright protection for your work, whether it's art or words or code, you literally have to modify it to make it copyright protect. A bull which by the way, is going to be a huge in the US Copyright office is coming out with new regs on this. So if you're an organization large enough to be worried about copyright, make sure your, you and your legal teams and your external counsel are paying attention to the copyright should be late January newest Copyright Office statements on not infringing copyright. There's been a lot of talk about that, but actually receiving copyright when you're using AI for whatever it is you're doing, coding or otherwise.
- [12:47]
Greg Kilstrom (Interviewer)
So you created something called the generative AI bill of materials. Can you walk us through how it works and, and why it's a game changer for software development?
- [12:58]
Matt Van Italy
Yeah, sure. So I love using analogies. If you think about the ingredients list on packaged goods, packaged foods, excuse me, calories, sugar, how much of it is vitamin D, et cetera, the generative AI bill of materials is an ingredients list and that breaks your code into three parts. How much was completely Generated by genai and not modified. We call that genai pure. How much was generated by genai and then modified which is much safer. We call that blended Genai code and then all other code non genai originated. The reason we call it the Generative AI Bill of Materials or GBoM. This has a pretty nice nickname. But the, but the real reason is there already is something called a software bill of materials which is about how much open source code is in a code base. So already last decade plus decades organizations already know that in high stakes situations in diligences, insurance, procurement, coding, teams have to show the provenance of their code. How much did their team write versus how much came from the open source community. And all we did, I mean incredibly hard engineering work. I'm so proud of our team and our research and engineering and product. But what we did is extended that idea. Now you need an ingredients list that's not just open source versus your team's creation but of the human code. How much of it came from, of the, of the code, how much was gen AI or not? So that the, the, the GBOM if you will is really just an ingredients list to help understand are you in that, that too little zone and you should use it more that too much zone and you should manage it better or blend it at least or in the just right Goldilocks zone in the middle.
- [14:41]
Greg Kilstrom (Interviewer)
Yeah. And so using that analogy to the open source software world, probably a lot of people familiar with that as well I guess. How much overlap is there in the way that organizations should think? I mean you mentioned from the copyright standpoint and the diligence standpoint. Does that always hold through or are there some differences or things like that?
- [15:02]
Matt Van Italy
Yeah, I would love everyone to listen to this a year from now, January 2026 and you can see how accurate these predictions are. Right now we are in the early adoption phases of GENAI for coding. Even though it's been around for several years, it's really just starting to take off. And organizations are most interested in understanding usage to increase adoption on the open source world. It would be crazy to say, you know what we need to do? We need to figure out to explain to developers why they should use open source. Everyone knows, right, knows that they have to do that. It did not have to. It's why would you reinvent the wheel? We can go to the open source community and use something. And so today the challenge is increasing adoption. We predict, we predict that exactly identical, a nearly identical, let's call it risk profile will be developed for gen Code. So in open source you have to worry about security risk through CVEs that is coming for Genai code you have to worry about intellectual property risk, that's GPL licenses for open source. Here it's copyrightability and so forth. You definitely have to worry about maintainability and keeping it up to date. True for both. And then exit risk. You absolutely. This is a little bit arcane but if you are selling a software company to a medium or large size investor or acquirer, you have to show your open source provenance. You have to. The SBOM is a huge part of technical due diligence. Actually that part's already come true. So SEMA's first product does technical due diligence and we, we built SBoM, we did S BoMS for years and then some of our major clients asked us to build a G bomb as well. So if you're getting bought by one of our clients, you will get a G bomb produced for you and you'll, you'll have to talk about it. But we think all the ways that this is the prediction that procurement offices, insurers, investors, all the folks who care about open source provenance, the open source ingredients list will increasingly care about Genai provenance, Gen AI transparency as well. Yeah, that was a little bit weaselly because I didn't put numbers on it. I bet by January 14, 2026 at least 10 major Fortune 500 procurement offices will care and insurance companies who, at least some insurance companies who ask for an S bomb will be asking for G bomb. So there hold me to it. I'll put a stick in the ground.
- [17:28]
Greg Kilstrom (Interviewer)
We'll have you back next year. Nice, nice. So you know, one of the other topics around AI that's been coming up more, more often lately is agentic AI. And you know, I've had a few folks on talking about that topic already. But looking at it from this perspective, you know, with the emergence of, of agentic AI, pursuing kind of, I don't know if it's open ended objectives but it's at least multi step and by that token maybe a little harder to trace in some aspects. What are some of the new and ethical and operational challenges that you foresee with the dawn of agentic AI?
- [18:10]
Matt Van Italy
Yeah, I think it just increases the need for transparency. The more that you are putting your organization's health in the hands of something else, the more you're going to need to understand what's, what's going on. You know, SEMA has spent a lot of time thinking about metrics, about coders and bottom line, they largely are used for evil, not for good. Because code really is a craft, not, not a competition. It doesn't lend itself to, you know, metrics like salespeople, my, my sales hat, how many bookings I have, how many calls I make is really indicative of my performance in coding. If someone adds more code. Yeah, actually it's not certain that that's a good thing. Right. Some of the best coding there is is removing. Right. So. So metrics about people we're really deeply skeptical of. There's some limited uses where it makes sense, but metrics about individual agents and exactly what they're doing and exactly what they're changing. One, I think it just comes with less objections, but it's also just that much more important. You need to know what all these things are doing on your behalf and how they're interacting with each other. I still, I am still passionate about the need for humans in the loop. I don't care how, how good the tools are. If the stakes matter for what the code's going to do on the other end, you, you really need something looking at it. And I'm sure this is a, you know, a bastardization, but my personal AI usage journey, I started by asking, by asking an LLM, tell me the answer to this question. Right. I started with that. Now I'm at. If I'm solving. If I'm working on a new product brief, I have a task for that product brief with 10 different Personas who interact with each other and then give me back a different answer under both circumstances. The second one is a lot better than the first one, but my goodness, do I need to know and carefully read and review what that final answer is? I just don't see that not changing. I see that changing. I just think you're going to have to. The more important the stakes, the more carefully you're going to have to look at the output and under certain circumstances, looking under the hood about how it was made to make sure it was done in a safe and appropriate way.
- [20:24]
Greg Kilstrom (Interviewer)
Yeah, yeah. I mean, again though, I wonder if the analogy going back to open source, you know, it's like the big open source projects that are well documented and well used and at least reasonably secure and all of that. Those ha. Don't they have processes similar. It's the human analog to those. Right. So it's like again, I think there's, there's an analog here for all of us in, in this. Of like, okay, we, we invented this and maybe it looks differently. I mean, you know, to go out of the software world, you know, Wikipedia has a process for reviewing, you know, for reviewing content and things like that. So like there's analogs here to be able to use it just, you know, to, I think to what you're saying, this moves quickly and it's. If not, if not but by design, it's not always as transparent. But again I, I think the, the reassuring thing is that if there are tools that help us to, you know, to have some of that transparency there, you know, it's, it seems like there's, there will be some good precedent. Right. So, and, and I guess you know, in along those lines, you know, talking about accountability and talking about quality control here. So I know you've touched on this a bit, but how do you look at this, organizations establishing that accountability for code created by AI?
- [21:50]
Matt Van Italy
Well, I do have to share a brief story. Before I did coding work and before I was in enterprise software, I was in school district reform. Deeply passionate about trying to make systems work and at the time systems now coding systems at the time helping make school districts, trying to help increase teaching and learning on behalf of low income kids. I don't think there's a harder problem on earth or a more important one. But I came to an organization where my title was Chief Achievement and Accountability Officer and I said, well that's a mouthful, I'm going to shorten it. And I shortened it to Chief Accountability Officer and that was among the worst decisions of my entire life if you ask me. I personally love accountability. I have to do lists. I have coach I love like I'm a competitor. Right back to the craft versus competition. Code is not like that by the way. Teachers are not like that. And so I am less about accountability for coders using, for coders using gen tools or otherwise and more about my goodness, smart, passionate professionals who get to create for a living. How amazing is that? Let's meet them where they are and explain as professionals this really matters. This really matters. And here's why it matters. And here's why it matters. So we talk in the coding world about functional requirements and non functional requirements and sort of over explaining functional is what does the product do and non functional is security, quality, maintainability, et cetera. Most coders love worrying about the non functional requirements and they just wish they had more time to do it. So if you can explain, you know, it's the craft, it's, they're not going to Ikea to buy a mug. They want to build the mug themselves and they want it built the right way, that process, that craftsmanship really matters. And so I always like to start with explaining to coders why this should matter and why they should think about it and then giving them data. There was some magic way to give coders data about their work without having big brother looking over their shoulder. I might be talking, but folks don't really believe that, that if there's going to be developer level metrics, there's going to be top down metrics buying. And so I would say to organizations find a way for developers to know, to have data without, without spying on them is really the, is the fine point. And I'd start with getting to full understanding before, before worrying about accountability.
- [24:11]
Greg Kilstrom (Interviewer)
Yeah, yeah, well. And so for those that sit outside of the engineering and technology parts of the business there, you know, rather than micromanaging and, and mistrust and you know, all the negative, what should they be doing in relation to that, you know, they're using their own gen AI tools for their own stuff but what should they be doing in relation to, you know, this part of the business?
- [24:36]
Matt Van Italy
Yeah, it depends on where you're sitting. I'd say CEOs probably the most important things that CEOs can do is role model the use of gen AI tools, whatever those things are. I use it on the product side and the communication side and the fun side. It's amazing for writing limericks as everybody knows it is explaining the why behind it. And I think one of the things that is leading to gen adoption going slower than I, I think would be better for developers and for the organizations is this identity conversation is a sense of am I really a coder if I'm, if I'm so heavily relying on, on this. And what I say to that is I do understand but are you a real coder if you use open source? Of course I am. Why would I reinvent the wheel if something already exists? Well, same thing. Are you a real coder if you use GitHub as a version control system or if you use an ide right. These. It is a very powerful tool. I don't, I don't want to put that aside but we've developed so many tools that have advanced the work and just this one just feels so different that really supporting colleagues in, in understanding that to be a professional is use the best available tools. And it's not a, it's not a dig, it's actually a plus to go off and see how you can use these tools. I really think that's the right way to think about It, Yeah.
- [25:58]
Greg Kilstrom (Interviewer)
I mean, it seems like in, in a few years it's going to be like using like Autocorrect when you type in Microsoft Word. Right.
- [26:06]
Matt Van Italy
Why would you possibly turn that off? Exactly. Right, right.
- [26:09]
Greg Kilstrom (Interviewer)
I mean, that's technically AI, Right. You know, Grammarly, you know, what are all those tools? It's like, yeah, why, why would you turn that off? Yeah. So definitely. What, I mean, you're, you think about this stuff a lot. What are you excited about on the horizon? You know, what, what should we be looking out for here?
- [26:27]
Matt Van Italy
I am really excited about how the power of AI can be used to understand code, not just to be used to help coders carry out their vision. You know, there's a lot of, there's a lot of drudgery. I mean there's so much about coding that is great and engineering management that can be really fun, but man, there is a lot of drudgery. You know, I had a dear friend we're talking about. He had 25 engineering teams that he was trying to keep track of what they were up to and by hand. Even with a traditional metrics dashboard, it's either nearly impossible or extremely unpleasant because you'd have to look through 25 things and compare and contrast.
- [27:08]
Greg Kilstrom (Interviewer)
Probably both, right?
- [27:09]
Matt Van Italy
Yeah, probably both right. And so instead using, using the power of AI to augment humans and understanding what is going on in engineering teams. I am thrilled by that. You'll be shocked to hear we're working on it. So any of your listeners want to want a taste, feel free to reach out to us. We'll be happy to show you what we're working on. But using AI to let professionals do what they do best and take out some of the drudgery work and in particular in understanding the engineering and the delivery roadmap. I am over the moon, excited about that.
- [27:41]
Greg Kilstrom (Interviewer)
Yeah, love it. Well, thanks so much for all your insights here. One last question before we wrap up. I like to ask everybody, what do you do to stay agile in your role and how do you find a way to do it consistently?
- [27:53]
Matt Van Italy
I mean, I, I spend a lot of time with Claude these days. I have Claude prompts that help me figure out the next prompt for the next problem. I have self improvement prompts. We have a Slack channel to share ideas. I cannot stress, I personally just how transformational this technology has been. Certainly Internet level disruption of my, in a good way of my workflows and we're only, I'm only scratching the surface of it. So I do continue to make time for my family but on the margin, my goodness, there's a lot to do with LLMs to learn and and try to become a better a better learner and a better professional.
- [28:32]
Greg Kilstrom (Interviewer)
Yeah. Love it. Well, again, I'd like to thank Matt Van Italy, CEO of SEMA Software, for joining the show. You can learn more about Matt and SEMA Software by following the links in the show Notes.
- [28:44]
Greg Kilstrom
Thanks again for listening to the Agile Brand brought to you by Tech Systems. If you enjoyed the show, please take a minute to subscribe and leave us a rating so that others can find
- [28:52]
Greg Kilstrom (Interviewer)
the show as well.
- [28:53]
Greg Kilstrom
You can access more episodes of the show@theagilebrand.com that's the theagile brand.com and contact me. If you're interested in consulting or advisory services or are looking for a speaker for your next event, go to www.gregkillstrom.com that's G R E G K I H L S t r o m.com the Agile brand is produced by Missing Link, a Latina owned, strategy driven, creatively fueled production co op. From ideation to creation, they craft human connections through intelligent, engaging and informative content. Until next time, stay curious and stay agile.
- [29:38]
Strayer University Announcer
The agile brand.
- [29:47]
Liberty Mutual Advertiser
And Doug there's nowhere I wouldn't go to help someone customize and save on car insurance with Liberty Mutual. Even if it means sitting front row at a comedy show.
- [29:56]
Greg Kilstrom (Interviewer)
Hey everyone, check out this guy and his bird. What is this, your first date?
- [30:00]
Liberty Mutual Advertiser
Oh no. We help people customize and save on car insurance with Liberty Mutual together. We're married. Me to a human, him to a bird.
- [30:07]
Matt Van Italy
Yeah, the bird looks out of your league.
- [30:08]
Liberty Mutual Advertiser
Anyways, get a quote@libertymutual.com or with your local agent.
- [30:12]
Matt Van Italy
Liberty Liberty Liberty Liberty.
- [30:15]
Home Depot Advertiser
Kick off Memorial Day Savings with up to an extra thousand dollars off select top brand appliances like Frigidaire. Plus get free delivery at the Home Depot. Stack your appliance lineup with a fridge with an extra large ice capacity and an oven with over 15 cooking modes. Then let the real MVP handle cleanup with a dishwasher's 50 minute wash. Cycle. Shop Appliances at the Home Depot offer valid May 14th through June 3rd US only. Free delivery on appliance purchases of $998 or more. See Store online for details.