← CMO Confidential
CMO Confidential
Dr. Joel Shapiro | Kellogg School | What an NFL Injury Analysis Can Teach Business About Resilience
00:30:57
Loading summary
Transcript75 lines
- [00:00]
A
The CMO Confidential Podcast is a proud member of the I Hear Everything Podcast network. Looking to launch or scale your podcast, I Hear Everything delivers podcast production, growth and monetization solutions that transform your words into profit. Ready to give your brand a voice then visit iheareverything.com welcome to CMO Confidential.
- [00:25]
B
The podcast that takes you inside the drama, decisions and choices that go with being the head of marketing. Hosted by five time CMO Mike Linton.
- [00:36]
A
Welcome marketers, advertisers and those who love them to Chief Marketing Officer, Confidential. CMO Confidential is a program that takes you inside the drama, the decisions and the politics that go with being the head of marketing at any company in what is one of the most scrutinized jobs in the executive suite. I'm Mike Linton, the former Chief Marketing Officer of Best Buy, ebay, Farmers Insurance and Ancestry.com here today with my guest, Dr. Joel Shapiro. Today's topic, what an NFL Injury Analysis can teach business about resilience. Now, Joel is a professor at the Kellogg School of Business at Northwestern where he has taught data and decision science for over a decade. He was also the chief analytics officer of a sales software company and has a degree in law. This is his second time on the show. Welcome back, Joel.
- [01:30]
B
Thanks for having me.
- [01:32]
A
All right, before we dig into this case, let's remind our listeners about what you teach and why.
- [01:41]
B
Yeah, so I basically teach organizations how to be successful with data and AI. So it's sort of a combination of basic analytics to advanced data science and now AI and large language models. All those tools and across lots of different functions, lots of different industries. I teach them. How do they actually leverage these things to get good stuff to happen? And, you know, I think with a keen focus on how to make smart investments in those kinds of people and tools and systems to get good business results, that's sort of my overall vibe.
- [02:11]
A
And why you went into this? Because this is the kind of thing that's applicable in almost anything. Politics, medicine, you know, why did you decide to go in this direction with this kind of teaching and, and, you know, degree that you have?
- [02:28]
B
Yeah, well, I think so. I started out my, my PhD is in policy analysis. And so I started out doing work in public policy and specifically around education and which I still feel very passionately about and I still care deeply about. Part of the problem for me was that I felt like I was doing a lot of work and writing a lot of reports that just sort of sat on a lot of shelves and it was hard to feel like you were Making a difference. And one of the real nice things about business is, you know, business moves. It moves fast. It moves fast. Sorry. And I'm a pretty deliberate guy, and so working in business sort of keeps me moving fast. And I think that's a really nice sort of dynamic, and it makes sure that my work is good and timely and really has some real effect.
- [03:11]
A
And so when I. When I leave your class, I am better armed to take data and AI and everything else and practically apply it to situations. Is that a good way to think about it?
- [03:24]
B
Yeah. I mean, you know, one of the things you're good at is finding the business problems that you can solve with data and AI, because that's always step one is, what should we be working on? Let's not be data driven just to be data driven, but let's make sure that we are finding and articulating our important business problems. And then the data strategy and the analysis and the use of data flows from there. So I think that's a good way of putting it, for sure. Excellent.
- [03:46]
A
All right, let's go with your recent study, which was just too interesting for me not to drill into it a little bit, the idea that predicting injury in pro sports has always been, as you call it, a failed use case, that nobody can do it. And you decided, maybe I'm going to take a run at this. Tell us why you decided to look at pro sports and then how it relates to business.
- [04:15]
B
Yeah, sure. So look, the fact that when data first got hot, data, science and prediction first got hot, you know, 10, 12, 15 years ago, something like that, people started talking about predicting injury in sport. Wouldn't that be cool if we could predict injury and all the things that a team could do better? And then for whatever reason, over the last handful of years, there's been, as you alluded to this narrative, that it's a failed use case, we just can't do it. And that's flat out wrong. Part of the reason that I know it's wrong is because I've built models and I've been advising a startup that builds models where you can predict injury. And I think sometimes where people get a little bit hung up is, you know, people will say, come on, you're telling me that you can point out exactly what body part and what game of the season a guy's going to get injured? No, that's much too extreme. But what's interesting is that we can find risk factors that just aren't otherwise obvious to it. Right. The data tells us stories that our Human mind simply can't see because the data is too complex. And sometimes the things that predict injury are pretty straightforward. Prior injury predicts injury, age sometimes predicts injury. Although there's sort of a survivorship bias. When people get long, you know, later in their careers, their pro sports careers, maybe it means that they're less prone to injury, so they're less likely to get injury. But in general, the older you get, the more likely you are to be injured. And then weird things too. It's like in the NFL where you went to college, can be a predictor of how likely you are to get injured in your first couple of years. And then you're like, is that because of the training program at the college they were in? Or maybe it's just that the college was selecting certain kinds of players, smaller, faster players, and they're more likely to get injured. Point is, there are a lot of things that kind of surprise us and that we can't see otherwise. And injury and risk factors are in fact predictable. And I know because we've predicted it.
- [06:08]
A
Yes. No one has been injured on CMO Confidential yet though, so I feel pretty.
- [06:12]
B
Not yet.
- [06:13]
A
Not yet. There's still time. So let's talk about the methodology on this and the NFL. Why you picked the NFL and all the data. How did you go about tearing this apart?
- [06:25]
B
Yeah, so the model. Yeah, well, okay. So all good models rest on good data. And one of the best things for me about this project was that I was working with a company that had put a ton of labor into building a great data set. And so I myself didn't have to build it. And so what they did was they went. I'll tell. I'll explain why the NFL in just a second. But what they did was they went to a tremendous number of public sites and private sites and pay for sites and so forth, and put together a spectacular data set over an 11 year period from about 2012 to 2023, of layer level data that had injury data, all sorts of other information about the individual players and who they were playing for and the scope of injury and the nature of injury and number of games missed and so forth. Now, of course, I could take that, and as we'll talk about, I can relate it to team success because that's pretty easy data to get. The reason we did it on the NFL was because the NFL has very strict injury reporting protocols. And so the data is much cleaner than pretty much every other sport. And so good models rely on good data. And in this case those strict injury Reporting protocols were key, and it's really unique to the NFL.
- [07:38]
A
So this thing takes everything, including the college you played at and, I assume, training regimens, all kinds of stuff. And you put it in your model, and.
- [07:51]
It starts predicting things. And you create a metric here, I think, called cash wasted.
- [07:59]
Tell us about how the model takes all this out, and then what does it predict? And then we'll flip this to business. But the NFL part is just too interesting to leave.
- [08:09]
B
Yeah. So when we talk about predicting injury, you're exactly right. You take all this information that you can about a given player and its history and its training and its team, and are they playing on turf or real grass or whatever else they're playing on, and you put all these kinds of things into a model, and you can do a good job of predicting how many games they have left in the tank. Are they likely to get injured at a certain point in a season? Where I got interested, though, was I thought to myself, okay, fine, it looks like injury is reasonably predictable, but how much does it actually matter to a team? Because we can tell ourselves a story that, of course it should matter. The more injuries, the worse the team does. And I felt like the very first thing that I had to do was establish, does injury actually matter? And if so, how much?
- [08:53]
A
Can I ask one. One question? When you say it predicts injury. So it would say, like, X percent of the team will get injured, or on average, you will have this. Like, it predicts, like when. When. When the. The model spits something out, it says, all right, you know, I'm the Dallas Cowboys or whatever. Here. Here's kind of my answer. Is it like, what does it do? And then. Then I want to go back to where you were going with. With this answer.
- [09:16]
B
Yeah. What it does is it says for any given person, it tells us how many games they are likely to miss in the next season. That's what it predicts.
- [09:26]
A
They're pretty specific.
- [09:27]
B
Yeah. And, you know, it's certainly not perfect, but what we see is that it's a whole lot better than, you know, sort of using intuition and gut. So it's a person specific. How many games are you likely to miss in the upcoming season? That's really the output that it spits out.
- [09:42]
So. And by the way, you can aggregate that over teams, of course, which is sort of a nice thing to be able to do. So I'm back to this question of do injuries actually matter? And I started talking to some executives across the NFL, and what I built was this. This metric called percent cash wasted. And I was trying to figure out a way to blend both injury and resource allocation. Because if a third string running back gets hurt, that's very different than your starting quarterback. And the best way that I could figure out how to build in resource allocation was simply to look at the amount of money expressed as a percentage of total payroll that a team spends on players while they're out due to injury. So a guy makes $10 million a year, misses 30% of the season, that's $3 million cash wasted. Some people don't like the word wasted. But when I started to get pushback and like, oh, I'm onto something because people don't like it, which is how I like to play. And so what I did was I said, does percent cash wasted matter relative to winning? And the answer is, yes, it does. The more cash you waste, the worse you do. But what was interesting about it is that it was actually kind of a flatter relationship than a lot of people expected. In other words, if you get really injured, it doesn't impact winning as much as you might think. The average NFL team wastes about 10% of their payroll, and in order to win another game, you'd have to get so much healthier that it almost feels impossible to, you know, sort of do anything meaningful to do it. So that whole notion of injury impacting winning is true, but all sorts of interesting implications because it's not as big an effect as a lot of people, including me, thought it was going to be.
- [11:34]
A
And when I look at the cash wasted thing, if there's a correlation to winning, by definition, does that say there's a correlation between the contracts and the pay, that this is actually a good market value for how people get paid or not?
- [11:51]
B
Well, I don't know that I would take it that far. I feel comfortable simply saying that.
- [12:00]
Contract value tends to be a pretty good proxy for quality of player. Yeah, players get paid more if they're better and more valuable to a team. And so the fact that they end up, you know, they're being hurt hurts the team. I don't know that that does a lot to sort of, you know, sort of nail down market value. But I see where you're going with it. I don't feel super comfortable saying that.
- [12:22]
A
Okay. And so there's probably a lot of people that are interested in this type of research. Right? So you have this conclusion. What, what do you, what do you do with it? And, and then what, what happens next when teams start taking this data and applying it to things that they do.
- [12:42]
B
Yeah. So one of the super interesting things that happened when I started sharing these results with. With teams. And by the way, there's a little bit of nuance to this. If you waste cash on offense, that's more detrimental to your team than if you waste cash on defensive players. And so there's a little bit of, you know, sort of back and forth here with some nuance going on. But one of the executives in the NFL said to me, he goes, it's really interesting that there's not as big an impact as you thought there was going to be. He said, it's not as big as I thought either. I kind of feel like whenever we give a coach a break, we say, oh, they had a bad season, but they were injured. Maybe we give them too much of a break. Like, maybe we should hold them more responsible for a bad season, even when they get injured. And so different people are sort of looking at this and sort of taking away different from conclusions from it. What I think it's really interesting is, and where I tried to take this was, can you be very injured and still be successful?
- [13:37]
A
Yeah, right.
- [13:38]
B
Because that's sort of what I wanted to figure out. Like, of the teams who are badly hurt, can they still make the playoffs? Can they still still do really well? And I got a little stuck there. And the reason was that disproportionately, in my data, teams that were injured and did really well just happened to be the Kansas City Chiefs in a handful.
- [13:57]
A
There you go.
- [13:57]
B
And then you're like, so what's the secret having Pat Mahomes? Like. Like, are you describing the Chief? Are you describing a trend? It got a little bit funny. And so I couldn't really take a whole lot out of that. So I. I tried a slightly different tactic. What I said was this. Imagine that you are a team that in a given year, you're healthy and you do really well. You're healthy and you make the playoffs. What if the next year you're really injured? What happens to those teams? Does anyone stay good? Do they all do better, poorly? And this is where things got super interesting. Teams that one year were healthy and good and the next year were injured. About 25% of them stayed good. Yeah, about 10% of them did a little bit worse, and about 65% of them completely tanked. And that became interesting to me because now I'm thinking to myself, why do you have that separation of what's variable driving that? And. And you know, I have looked at that 100 different ways and there's really only one big thing that I've come to the conclusion of, and that is if you have a highly paid quarterback and the quarterback gets badly injured, you are going to tank. No team has been able to withstand the loss of a highly paid quarterback and still have a good season. That's about the only thing in the data that I found. But that in and of itself is interesting because now I'm starting to think about things like leadership and CE is then how do we plan for resilience more broadly? And that's sort of the connection that you asked about before. The connection to businesses. Injury for a team is unexpected.
- [15:43]
A
Right.
- [15:44]
B
And trying to sort of fight through the unexpected bad stuff is incredibly important for any team, whether it's a business team or a sports team. And this notion of how do we actually plan to be resilient, because I don't love the way that a lot of people talk about resilience. Resilience is a big buzzword right now. And when people talk about resilience, what they typically mean is that when things go bad, you fight your way through it. You've got the character and you got the moxie to fight your way through it. And I think that sort of undersells the concept of purposeful resilience. Like, you want to build resilient organizations, so when things go bad, you can survive through them. And, you know, we can say that we build our teams for shocks, but what the real sort of secret is, what the ingredient is that allows us to build for shocks. I'm still trying to figure that out. And I wish I had that silver bullet, but that's where I am in the research right now, is trying to figure out what exactly is it that makes teams resilient in sports and business alike?
- [16:47]
A
And do you have any, like, variables where you're thinking, this feels like it's a good lead here.
- [16:54]
And you know, and how companies recruit and staff and think about it?
- [17:01]
B
Yeah. So I have some things that I'm still sort of playing with. And I gotta tell you that I'm very cautious about taking results from one context and extrapolating them to the other. There are some people who feel very free about sort of just making claims. And I'm always. I always hold myself to a very high standard in this regard. But I will say something that I think is sort of fleshing out in the data is that in the NFL, backups matter.
- [17:25]
A
Yeah.
- [17:25]
B
Having a good backup plan matters. The translation to business teams, I think, sort of speaks for itself. And and teams that, you know, you can't, nobody has the right resources to be able to have a highly paid quarterback them go down and still have a good quarterback step up. But in all the other contexts, it seems to me that good, solid backups matter. And that is an important lesson in and of itself about resource deployment. I will tell you that I talked to a bunch of, of trainers in the NFL because you can imagine that if you could predict injury, one of the questions is what are you going to do about it? And that's the same question in business. If you can predict that something bad is going to happen, what do you think?
- [18:04]
A
People are going to leave, you're going to get raided and competitors, like, there's a lot of things that's, you know, that, that mess up with any annual plan. And almost immediately. And you see some companies always win. Like some teams always manage to show resilience and other companies don't.
- [18:21]
B
Yeah. And so in the NFL, it was interesting because I said to some trainers that I was talking to, imagine that I could tell you that, that a certain player was at higher risk of injury next year than you would have thought. What are you going to do differently? And every person to a T said, I don't know. Like if somebody is at higher risk of a soft tissue injury. We already do everything we know to keep players flexible and strong and so forth. And if you're talking about like a broken bone, well, I got nothing. Because accidents happen. And the only way you keep those things from happening is from keeping people off the playing field. You don't play them, which is kind of a ridiculous solution, of course.
- [18:55]
A
Right.
- [18:56]
B
And so you can't really, according to these trainers, do a whole lot more to keep players healthy. What does that leave you with? You got to bring the right players on your team who are unlikely to get injured. And that's part of the roster construction process. When you think about spending your big dollars to bring the right players on, it's not just about productivity. It's not just about talent. It's about staying healthy and staying on the field. Because you can't be productive if you're not on the field.
- [19:25]
A
So flip this over to the business side and let's talk about resilience in business, because within this there's a whole bunch of recruiting. Longevity training is a different thing, but still training. Sal, there's lots of stuff. Flip this over and just, you know, opine on how our listeners should be thinking about this kind of data where they're sitting looking at their Business orgs.
- [19:53]
B
Yeah. So I think it's an imperfect analogy to just think of roster construction as team construction because the truth is if you have a quarterback that plays half your games, they could still be valuable to you. If you have a product manager who shows up 50% of the time, you're probably not going to keep.
- [20:08]
A
Probably not.
- [20:08]
B
Good. Not. Yeah, it's not, it's not a great analogy here, but. So I think of resilience a little bit differently. I think about resilience as things like, you know, supply chain disruption. Have you built, do you have enough foresight, enough intelligence, predictive work, AI kinds of things to really be able to identify where you might see and when you might see some supply chain disruptions so that you can ensure that your processes are really set up to withstand those kinds of problems. Problems, right. It's not about the people like it is in football, it's about the business processes or you know, I've done a lot of work with a lot of sales teams and this is a little bit more of a staffing one, but sales teams who are really reliant on that one rainmaker and that rainmaker leaves and all of a sudden they're kind of screwed. Right?
- [20:56]
A
Yeah.
- [20:57]
B
Their leadership questions too. Apple has remained kind of a great company even though Steve Jobs is no longer with them. Right. And I don't know, I think about it in startup context too. You startups that are reliant on very easy, you know, free flowing capital, what happens if it dries up? Are they resilient enough to still be able to move forward and capture new opportunities without that capital? So I like to think of resilience sort of writ large and the role of data and AI in helping us predict the bad stuff so that we can build these resilient teams and these resilient organizations.
- [21:33]
A
So if you don't mind, give our listeners a little primer on how they might think through from their seat, how they could apply. What you just said about resilience is it's, it's kind of like it's forward thinking, it's looking at the pain points, it's thinking through things that can go wrong and the backup plans give our, give our listeners kind of a little dose of what you might tell your, your class or your corporate clients. Hey, here's how you should think about resilience from your chair.
- [22:07]
B
Yeah. So first I, I will do that, but let me just also point out that sometimes people think about resilience as like a back office, almost like, you know, business continuity thing and they're like, okay, that's fine, resilience is fine.
- [22:20]
A
Wipes us out. We have to have a plan.
- [22:22]
B
Yeah, but, but it's just, you know, it's sort of a back office idea. And I think that's a mistake because I think resilience needs to be thought of really as a strategy unto itself. Like when things go bad, you want to be able to stay aggressive, you want to be able to seize opportunities, you want to be grow when others can't. So look, the short of it is for any given business, you've identified your opportunities and you've identified some of the potential weaknesses and threats. What you want to be able to do is you want to have some sense of how likely some of those threats are to come to fruition. It's about having people who are really in tune with the marketplace. Sometimes it's about having good data scientists. But at some point you've got to be able to understand what risks are real, what the actual sort of magnitude of those risks are and how much harm they're going to do to us. I think we're really bad at thinking about the real potential harm when things go wrong and planning for it, because things don't always go wrong. So it becomes sort of a second nature to us. So I think it's really about having a good sense of truly what are the most important threats. Doing as good a job as you can is getting a sense of the likelihood of those things, whether that's with dedicated data teams or market experts, and then building in select places where if things go wrong, you're most likely to pay the biggest price. It's just a trade off of sort of the, you know, the costs against the benefits.
- [23:47]
A
And then one of the things that's kind of beneath the surface on all this is companies that actually are really truly resilient are going to perform in that top quartile a lot longer than other companies. You mentioned Apple. Are there any other companies or industries you would call out that say, good God, these people were super resilient or they weren't very resilient when you look out there at different things?
- [24:14]
B
Well, once you hit a certain level of scale, it's easier to be resilient. I think of a lot of tech companies, Salesforce and those kinds of places that do a nice job. But part of that is scale. I guess resilience tends to be a little bit easier when you are selling goods and services that are more sort of keep the lights on sorts of things. Right. You're selling software that, you know, companies truly can't do without because they're just, you know, table stakes for simply existing. So some of those places almost have resilience a little bit more built in. So for those that are not sort of keep the lights on kind of goods and services, there's a greater need to sort of look further into the future and make some better guesses and to plan for when things do go wrong, if they do go wrong.
- [24:58]
A
Yes. Okay. So you're looking at all this and you know, I, I must ask, when you look at the NFL, we're closing in on, you know, I guess we're what, 10 games or 11 games in, 12 games in when this thing airs. Any super bowl picks or any interesting things leaping out to you when you look at your studies or.
- [25:20]
I know you live in Chicago, so the Bears, you know, snatching a lot of victories.
- [25:25]
B
Well, I mean, you know, so I, I found that I am about as bad as it gets, actually. Predicting winners in sports don't hold that against me. When I start talking about resilience and predicting things going wrong. I can predict injury, but predicting winning I seem to be bad at. The Bears have had the weirdest 7 and 3 start in the history of the NFL as far as I'm concerned, playing some pretty mediocre football. So I'm going with simply, it is their year, they're winning it all. I'll take all the, the heat from the people who think I'm crazy, but that's my prediction.
- [25:59]
A
It's hard for me not to look at the data you are creating here on predictive modeling of percent cash wasted and think this isn't going to connect eventually to betting?
- [26:13]
How do I even. Is that the kind of thing that just inevitable given big data and AI and the power of DraftKings and FanDuel.
- [26:23]
Or how do I think about all of that?
- [26:25]
B
That. Well, I think you're right. I mean, those two things are coming together very, very quickly and the more data that we have, the more predictable things get. And therefore the sort of, the different dynamics in the, the betting systems and the bets that exist, you know, everybody has access to the same intelligence, of course, and bets tend to be more easily won or, you know, lost, I suppose, when there's asymmetry in that. So the fact that everybody has access to the same information means that maybe there won't be so much change. I just get nervous, you know, with a lot of the, the little bets. And we're seeing some things right now about, you Know, some, some collusion and people doing some cheating, you know, with pitches thrown. And so. Yeah, yeah. So you know that stuff is sort of independent of the big data. Right. It's just that. But because you can bet in such granular, small scale kinds of things that it's easier and easier for athletes to sort of, you know, throw a game or throw a certain kind of pitch, I suppose, or whatever it is. So that concerns me a little bit. But yes, you're right. The, the, the prevalence of better data, better intelligence, better predictions certainly changes the way that betting.
- [27:32]
A
Inevitable, inevitable march of big data on yet another front.
- [27:37]
B
Yep, totally.
- [27:38]
A
All right, so this brings us to our last question, practical advice for our audience we haven't discussed yet and, or the funniest story you can share on the air. You can pick one or both, but you must pick at least one.
- [27:53]
B
Well, I guess I'll go. I don't know how funny it is, but it's certainly something that I was quite tickled by in a recent class that I was teaching. So I had a group of students who were, we're talking about betting, so I might as well share it. We had a group of students who are working on a project where they were trying to predict NHL playoff teams, looking at the halfway point of the season, predicting playoff teams at the end of the season. And of course you can do a pretty good job with that because at the halfway point of the season, you know who the good teams are. And so good prediction in that case tends to be sort of the last teams in and out and so forth. And I had three teams, three different student teams working on it and every single one of them beat Vegas pretty handily with their machine learning models to predict who is going to make the NHL playoffs. So I was quite proud of the fact that my, my students were nailing it. And had they placed bets, I don't think they did, but had they placed bets, they would have beat Vegas pretty handy.
- [28:47]
A
They could have almost paid tuition.
- [28:49]
B
Seriously, almost.
- [28:50]
A
And then I've come like, how much data does a student team plug into one of those models? I'd love to give our listeners some understanding of, of what these students are doing when they are doing a predictive model like that.
- [29:05]
B
Well, in that case, you know, all you're really doing is you're trying to predict team wins. And so you don't have a tremendous amount if you think of data in terms of rows and columns, you don't have a tremendous number of rows because you have what, 20 years of historical data and there aren't that many teams, like 30 teams. So it's 20 years times 30 teams, about 600 rows. We have a tremendous amount of information about each team though. And so they have, you know, 500 columns or so that can go into a model like this. But I will tell you that Most of the 500 things aren't all that relevant. There tend to be, you know, a handful of things that in isolation or in combination really matter deeply. So a lot of information about each team, but ultimately there aren't that many teams to use for predictive purposes.
- [29:46]
A
But, but this also, when you say 500 rows and 20 years, that's not a little bit of data for someone sitting at their desk in a small company, right?
- [29:56]
B
Yeah, you're certainly not going to sort of eyeball things and be able to make sense of it, you know, other than, you know, being able to use some sort of computational sophistication. And you know, these students would import this into a machine learning tool. They would do some of their own programming. So it's not like you're in Excel and building these models and sort of eyeballing it for sure.
- [30:16]
A
All right, well, well, you know, I think that's a great way to end the show, which is there's a lot coming down the road for everybody on the big data front and predictive modeling. And this is probably the most we've ever talked about the NFL on the show. So thank you Joel and thanks to everyone for listening to CMO Confidential. If you're enjoying the show, please like subscribe and share new episodes. Drop every Tuesday on Spotify, Apple and YouTube. Our catalog gives you access to over 150 shows including Colonel Mustard in the study with the job spec. How poor design shorten CMO lifespans Why can can't dissecting compensation A primer on understanding and negotiating pay and Joel's first show, the grocery prediction case. It's not just about the data. Hey, all you marketers, stay safe out there. This is Mike Linton signing off for CMO Confidential.