← School Business Insider
Loading summary
Transcript72 lines
- [00:01]
A
You're listening to ASBO International's School Business Insider. I'm your host, John Brucato. Each week on School Business Insider, I sit down with school business officials and industry experts from around the world to share their stories and explore the topics that matter most to you. Find out what it means to be a school business official and get your insider pass on all things school business. Hello and welcome back to School of Business Insider. Today we're tackling an increasingly critical topic for school districts, Academic Return on Investment, or aroi. As resources grow tighter and districts face pressure to make strategic data informed decisions, Academic Return on Investment offers a framework to evaluate programs and ensure alignment between financial and academic priorities. Joining us are two experts in this field. Tim Hilker, Assistant Superintendent for Administrative Services at Washington Saratoga, Warren Hamilton, Essex BOCES, and Dr. Joe Greco, Director of Budget and Program Integration at Saratoga Springs City School District. Tim and Joe have presented extensively on ARoI, helping school business officials integrate financial and academic data to to drive strategic decision making, foster constructive data cultures, and navigate challenging conversations about resource allocation. Tim and Joe, welcome to the podcast. I'm happy to have you.
- [01:31]
B
Thanks for having us.
- [01:32]
C
Hey, John, thanks for having us.
- [01:33]
A
Absolutely. Tim, you've been on before. So what's been going on since your last episode where you, myself, and Christine really talked about the evolution of just board leadership? Give our listeners a little update on what's been going on with you.
- [01:47]
B
Well, we presented in Nashville and we kicked off the campaign for my candidacy for the Esbud International Board in 2026. So we've been busy with that in addition to our day jobs. So lots going on.
- [02:01]
A
Awesome. Well, glad to have you back on. And Joe, you're new to the podcast. Maybe you can give our listeners a little bit of your background and really what led you here today.
- [02:09]
C
Sure. So I'm probably not the most traditional school business official. My first 20 years was on the curriculum side of the house. So I started out as a high school science teacher and department head, went on to become a building administrator at the high school and middle school level, did several years as a K12 curriculum director of STEM education and essentially was on the traditional pathway, if you will, of a superintendency. And as I spent more and more time kind of talking with superintendents in the field, the bit of advice that they kept giving me was make sure you have a really strong school business official, because if you don't, you're gonna be in a lot of trouble. And I really took it to heart and I knew I wanted to sharpen my skills just a bit more. So I had a good relationship with Tim Hilker, and he kind of had this brainchild of maybe kind of creating a position in the business office to kind of use the skills that I had as a curriculum director and kind of leverage that to kind of bring the two sides of the house together, if you will. So I agreed to work with Tim. I did have my school business official certification, so ended up working in the position of director of Budget and Program integration. So essentially, that's my job, is kind of bridging the communication gap that often exists between curriculum, instruction, and school budget and really trying to tie our programming goals. Goals. To our budget development goals. So as I started with Tim, I said this just for a couple of years, just so I can kind of sharpen the saw a bit. And now I'm in year seven.
- [03:57]
A
Look at you now.
- [03:58]
C
Realizing how little I know. But to address my imposter syndrome, I recently did obtain my sfo. So at least I feel like I'm part of the conversation now.
- [04:08]
A
All right, so you're totally in it now. I mean, I think so. Once was just maybe kind of bridging the gap. I mean, you're all in. So. Awesome. Well, glad to have both of you on. And again, thanks for carving out a little bit of time today. So to really kind of kick this conversation off, what is academic return on investment, and why is it such an important framework for school business officials to really kind of focus on in really today's resource constraint environment?
- [04:33]
B
My passion from IT for it comes from the unification of instructional data and financial data. And I think more traditionally, those two really only intersect during budget season when you have competing priorities and decisions have to be made. But we feel strongly that we need to look at academic programs through the lens of both what it's doing for students and what it's costing and whether we're getting out of it what we want for what we're paying. So, you know, that's sort of a canned answer, but honestly, I think it's a conversation that we can't afford not to have. It forces us to focus the conversation among the administrative team, and it forces us to look inward at some of our systems, particularly around our data systems and our data culture. So we'll get into more of that. But they said it's just. It's an increasingly important conversation that I think intuitively we all kind of know that we need to be having. But it's just. It's getting down to it and making sure. That we're having meaningful conversations around it.
- [05:54]
A
And, Tim, did you find it challenging to really kind of make those correlations before Joe came along? Because, you know, to Joe's point, there is oftentimes a communication gap between the business side of the house and the curriculum side of house. But were you. Were you finding that as well just. Just before you kind of created this position to. To bridge that communication gap?
- [06:15]
B
Yeah, I was. And that was part of the origin of the position. And Joe, ultimately being hired at the position is. So much of this hinges on communication and collaboration. And I have the. The background to know the. The financial part of it and some of the data aspects of it. But what I didn't have was one, that instructional background to understand what the needs of staff and students were, but also the institutional knowledge, and then, you know, having somebody who has a little bit of both or a lot of one and a little bit of business to help me cross that chasm and get everybody talking the same language and having, like I said, meaningful conversations around that has been huge.
- [07:16]
A
And, Joe, on your side of the coin, I mean, coming from the curriculum world for so long, what was your experience kind of trying to meet Tim in the middle here to really bridge that gap? I mean, did you find it challenging to digest these financial concepts and correlate them to curriculum, or what was your experience in that regard?
- [07:34]
C
I mean, I will say, honestly, the financial piece of it, I feel like it was a completely different world. The language was completely different, but it allowed me to kind of take that step back and kind of understand how confusing it must be for folks on the financial side when the instructional folks are talking in all their acronyms and their different programs, just assuming that everyone has kind of that same base knowledge. So it did allow me to kind of understand both sides. But, you know, Tim was fantastic. And again, I knew I didn't have to match Tim's ability. I just needed to complement it. So, again, just bringing what my skill set was, what Tim's skill set was, so we could have rich conversations about, well, what are we doing? Why are we doing it? How do we know it's working? I'd say the biggest difference that I saw coming to the business office was the level of accountability being in curriculum instruction. I think that I was under the perception there was high accountability, and I think some of that was just my own integrity for the position and the work that I was doing. But I had never experienced things like audits or the frequency of audits and just how critical almost every Keystroke that you're making is in school business. And really now trying to understand that and relay that information to my friends in curriculum instruction to start to bring about some accountability for the decisions that we make and how the goals that we're setting instructionally, how they relate to the budget development process. I definitely felt that from the curriculum side, something that we do very well and quite proficiently is data collection. But I always felt that we were data rich and information poor. And that's really the work that I was doing with Tim was to say, all right, well, how do we actually look at this data a little bit more critically and start to tease out what's important, what's not important, and then make some sound decisions off of that data?
- [09:44]
A
So, as public school districts, we are evaluated pretty much all year by many different agencies and taking the same data and splitting it apart in so many different ways. So tell me how academic return on investment differs from maybe some traditional methods of evaluating your programs, your initiatives, and what kind of empirical evidence are you finding when. When implementing these. These returns on investment?
- [10:08]
B
I think in a word, it comes down to intentionality. But the way it differs in that is that we in education have a tendency to implement programs based a lot of times on feelings or perceptions or things we've heard, but we put them in, and we start to collect the data in the ways that the designers of the product have said that we probably should. And then we start to assemble that data in a way that supports what we're doing. And that's just. I mean, it's been happening for decades. I didn't make that up. But. But the academic return and investment model walks that process all the way back to identifying what programs you're using and then identifying the strengths and the weaknesses. And it analyzes whether changes need to be made to the existing program or the program needs to be perhaps overhauled or replaced. And then it looks at what are those problems that we're trying to solve or those weaknesses that we're trying to mitigate and looking at all viable options to do that, and then setting goals for when we implement the program and assigning timelines to those to make sure that if this happens, then we know we've reached it, or if this didn't happen, here's how we know it's not working, and what are we going to do? And there's a lot of work. I mean, I spent about half a minute on that, but there's a lot of work involved. When you take that and you and you overlay that with dozens of programs.
- [11:50]
A
You mentioned setting these goals really on the onset of any kind of program or initiative. Are all of these goals really objective in nature, or is there a certain level of subjectivity that comes along with academic return on investment?
- [12:04]
B
Well, I think that's why you have all of the team members in the room, because from the business side, we tend to want to look more, more at quantitative and objective measures. And you do have to have that, there's no doubt. But you also have to have the educators in the room who will help us understand what some of the more qualitative and subjective aspects of that are.
- [12:29]
A
And who is in the room. I mean, when you're setting out a lofty goal like this, I want to assume that there's buy in and everyone's really on the same page. But how do you select who's going to be a part of an initiative like this? Who's accountable for collecting this data, reviewing this data, and ultimately making a decision to maybe course correct if need be?
- [12:53]
B
You know, if you go back to the beginning of this process, the first thing you do is you've got to identify your readiness. And part of that means engaging everybody. For example, when we first started out, we didn't have our human resources person in the room and we realized quickly that we needed to. We also didn't have an administrator in charge of data. We realized quickly that we needed to have one and that person needed to be integral to the process. So you, you learn a lot as you go through this process, but really you have to have stakeholders which represent all aspects of the organization. You know, technology is another one. You just, you have to have all those people in the room and you have to have conversations about readiness and be on the same page before, before you can really get some traction towards this.
- [13:42]
A
So you've both been doing this for some time now. I mean, can you share any common misconceptions about return on investment that you've encountered that maybe when you had started out, had one idea of it, but realized very much it evolved into something else? What did you kind of have in terms of stumbling blocks or misconceptions?
- [14:00]
C
I think some initial misconceptions with it is, or at least people hearing about academic return on investment is that this is something that central office has come up with and it's their way of cutting programs. And that's a biggie because when people think that their program or their baby, anything that they've kind of brought in and they've nurtured over time is under attack. There's this natural tendency to kind of circle the wagons and protect it at all cost. So that's a hard thing when you're rolling something out like this and you're very excited about it is how do you kind of phrase certain things? How do you get people involved early in the process? So that doesn't come across. The other piece is that I think even from central office perspective, the thought of academic return of investment and to go through a process is for your high cost programs. And that's not always the case. Sometimes it's not even a high dollar amount could be a high time factor. It could be just something that over a natural period of time we would want to kind of reevaluate programs and make sure that they are continuing to provide us with the results that we thought. I think it's the other piece that we've dealt with with a misconception is we think everyone's going to love this.
- [15:28]
A
I was gonna say how did you introduce this and that? It just be another thing where people are rolling their eyes like, okay, here's another district initiative. But how did you really get that buy in? Because this sounds like a really lofty initiative which will ultimately pay dividends if you are ready to. Tim, your point? But how is it just not another thing?
- [15:48]
B
So we spent a lot of time on this and actually Joe recently wrote an article about this. But it's. You heard me say earlier that this is a conversation that we can't afford not to be having. Part of the initial steps of identifying your readiness is taking inventory of what programs you have. And so for example, where you discover that you had three different third party assessments or five or six different screening tools, not only those cost money, they typically would lack the ability to compare results. And they probably are consuming time in the classroom that we just don't even realize or account for properly. And so, you know, getting people to understand the efficiencies that can be gained from this, it's not just for the sake of cutting. It's for streamlining and getting comparable data so. So that we can analyze results. And that's really what is at the heart of this.
- [16:53]
A
And what are those steps to really kind of start integrating the financial and the academic? I mean, Tim, you just talked a little bit about readiness, but I'm thinking, how are people speaking the same language? How are people really buying into this? Like, where are those first steps?
- [17:08]
C
You want me to take this, Tim? So I think one of the first things that we try to introduce with. With folks is just to get the conceptual understanding of academic return on investment or just return on investment in general. So one of the examples that we use when we present is we talk about streaming services. It's something very accessible. Everyone has streaming services. So then we start asking those questions of, all right, well, which streaming services do you have? And then when you start to look at the ones that you currently have, how much do they cost you? And then start going a step further. So now you're looking at all these streaming services that you have, you have a general sense of how much each one might cost. Maybe you do. Oftentimes we don't. And then you start to say, well, which one's the best one? Which one is giving you the best bang for your buck, or your return on your investment? And what we always hear from folks, well, that's almost impossible because my HBO provides me with this particular show that I really like. But then I want Apple TV for something else, and then I want Paramount for whenever Yellowstone may return. So there's all these different things that we like. A little chunk of this and a little chunk of that. And then people start to make that connection to academics and start to see the. The detail that goes behind this and that. This is a very complex thing to look at. And I know Tim had mentioned that from the business world, we try to jump at the quantitative immediately, but there's so many nuances to the qualitative, and that makes it very challenging. But I think opening that up and having that conversation and being transparent about the challenges ahead of time kind of relaxes folks a little bit into the process. On the financial side, though, I think one of the most important things that Tim and I did was our chart of accounts. I think building your chart of accounts specifically with areas where you can identify programs and kind of put all of those particular expenses for that program and easily to kind of decipher exactly how much this is costing us, whether it be on salary, contractual costs, supplies, equipment, even transportation. Having all of that at your fingertips allows us to get a better sense of what are some of the direct and indirect costs associated with particular initiatives.
- [19:43]
A
So it's not even just what data you're collecting, but how you're organizing it. Because, Joe, to your point, you could run a simple appropriations report and see at a glance this is what X program is costing us, rather than doing a full budget, export, filtering, and doing all of this. So you're really kind of setting yourself up for success by really organizing your data with purpose. That's a great piece, which kind of leads me into my next question. So, Tim, you had mentioned who's involved and who should be involved, specifically with technology and your data people, but can you talk to me a little bit more about maybe what tools or resources you found most helpful with collecting and analyzing AROI data?
- [20:26]
B
Yeah. So, I mean, in the most basic form, it's your financial management software and it's your student management software. But I think getting a handle on what systems you're using for different purposes and if there are gaps and overlaps between them, and perhaps either streamlining or somehow making that more. More efficient, because that was really the lesson learned for us was that we had tons of data, but we just. The data wasn't really consistent and it wasn't really telling us much. And so that was really our first mountain that we had to climb. You know, we use all the traditional tools like Excel and such, but as Joe mentioned, it was for us on the business side getting our accounting system set up to be able to provide really good information and somewhat in real time. Because this process, as you start to go through it, tends to evolve, sometimes quickly, but sometimes just in different ways than we anticipated. But even on the student data side, it's sometimes used differently in different areas of the organization, or there are different ways of understanding what the intent behind it is. And so you kind of have to walk that all the way back to how the systems are used, how they're managed, and what the beliefs are around the purpose and the outcomes of using those systems.
- [22:00]
A
Just by peeling back the onion a little bit, like you have been with aroi, did you regain some efficiencies just in your business operations and your data operations? Because to your point, you said you were data rich, but information poor poor. Were you just finding yourself on the hamster wheel, just collecting data maybe for the sake of doing so?
- [22:19]
B
I'd say somewhat, and Joe may be able to speak to this a little better, but having rebuilt our accounting system after we used it for a year to really helped improve our budget development process. So that was one very noticeable way in which it helped us out, even if we had only done it for that purpose. But then we were able to gather some of the financial data and the outcomes relative to what we had planned when we developed the budget and things like that, and then we could take those and overlay those with some of the academic outcomes.
- [22:59]
C
Yeah, I mean, as good as those things were, I think probably the most powerful tool that we ended up using was our initiative inventory because that's a term you hear all the time is initiative fatigue. And I really didn't give it too much thought originally because I just thought it was just people just belly aching that they got too much to do. And then when we started to do the initiative Inventory, as I wrote, well, let's, let's quantify this. What is actually on your plate, right. And what we found from the results of this, and we asked principals and we asked teachers, and it was just this, I mean, massive, massive laundry list of all sorts of things it could be from, you know, academic coaching, integrated classrooms, universal screeners, responsive classroom learning management systems. They were listing basically the kitchen sink of everything that they possibly touch in a school day, in a school year. And the biggest takeaway that I think Tim and I got from that was, well, one, there is some legitimacy to when someone says, I've got a lot on my plate. But we quickly realized that we were naming everything in initiative.
- [24:26]
A
I was going to say, how do you differentiate, differentiate between initiative and just what you do day to day? Right.
- [24:31]
C
Yeah. So that's where we started to kind of peel this down and realize that we needed to. Again, we have a good chart of accounts. We needed to really start to pare down what our language was. So we made sure that we were speaking the same language in the business office that we were with our curriculum folks and our teachers and building principals. So what we started to do is we looked at this whole laundry list of all these things that people were listing as initiatives. We said, well, let's, let's get back to basics. Let's trace this to what's our vision. And I think one of the universal things that you're going to find in most vision statements at a district is talking about increasing academic achievement. Pretty basic. So then we say, all right, well, what's the actual initiative that helps to support that vision, the increasing academic achievement? So then we said, all right, well, high quality core curriculum is really the initiative to support that vision. And then the programs that really help you do that is going to be like your special education programs for your hardest to learn students, your multi tiered system of supports for those students that kind of sit on the cusp of not meeting state standards. And then you should also be providing programs of enrichment for your students that have already mastered that content. So then what was all of these other things everyone else was listing? And we decided to refer to those as supports. These are all additional supports that were really kind of churning the water here in the individual classrooms. Again, they were there to support programs, the actual initiative or the vision of increasing academic achievement. But it was so cluttered that teachers didn't really know which one was working, which one wasn't working. We were holding on to things that we've just done because that's the way it's always been been done. Sometimes we were holding on to things because it helped a couple of students in the past, but it, there was no mechanism in there for collecting data and really looking at these things critically. And again, it wasn't anyone's fault in particularly, I think, you know, having been a classroom teacher and been a building administrator. I think educators have big hearts and our tendency is to kind of have a kitchen sink mentality that if it's good for one student, it could be good for all my students. It would be preposterous to think of doing the same thing in the medical field when we say, well, one pill is good, so 10 pills must be 10 times as good. But that's kind of where we're at. We want to help all of our students. We don't want any of them to struggle or fail. So we're kind of adding more and more of these supports on. Some of them do come from central office or they come from a building principal, but oftentimes these are. We were finding they were coming right from the classrooms themselves. So some of this I have too much on my plate or initial fatigue was actually self inflicted. So by having all this information and then kind of talking more openly about the this and again acknowledging, seeing that people are stressed, they have a lot going on. Now starting the conversation of, okay, now that we've established what these things are, let's talk about where is their redundancy and then how can we start putting mechanisms in place to see, well, which one is yielding the highest return on our investments so we could be more strategic in our next move.
- [28:04]
A
So how did you collect the data for this initiative inventory? Who was looking at it and how long did it take? Because when you were reciting the process you went through, my head was spinning at it. Must have taken quite a herculean effort to really dissect this data purposefully. Tell me about that process.
- [28:24]
C
Yeah, I'm just trying to think back on it when we initially did it.
- [28:27]
A
Unless you blacked it out because it was too much.
- [28:28]
C
Yeah, it was a little traumatic. This started out really as a survey out to our teachers and to our principals and central office and then again just kind of talking about, so here are some like major areas and then Having them list out some things that again, that they felt were initiatives, and then kind of taking that massive list and then start to look for similarities, categorize it, and again, just continue to distill it down further and further. Again, I went into it initially thinking there was going to be several initiatives. And then we can kind of talk about, well, which one was the one we wanted to do an academic return on investment study. And it just turned. The data that I received turned out to be so massive that I knew we had to kind of pump the brakes on getting to the next step of a return on investment study and really start to identify what are all these things. And then now our academic return on investment study is really more focused on some of the supports rather than the initiatives. I think something like an initiative, high quality core curriculum, I don't need to do an academic return investment study on that in particular, but maybe our after school tutoring, which is a support of that, is something that we do want to do a study on. And that's where we start to get into our next phases.
- [30:04]
A
This perfect segue to Max's question. You know, when you're both working together at Saratoga, can you recall an example of how academic return on investment was used successfully?
- [30:15]
B
Yeah, when we first started this, we. We wanted to really look at our elementary ELA curriculum. And we realized fairly quickly that that was a really broad stroke. And we started whittling that down to what if we just looked at the third party assessments and comparing the predictability of those two state outcomes. And there were a lot of iterations in between, but like Joe was kind of talking about, we had to go from big picture down to one of those smaller supports. And then it was something small enough that we could tackle. And so we were actually able to successfully implement one third party assessment across all six elementary buildings at Saratoga so that we could compare results and so that we could set academic goals and measure our progress relative to those goals to see if what we were spending on those products were really worth the value. That was the way that it was kind of stated to us as, we can't live without these. And so that was just one example. We also had, I think, some pretty good success looking at some of our secondary tutoring programs. So, you know, we've done a number of these, but those are two that come to mind.
- [31:56]
C
I will say an interesting caveat with our initial study. When we were looking at third party assessments, One of the things that I had done on that study was basically I wanted to look at A correlation between how students were doing on an assessment and seeing whether or not it was a good predictor of how students would do on state assessments. So essentially I was using a Pearson coefficient to say, all right, this is what the Lexile level, reading level was for these particular students. And then how are they performing on a raw score on the state assessment? And the data showed us that one of the assessments was a better predictor. We also looked at the cost per pupil, and that same one that was a better predictor was actually less of a cost for us. One additional data point that we did look at was teacher satisfaction with the particular software. Usability, user friendliness, however you want to put it. And the other software was actually preferred by teachers. So even though we were collecting that data, it goes back to kind of Tim's original thing of the business office. We look at data and we go, oh, well, this is clear as day. This one is cheaper and it performs better. We should obviously go with this choice. And what we learned early in this process is that is not always the case. So, long story endless. We ended up going with the other program that was more expensive and not as solid as a predictor. But the teachers were more comfortable with that particular software. They liked the usability of it. So if there's anything to be gained, the one piece is that we are consistent, at least that we are using a single program instead of two or three different assessment tools. But it allowed us to kind of go into this understanding that this is a messy business, that we want to try to collect as much data as possible to make informed decisions. But at the end of the day, that's the best we can do, is provide the data for folks to make informed decisions. And sometimes it might not be the decision that we would go with, but we again, are setting up the template for them to use this to be, again, more deliberate with the decisions that they're making.
- [34:22]
A
Well, I mean, there's that subjectivity we were kind of talking about earlier. Even though the data black and white may suggest to use software A, you're using your professional judgment along with the input from the teachers that maybe software B, although on paper may not look like the best solution, truly did work out best for Saratoga. So. So you gotta be a little bit fluid with something like this. Can you tell me a little bit about some of the biggest obstacles you face with. With implementing academic return on investment and, you know, how were you able to overcome them?
- [34:54]
B
Yeah, I'd say paralysis of analysis is definitely something we suffer from. I think this will resonate with most of your listeners. If you get a group of administrators in a room. There's no shortage of reasons why we can't build an accountability, you know, but, you know, I, I think that we, there's a natural presupposition that if it's for kids, it must be good. And I think a lot of times that's true. But you'll hear us say a lot of times in our presentations, just because you can doesn't mean you should. And so I think getting beyond that initial fear of what's the data going to tell us and how am I going to respond is certainly an obstacle we found. And I have to imagine that we're not alone in this, that our data system and our data culture was absolutely a barrier to us being able to do this, because otherwise you're just constantly trying to chase information and it's not really productive for anybody. Which I think is probably leads me to my last one that comes to mind and that's the lack of immediate progress. This is a very long, well, it can be, I should say, a very long process. It requires a lot of collaboration, which takes time, but it's, it's hard to see results immediately. And that's something that I think in education we often look for is, are we getting immediate, noticeable results? And that's just not how economic return on investment works. I think you really have to build in the systems and the processes and get the collaboration working in order to, to make it work.
- [36:49]
C
The, the other obstacle I'll mention in here is the weaponization of, of data in the spirit of academic return on investment. And I think sometimes folks will see this as an opportunity to go after a program that they've already made the determination in their head that this is something that has to go. And now we're going to try to collect data to support that hypothesis. And similarly, we have some people that have a particular program and they know in their heart of hearts that is the best program because we know that it, it was such a success for, you know, this particular student and they tend to cherry pick certain data in order to support the hypothesis. So we've really tried to push that when we're rolling this out and we're having data collected, that the person collecting the data is not someone who is directly involved with the particular program or the person who's analyzing that data is not directly involved with it. So it might be your math curriculum person is helping with the ELA data collection or vice versa, or if your district has A central data person, that's always the ideal, but again, try to move that around to prevent people from, again, not that they would naturally want to skew results, but there's inherent bias that exists anytime we're collecting data like this and when folks feel that this could potentially either eliminate a program or expand that program.
- [38:24]
A
And to your earlier point, maybe one program worked really well for one student, but that doesn't necessarily correlate to it working for all students, which would lend itself to. There's probably going to be some kind of resource reallocation when engaging in this study. How do you approach those ultimately difficult conversations and having to speak to somebody who may have inherent bias on a program and cherry picking the data. How do you, how do you approach those conversations?
- [38:54]
B
Well, this is where the readiness comes in handy. Because if you have properly set out your objectives and set goals and furthermore put in processes to measure and agreed upon on what those outcomes will look like and how we'll respond, I think it's easier than to focus on what the outcome is or should have been. I think also understanding what's important to all of those involved in the process, which presupposes that you've properly identified all of the stakeholders who should be involved in the process, but just kind of understanding what things people are passionate about and why they're so passionate about, because I found it's not uncommon that once you get past that and they realize that this is not some sort of a personal attack on their program, that just by asking a few questions that they come around to, oh, this isn't so bad. And so taking that sort of team approach and not having it be because I said or because Joe said, but because this is what we agreed upon and this is what we said we were going to do and if the plan changes, that's okay, but we all need to agree on that.
- [40:22]
C
So a personal tip on when you're trying to address this with folks, because I think everyone tends to see this as the dichotomy of they're either going to leave me alone or they're going after my program. I always try to approach it with folks to say, I want to start collecting data on this particular initiative, this particular support or program, so we have that data. So if we potentially want to justify the expansion of that support or that program in the future, we have that available. And I think when you approach things as there's a potential to expand this, I think it relaxes folks with the process. They know that this isn't central office or the business office in particular, looking at how much are we spending and how are we going to cut this to balance the budget. We have a fairly large budget, and we should be supporting programs, and we certainly can support programs. But we want to be judicious on how we're allocating those dollars, and we want to make sure that those dollars are going towards things that are giving us the highest yield. So when we say, all right, let's look at some data, and if this is checking the box and it's doing what we think it's going to be doing, that's great, and we have evidence to support that should budgets get a little tighter in the future, or it also gives us the information that maybe there's something we need to fix. That's another way of kind of softening this conversation is sometimes you might have had a great, let's say, math program that came in, but you had a lot of teacher turnover over the last few years. And because of that, those new teachers coming in are lacking some of the professional development. So it may be giving you some false data that the program is ineffective, when in fact, it's part of our implementation that just needs some tweaking, a little bit of fixing. We invest a little in professional development, which from the business side, is going to cost me a heck of a lot less money to fix an existing program than to actually scrap it and bring in an entirely new one.
- [42:28]
A
And with such a novel framework, what did your conversations look like with your board of education, your community? We've been talking a lot on this episode so far about the internal conversations of readiness, who's involved in the process, how are you evaluating the data, but how are you ultimately messaging this to the community at large? Because it's a huge undertaking.
- [42:48]
B
I think the same is true with some of our colleagues as we presented this in various settings, as it is with board members and potentially community members who maybe are interested in this. And when you really listen to what this is about and what it's for, intuitively, it just makes a ton of sense. And people are often walking away thirsty for more. They want to hear more, they want to see results of the work you're doing because they know it makes sense. The harder part, though, is when we do really get past those first few initial steps in the process process, and then we start talking about either instructional outcomes or financial outcomes, and people start to get a little uncomfortable, then you have to. When. When that process starts to get a little sideways, you have to kind of take a step back and manage that.
- [43:53]
A
And.
- [43:54]
B
And so much of this is that management of that process. But like I said, to the outside observer, this just makes so much sense and should be. It shouldn't be so novel to steal your word. It really, our whole passion behind this is that we hope that this will kind of become the standard, that this is how people are thinking about these things and how they're talking about financial and instructional outcomes.
- [44:27]
A
It just seems obvious that academic return on investment be incorporated throughout the budgeting process. Joe, you spoke about a little bit, but how did you kind of now weave this into you? What was your budget process before and how has that that landscape changed a little bit for you, your superintendent, and your board?
- [44:48]
C
I think one of the most important things we're doing now, now, and this really came about with the infusion of the COVID relief dollars, is that immediately we said, all right, so if this is going to be something new that's coming in, that we're applying new dollars towards, that. We had a kind of a budget process where when they were making that budget request, they were identifying, starting with the analysis of what's the current reality. And now we're doing that with all of our new programs, even after the COVID relief money has fallen off. But we're looking at the initial analysis. So asking folks to give us what the current reality is. And when I say current reality, I'm asking for some data on what's working and how do they know. And I always start with the what's working, because again, if I open it up to say, well, what's broken? It's very easy to say everything. But when you start looking at what's working, it allows me then to leverage things like, well, can we streamline that existing program? Can we fix the existing program, again, being much more cost effective? But I do ask for that in the initial analysis of what's working, how do we know? And asking for some of that data. And then if we're looking to implement something new or to change direction, then I'm looking for the actual implementation plan. What's the vision? What are the prerequisite skills that our teachers are going to need? The other piece we're looking at is incentives. So how are we going to convince folks that have been doing something for a very long time, how are we going to convince them to do something differently? And not because I went to the ASBO International Conference in Fort Worth, Texas in October, and I came back with this fantastic idea.
- [46:42]
A
Great plug, Joe. Real nice.
- [46:44]
C
Not enough, you know, but that's simply not enough for me to come back with a great idea for everyone to be like, oh yeah, absolutely, let's do that. So there has to be some built in incentives so people want to change direction. What resources are folks going to need? So is this going to be textbook? Is it going to be supplies, equipment, electronics, whatever that goes with it? And then eventually you need that plan, the action plan of who's doing what, who's going to be responsible, how's the data being collected, who's collecting that data? How is that analysis going to look? And in that implementation plan is really the measurement tool that we've been talking about for academic return investment. And as simply put, what does success look like? And identifying what success looks like before you actually roll a program out is critical to this process. I cannot stress that enough identifying with all the stakeholders in the room at the same time to say, this is what success is going to look like. This is how we're going to measure it, this is how it's going to be quantified. And even if you say we're going to expand this over one year or two years before we make any program decisions, but again, so folks are very clear about the data and how we're going to collect it, and then the final piece in there is how we're going to respond to it. I know Tim has mentioned this a couple of times, but that response is going to be critical because once we have the data and it tells us what this particular story is, then we have to stick to our guns and say if we are going to fix it, if we're going to expand it, are we going to streamline it potentially, are we going to eliminate this program and go in a different direction? But I think for us to be brave about this in our conversations with our board members, with our superintendent, with our other stakeholders, I think that's critically important because we found that when you try to look backwards at existing programs and try to collect data on those, that's where folks get, again, an inherent.
- [48:53]
A
Bias where they say tribalism comes out and they want to preserve that programming.
- [48:56]
C
Yeah, absolutely. So then they start to get particular data points that support that original hypothesis, which kind of clouds the whole idea of setting up an experiment here.
- [49:11]
A
And with academic Return on investment, I guess specifically in Saratoga, you've been doing it for some time now. Do you feel that this is now an inherent part of the district's culture and will outlive your tenure, your superintendent's tenure there, your board members? I mean, is this really kind of just baked into who Saratoga is now.
- [49:31]
C
I would say not yet. And kind of have a growth mindset on this. This is challenging. And I think Tim's mentioned it multiple times. This, this takes a lot, lot of work. And I don't think we've gotten to the point quite yet. And I know we were making some progress on it. And then once Covid hit that kind of slowed down that process. And I'll be honest, even the infusion of the COVID relief dollars actually did more harm than good in an instance like this, because it really reduces the sense of urgency that we're dealing with finite resources here, and we need to be more judicious on how we spend those dollars. I think moving forward, now that you're having declining enrollments happening across most districts, that you have your kind of fiscal cliff of the COVID dollars coming off, you have political pressures of trying to privatize education. And then of course, you're going to have increasing expenditures, whether that be in salaries, in healthcare, general inflation. I think all of those pieces together now is starting to tighten the belt a bit on school budgets. So something like academic return on investment becomes more in favor in those particular times. I think the best thing that we can do right now is just be ready so that as we start to get into those tougher budget discussions that we've already identified, this is what success looks like. This is the data, and we can make more precise decisions and stretch our dollars as far as possible.
- [51:08]
B
I would agree with Joe's analysis of that, but I will say this, that. And I said, I've said this for a long time. In order to do some of the work that we've done, we had to work on some of those systems. Like we talked about our financial accounting system and our data system and our process and our culture around data. Having established some of those systems and processes, those are things that I think would live on past any of us. And so if that were the only thing that came out of this, I still think that a lot of good came out of it for kids.
- [51:49]
A
You know, you bring up a great point about the federal Covid money that hit a lot of districts. I guess I didn't really think about how in a framework like this could almost really derail the thinking and the mindset behind why you're doing it. Did you find that you had to almost start over and re educate everybody as to why return on academic, return on investment was a priority? Because the purse strings didn't feel so tight for a couple Years, but we're kind of. We're getting back to, if not already, to where we were in 2019 and prior. Did you have to do some kind of re education and start from scratch? What did that look like?
- [52:24]
B
We definitely lost a little bit of our foothold with COVID relief funds, but we. We kind of pivoted quickly and the conversation turned to, okay, if we're bringing in new programs with this money, let's look at it through this framework. And I think people were generally receptive to that. But we also, we did it with the end result in mind that eventually that money's not going to be there. And so we're going to have to make some difficult decisions about either programs that were implemented or staff that were hired or whatever it may be. And we need to have this information readily available when that time comes, because there are going to be some pretty deep conversations happening in relatively short amount of time. So I think we navigated that successfully, and just probably in the last year or two have kind of gotten back to where we were pre Covid and making this more of a language that people are using and getting some common understanding about what we're doing, why we're doing it.
- [53:40]
A
So what do you see on the horizon for school districts in the next three to five years, and how do you think academic return on investment is going to be able to help address those?
- [53:50]
B
Well, you know, I think the next few years are going to be particularly challenging. I think part of that is related to the COVID stimulus funds going away. But I also think there's some other pressures that schools are facing, both from a revenue and an expenditure standpoint. And I just think it makes it all that much more critical that we're thinking about how we're spending money in these ways and making sure that while the cheapest option isn't always the best, that we have good justification and we are intentional about how we're allocating our resources. And. And I think to balance that out, we talked earlier about the misconception that this is about cutting the budget. And you do have to kind of walk both sides of that line, because when resources get tight, you are really using aspects of this framework to make some of those difficult decisions. But that's not the intent behind it. The intent behind it is to make sure that the programs we're keeping are the most meaningful for kids. And if there's one takeaway from this discussion, I hope it's that, because there really is a lot of value in these conversations, and the value is really, for students.
- [55:18]
C
One additional thing, Tim, I'd add, is that this isn't always just about money. Sometimes it's about time. And we hear a lot that teachers are burnt out and that our students are burnt out. And some of that is because of that kitchen sink mentality of throwing all sorts of resources to students and some students and teachers that may not need that particular resource. So where there is money to be gained in some of these, there's also time to be gained. And hopefully, as we start to reduce some of the clutter and reduce some of those items that are on the plates of our teachers, hopefully that starts to improve their general feelings about what they're doing in the classroom, reduces some of that anxiety, and hopefully helps with our recruitment and retention of our faculty.
- [56:08]
A
And if someone's inspired by our conversation today and really wants to take a good look at academic return on investment, where would you recommend they start?
- [56:18]
C
I think there's a lot of information that's out there. Certainly you could reach out to Tim or myself on this topic. You know, we've both published on it. We've, We've, you know, attended and presented at both the. Our state ASBO and ASBO International. We're always willing to share our experiences, our successes, as well as our pitfalls to kind of help, you know, bring this information out and, and support our fellow school business officials.
- [56:51]
B
I think, you know, we could certainly share some tools and some best practices. But I think starting by just taking inventory of all of the different programs you have, identifying the costs, both direct and indirect, that are associated with those programs and, and just having that information to bring to your administrative team and have this conversation about how are we looking at this both on the instructional and the financial side. You know, we had a state of goal when we were bringing a new ELA program of increasing the number of students who are or above grade level in reading by third grade. And I recall distinctly about two years into that process, in a meeting I asked the question, how are we measuring on or above grade level? And I got two or three different answers. And so I bring that up, not to point fingers or anything, but because you have to stay focused on a common goal and a common understanding of whether we're reaching that goal or not. And you do. You have to be realistic about it. So, you know, that doesn't point people to a specific tool or resource. Like Joe said, there's a number of different authors and consultants out there who do work on. We are certainly not the leaders in that field, but we are passionate about what we do, and we're happy to help anybody who just is really looking for that jumping off point of what can I do to begin this conversation or if they're facing a particular obstacle, how they can overcome that, we'd be happy to help. And I know there are many others out there.
- [58:54]
A
Well, gentlemen, it's always a pleasure to speak to you both. And I appreciate you really diving into the details of academic return on investment. And if anybody is interested, I'm sure you're going to be presenting in 2025 at Fort Worth. So take a look out for that. But thank you both again for spending a little bit of time today and discussing such an important topic.
- [59:13]
B
Thank you for having us.
- [59:14]
C
This is great. Thanks for having us.
- [59:18]
A
Thank you for tuning in to School Business Insider. Make sure to check back each week for your favorite topics on school business.