Loading summary
Transcript144 lines
- [00:00]
A
Foreign. I'm Santi Ruiz and you're listening to Statecraft. As a reminder, the complete annotated transcript for this episode, as always, is at www.statecraft.pub. today I have the privilege of talking to Jeremy Singer, former and current president of the College Board. Jeremy, it's good to have you on.
- [00:23]
B
Thanks. Excited to be here.
- [00:25]
A
Before we get into the story of the day, which is the FAFSA salvage operation that you did a couple years ago, maybe the most urgent question here is my SAT scores, which I'm sure you saw in your role as president of College Board a few years ago. What did you make of them? Room for improvement.
- [00:44]
B
Excellent scores. Yeah. Well equipped for the modern world.
- [00:48]
A
Gosh, Jeremy. Well, thank you.
- [00:50]
B
That is a very common question. We do not have access to scores, so your listeners don't have to fear.
- [00:55]
A
No, you're not. Name searching.
- [00:56]
B
No. Even very important people, we cannot.
- [00:59]
A
Okay, that's good. Institutional trust building right there. Jeremy. I'll just narrate for listeners. You've been president of College Board for a while. A little more than 10 years, I want to say. Is that right?
- [01:09]
B
Yeah, I joined. I'll be 13 years. I'll join the College Board in March. So long as I've ever been in any one place.
- [01:16]
A
That period as president of College Board was punctuated by a roughly six month stint in the federal government. And that's what I want talk to you about today. So we'll get into the history in a second for listeners. Let me narrate like a high level picture of what, of what I want to talk about. And you, and you correct me here. Fafsa, which is the main system for student financial aid for college applicants. There's a big effort around 2020, a big bipartisan bill to simplify it, to make it more streamlined for applicants to go from around 100 questions to around 36 questions, for instance. And when that effort was finally rolled out at the very end of 2023 and into 2024, it was totally botched. And we'll get into the details there in a second. But tons of applicants couldn't finish filling out their forms. Tons of applicants couldn't access the site. Colleges had to wait months to get initial financial aid information from students and national news. There will be a chunk of listeners who are trying to get financial aid right around this time.
- [02:18]
B
For a number of listeners. It will bring back ptsd, but yeah, that's right. Framing, I will say my six months, I call it the world's worst sabbatical. Not the way, you know, it was good to have a little bit apart from College Board, but it was like jumping into a fire. Let me start on, you know, even at a higher level, which is despite a lot of noise and claims about the value of a college degree and so forth, there's a ton of research and Raj Chetty out of Harvard is one of the best that shows a higher ed degree is the surest pathway to better life. And for a huge number of students. FAFSA is the tool that unlocks financial aid that allows for a large number of people in the US to actually fulfill that, go to college, get a degree and improve their pathway.
- [03:07]
A
Will you give a sense of the scale there? How many students, American students are filling out FAFSA in a given year?
- [03:13]
B
In total, There are roughly 17 million people access the system on an annual basis, but that's both students and parents or families. So I should get the number. I think it's at least 6 million. Go to it. What unlocks is everyone's very familiar with Pell Grants. So it unlocks Pell, which is federal aid. There are federal loans you can get. There are work study where you get paid to work in college that the feds pay for. But it also FAFSA is used by many states and even some colleges to open up their aid. So the state aid to students or an individual institutional aid to students. So it is sort of the linchpin of financial aid for so many families in the US and will you just
- [04:00]
A
explain that federal state relationship for me? Just I fill out FAFSA as an applicant. That's a federal program. And then a state like Louisiana, maybe Montana needs that information to prime its own financial aid decisions.
- [04:15]
B
Students could either be an independent student so they are no longer dependent on their, any, any family for financial support, or they're dependent like many younger students are. And the FAFSA itself is a form that people fill out. If you think about you get a car loan or you're buying a home, you fill out a lot of forms to get a loan. You know, you could think somewhat analogous to that. So a student, a prospective college student and or higher ed student and their family will complete this form. It will be analyzed to determine what level of aid, what kind of need they have based on their income and family income and all these other factors. And then it will identify what level of federal aid they will get. And it generates something called an icer, which is an institutional student information record, but generally known isir. And that for that student is then the Federal government facilitates it and sends it to states and to colleges. So they consume the icer and then based on that, they will know what kind of federal aid the student is going to be eligible for. But in addition, they'll know they can say if Louisiana is giving some state aid to students in Louisiana, they could use the ICER and the evaluation to determine how much does Louisiana give to that student if they go to Louisiana State school and then even an individual institution sometimes. You know, many institutions give institutional aid to students in need. And so they may use this as the best proxy for that need and say, okay, this student, the tuition is going to be less or we're going to give them this scholarship based on this icer. But it's all contingent on the information originally filled out by the student and
- [05:59]
A
their family, which millions of kids are filling out every academic year.
- [06:04]
B
Yeah, and I would say there was an old system that before FAFSA was redesigned in the last couple years, there was good research that there were literally millions of students who would be eligible for federal aid and potentially state and institutional aid, but never completed the fafsa. And some of it was, it was too complex, some of it they didn't know about. There's a lot of reasons. And so this had been an issue for a long time. And Lamar Alexander deserves a lot of credit.
- [06:33]
A
Republican senator from Tennessee.
- [06:35]
B
Yep. And he had a lot of desire to try to simplify this process so it'd be easier and more accessible to a lot more families. So a lot more families would be able to complete it and then get aid and it would help a lot more low income students. Well, you'd open up college to them. And so when he was on his way to retire, it was A rare as 2019, there were two, as you pointed out, two congressional bills that were basically about FAFSA simplification. And the question was, hey, can we take this cumbersome 100 plus question form process where a lot of it was taking actual data from your tax returns and figuring out, oh, this field 38 needs to go here and mapping it in and actually physically a family inputting it and potentially making mistakes and so forth to modernize that and make it easier. So it'd be significantly fewer questions. There would be logic built into it. So if you think TurboTax, if you ever do that, based on if they find out some information and you, they say, hey, these six questions are no longer relevant, so I no longer need to ask them. So they did some piece of that logic into making it more Streamlined the questions that are relevant as it figures out what you put in. Most importantly, and this becomes part of the storyline later, was I think the single biggest breakthrough was going to pull data from the irs. So the IRS has, Santa, your tax. You did a tax return. Hopefully.
- [08:02]
A
You're making me second guess. Did I leave the oven on?
- [08:04]
B
Yeah, I'm trusting you here, but I know so. But let's say you filed and you have a child. Now instead of you trying to figure out what parts of your tax return you need to physically enter into the form, it would just ingest that from the irs. And so that dramatically reduced the time and amount of input. Literally families going from over an hour, hours to complete the form and find all the information to literally tens of minutes, some under 10 minutes, depending on the complexity of the individual.
- [08:33]
A
And as you point out, the instinct here behind that kind of bipartisan congressional push was very straightforward, obvious reform. One of the federal government has my tax information. It has all this information that's relevant to fafsa. There's no reason why we should throw that back on the taxpayer or on the American citizen and say, go dig that up again. We've already got it. It's a matter of linking up these massive internal federal systems.
- [08:59]
B
That's right. And so all well intentioned, all made sense. But as you noted then, it became the biggest software project that the Department of Ed had ever done and their initial take was not successful. And so it was supposed to launch in October of 2023. And as you pointed out, it didn't launch October 2023 to launch much later. And then it was launched in pieces. And I can get into that if you're interested.
- [09:25]
A
That's basically where I want to go here today. I don't think you're going to toot your own horn like this, but I'll just say you were brought in was June 2024, so six months after the actual botched rollout. And Jeremy was responsible for running what was basically the six month cleanup job of like, let's salvage FAFSA so that this next calendar year for students, students get access to the aid that they are fully federally eligible for. So I guess I want to start here without casting aspersions or we don't have to kind of name names or place blame, but talk to me about basically 2019 is a 2020 Congress says Department of Ed. You've got the better part of three years to set up this new system and we want it ready for the kids in the fall of 2023 to apply so that in the fall of 2024 they go to college with however much in federal aid. What happened between those bipartisan bills passing to great fanfare, Lamar Alexander retiring, and this kind of nightmare rollout?
- [10:21]
B
Yeah. So I've spent a lot of time trying to uncover that and discover it. So I'll walk you through it. I do want to say I appreciate the recognition. You know, I got a call from the White House in May of 2024 and sort of they were modeling it after what the American Health Care Act. When they, you know, initially launched the exchanges, they weren't working. They kept trying to fix it. And eventually President Obama had the foresight to bring in a team from the private sector with software experience. And they came in and they fixed it. And so that was the same. They got to a point where they were like, we need people with software and large scale software experience to come in. And that's what they hid in my name got floated. I got asked. I felt from College Board role the biggest thing I could do both from a mission standpoint because all the students that couldn't complete FAFSA and the families remember or students were trying to serve as College Board. And they were a membership organization. So every higher ed institution is a member. And that was maddening for them. It made their job next impossible that year. So we decided the best thing I could do more than anything I could do at College Board in the next six months was to try to write fafsa.
- [11:26]
A
Just jump over and I'll just say.
- [11:27]
B
Last thing I'll say is I called anybody any brilliant person I work with, my career friends, colleagues, former colleagues, and had a great team of eight people. Jeff Olson, who's a CTO for College Board, I think he mainly came because he was like, you're way over your head. You need me. But other great brilliant people, including someone who used to. I worked with in multiple jobs, but was at a different organization, Aaron Lemon Strauss, who came on board and he's actually, when I left, he stayed and he's sort of now the GM of fafsa, which is what essentially I was with Jeff being sort of the CTO of fafsa, which they had never had.
- [11:59]
A
They never had a cto, just to
- [12:00]
B
be clear, they had. So I don't know if you. We can decide how. How deep to go here. But. So there's Department of Ed and then within that there's fsa.
- [12:09]
A
In fsa, which is Federal Student Aid.
- [12:12]
B
Yeah. And it was one of the. I think there are Three or four set up by government to be a performance based organization. Right. And so the idea there, they want it to run more efficiently and so forth. I think the post office, something in NASA, maybe there's a couple that are set up like that. I should know the list. You can correct me afterwards, we'll go check. But the idea is to run it a little more like a private sector. And why is that? Well, if you think about it, fsa, there's trillions of dollars of loans. So it's one of the world's largest lenders. And so it's not just another branch of the government. And it's a little odd, it's in the Department of ed, but it's really a huge financial loan organization. And so they had within FSA there was a chief operating officer, which was in government they call coup. I never heard it before. It's referred to, okay, chief operator, I guess that's the government.
- [13:08]
A
No coo, we say coup.
- [13:10]
B
Yeah, coup. I had to get Deputy Q. But anyway, that's so funny. So that the COUP was supposed to be the person to run all of fsa, which isn't just fafsa. FAFSA is a big piece of it, but there's a lot of other, like how you manage outstanding loans and so forth. So there was that job who you could say is it should have been the GM of fafsa, but they had a lot of other responsibilities and there was no one then specific to FAFSA or any fafsa. And then there was there was a CTO of fsa and then there's someone at the Department of ed, but not specifically to fafsa. And so that becomes a big issue. But let me go back to your original question, which is like sort of what went wrong? And a bucket into three large areas and we can go as far or deep as you want. I should say none of these are super new. So one is the bills we talked about, the two bills that were bipartisan, that were incredibly well intentioned, passed in 2019, were overly prescriptive on how the system had to work. And they basically hard coded product requirements. And 30 years ago, 40 years ago, everything was what calls waterfall, where team one defines every requirement. You know, there's a team that defines exactly what the product has to look like or the output has to look like. And then they hand it over to developers, it's going to code it and then they hand it over to QA who's going to check it and then they launch it and that process While very large in the 80s and some in the 90s, was replaced in the most cases by some new version of Agile and Agile. The idea is the core here. My cto, Jeff Olson, will always lecture me, your future self is almost always going to be smarter than your current self. And so if you try to define everything up front, you're going to learn in the process and you're going to miss out if everything's hard coded or defined. And so Agile starts, instead of trying to define everything, you start iteratively producing code, getting feedback, learning, and that loop keeps going. Now, for some projects where there's a very specific deliverable on a date to a budget, it's hard to do agile because Agile, as you learn, it's harder to predict those dates. So people will use waterfall occasionally. If you imagine you're going to build a skyscraper, the big. We both live in New York, there's a. The big skyscraper Manhattan that is waving. Well, that was a waterfall. They tried to define everything, but you can't. And you sort of have to, right? Because you can't say, I'm going to decide, we'll figure it out.
- [15:42]
A
50 floors up, right?
- [15:43]
B
Yeah, exactly. Where the ducks are, where the. That all has to be defined. What you do in the foundation depends on what, how high a building, what that's going to be constructed. So buildings, yes, or waterfall. But many other things don't need to
- [15:54]
A
be as an aside here, because this is actually something that's been on my mind a lot recently. It's been interesting talking to my colleagues at IFP on the infrastructure team about what better planning looks like there, because it's interesting in some of those domains, what you'd actually like to see to build products faster and more efficiently is more of the upfront planning. But that's as you say, in places where you get one chance to dig or one chance to build some physical constraints. And everything is dependent on the initial constraints and often some of the budget bloat. In those cases, things like the Boston Big Dig are a result of not understanding that you're not going to get to go back and iterate because of the physical constraints of the process. Whereas here in software you can write new code, right? You are allowed to learn from your, from your mistakes.
- [16:42]
B
You're not punished, no 100%. And it'd be interesting just as a side to your analogy, like the Big Dig which Governor Shapiro did in Pennsylvania, remember when the. There was the part of the highway to 95 and how fast they fixed that. And I think they went in a, you know, we're stretching it here because it's not software but it was a more agile. How do we get it fixed correctly, quickly and that kind of thing. Maybe there is some application in certain instances.
- [17:09]
A
Totally. I think it's more, I think it's more complicated than my simple gloss. I've just been interested recently in domains where high speed rail in California there was not a lot of planning inside the California Department of Transportation and they didn't have the technical capacity internally. And so as a result you had to go back after you've started finding your contracts and you realize things you didn't know about the project. There's a lot more to do there than I think I'm qualified to untease.
- [17:30]
B
Yeah, me too. I'll give you, your listeners, a very specific example. So there was a. And I'm not going to call out individuals but there was a very well intentioned Democratic senator who cares a lot about the unhoused population, particularly unhoused applicants for fafsa. So unhoused being the homeless. The term of ARC for homeless, if your listeners aren't familiar. So in the bill they said, hey, for an unhoused person there's different things they have to explain because they don't have a home to list. And so you have to give them more scaffolding so it's easier for them to complete 100% makes sense, right? The problem was the way it got defined. The team that was executing had to put this interstitial direction that was very specific to a very, very small like a percent of a percent of the FAFSA applicants were unhoused. And I'm not saying they're not important, they're critically important, but for that population they put this language, but everybody had to see that language. And if you didn't know, if you weren't unhoused, it was very like wait, what are they trying to say? What do I do? Do I not list my address? Like. And so it was like well intentioned, they executed but it was, we quickly saw when we got in front of students, so many students got confused by these pages. So good intention but. And they couldn't just take it out because it was the way the statutory was written. Similarly we, we had for a Republican senator who was actually very active in writing the bills. I was with her in front of a student actually filling it when we were doing beta testing and more on that later. And, and she was a very. Not a fan of us for the record. She actually thought we were. When we first came, she went out to the press and said, this was a College Board. There are people from College Board that we were going to further torpedo the FAFSA to help College Board somehow, like that we were there to be sabotage.
- [19:26]
A
That's a strange allegation.
- [19:29]
B
Yeah, yeah, yeah. But anyway, but so she's huge critic of us. And it was interesting because there was a light that went off which was front of a user and there was some confusable in the software. And she turned to me and credit to her, she said, oh, my God, this is something I put in, not knowing that the user would. This would be confusable to the user.
- [19:48]
A
So do you remember what that was?
- [19:49]
B
It was some language that was very specific about an instance that a student had to answer. But the way it was presented, which we had to present it, the exact language was quite confusing. So you could have got that information in a much less confusing way. And then she immediately saw it because she saw the student struggling with it and she's like, ah, I didn't think it could be interpreted that. I wish I could remember this specific. But it was a very obviously meaningful moment. And again, I think the legislators are trying to, you know, their view is, hey, we can't leave this up to career people to define all these things. But then they've handcuffed them. The second big issue is what's the vendor? Who's producing the software? So as I said, this was the biggest software project in the history of Department of ed. They had limited deep technical experience in the department or in fsa.
- [20:41]
A
When you say limited technical expertise, put some numbers on that for me. How many engineers were there at fsa?
- [20:49]
B
They had four primary software vendors. GDIT was the biggest. They had a very large contract with Accenture and then two smaller tech firms. And my father worked at IBM back in the day. And you know, the idea. You never get fired for hiring IBM. I think probably government. You rarely get fired for hiring Accenture gdit because they're big, you know, monoliths that the government uses a lot.
- [21:12]
A
Beltway bandits is the. Is the term, you know, maybe. Maybe the critical term that people use to describe these big consultant firms.
- [21:19]
B
Right. But I would say, like, I don't know how good these firms are technically, but GDIT had built the old FAFSA software and it was built in archaic software languages that were no longer being used.
- [21:34]
A
Cobalt.
- [21:35]
B
Right, Cobalt. And it built on mainframes back in the day. And unfortunately, a lot of the people that we were getting were the same people that had built the system and frankly didn't know Python, didn't know modern software techniques. So they were ill equipped for the job. But basically there weren't what was clearly needed. And I think what they actually, Aaron Lemonstrausse, who's here now, has successfully built is a 15, 20 senior technical people who understand the architecture, understand the software and can then effectively keep the vendors accountable. And the problem was there was no ability to sort of check or determine the veracity of what the vendor said as far as status, the quality of code, all the things you'd want to do. And so they had these four large vendors and even one of the challenges is the vendors did not work well together. So listeners probably hear about this, but like there's no perfect solution. Sometimes government, they give one big contractor, but the problem of one big contractor is then you're sort of hostage to that contractor in the future because they know everything, they control and you can't really do much.
- [22:51]
A
The argument for giving it to one contractor is, as I've heard many times, the expression you have one neck to squeeze when things go wrong. You don't have to wrangle, you know, to herd cats. You say Accenture or whoever it is, we're holding you to account. There's no way to diffuse responsibility. You're responsible for delivering this and we can try and hold you the single defender to account.
- [23:10]
B
That is the argument because otherwise you get, you know, five people pointing in 12 different directions of who to blame. I think on that model, by the way, there was such ton of people that were like our guides when we took this because we none of, we all had software experience, we all had education experience, but none of us had government experience. And so, you know, while Jen Polka was like, she was our guru, her book was our bible. But also Danny Warfell from the irs, he was willing to meet with us and advise and he was fantastic. His take on this, by the way, is it's costly, but if you're going to pick a single vendor so you have a single ringable neck, as you say, he would keep a second company sort of on the side, but known and they pay to keep them up to speed so that the threat of if you don't do this better, I'm going to move off you was credible
- [24:04]
A
and you've already prepared the ground to do that.
- [24:06]
B
You're constantly paying them to stay up. Yeah. So there's a cost to that because you're constantly paying a backup to be ready, you know, to come in you have a backup quarterback who's always doing reps and stuff and you're paying them something not insignificant. But then you can say, you know, do better, I can bench you or I can fire you. And I think that if I was going to do one, I would do that. But then the other model is hey. And there was a recommendation to government a while ago. It for a short time became the thing where you have four or five vendors and they're all involved and so you can sort of move the work. If one's not performing, you move it over. The problem with that is, well, one is procurement and the government is its own issue and I don't know enough other than it was. There's archaic rules and it becomes political and it was a mess. So very frustrating to work with procurement. But the other issue is if you're going to have four or five vendors, you need someone who gets it all to make them work together. You need systems that, that allow them to work together. So here's a very specific example for your listeners. When you release software, and you may know this, the release process is a, is an evolved process to make sure you do it correctly. And so it's going to perform the way you want. And you have release notes, you have all these steps. Each of the four vendors, when they wanted to release code, each of the four vendors had to do something, but they had to do it independent. There was no, they weren't on the same systems, etc. And they were actually either emailing or texting things that shouldn't be in a system.
- [25:30]
A
Because the different, just to be clear, the different parts that they were building were things like one of the vendors does like the security verification and one of them does the plugin to get the IRS data and one of them does the front end that I. Yeah,
- [25:42]
B
the analysis to determine what their the score is so they can generate an ICE or another is to generate the ICERs. You got it. It's all those things.
- [25:50]
A
So all those have to. Just to make sure I've got it clear, all those systems have to be like fundamentally interlinked. They're all part of the package that I, the 17 year old engage with as I'm filling out and all that has to just sing on the back end.
- [26:03]
B
And they may be back in systems you never. That you don't see but are critical. And none of this is visible to the user but is essential for it to work well. Exactly. And so one, your question is a good one because you try to get a common Interface so that the ability to systems to talk to each other is easy. Everyone talks about that. Easier said than done sometimes. But you use these APIs, you know what to expect if you're one end of software. Adjusting information from another, both sides know doesn't always work that easily. But equally importantly, is this even just something as simple as communication, release and coordination? That was terrible. Like, and after I left a big victory. We got a very happy note from Aaron Lemon Strauss to the group that worked in this that they had finally all gone on Slack, which was hard. We were trying to get everyone on
- [26:54]
A
the same system, all the vendors, all
- [26:56]
B
the vendors and D.O. and FSA. Yeah, yeah, yeah. So that was like a huge breakthrough. It sounds so freaking obvious. Like if you're at the private sector you would obviously everybody working on a software project would be on the same system so you could communicate first. Imagine like, oh, I got this in Slack, I got to copy it to teams so this person can hear it. Now I got to copy it into whatever. Just it's absurd.
- [27:17]
A
It's a nightmare.
- [27:18]
B
So the vendor issue and the lack of sort of a centralized group and that that goes to the third. So again, first was some of the requirements in the bills themselves being overly determinative. Second was sort of the vendor and the vendor fragmentation and vendor quality. And then the third was just organizational within fsa. I think some of the classic unclear decision rights escalation took a long time. Very risk averse culture. I think on both parties there's a lot of politics that gets into it. And so all this compounds the same problem. And what was interesting, there was a train wreck happening. I don't think anyone realized and to, I don't know, fsa, Department of Ed, the White House, credit or explanation, not credit. I don't think anyone there knew. The vendors were like if you go back, so remember October 1, 2023 is when they're supposed to launch. And my guess is spring of 2023, the vendors are like, oh, we're doing well, everything's well. It was completely non transparent to the department of like how screwed they were. And by the way, like if I was doing a release on October 1st of that scale when I was at College Board or when I used to be a Kaplan or McGraw Hill, we would have been code complete probably in May of 2020. And then we would have been just testing, load testing, user testing, et cetera and just improving, improving, proven on the edges. So that October 1st, when you open to 17 million people works and they were still building like core functionality very late, even past October it turns out. And so what happened is I don't think the department knew how screwed they were. And so they were somewhat surprised as it got closer of like how bad it was. And even then they didn't know how bad it was. And they, and so really well intentioned people in the department of the White House were set up to be. If you don't have the information, you're reactive in the process. We can talk about later. They really pissed off the community higher ed, the CBOs involved, the high schools, the general public because they didn't, they couldn't adequately communicate the situation and so they kept putting stuff out that was just ended up being wrong.
- [29:27]
A
I don't want to make this a, a blame exercise here and I'm more interested I think in your perspective on the, the cleanup. But I am just for my own personal education, I'm curious to kind of understand how something like this happens because as you mentioned earlier, the healthcare.gov debacle and then that successful cleanup happened, you know, two administrations prior in the Obama years, I've always understood that as the beginning of a big sea change in American policy around how you build tools and service delivery. It was such a kind of catastrophic debacle and then it got cleaned up and people were able to apply for their, their healthcare insurance. I guess one question is why didn't those lessons percolate such that it didn't happen in a similar way? Round two? Like what was it about, as you understand it, that meant that those kind of really big painful political lessons weren't learned by the next Democratic administration?
- [30:18]
B
Yeah, I mean I, I don't think they were absorbed by either party fully. And I think it's complex. So developing good software particularly that has the complexity of FAFSA or other software that many of the government does is not easy, even in the private sector. So just to be clear, there's a lot of private well run companies that botch software launching and I've had a few in my career. So not to excuse it but, but so it's a hard thing. And then the lessons like Jen Polka was one of the, as I said, was like one who was lived through that and really has dedicated her career to trying to define how to improve this. But it's not always like an easy prescription or easy to follow. So to our earlier discussion about if you're going to build software, what do you do? Well, yeah, in an ideal world because the other alternative by the way is don't use Vendors just build it yourself. And some have tried that in government. That's really hard because can you attract, you're competing then to get the best software talent. You're then competing with Google and Facebook and private enterprise meta, I guess that can pay a lot of money and it's quite hard to compete for those. And if you're not getting those, frankly, unless it's someone who's just so civic minded, which there are those and there's brilliant people who are civic minded, Jen being one of them who could do better in the private sector but see the value of what they do. They'll make less, but they want to have impact.
- [31:48]
A
But by and large you have trouble attracting talent.
- [31:51]
B
By and large you're going to get a lower quality of person who's going to see this more as just going through the motions. And so a, it's very hard to build it internally and support it long term. So the hope is you can find that smaller slice of people with very strong technical that they didn't have. So that's part of the issue. I don't know all the details, but I think part of what came out of the health care issues was this idea of trying to use multiple vendors. Like that was one of the recommendations. So I think in some ways the recommendation was yeah, use number of vendors, but not that simple. Like if you ask someone to say yeah, use multiple vendors, but you need really clear leadership, expert leadership. Although what's interesting is you can potentially use a third party vendor like an Accenture or someone to be. If you really don't have the technical expertise and you want a single riggable neck. There is a world where you say, okay Accenture, you're going to be over all these vendors and it's your job to make sure it delivers. So that if what happened in the fall of 2023, you could have gone to 1org and say you really screwed it up, you're accountable. But they didn't have either of those in this situation.
- [33:02]
A
Right? I mean it's my understanding and correct me if I'm wrong here, but there were no product professionals at FAFSA in the fall of 2023. There was nobody there whose job it was. You're slowly shaking your head.
- [33:14]
B
Yeah, I mean I don't there, I mean like there were product people, there were engineers, were they the right product people? Like I would argue there were people that were product experts in the old FAFSA system. So they understood work workarounds and very minutiae of. Because there were Some things that the old software had issues with and oh, this is this, you know, and they knew that stuff inside out. Are those the people I would put on developing, rethinking the software and developing something modern? No, because they were so seeped in there, just like the Gdit engineers that were, you know, were Cobalt engineers and sort of knew how to, how to code something in 1980. These people were not, did not know, were not the type of people I'd want to develop an innovative product in 2023. That was part of it, I think what happened. So no question they didn't have the right people on staff either in fsa, in the department. And then it became a crisis. And then once it became a crisis, the next thing was, was what happens in crisis was a lot of people, again, well intentioned people smarter than us in policy and many other things, or at least smarter than me, jumped in to try to fix it. But they also were not software people, they were not operational people. You know, they were, they had very different set of skills but they were trying to save a really bad situation. And so they didn't know what they didn't know and it. And I think they made some improvements but they couldn't figure out how to hold the vendors accountable. They didn't know enough about software. So what was interesting is when they finally came around to bringing, asking me to join and letting me bring my team in was a big part of my job was trying to keep some of these well intentioned people, but who really didn't understand software, sort of away from the software to give the rest of the team the opportunity to do work. And we were lucky. There was U.S. digital Service people which were, these are civic, like a civic Peace Corps. These are very successful technically, whether engineers, product designers, data scientists who come from the private sector and come to the government and they do two or three year stints helping veterans get benefits, you name it. Very effective. Jen Polk again, one of the origins of that. But there was about 200 of them in the government before this current administration and we had about 15 were working on FAFSA. But they were sort of frustrated because they didn't have the room to do their work. So big part of what I was doing is just, and this is not unusual, sort of giving coverage. So Jeff Olson, my CTO and that team could actually get the technical work done.
- [36:13]
A
Can you just explain, I didn't plan on asking this, but you're making me curious. What did that mean to give those folks the COVID I mean you had Politicals. The White House suddenly really hyper engaged in this because it was a, you know, a front page news story. What did you have to do to give these people cover?
- [36:30]
B
It was a lot of selling. I think I was lucky because I was there. You know, it was an intergovernmental party partnership agreement, which is an agreement where ipa. Yeah, yeah, I'd like to be a better. But where College Board was still paying me and let me to work for the government. And I love my job at College Board. I love what college does. So I was sort of of the mindset of like you brought me in to fix this and if you don't like it, you know, fire me. I'm happy to go back to my job. Like, this is not, I'm not, I'm not enjoying this work. This was brutal amount of hours and stress.
- [37:04]
A
You had your own other vendor lurking in the background in the form of, you know, we could. I can walk at any time.
- [37:10]
B
Yeah. And, and so I was a bit of an asshole. I don't know if we can curse in this pocket but, but not. I'm not proud of some of the
- [37:15]
A
things we can say asshole.
- [37:17]
B
I think on this podcast. Okay. So I, you know I had that kind of like. And frankly I brought a lot of credibility with higher ed, with K12, with CBOs. They're very essential to fast. And we could talk about it, but.
- [37:30]
A
So will you just define CBO's for me real quick? I'm realizing I don't know what that means.
- [37:33]
B
Yeah, you community based organizations, these are a lot of nonprofits that really are part of the ecosystem that helps families and students complete the fafsa. They sort of ride shotgun. They go to schools, they go to different communities to help them complete it and are really so essential to how it works. NCAT is the probably biggest, most critical and we should come back to that because part of the success we had was thanks to all those organizations. And when I got there, there's such a fractured relationship. We ended up meeting weekly call with staffers from the House and Senate who the top eight top four Democrat, top four Republican who work on education in the House and Senate. Their staffers was a group of 20. We had to present to them weekly. And it's a very. Was a very critical like very questioning us. And they're very frustrated because they wrote the bills and now the software wasn't working. And you know, I was now the face of that. So I was getting. They were unpleasant calls and again, all well intentioned, but they didn't they were pushing us in a lot of different directions. And so I had to constantly explain to them. So. And then similarly we were in a weekly White House call and then we met with the secretary. I'll tell you the great, the great example. We got there in June and we realized there was no way we could launch the next year's form on October 1st. And as I said earlier, if I was at any of my old jobs software this significance, I'd probably be code complete in May, maybe June to launch something to this size population. And they were not only not code complete, they were still fixing bugs from the prior year and there was still functionality. So we got there, we're like, there's no way we can hit October. That makes sense. But the system was all going toward a October 1st launch. And in fact, while even in there the department knew that was such a Hail Mary to make October 1st first. There is pressures not to acknowledge this
- [39:42]
A
within the department, within the department, within
- [39:45]
B
the White House, within the community. So you had these well intentioned associations of college presidents, high school associations, these community based organizations. They actually wrote a letter to the secretary and to Congress to say you have to launch on October 1st. And you know, I come in and say that letter I think went in April or May. And so I get there and I'm like, there's a phrase in software like where. And I've been on the other side of this where I'm demanding something that's unreasonable, but two moms can't produce a baby in four and a half months. So there are times where you can throw the kitchen sink as much money, whatever, and you still can't produce a quality software in a certain time.
- [40:27]
A
And you realize this after looking under the hood. You came in and take a look and then you're like, oh, that's, that's absolutely not possible.
- [40:34]
B
Yeah. I said we're, we're, this is a suicide mission. And by the way, last thing I wanted to do was work my ass off. And then all these constituents that college were serves that I know and I built credibility over my whole career like disaster.
- [40:49]
A
You got it. You had a very strong incentive professionally as well as in the role to just, you had to tell the truth. You couldn't, yeah, you couldn't lose credibility on this. You had to just be honest and be the, be the, be the messenger on this 100%.
- [41:02]
B
And if you think about all, when I talk to all the constituents, they wanted October 1, but October 1 is not a magic date six years earlier. It used to be January 1, and they moved the deadline forward maybe 10 years ago now, which made sense because then it gives more time for families to complete the form, and it gives colleges more time. So earlier is better. But it's not like if it was November 18, which is when we launch, it was. It was a disaster, like. And so many colleges, high school CBOs, do stuff to help students and families complete the form, and they plan around when it's going to be released. So they told me, hey, if it's not October 1st, we'd much rather know a firm date that you're going to hit. And it's going to be quality software, not buggy. So we can plan around that, then learning on October 1st. If there's a 70% chance you're not going to return first, but you don't tell some doctor first, that's a disaster. And so I frankly got. I couldn't convince the White House. I couldn't. The secretary. The secretary had gone in front of Congress in May, and what I was told was he was told, do not commit to October 1st, no matter what they ask you. And he's like. Because they knew they. There was a real risk. And of course, he goes in front of there, and in the moment, he says, we're going to make October 1st. Or we're, you know, this is.
- [42:20]
A
This is Secretary Cardona, Secretary of Education.
- [42:22]
B
Yeah, yeah, yeah, yeah. So Miguel Cardona, the bright lights of the.
- [42:26]
A
Of the congressional oversight hearing, just incredibly stressful.
- [42:30]
B
It's hard not to. And you want it, and it's a good thing. You want it to do good. So I get in there and I. I'm trying to convince the White House, and they're like, well, they're. They're worried they're going to get crushed because they have this letter from all these constituents, blah, blah. So I literally start calling all the people I know and say, look, we're not gonna be able to do this. We can pretend and try and miss, but it's gonna be very similar to last year. So the first year, they didn't really open the software. They claim it was December. It was a little bit December, but it's into January. But that was only the part of the software where the student family could enter their information. The production of these ICers that we talked about, which go to colleges, that functionality didn't open until many months after January, in 2024, for that first cycle. And when they produce the icers, they were like, oh, shit, there's an error in the submission process that is generating faulty ICers. So literally millions of students had to go back in and redo this. And now we're talking spring of 24. So I basically. We knew that if we tried to go for October 1st at best, we could only do that front end again where people could submit. And there was a chance that what they submitted we didn't have to reach. So we were like, this is disaster. But it took so much gamesmanship and political capital and triangulating to get everybody to accept that, and we eventually did. So when you asked, like, what was my role was that it was sort of trying to. Not just in the government, but with the community that cared, the whole ecosystem trying to help them be realistic. Because all these people were aligned, like, everybody. No One was anti FAFSA. Everybody wants FAFSA to work. Republicans, Democrats, higher ed, CBOs, high schools,
- [44:11]
A
even people who are skeptical, who I know who are skeptical of, you know, the amount that we give in student federal aid or the political economy is there. Nobody wants us to make a promise to the American people. And then that promise not to be kept. You know, like, it's hard to find anybody who would say it's, this was a good state of affairs.
- [44:27]
B
So that was a big piece of being that buffer to this. And then there were just like, I'll give another example. There was a situation where. And your software, people like this. So we ended up doing beta. And this is a very important part of the story. So whenever you're doing software, you want to. Before you open something to 17 million, if you can, you would want to test this with as many people as possible so that you can find bugs, you can fix it so that it works. And so we had this concept very early. We want a beta test. And there's a rule of thumb in software that if there is a bug, like an issue with your software, if it's in a frequency of 1 in 10 users, meaning for every 10 users one hits that bug, you usually will be able to diagnose it and fix it effectively if you have 100 users. So you want 10 people to experience that bug, and then you can probably figure out what it is and fix it. So a 1 in 10 bug is a disaster. But we started our first beta with a few hundred students so that we could say, hey, if there's that frequent a bug, luckily there was. We would find and fix it. But we found little things that we did fix. Usability issues. Then the second beta, two weeks later, was thousands of students. So now we're Finding one in a hundred issues. And then our last beta was tens of thousands of students, so we would find one in a thousand. So we weren't going to find one in a million or even one in 100,000, which, when you have 17 million people, there's still going to be bugs. But that six, seven weeks of beta testing was so essential. And we went out to schools, we watched them complete with the help of CBOs, with the higher ed, with K12. We all participate in seeing them complete the form. And we discovered so much. It was back to your product point. I took one of the FSA product people that had been there 20 years. It was her first time she'd ever been in front of a user, which in 20 years. It was probably like watching a user actually interface with software, which was very upsetting. So we're building the software, it's getting close. It's a. It's a week before we release. As anyone who knows the software, whenever you do anything to the code, there's a risk that you create some other issue. And so there's something called regression testing, which is supposed to help reduce that risk because you test the software in a million ways, but it's not foolproof. And so I'm very conservative, not very conservative, but I like to err conservatively. So if I have a system that's working, I don't want to mess around with the code a week, 10 days, two weeks, three weeks before it's going to launch because there's just too much. We introduce too much risk. And so there was a small population and I can't remember they could complete the form, but it was kind of. They had to do something above and be. It was not a great user experience for some population, but it was less than a percent of users. It wasn't the unhoused, but it was another group. And I can't remember right now, it actually may have been incarcerated students, but it was some group that the steps were complex. So we were like, we're going to defer because we're about to launch 17 million people. Last thing we want to do is try to fix this, even if we can, and introduce something that affects all 17 million. And so I got a call from someone very senior in the government who said, you have to fix this. And I said, no, it's just too risky. No one in their right mind who knows software would ever play this game, but we'll fix it later. Let's get it out there and then we can fix it. We can be Very confident when we release the fix in six weeks or whatever. And we went back and forth, and finally I said, look, my CTO and I, Jeff and I, we say, this is a disaster. It's a mistake. It's not gray, it's black and white. This is a big mistake. But if you want us to do it, all you have to do is send me an email saying that despite our recommendation and hard confidence, you know more about software. And you're telling us that we should do this. And if something goes wrong because of this, and of course, then finally they back down. But, like, there's a lot more of that shit I had to do than I wish I had.
- [48:22]
A
I'm guessing if I ask you for the name of that very senior political official, I'm not going to get it. But what's the larger lesson there? Is it, like you can. Politicals don't want to put things in writing. They don't want the paper trail. What did you take away from that?
- [48:37]
B
I mean, I think both parties face such pressure now that it's not very conducive. So I'd say one is listen to your experts. You're bringing your experts for a reason, and you got to listen to their. And this person I won't name, but again, is incredibly. Did incredible great things. Important person who I'd endorse in a minute. But in this one area, they were out of their depth. And I don't think they would have done that if not for the. What had happened the previous year and all the heat they'd gotten. I would say the thing that's depressing is just how political all this is. And so let me just give another anecdote on this. So we talked. We're talking to the aides, the staffers for the House and Senate, and we. I had to pitch to them Beta. We had to pitch to them. Why Beta makes sense. Sense. And there's a lot of pushback at first. Oh, it doesn't make sense. You should just launch. Blah, blah. And so finally we laid it out in such a convincing way. And the staffer I mentioned earlier, who. For a prominent Republican senator who's still there, who helped really write. She had been close to Lamar Alexander, so had really written a lot of the stuff and I think really cared and was really, really, really good. Who. The one who would realize that some of the things she requested, acquired, created confusables later. But she loved Beta when she finally understood and she said, you know, Jeremy, Jeff, can you help us figure out a Way to a requirement in any government software that they do a beta launch before they launch fully like that level. And we were like, we gotta meet as like this was the person who told us early we should quit because we're from College Board and we're trying to undermine government.
- [50:19]
A
You won her over.
- [50:21]
B
We were like so happy. And we rented an apartment in D.C. for the six months. And so we went to the apartment and we were having dinner before we did more work and we're really happy. And then her boss, the Senator, tweets out, you know, again, failure of the Biden administration there. This beta thing is just an attempt to confuse the public. And you know, like totally had weaponized this in a bad way. Despite this, his main staffer Ned like loving it so much that she wanted her help. So it's like stuff like that that your just head fucking explodes.
- [50:54]
A
That is a nightmare. That's an awful one. On the politics of this. I'm curious. This is not, not exactly a political question, but every administration has priorities. Biden Admin. When it came to student loans, student education, there was a big other political priority which was student loan forgiveness. Particularly this was during COVID And so a lot of loans were suspended. And over the course of 2023 and especially 24 when this rollout got botched, there were a lot of allegations from Republicans in the House and Senate that it was that focus that had led to Department of Education politicals taking their eye off the ball and not being good doers of fafsa. I am curious how much credence you'd give to that political prioritization question.
- [51:38]
B
It's an interesting question. I think it goes deeper. Santi, like. Like what you think the role of government is. And you know, I think, not to oversimplify, but if your view is you're maximalist and you want government to really help people in, you know, citizens of the United States, like succeed, including get into, you know, experience quality education, get a degree and with an affordable amount of debt that they can repay, have a system that makes sense in how they repay it. I think the Democrats are more ambitious on the things they think government can do. I'm not breaking news here.
- [52:16]
A
And I think if we're going to make that the headline of the episode,
- [52:20]
B
and if you're in the Republican Party and if you're very specifically ones that are very minimalistic of what the government should do, you have a lot less ambition. And so I think there's something fair to say, which is when you talk to the lifers at the Department of Ed. I think they are more inspired by on average what the Democrats want to do than what the Republicans want to do. But they sometimes have more success in Republican administrations because Republicans have many fewer priorities. They're trying to do many fewer things because they don't believe in the same way of the way government can help individuals. And so to that extent, I think there's some veracity that the Biden administration just tried to do too many things at once in education. Now, unfortunately, they're living through Covid, so we could debate of where most of those things required. I'd also say what's interesting is the loan forgiveness. The current administration, they're working on a system people who had loans who hadn't repaid to sort of dock their, like automatically dock their pay to get that money back. Which question if it's constitutional, but I'm not a constitutional scholar, but they went pretty far, my understanding, and then they ended up not doing it for political reasons. So I think they're both parties play the politics game.
- [53:41]
A
It is a political minefield to do anything novel on student loans. And I think, as you say, I think the part where I definitely agree with you here is it's very easy to take your eye off the ball of some of the core functions. You know, in this case, this was a bipartisan, statutorily required thing for the fafsa, had to do. And as you say, it was incredibly big and ambitious modernization, you know, but it's also something that needed to get done.
- [54:09]
B
I want to just double click on that because, like, I don't think people understood the complexity and the magnitude of it, which happens all the time. Again, like there are projects I've been involved with where we didn't understand the magnitude until later. And like, it's easy looking back and saying it was such a big project, it should have been the priority or, you know, but it's also hard to know when, you know, you, you get a crisis like Covid. Like, I don't want to. It's hard for me to second guess that that stuff because they were also dealing with a million curveballs that Covid created and try to helping the population. That so like, easy for us to sit back here and say, oh, I wouldn't have done that. I would just focus. But like, yeah, maybe in retrospect, but in the moment, it's a very hard thing to have done.
- [54:55]
A
I ask mostly to try and get a picture of what happened for next time. Less so to kind of armchair quarterback. I have My own personal views on was avoidable and not. But I think that's neither here nor there. I'm curious, I want to change gears a little bit for your perspective on the Government accountability office, or GAO, which issued a very critical report in 2024. 2024, 2025. They had their big roundup report.
- [55:18]
B
Actually it came out, I'm trying to remember. I was still there when they were talking about it, but it may not have got 25.
- [55:27]
A
It was, it was 25 of this. The actual big report came out and I just got it up right here. And this was a report criticizing the whole botch rollout and some things that you guys did in your cleanup operation. And your colleague Aaron Lemon Strauss, who as you mentioned is still there, wrote a response to GAO like a nine or ten page report saying, no, you've got it wrong. And actually I'm paraphrasing here, but he basically says like, you don't know anything about building software and if you did understand building software, you wouldn't say this stuff. I'm curious both your reflections on that whole back and forth with GAO and just your experience of working with that oversight body while you were there for the six months.
- [56:08]
B
That response everyone should read because it's is genius and it's 100% right and Aaron is brilliant in that way. I think Jo, you know, it's kind of interesting because as I said, I was never involved in government. So this was like wide eyed Jeremy going to dc, thinking about how things work and then seeing the reality and being like, oh shit, this is how the sausage is made. So the biggest hit, I think was gao because I always had viewed him as this, is this idealistic, like, you know, their place that keeps everybody accountable and is really smart about, you know,
- [56:41]
A
who doesn't like accountability.
- [56:43]
B
Yeah, exactly. And then I, I see it, I'm like, holy. Like if we had followed what they suggested, it would have been a much larger disaster. And it was almost like they were, you know, flashback to 1990 and said, okay, this is how you make software. And then that was what, you know, they had programmed quad or whatever they used in 1990 practices to give a recommendation because it was such a mess and incredibly, and I don't want to say, please, that oversight isn't important. I think oversight is a critical function. But the way at least the GAO team that did this was so bad. I mean, very process heavy. It was very compliance oriented, which compliance does not lead to successful software.
- [57:29]
A
What do they ding you for on the process and compliance of what were they. What were they upset about?
- [57:33]
B
There were stuff of, like, if people had taken certain trainings, we would have avoided this. They were critical that in these moments of crises, we didn't, you know, not we, but there's part of me. But people didn't go through the right steps to document everything. But as you know, like, if you're dealing with a house on fire, you're not going to go, okay, I'm going to first go to the second floor. And it's just so detached from any reality. So I was there, and they were requiring key people that I needed to launch the software in the fall of 24. And they were having us produce old documents and review stuff and be interviewed. And I'm like, guys, we need to do this. And to the credit of the department and the white, like, they tried to help at least make sure that the critical staff we needed didn't get too distracted by the these, you know, exercises. But it just seemed like when I talked to them, it would just seem like they just didn't get it. You know, it's not people I would want to give a recommendation on how to produce good software.
- [58:35]
A
Now, is that mostly, as you see it, a lack of expertise with building software in the modern age, or are there other things about gao, about its culture, about its approach that just mean whether software. Otherwise they were going to give you bad feedback?
- [58:51]
B
It's probably mixed. And I want to be clear because I don't know, I haven't been involved in other. This is my one instance of a GAO experience, and it was limited. I did read the report and I read Aaron's response, and I did see the process.
- [59:03]
A
I'll just. I'll just say these are not unique criticisms of GAO to you and your team here.
- [59:09]
B
Yeah.
- [59:10]
A
Similar complaints from people in different domains. You're not out on an island here.
- [59:15]
B
No, no, I know. Well, so here's a question for you, like a question for all of us to ponder, which is, you know, when you're trying to evaluate something and you want it to done effectively and fairly, you often have rubrics or frameworks of how you think about how is this being done. So I do think their framework is broken. I also wonder whether they have the quality of mind of people that can understand there's a framework, but understand nuance and say, okay, yeah, you do want really good documentation. But when you're in an instance like this, that's not possible. So I'm not gonna ding them for that. But what I would have wanted to see is X or understand waterfall versus Agile and say, well get it that less documentation because of this. But then if you're doing agile, you should be doing these six things that are essential for agile success, which are different from trying to force a waterfall process or something like that.
- [60:05]
A
Right. I want to hear the kind of broad lessons you take away from this experience and how you would advise people kind of doing big software projects in government to approach them. But I guess before we get to that, curious for what you'd want to see different from oversight. What would a better jail look like? And what would in particular congressional oversight? You guys had Congress breathing down your necks after the failed rollout. What would a good version of that that actually did all the things you want out of oversight and stopped you guys from wasting money or fraud, et cetera, kept you on mission. What would a good version of that look like?
- [60:41]
B
Yeah, by the way, I don't think there's a lot of fraud involved.
- [60:43]
A
No. But putatively that's why a big part of the reason we have this whole oversight system is to stop you from taking trips to Vegas or whatever it is.
- [60:50]
B
Right. And I agree in that. And there's rules and it's just again, it's this. Yeah. I mean other people have talked more eloquently about this, I think. So one, one piece of it is, which I don't know how we do, but if we could depoliticize things, that would help. So I don't think it was bad. And just for listeners, it's not normal that on a project like this, the team reports to 20 congressional staffers once a week. That was again a reaction to the early issues and said we got to do something. And so it wasn't even bad intention and it wasn't even a bad thing for Congress to want to be closer to something that has shown had issues. But I wish it was less political and I wish the instances where they weren't playing roles, but they were, you know, and there were moments of that, like I said, like where they could really appreciate the good things and give us, you know, and ask questions. And it wasn't. It didn't have to be a stage or as theatrical like as it was. And I think over time again here, Jeff and I benefited because we didn't want careers in politics and in fact the opposite. We do it so we could be pretty honest. We didn't care about our brand to some of these people, but if we did, we would have to be even more political in what we said and how we said it and maybe we don't share as much. And so we, we had a, this whole concept of working in public and Aaron Lemonstrass really started this but we wanted to put up a blog that gave everybody all the constituents because there's so many people and as I said, there's the ecosystem so large we wanted to galvanize them. And so we said we're going to work in public, we're going to put all this stuff out there, we'll talk about what bugs there are so people can help their students or whatever work around these. And we got such resistance from that openness in many ways for good reason because there's a danger the more, the more information you give out, the more stuff that can be weaponized politically. And I wish that wasn't there because I think if you didn't have that there would be a tendency for more openness.
- [62:49]
A
I'll just interrupt there and I want to hear the rest of your answer. But I do want to push back here a little bit in part because I'm just such a skeptic that depoliticization of oversight is even possible. Like it's hard for me to imagine a world where in a two party system when there's high upside potentially to dinging the admin for the way they've done something, that either side will bind their hands there. You know what I mean?
- [63:12]
B
Yeah. And it's all degrees. Right. And I mean we may a year from now say hard to imagine the Federal Reserve being independent. But like at one point there was no question the Federal Reserve was an independent org. Right. We didn't want that political. So it's all where you put the
- [63:27]
A
dial and the question is, you know, can you wind the clock back? You know, it's great to have an independent Federal Reserve, but it will be hard to depoliticize it if Supreme Court rules a certain way. Right. It'll be hard to undo certain kinds of politicization.
- [63:41]
B
Yeah. But it's harmful. So I, I, my point is it's all degrees. So to your point, I'm living in a fake world if I think politicians aren't gonna take advantage of miscues of the other of instances from the other.
- [63:55]
A
I don't want to be too cynical
- [63:57]
B
but, but, but so I agree with that. Like that but, but how it's but there are things where they're shared aims and I do think, and so I do think the gao, I would have to believe that the GAO could be independent despite whichever administration but maybe that too is no longer. Then if the GAO becomes just a branch of the current party in power to try to drum up dirt that can be used politically, then I don't even think it's worth having a gao, because then it breaks. So unless there's some level of impartiality and independence of the gao, you know, I don't even know of the function. So that. That's part one. So you need enough of that. And then I think we need the sort of people that have systems that they use, but also have the flexibility to understand nuance and really understand the system. Because what happened, at least with fafsa, was not helpful and was actually counterproductive and potentially just destructive.
- [64:59]
A
Are there things that you observe about the statute that told fsa, go fix fafsa, that looking back, you think, okay, if you phrase that a different way or given us more room to achieve some broad outcome, instead of giving us that line by line, the initial pass would have done a better job. What could Congress have done in writing the statute to avoid some of these failures?
- [65:21]
B
Well, I. I think so. You know, in software, you develop use cases, but you don't specifically specify how it works. So a use case could have been. I want an unhoused applicant to easily understand how to fill out what they should do in this section that is complex. If you don't have a home, and you could have defined it like that, you have to put this language to everybody that explains this, you know, like very prescriptive. So there are a million examples like that where you could say what you want the software to do, but then let the team figure it out and do it most effectively and test it and iterate and put it in front of users and see how well people understand it. But not hard code, the actual how the software is going to present.
- [66:08]
A
Let me zoom out a little bit. In tackling a big project like this in general, let's say you can't change how Congress behaves, you can't change how the other oversight bodies in the federal government behave, but you're thrown in at the beginning of another one of these modernization projects somewhere else in the federal government. What's your kind of basic laundry list to avoid having to, you know, somebody else come in and do a cleanup for you afterwards?
- [66:31]
B
Some of the stuff we talked about. So I'll just go through it quickly. I'm not giving up, and you could correct me, but my understanding, I started to learn about this in real time. But look back is there was a movement in the last few decades to make bills more prescriptive because there was sort of an increasing distrust and maybe some of it was warranted of bureaucracy or lifetime people. So can you unwind that some and can you learn? I do think the people involved from Congress, the staffers at least sort of learned a little bit about how software is produced in the process would be more amenable to not overly defining. Now could you do that? Could you go back? And then you need controls that it's built in a way that meets there. But there are ways to do that, there are ways to build use cases and software has solved some of this. And so I'd like to see that. The second is I do think you need a cadre of people. If a department is doing a software project, they need a cadre of people that are technical experts and know how to manage and manage vendors and understand architecture. And that isn't always the case. And so that may be figuring out a way to attract more of these type of people. It may be ability to pay them more, whatever it may be. And USDs I think had been a really useful version of this, but unfortunately I think that, you know, their presence has been significantly reduced now.
- [68:00]
A
Definitely. Although you've got now a different, you know, in some ways a different version of USDS and Tech Force, which is new admins, I'm not going to say a rebrand, but it looks an awful lot like the old USDS in a lot of ways. Cycles of new initiatives of this sort.
- [68:16]
B
I mean, USDS had a great track record. We'll see.
- [68:18]
A
It did.
- [68:19]
B
And one thing is, and this is tricky because there's two sides to this, as there is with a lot of this, but like when you make public commitments. And so again, should they have launched, should they have thought harder, could have been some process about what a timeframe that would have made sense and how to do that. Again, they didn't have the technical people, they didn't have the vendors. I think a lot is around vendor management and choosing vendors. That whole process has to be changed and reinvented. It happens well sometimes, but it often happens too frequently.
- [68:53]
A
Two things, I mean, one is I have to imagine looking back the political leadership at the Department of Education that promised it would hit these dates that it couldn't. In retrospect, I'm sure all those people would rather have taken the heat up front for saying we're not gonna be able hit that date, we're gonna have to push and avoided the actual rollout debacle than the other way around. And so it's an interesting. You know, in retrospect, you could go back. You. You might say take your lumps from the congressional folks but get your timeline right instead of desperately trying to hit a target. You may not be able to.
- [69:29]
B
So you have so much more experience here.
- [69:32]
A
Really. No, really, it's not really.
- [69:33]
B
Well, I mean you follow this much closer than I do. What I would pause my one argument is I think today maybe this has always been but and it goes back to our political polarization and making use but it's so punitive to show any weakness or any sign. And it's why you have some presidents who say I did everything right. I've never done anything wrong which no one is infallible or I don't believe anyone's helped because anything could be potentially used. And so some people are unwilling to admit there is a culture and I think this permeates both parties where you would think logically, you'd take a hit early, then take a risk, you take a much bigger black eye later.
- [70:17]
A
But it's hard to do that in practice.
- [70:19]
B
But I don't always see that. I think like there are incentives. Like again, there was a lot of reticence. I look, I had to go pretty hard to get the acceptance that we weren't going to launch October 1st, even when it was crystal clear to me and I had a ton of evidence to show that we were likely going to be screwed. And there was almost like I'd rather take that flyer that it may work. It's small chance and if it doesn't take the heat then take known heat today even if it's smaller. And that is really problematic because you can't make that's not how anyone else
- [70:52]
A
runs your case is especially interesting there as a counterpoint to my view because you were warning the exact same things that happened last time are going to happen this year. You know, exact same kinds of problems and you still had an uphill battle to convince folks that good to wait.
- [71:07]
B
Yeah. Because it would have looked bad. And we eventually did. Understanding the situation happened very quickly. I uncovered a memo I'd written the first week I was there in June or the team had written and it basically said we should beta we can't it October 1st. Here's a plan. We need to engage a community. Like all the stuff we ended up doing, we had to defer A very tough decision was to defer some functionality that hired really wanted to do mass uploads of ICers that was never launched in the last system. And we had to defer till after we launched because if we tried to do that we would have even be later which again would say no brainer. But I can't tell you how much capital I had to use and how much time I had to spend to convince everybody that this was the right decision. And again I had to go to higher ed, get them to know this is terrible medicine but why we had to do it and then get my back to tell the department, hey you guys, you got to do it. What Jeremy said is right even though it hurts us, you know. So I literally had to get. And luckily I had those connections or we had those connections. But it was amazing the resistance of not saying we aren't going to launch October 1st, knowing that the other party is going to use it as a huge political weapon. But it would have been much better politically it turned out much better than let's say we tried October 1st and then we didn't launch and in October we're taking a beating that we. But the Democrats would take a beating like that would have been much worse. But to get them to. They'd almost rather roll the dice kind of mentality which is a bit sad to me.
- [72:37]
A
It is. It's fascinating though. I mean just as a kind of human thing. Last question for you. You spent about six months in. You were kind of in some ways the epitome of this, of one model of going into the federal government. You come in on an ipa, you were there six months, you were never paid by the government. Your salary was paid by your private sector employer and you were out, you did a thing, chop chop, you're done. What are the strengths of that model and the weaknesses of that model versus the other thing that we've talked about of we just need a lot more long term technical capacity in the federal government. Like if you were trying to get the balance right tomorrow, how would you try and apportion flyers in from the private sector versus spending more money to try and get good talent in for five years or 10 years?
- [73:23]
B
Yeah, I mean I definitely think this model is only used in emergency. Like it is much better to have the talent in the government and figuring out how to make it conducive for people. And there are a ton of people in government with these skills, even USDs or these new effort of it. They're brought in for a limited period and it's really valuable. But I wouldn't want those people running things. I'd want them and nice thing with USDs is they intentionally didn't try to run stuff because they knew they were temporary. So you really need those skills in the government. You need those people driving it. This is a measure of last resort because, like, there's so much stuff we didn't know that would have been useful and if we had been working there. And to Aaron's credit, there's a guy, Chris Cummings and Nita Sokolar. There's a lot of people that we were working with that stayed. And one, they know so much more now than we knew and they're getting so much done thanks to that knowledge and experience, and they could build on it. So I'd much prefer figuring out enough talent that can lead and then bringing in, utilizing consultants or whatever as needed, but not to drive it.
- [74:30]
A
That makes a lot of sense. Well, Jeremy, this has been a pleasure. If there's anything I can do to improve my almost 3 year old's chances of acing the SAT in 15 years, shoot me a line, please.
- [74:42]
B
Yeah, I'll send you the score. I'll send you the test in, you know, at some point.
- [74:45]
A
Yeah, if you, if you. If you've got advanced notice, that'd be great. No, but really, it's been a pleasure. Thank you for making time, Santi.
- [74:50]
B
My pleasure. Thanks for the conversation.