Loading summary
Transcript45 lines
- [00:00]
A
You're listening to TechTank, a biweekly podcast from the Brookings Institution exploring the most consequential technology issues of our time. From racial bias and algorithms to the future of work, TechTank takes big ideas and makes them accessible. Welcome to the Tech Tank podcast. I'm co host Nicole Turner Lee, the director of the center for Technology Innovation at the Brookings Institution. Headlines are currently focusing on the introduction of generative AI across various sectors. And some industries have even employed AI and machine learning algorithms for years, the financial sector being one of them. AI is being used in banking fraud detection, credit underwriting, and data analytics. And even more so, spending on technology within financial services firms is expected to grow in the coming years. Now, this doesn't come without some policy concerns. In fact, I don't know how many of you really, really follow me. I know that some of you say that you just keep an eye on me wherever I go. Well, last week I was before the House Financial Services Committee talking about AI use and financial innovation. Listen, I think there are so many promises for the efficiency in the sector when it comes to the use of artificial intelligence. But I also want us to be mindful that there are some risk as the industry is also highly regulated and responsible for decisions that have serious implications for those of us that may end up not qualifying or being rejected or having our information stolen. And the list goes on. So these discussions are challenging. And, you know, since I testified not too long ago, I figured I'd have more conversation about this with my dear friend and colleague who's also a CTI Resident Fellow, Aaron Klein. Aaron is a senior Fellow within the center on Regulation and Markets and the Miriam K. Carliner Chair in Economic Studies at the Brookings Institution. I told you, he's not too far away from me. He's an expert on financial technology, regulation, payments, macroeconomics, infrastructure, finance and policy. Listen, if you don't know who he is, I would highly recommend that you go to his page and you see all of the wonderful stuff that he's talking about, not just in AI, but in the economy overall. So I want to welcome Aaron to the platform. He's somebody that those of you here have heard him before. But, Aaron, thank you so much for being here today.
- [02:42]
B
Nicole, it's a pleasure to be back.
- [02:44]
A
I know I always enjoy talking to you for a variety of reasons. And I think today, October 1st, is a quite special day that hopefully we'll have a time to get into as well. So let's jump into this. I testified before House Financial Services Committee last week talking about AI in the financial sector. I laid out some broad categories where the technology is being used. But, Aaron, explain to us in more detail since I gave an overarching explanation of how and where we're seeing AI in your sector.
- [03:15]
B
Yeah. So, Nicole, thanks. Your testimony was fantastic. And I encourage everyone to go back and, and read it because you make some really good and nuanced points that I think a lot of folks would benefit from appreciating that AI has some really cool potentials to overcome problems in the current system and then some pitfalls to make what's already a problem even worse. And instead of reacting into one or two base groups of, you know, future fear mongering or, you know, technological transcendence, you scout in your testimony kind of the path to harness the good and hold off the bad. But to answer your question, look, we have AI and financial services. Most people know there's an artificial intelligence that's been dominating the world of credit allocation for a long time. And it's called fico. Fico, you know, is stands for Fair Isaac Corporation, the company that invented it. But by most measures or metrics of machine learning or artificial intelligence, it's been one for a while. One metric I like about AI is if the computer can do something and we're not really sure why or what it means. You know, people come to me all the time and they say, Aaron, you know, would it help my FICO credit score or hurt it if I took out a new credit card line? The answer is I don't know. And neither does FICO because it depends on 10 other things, which all depend on 10 other things. And this, this AI is pretty dumb and pretty bad. Most people have heard of fico. A lot of people know their score or think they know their score. I don't know. Nicole, do you know your fico?
- [05:01]
A
Of course. I check it all the time.
- [05:04]
B
And you know what, what is a. What is a really good, like, aspirational, you know, really good fico.
- [05:13]
A
Well, you know, Aaron, I hope you're not trying to get into my personal business, but from what I understand, it's upwards of 800, which I'm not sure if anybody is that perfect over the course. Course of lifetime. Except my mother. Aaron, I have to say, my mother is perfect. She says that she has great FICO
- [05:28]
B
score, God bless her and congratulations. But.
- [05:31]
A
Right.
- [05:32]
B
You know, in my life, I can only ever think of one other thing where 800 is the goal. It's a really weird goal when you stop and think of it, right? Shouldn't the goal be a thousand or 100 or. Right. The only other thing I can think of where 800 is, the goal is.
- [05:50]
A
Okay, what?
- [05:51]
B
SAT?
- [05:52]
A
Oh, LSAT.
- [05:54]
B
Standardized tests.
- [05:56]
A
I haven't taken one of those in such a long time. You know, they're also not required, so my kids are actually in college right now. And she didn't have to take it. She took it, but she didn't really have to submit it. Go figure. I still had to pay for it, though.
- [06:09]
B
Well, you know, there was a period where it wasn't. I think schools are bringing it back now. But for most of us of a certain generation, we remember the SAT often. It may. It may scare some people, there may be some shutters right now with post traumatic stress. But you ask yourself, why did. Why did Fico make 800 the goal? And I think the answer is they want to subconsciously appear to be more legitimate. Look, you know, you and I can have a whole separate conversation on whether the SAT is really productive, is a good standardized test, whether we should use it or not. But if you're of our era, your SAT score mattered in terms of where you went to college, just like your FICO matters in terms of whether you get credit or not.
- [06:52]
A
I want to slow you down on this for a minute. Just. Just stay with me for a second, you know, and. And folks, you can see we have a great relationship. I actually adore this colleague of mine because he's so brilliant in the way he thinks about stuff. Wait till you hear more of this episode. I'm sure you're going to actually give us some more insight, but I want to slow you down. Like. But FICO compared to AI, Aaron, don't you think that that's actually a standard metric that most people know about? Right? Like I said, you have all these resources now. You can check your FICO score. If you don't get credit, you can actually appeal it or you can dispute it. I mean, AI is looking at your social media profile you write about. Historically, AI is attached to the computer devices that people use that you're more likely to get credit if you're on a Mac versus a PC? I mean, that's the part that scares me. Who do I run to when AI basically suggests in some way or form that maybe because I'm a black woman and I look at my credit cards all the time or I use them all the time because I'm always shopping, because the AI knows that about me, that I might not be credit worthy. I mean, help me understand that. Latonya Sweeney tells us, like if your name is a certain racially ethnic sounding name that you'll be denied credit. Help me.
- [08:05]
B
So I'll tell you a couple things on that. One is if you're shopping all the time, that's actually not. That's a reasonable little flag. It may not be red, it may be a little yellow for a bank to kind of go, I don't know. But look, FICO's are three digit numbers that are based on input data from credit reporting agencies. Equifax, TransUnion, Experian are the big three. Those entities are full of errors. Look, there's an Aaron Klein in New Jersey who didn't pay his cell phone bill, who held down my FICO score for 10 years. And I was absolutely powerless to get that off my credit report because I'd lived in New Jersey too. And the credit bureaus have no legal requirement for accuracy. In fact, they have every economic incentive to be inaccurate because accurate is expensive and costly. And the math will tell you that as long as you're inaccurate in both directions, that is that Aaron Klein in New Jersey got something good on his report from me and I got something bad from him. That it all comes out in the wash. Because at the end of the day, these FICO scores are all getting aggregated and it just cares whether 1000 loans at a score of 750 perform this much better than 1000 loans at 700. The point I would make to you is there are good reasons that you lay out to be concerned about other pieces of information being used in credit underwriting. But if you block all those new pieces out and you just use the old data, you know, a lot of people are born and they turn 18 and their fico's been trashed because their parent was poor and took out a loan in their name or took out a bill, an electricity bill, a gas bill, a phone bill in their kid's name and then defaulted on that. And so FICO is full of problems. Credit reporting is full of problems. In fact, there's a really good study, and I think you cited it in your testimony from a group called Finreg Labs that looked at something called cash flow underwriting, which basically said if I take a computer and I just look at the amount of money in your bank account every day for the last two years, I have a better guess as to whether or not you're going to pay back that loan than if I look at your FICO score.
- [10:34]
A
That says to me then that in the financial sector, Aaron, it's highly regulated. People like you have sort of settled on the research and use cases as to what are good indicators for how we determine creditworthiness and economic viability. But then you have again, Aaron, I mean, you still have AI. I mean, help me, where does AI come into this play? Because what I'm thinking is maybe there's a space for it or maybe not. But we also know that banks are using it for processing back office processing, you know, fraud detection, et cetera.
- [11:12]
B
Yeah. So look, I think we ought to incorporate AI and we ought to do so wisely and carefully with guardrails and with some safety. But fraud detection is a perfect example.
- [11:25]
A
Right.
- [11:25]
B
I just came from an event this morning at the Aspen Institute looking at frauds and scams and their estimates that one in five Americans has been fraud or scammed. But sometimes when, you know, when something feels too good to be true, it usually is, but you kind of get suckered. There's a psychology of it all. You lose your mind for a minute and you go to start to send money and it's an AI. You know, you can't expect a bank to have a person looking at each of your transactions. One, that's very expensive. Two, there's some real privacy issues. But if you run something through a giant algorithm and you go, huh, Nicole, that's really odd. You've never spent sent money to Nigeria or Myanmar before. Why are you. And by the way, this, this account that you're sending money to has had three complaints about fraud. Maybe you should think twice before you buy this person's concert tickets that they've offered to you in Craigslist, an AI to detect fraud and scams, the company known as Block. A lot of us call Square from those devices. They're doing some interesting stuff. Zelle Plaid. There are all these companies in the back office and, and they're doing a great job, number one. Number two, financial services is more than just banks and borrowing. It's also insurance. And this gets into a tricky question, which is that if you think about it, there's a very core tension in all of our world. One is most people believe in some form of risk based pricing. If you're a riskier credit risk, you should pay a higher interest rate. If you're a riskier driver, you should pay more in car insurance. Most of us also believe in non discrimination. We shouldn't assume that because of an immutable characteristic of a person that that means like they're going to behave in a certain way or group. But people in our legal and regulatory structure are schizophrenic in this. And let me give you an example, Nicole. If I said to you women are better credit risks than men in the aggregate, Men default more. We make a lot of dumb choices. Thus women should all get a cheaper interest rate. You might be excited or not, but it's blatantly illegal. Congress has put in a law and it said you cannot discriminate based on gender. And I think most of us would say that's a good thing. Now I have two teenage daughters. A lot of people have teenage sons out there when they go to get their driver's license and you have to buy car insurance for them. We know that teenage boys, and you know, forgive me because I used to be one. So I feel like I can say this. They make a lot of dumb, risky choices more likely than teenage girls, especially behind a wheel. This has been proven. Teenage boys are more likely to get in a car accident. When you're 16, you're two identical people. You could have identical twins, same everything. The boy will pay more in car insurance. Why? Because of their gender. That's totally legal. Why? In large part because states regulate insurance. In fact, some states have created gender neutral insurance and often they get a political backlash from the women who say, why'd my rate just go up? And so my point here being is that I've picked two metrics. I picked the same metric gender, I picked two different financial products alone and car insurance. And I've told you that in one aspect society has said discrimination on this characteristic is illegal. And on the other, I've said it's totally not just legal, it's the norm. And by the way, it's real money. And I think that gives a lot of people a pause because they ask themselves, okay, how do we want to draw these lines between protected non discrimination and risk based pricing? And what you're going to find with AI is AI is going to uncover lots of weird things correlated with risk. Some of which I think we would say that's fair. If you're on your, if you're texting while you're driving and the AI knows that you like to text and drive, you should pay more in car insurance. I think most people would say that's a risky behavior, that's on you, right? But now the AI goes and says, well, your, your glasses prescription is a little higher than someone else's. Should you pay more in AI? Well, if you're both wearing your glasses, but what if the Computer says that people with worse vision or, you know, we've, we've noticed that your reaction time's a little slower because you got cut from a few sports teams. I think we get a little bit pause. And so this is one where we don't understand the technology is going to start to find these connections, and it's going to find them faster than we can sit around in a room and say, okay, not okay.
- [16:42]
A
I mean, I love the way that you point that out, because a lot of my work, as you know, is sort of identifying where this discrimination might occur, particularly for protected classes. Right. Because the same correlations that we're making can show up if you happen to be a person of color or you're low income or you've got marginal credit. That could determine the extent to which you'll be a riskier investment, not just to get a bank account, but to get that car insurance that you're speaking about. Now, Congress, you know, in the last administration, there was a lot of talk among several members to sort of come up with more accountability structure and legislative directives that help to avert some of the discrimination that you're speaking about. This, this Congress, under the Trump administration is basically suggesting, going to sort of move through with innovation and put the foot on the gas as opposed to thinking about some of the things that you're suggesting. And one interesting thing that's come out that came up in my hearing as well is this use of regulatory sandboxes. So the idea that, again, under the current AI action plan of this administration, just to sort of preface that for our listeners who may not be following D.C. politics quite closely, it's allowing for companies to, to sort of work on solutions, perhaps to some of the problems you're talking about in terms of bias, mitigation, products going to market quicker, new products that we've not even thought of. But to do it in what's called a regulatory sandbox, where there's some protection. Now, under this administration, the protection is do whatever you want because we're really not going to give you any oversight on this. Right. That's a simple way of putting it. I love to hear your take on regulatory sandboxes, particularly at this time, and whether or not that's a really good idea to sort of do the work. You're talking about really thinking through the role that I could potentially play as we look at this myriad of proxies that could show up.
- [18:40]
B
Yeah. So, you know, let's talk about this for a little bit, because I appreciated your testimony was very nuanced on sandboxes.
- [18:48]
A
And you saw I quoted you, Aaron Oral. I said your name. I just want to let you know,
- [18:54]
B
I always, I always appreciate that because, you know, sandboxes were tried and were pretty successful in England, but in the British context there were consequences and there was government oversight there. And they've also been tried in several states. Arizona, notably state based sandboxes are a little difficult because a lot of times in financial services we're talking about size and scale, you know, and products. It's also hard sometimes in states to, you know, make sure that your product is just being used by residents of your state or driving into, etc. You know, when my kids were little, I always thought of the sandbox as the place where I dropped them off to play unsupervised while I talked with their parents. And every experience of mine ended up in a sandbox. And it started out great, the kids were having a good time, then I kind of ignored them. And it either ended with them throwing sand at each other or one kid sitting in a wet puddle of sand. And I've always kind of thought that there's an interesting way to describe them as greenhouses. When I think of a greenhouse, I think of a controlled environment with maximum transparency and sunlight so everyone can look in and see what's growing in a special controlled environment. And then if it's successful, perhaps you can transplant it outside into the wilderness and let it take off. And so I tend to think that the right answer is to try some of these things in more controlled environments. You know, we do that with medicine and other scientific examples. People, you know, have to be informed
- [20:35]
A
when they go in.
- [20:36]
B
This is a new product. We're not quite sure how this is going to work. Do you consent to letting these other, you know, other things look in? And we're seeing some of that. Some car insurances will let you opt in. If you give them access to your phone while you drive, and it can tell how fast you're driving and whether you're texting, then you'll get a lower rate. That seems to be okay. Now there's a reasonable concern in which some of this is a bit of a zero sum, right? Particularly in the world of insurance, but also somewhat in the world of lending that if one group's getting a better deal, that means I'm getting worse. And so if I'm not opted in, then I'm doing worse. Well, maybe. But you know, this gets back to that core tension, which is how much do you want to be risk based, which is, you know, people who drive faster are riskier drivers who should pay more in car insurance versus, you know, how much do you just want to be immutable? I have a right to privacy and I don't want my car insurance to be able to look at my phone all the time. I get that. The same with, with, with banks. You know, maybe I don't want this, you know, my, this auto dealer to look and see how much money I have in my bank account every day because that's going to change how they negotiate the car price when I go to the loan. So there's a tension in here, but I tend to think that responsibly moving forward is a better solution than either a wild west where you just drop off the kids in the sandbox and come, come back in 20 minutes, or a reflective. No, no, no. There are too many downsides. We can't move forward, which I think, unfortunately the last administration got caught in a little bit too much of analysis paralysis. And technology is moving faster. Moore's Law is pretty, pretty brutal. And, you know, the current status quo is pretty bad. One of my frustrations with a lot of my friends on there's a simultaneous argument that the status quo is really problematic and we should be really afraid about adopting change. I just see a core tension in those two positions.
- [22:48]
A
And that makes me wonder then what's the better solution, right? If it's not a regulatory sandbox. And I don't want to give away what you told me the other day as we were passing each other, but is there another solution that we should be looking at that makes this process much more transparent?
- [23:05]
B
So I kind of like this greenhouse idea where folks.
- [23:07]
A
Bingo. You said it. You said it. Because I think it's brilliant. Go ahead.
- [23:12]
B
Yeah, I mean, folks can go in, they can set up a controlled environment, they can let the regulators look in, they can let the consumer groups look in, because there are groups out here that are looking for consumer abuse. There can be an agreement that this is a trial. If something bad happens. There's some safe harbors here. Just like what people do with drugs when you've tested something enough that you think it's going to work and there's a reasonable likelihood of success. You have trials and, you know, we're thrilled when we find a good trial. And there's a brand new drug that, that helps millions of Americans control a chronic disease. But sometimes the drug trial doesn't work and some people have some pretty nasty side effects. And that's Horrible. But that's the cause. That's what happens when you, you know, move forward and have advances. So I like this idea of allowing new technology we tried in controlled and transparent environments. What I fear is the current crowd is, you know, with, especially with the defunding of the Consumer Financial Protection Bureau, they're going to fire every enforcement attorney and not bring any new case. If you don't, if you take the cops off the beat, you know, yes, you know, crime reports will go down. That doesn't mean crime has. I'm worried that this current crowd is too focused on innovation without protection, particularly if the word crypto is involved. But I was a little frustrated that the last crowd was too concerned with making sure that everything would work and wouldn't try anything new. And that boxed in the status quo. And let me tell you, I think FICO is, is, is a horrible system that we're all stuck on that is, has rampant discrimination and harms millions of people and doesn't protect us financially. You know, I don't have to go through with you the subprime mortgage crisis where they said everybody's credit score was, was good enough to get mortgages that, you know, common sense would tell you these people couldn't afford and we're going to blow up. So I don't think the current FICO system is that great and worth protecting. You know, I'm not going to give away the secret of the movie Fight Club, but for those of us who've seen it, and I think we're long enough in spoilers, the very end plot was to blow up all credit reporting. And you ask yourself, credit reporting is a very old technology and it is not clear that whether or not you paid a bill on time 12 years ago has anything to do with your likelihood of defaulting going forward. And it is clear that knowing how much money you have in your bank account does.
- [25:49]
A
You know, this is so interesting to me because when you start to put in some of this technological advance that is accelerating efficiency to your point, and this is sort of what I argue in my research, if you put it on top of fractured systems or systems that have not necessarily been updated in terms of policy or program, essentially AI is just going to augment that dysfunction. And so I really appreciate what you're saying. It's like, you know, going to, you know, dinner for a holiday and you still have that uncle there and you just know that they're not going to change their mind on the things that they believe or think about. So you might as well, just get ready that you're going to have that conversation again every holiday. Right. Because you've not necessarily solved the problem. You might have put out a few more entries and more appetizers. But I love how you're really talking about this. We really have to have these fundamental conversations around how we posit credit and lending and, you know, eligibility and systems that are pretty much archaic and outdated and how we integrate AI into this. And that's why I fear, too, Aaron, that financial services companies are sort of leveraging AI but not necessarily fixing things that need to be evaluated and reevaluated for the efficacy. You know what I mean?
- [27:05]
B
Oh, look, look. I wish I was wearing my sweater that would make me look like the drunk uncle character on the snl. But the reality of the situation, and one of the problems is that as the temperature in Washington gets hotter and hotter and you reference the shutdown that we're having right now, and not just the shutdown, but I mean, you know, you talk about the president using deep AI fakes on the official, his official channel that had clear racist overtones and put words quite literally in the mouth of the minority leader. And the boiling in symmetry, political anger that is being heated up on both sides, I don't believe symmetrically, but certainly it's on both sides to some degree. It precludes having an honest and safe space. You know, you and I can go, but go back and forth. We can be from very different backgrounds, but we can have an open and honest conversation and acknowledge certain facts and realities and then try to say how to address it going forward. It is very difficult to have this conversation in this type of environment. I'll close with it with another little interesting tidbit about AI and credit. It is. Do you remember the research that showed that the top two reasons that people default?
- [28:41]
A
Which one is that? Because it's been a lot.
- [28:44]
B
It's been a lot. It was a young Harvard law professor by the name of Elizabeth Warren. Already a few people may have closed their ears on, but she found, and this has been replicated by many others, that the real. The top two reasons people defaulted on loans were medical problems and divorce. And those were the two leading predictors of default. Now, medical problems, I think people tend to think, are a little. Well, that, you know, not necessarily your fault. Right. If somebody has, you know, should we not lend to somebody because we think they're at a high risk of cancer? You know, I think a lot of people would be very concerned about that and say that's unfair. Discrimination. Now, divorce is a little bit of a different story. If I used to. I used to give this example in conferences, and then I realized it made people uncomfortable in the audience. But there are AIs at your bank that have a pretty good idea whether you're cheating on your spouse. And, you know, let me give you a hint. If you're booking hotels in the city you live in, if you're, you know, ordering room service at 2am in a place that isn't your house, it. I've had bankers tell me that they've developed little AIs and started flagging people as credit risks, and they realized they were just picking up divorce. If you're in marriage counseling. Yeah, going to see a marriage counselor right now. Turns out that when we passed in the 70s, America had this big push in financial services to get rid of discrimination. And we put in all these protected classes and a raft of laws in the 1970s of a kind of rebirth of the Progressive Era. One of the things that we did was we protected on marital status. It's actually a protected class in bank lending. And it was done because unmarried working women, which was growing radically in the 70s, were not getting loans because the bank officer was saying, who's your husband? And they'd say, I'm not married. And they go, oh, come back. Come back, sweetheart, when you have a husband, and then we'll talk to him. Women were rightfully infuriated by that laws were passed to make that illegal. I have a single. I grew up with a single working mom, and I'm incredibly thankful for that, and that's the right thing to do. But now you say to yourself, huh, this is kind of. This is a tension of the use of AI. And it's one that, you know, is a little bit sometimes uncomfortable for people to talk about, just like race, just like gender. And I'm very pessimistic that we're going to be able to have an honest conversation at this moment with everything else that's going on in society. And so I'm kind of hoping that there'll be a calm after this storm where we can piece this thing together.
- [31:48]
A
Well, and I think that's so interesting. I mean, as a person who's gone through a divorce, you're completely correct that there are many economic vulnerabilities that come with that. And to your point, you know, part of the use of AI should be a positive narrative, right? On how it's helping people build wealth. But again, you know, you and I both know, you know, being denied credit or a homeownership loan or something else. It only widens, you know, your economic opportunities. But for racial minorities in particular, it widens the wealth gap, which I think at the end of the day, it's really not just about what you can buy today. I keep telling myself that, Erin. Like, I know I love certain things I want to buy, but it's really about the future and what you're going to invest in your children. And, you know, that's why having a home is important, all that stuff. In fact, we gotta have this conversation. I was in a conversation just recently with somebody, said maybe homeownership is not the only thing that actually creates wealth. But before you go there, I'm just gonna have you back on, my friend, and we'll talk about that as well, because I think AI will have something to do with that going forward. Aaron, it is always a pleasure to have you on the Tech Tank podcast, Nicole.
- [32:58]
B
I learned so much every time I talk to you and I look forward to doing this again sometimes.
- [33:03]
A
And, and really for our listeners out there, we actually do like each other. This is what civility looks like. So I want to stress what Aaron said. We have some differences on some things, but for the most part, we're greenhouse about our car. I think I'm gonna use that. Aaron, we're greenhouses, right?
- [33:17]
B
When it comes to our conversation, take anyone with it. You have, you have a gift of language.
- [33:22]
A
I know you think it'll make the Websters, you know, cool. Word of the year, Greenhouse. Everybody should be about the greenhouse effect, right?
- [33:29]
B
It's, it's in my world, it's 5786, 2026. We can work on Greenhouse for, for that as well.
- [33:40]
A
I love it. Well, thank you so much for joining me, Aaron.
- [33:43]
B
Thank you for having me, Nicole, and
- [33:45]
A
for all of you. You can find more of Aaron Klein's work on the Brookings webpage at www.brookings.edu. and while you're there, explore more content on tech policy issues from Tech Tank. The that is our signature newsletter that Aaron contributes to that keeps up to date issues in tech policy front and center. This concludes another insightful episode of the Tech Tank podcast where we make bits into palatable bites. Until next time, thank you for listening. Thank you for listening to Tech Tank, a series of roundtable discussions and interviews with technology experts and policymakers. For more conversations like this, subscribe to the podcast and sign up to receive the Tech Tank newsletter for more research and analysis from the center for Technology Innovation at Brookings.