← The Report Card with Nat Malkus
Loading summary
Transcript131 lines
- [00:12]
A
Welcome to the Report Card with Nat Malkus, the Education Policy podcast from the American Enterprise Institute. Recently there's been a lot of hand wringing over grade inflation, both at the higher education and K12 levels. But how big of a problem actually is grade inflation? What sort of effect does grade inflation have on student learning? Does grade inflation help or hurt college enrollment? And what impact, if any, does grade inflation have on lifetime earnings? To discuss these questions and more, I invited Jeff Denning on the podcast. Jeff Denning is an associate professor at the Lyndon B. Johnson School of Public affairs at the University of Texas at Austin, and along with Rachel Nesbitt, Nolan Pope, and Meryl Warnick, is the author of a new paper, Easy A's Less the Long Term Effects of Great Inflation. Jeff Denning, welcome to the Report Card.
- [01:11]
B
Thanks for having me.
- [01:13]
A
All right, Jeff, Many folks are concerned about grade inflation both at the high school and college levels. Basic question to start with, has there actually been great inflation in recent decades? And if so, do we know how much great inflation there's been?
- [01:32]
B
That's a great question to start with. It's actually a little bit harder to answer than I wish it was. I think the answer is yes, there has been great inflation. Certainly grades have gone up, average grades have gone up. But you know, it depends a little bit what you mean by inflation. If average grades have gone up, but so has performance, you know, student performance, then you might not want to call that grade inflation. I think there's some evidence that grade inflation, that is grades have gone up without an accompanying increase in performance that has happened. I have a paper that looks at this in the in college and I think does a good job establishing it going up. I think in high school we have seen grades go up, but I don't think there's been quite as much. I'm unaware of a paper that kind of documents, oh, this is how much of this is in excess of increased performance.
- [02:26]
A
So how much have grades gone up in recent decades? I mean, just from a pure observational standpoint, what do we know?
- [02:36]
B
That's a good question. The first figure in our paper kind of goes through both for college and high school using nationally representative data. And it's gone up. I'm looking at the figure now, trying to eyeball it a little bit. So in high school, the average GPA in the 80s was something like 2 point, little bit above 2.6 up to now above 3, so 3.1 or something like that. College has seen a similar increase, but
- [03:03]
A
maybe not quite as sharp, so substantial that's over decades. Has it been relatively smooth?
- [03:10]
B
Yeah, as best I can tell, it's been relatively smooth. There hasn't been some huge change. I think with the notable exception of the like during COVID things kind of got crazy and then have come back a little bit in some cases and not in others, but.
- [03:25]
A
And what do you think are the main concerns that education researchers have when it comes to great inflation?
- [03:32]
B
I think. I don't know if we have concern. I think we want to know what's happening and the consequences. At least that's me. I'm interested in knowing what the consequences of great inflation are for students. That includes consequences, kind of contemporaneous consequences for how much I'm learning, but then also how it follows me into the labor market. Those are the things that are top of mind for me.
- [03:55]
A
Do you hear folks arguing, well, look, there could be an upside to great inflation? And that's not just a obviously there's an upside to everybody getting better grades because they indeed deserve it. Is there an upside argument for great inflation?
- [04:11]
B
Yeah. So this may be a little bit longer answer than you want, but I think this started, this project started because we had an earlier project documenting great inflation in college, or at least that's one of the reasons it started. And people kept asking us if this was good or bad. And we said, okay, this great inflation in college led to higher graduation rates. And people are like, oh, is this good or bad? And we're like, well, I don't know. It could be good or bad. It could be good because it's helping more students graduate. And we generally think that might be a good thing. But it also could be bad if it's causing some students to study less and learn less. So I think you can make an argument that, you know, having students persist in education because they had higher grades is a good thing, might lead to more degrees, might lead to more learning. But you can also argue that it might be a bad thing that students have reduced incentives to learn, that sort of thing.
- [05:07]
A
Jeff, in your paper, you differentiate between mean grade inflation and passing grade inflation. What do you mean by mean and passing grade inflation?
- [05:18]
B
Yeah, so mean grade inflation just measures a teacher's average propensity to inflate grades. So on average, are they going to inflate grades a lot or a little? Passing grade inflation measures a particular margin, which is their propensity or their likelihood to pass a student. So you can imagine, I mean, as a teacher, I teach classes. I can choose to either make it really easy to get an A but hard to fail. You know, I could do all kinds of different choices, make it overall easy, but hard to fail. Overall easy, easy to fail. And we wanted to capture passing grade inflation. We wanted to focus on that because that's a really important measure for students. That is, you know, do I get credit for the class? This class maybe is required for graduating from high school and that sort of thing. So we have passing grade inflation, which is this kind of margin of F versus D and then just on average, great inflation.
- [06:17]
A
Jeff, I think these two terms capture a common way of thinking about great inflation in education research. It seems to me that many are ambivalent about great inflation. On the one hand, they think great inflation probably leads to less learning in a given year, but on the other hand, they think that great inflation probably leads to higher graduation and college attendance rates. Were these terms designed to capture this difference?
- [06:44]
B
Yeah, basically. I mean, it comes a lot from actually my own kind of experience in teaching. You know, I want to think very carefully before I fail a student. But I might also want to think about how easy is it for someone to get an A or to get A, to get grades, higher grades, just in general.
- [07:01]
A
So the paper you and your co authors wrote is called Easy A's Less Pay, the Long Term Effects of Great Inflation. What did you and your co authors set out to find?
- [07:16]
B
I wrote this earlier paper on great inflation and people kept asking if it was good or bad. And that paper, we weren't really able to answer that question. This paper was an attempt to try to answer that question of is this good or bad? And so we constructed these measures and wanted to see like kind of in the long term, was this a net good or net bad thing to have great inflating teachers. And with this important twist that you've mentioned with passing grade inflation being potentially different in its long term effects than kind of overall great inflation.
- [07:54]
A
And what was your gut going in? I mean, did you have an expectation?
- [08:01]
B
I think I don't. I mean, we would have been fine with whatever we found. I think it was a sufficiently interesting topic that whatever we found would be okay. I think for we thought that passing grade inflation might have this kind of positive, potentially positive impact because it helps students to stick around, you know, to not drop out of school to persist into college. That's kind of a, you know, we thought that might be the case. We thought mean grade inflation, we were less. You know, I didn't know that we thought for sure which way that would go. But we thought the main effect of that would be kind of a reduction in incentives to learn. And so we thought that might be negative. But I, you know, just to reiterate, I think we were fine finding whatever we found, but those were kind of our predictions going in, I think.
- [08:50]
A
All right, in the paper you look at students in, in both Los Angeles and in Maryland and these two data sets cover two different time periods and two pretty different sets of students. Can you say more about the data that you use in the paper?
- [09:11]
B
Sure, yeah, we use, I mean, you got the highlights. We look in Los Angeles. Los Angeles is a really large school district. It has some unique features. You know, comparatively low income, higher fraction Hispanic. And then we look at the state of Maryland. Maryland is also, you know, Maryland's a very diverse state in lots of ways. It has very high income districts, it has low income districts, it has students from lots of different backgrounds, rural, urban. And so we think they're on both coasts, you know, so we think we kind of have a broad coverage of different educational settings that people might be interested in.
- [09:53]
A
And in your two data sets, there's a huge difference in graduation rates in LAUSD and Maryland. How much does that difference matter when you're looking at outcomes in this paper?
- [10:08]
B
That's a, that's a great point. The first thing I'll note is there's the, are different years that we have coverage for. So the LA data is for earlier graduation rates are increasing over time. And so LA may be closer now to, you know, Maryland than it was because we're looking at Maryland at a later point. So I just want to point that out quickly. They are very different contexts. So I think that mostly comes down to interpreting when we find an effect on graduation in Maryland that might have, you know, that might be a, the same increase in graduation might be off of a very different base. Right. So a very large, it could be a, a 1% increase in Maryland is 1% more graduation when there's only 10% of people not graduating in LA. You know, it's a higher fraction. And so you want to interpret that 1% differently in that case
- [11:03]
A
between these two periods. You cover many years. Do you think the dynamics of grade inflation changed at all during the time period?
- [11:15]
B
Probably. Possibly. We're, we. Yeah, so that's a, we're looking over a long period of time, but we're making kind of comparisons within years. So our statements are kind of across teacher. Within a year there may be differences across time, but that's not the main focus of our, our paper.
- [11:33]
A
Jeff, you look at both Mean grade inflation and passing grade inflation. But what I'm sure some are wondering is how do you even measure grade inflation in the first place? I mean, how do you determine that a teacher was a stricter or a more lenient grader?
- [11:55]
B
It's a great question. That's what we spend a lot of time on. And this paper is trying to figure out a measure that would allow us to do that. Basically it's comparing a teacher's grades that they give out relative to their students performance as measured on things like standardized tests at the end of the course, but also their performance of students as measured by their prior performance in standardized tests in the last year, but also grades in the last year. So it's that, it's that different. There's a bunch of statistical stuff happening in the background to make sure that it has nice properties. But that's the basic idea is if this teacher gives out really high grades, but the students routinely get really mediocre test scores, then we'd say, okay, that looks like a grade inflating teacher. Whereas if you can do the opposite where like, oh, they have really high test scores and comparatively low grades, then we like, they don't inflate grades very much.
- [12:56]
A
How does this play out across teachers and across schools? And does that matter for how we measure this? I mean one could imagine that this out in the actual world, not in sort of a measuring or a measurement approach could happen in some schools, for some policy environments or for some teachers. In the context of the paper, what can you capture and why does that matter?
- [13:34]
B
Yeah, so we, we're looking within schools and within years, so we're kind of comparing, you know, teachers within schools. I think there's interesting questions about like maybe, you know, what's happening at the school level and how does that affect students? That's not this paper. This paper is all about kind of within schools, teachers that inflate versus teachers that don't.
- [13:55]
A
And that's important because if you had sort of omniscience from your statistical approach, then you could answer specific questions. But not all questions are game here.
- [14:12]
B
Yeah, there could that one, maybe that's our next paper, I don't know, but
- [14:15]
A
it's not this one in practice. Do mean grade inflation and passing grade inflation describe two separate patterns of teacher behavior? Or are teachers who are mean grade inflators also passing grade inflators?
- [14:33]
B
They tend to be correlated pretty highly, particularly in la. Less so in Maryland in part because Maryland has much less failing happening. But they are, they are highly correlated with Each other. So passing grade inflated passing grade inflation. Teachers tend to be mean grade inflating teachers on average, though that's not always the case. And so we use those that's not always the case examples to kind of tease out the difference and the two
- [15:02]
A
effects, from what you can tell, how much variation is there actually in grade inflation practices? In other words, are students experiencing a pretty consistent level of stringency or leniency across the board, or does it vary a lot from one teacher to another?
- [15:23]
B
There's substantial variation across teachers in terms of their. Our measure of grade inflation, it's a little hard to kind of put into, you know, plain English, but one standard deviation change in grade inflation represents like something like 0.2 or 0.3 GPA points difference. So, you know, that means that there's a lot of. There's a lot of variation, you know, from. You can go 0.2 or 0.3 GPA points up or down, and there's a lot of people kind of in that. In that range. It's not like all of the teachers are giving the same sorts of. They're not all the same in their great inflation practices.
- [16:06]
A
And from just a general approach, from a general standpoint of like a principal or someone who's looking at this from the outside, is there a reason to think that grades should be tied pretty clearly to standardized test scores? I mean, is there a mechanism where grades might be, you know, sort of standardized or bracketed, or is it kind of, you know, teachers kind of going by what they think is generally appropriate?
- [16:39]
B
Yeah. So standardized. Using standardized tests to generate this measure. There's some. Some good things about that. It's standardized, so we have some comparisons across classrooms, but there's some bad things about that, too. You can imagine a teacher does something that helps students learn in a way that's not captured by standardized tests, but then is rewarded in the grades they give out? Right. That would be like an example, I think, of what you're talking about, which is there's some kind of learning happening that's not really captured very well on standardized tests. So we acknowledge that possibility in the paper. It's an important thing to think about. That would tend to be people who we call grade inflating are going to have good outcomes for their students in the future. Right. So if they're teaching them something that's not captured that are raising their grades, then we're like, oh, that's. We have this measure of grade inflation actually has something else in it that's like something good about teaching, we're going to find the grade inflation is bad, mean grade inflation is bad. And so that bias goes the wrong way. Right. Like it. We don't really. We're not saying we have a perfect measure, but kind of those obvious stories about there's some other thing that teachers are teaching that are not incorporated in standardized tests would tend to kind of fight against what we find.
- [17:58]
A
Are there any sorts of teachers who are more or less likely to inflate grades? For instance, there's this stereotype of the old cranky teacher who's stingier about giving high marks. From what you can tell, is there any truth to that stereotype?
- [18:16]
B
Yes, there's a little bit of truth to that. So a little bit of truth, I should say. So as teachers gain more experience, they tend to inflate grades less. So but it's not a huge, it's not a huge change that we see for teachers. But it is the case that with more experience, teachers tend to inflate grades less.
- [18:37]
A
Are there any other traits that seem to be positively or negatively associated with grade inflating behaviors?
- [18:42]
B
We don't have a lot of teacher characteristics for us to look at that. So one we could calculate is is experience. And that's the one we looked at. We'd love to look at more other characteristics. We just don't have them.
- [18:54]
A
All right, Jeff, let's talk about the effects of grade inflation, starting with mean grade inflation. For starters, what effects does mean grade inflation have on test scores?
- [19:06]
B
So I want to point out one quick thing interpreting this. These when we're looking at the effects of mean grade inflation, what we describe in the paper is mean grade inflation holding fixed other characteristics of teachers, in particular their passing grade inflation, but also teacher value added estimates both for cognitive or test score value added and non cognitive value added. So we're saying kind of over and above these traditional measures of a teacher effectiveness, what's the effect of mean grade inflation? I just want to point that out because you might think, oh well they. And we actually do document that some teachers that grade inflating teachers have lower value added. For instance, we're saying we're holding fix to teachers value added and saying what is the effect of grade inflation? That's just an important caveat that I want to.
- [19:55]
A
Controlling for some pretty important non normal aspects of teachers.
- [20:00]
B
Yeah, so, yeah. So controlling for standard measures of how teachers are, you know, measured in academic literature. So the effects on test scores are negative mean grid inflating teachers holding those other things fixed reduce future test scores for their students.
- [20:20]
A
And, and this would somewhat make sense with folks who are concerned about grade inflation. In other words, if you don't hold the target out far enough so kids have to work and stretch, they might not work and stretch. And the lack of working and stretching could be a long term reduction in test scores to some degree.
- [20:41]
B
That's right. That's, that's very consistent with this reduced incentive to, to study.
- [20:47]
A
So in the paper it says having math and English teachers who are on average 1 standard deviation higher in mean grade inflation reduces test scores in the next year by approximately 0.02. So that's 2% of a standard deviation. How big of a deal do you think that is? I mean, it doesn't sound like a huge deal.
- [21:13]
B
Yeah, I, that's, you know, people have their own opinions about that. We mainly see it as directionally consistent with what we have in mind. And I would just point out again, the interpretation is conditional on all of these other teacher characteristics. This one additional thing, this grade inflation measure reduces test scores. So, you know, grade inflating teachers tend to have different characteristics. We're saying hold those fixed and just focus on grade inflation. So I don't know, you know, you can decide if you think that's a big or small effect on test scores. I think that's one of the benefits of this paper is we're able to look at future outcomes and not just rely on kind of a standard deviation of a test score.
- [21:57]
A
How does mean grade inflation affect other outcomes like graduating high school or taking the sat?
- [22:04]
B
Mean grade inflation reduces the chance that you graduate from high school and it reduces the chance that you take the sat.
- [22:13]
A
So I think this is pretty surprising. I mean, what do you think is the mechanism here? And I know you can't isolate that, but I mean, how is it, do you think that grade inflation, I could see it might lower test scores, but lowering graduation rates or rates of taking the sat, it just, I don't see the bridge. Do you have one? Even if you can't prove it in the paper? I mean, what's your theory of the case?
- [22:37]
B
You sound like an academic reviewer. I think if you, you can imagine the story is that students are learning less when their teacher is inflating grades. And we see that with test scores. And if you learn, I think if you learn less, you're less likely to graduate from high school, you're less likely to take the sat. You know, like once you kind of reduce your human capital, once you reduce how much you've learned, it's not surprising to me that there are follow on consequences of that.
- [23:11]
A
What effects does mean grade inflation have on college going college enrollment?
- [23:18]
B
Yeah, so we're able to look at that in Maryland and we find this reduces college enrollment. Again, consistent with this chain of you're less likely to graduate high school, you're probably less likely to go to college as well.
- [23:28]
A
Right. So we have this sort of small cascade of effects. Do the effects you only have college enrollment in Maryland, do the effects in LAUSD and Maryland do. Different times obviously different teachers and students. Are, are the other effects fairly comparable?
- [23:46]
B
I think so, yeah. They have, they're directionally very similar. There's some differences in kind of the magnitudes. You know, like I mentioned before, there's different graduation rates and then that sort of thing. But if you kind of adjust for those and think about those, I think that the results are very, are kind of remarkably similar. When we can look at both sets of results, which was didn't have to be the case. Right? We didn't. That didn't have to be true. So that's I think reassuring.
- [24:13]
A
And then you also find some effects on employment and wages eventually. What were those?
- [24:20]
B
Yeah, we find that employment is reduced and actually wages are also reduced when you have a mean grade inflating teacher.
- [24:30]
A
And the differences that you have, they're, they're large by the, the teacher effect. A standard deviation higher mean grade inflation reduces the. And, and this gets a little technical. But the present discounted value of lifetime earnings of their set of students by over $200,000 per year of teaching. That's a lot. Cumulatively it's not a lot per kid. But the fact that it is an effect on so many kids is a large net effect of grade inflation. Does mean grade inflation affect different students differently? I mean are low achievers more likely to be affected than high achieving students? How does the mean grade inflation affect students across the spectrum?
- [25:22]
B
We have several checks of this in the paper. We don't see strong differences by mean grade inflation by, by high or low ability students or performing students. We have course measures of that that we're using, but we don't see big differences there. I think your point about interpreting the magnitudes is right. It's kind of a small effect for any individual student. But teachers are important because they impact lots and lots of students.
- [25:50]
A
All right, Jeff, it's time for grade it. No inflating grades. Be smart on us and explain briefly your rationale for your grading. Are you ready?
- [26:00]
B
I'm ready.
- [26:01]
A
Recent changes to student loan Limits for graduate students.
- [26:10]
B
Trying to get me in trouble on this one. I work on this as well. I, I would say directionally, they're good. Directionally, it's good to reduce how much students can borrow the exact magnitudes. I'm, you know, there's debate about exactly what those should be, so I give
- [26:30]
A
it like a, A B, B. Texas's top 10% rule.
- [26:40]
B
I really like that A. I think it's A. It offers really direct incentives for students, makes it really clear. I think it's gives access to a great university that I'm very biased. They're a great university to lots of different students from different regions and the. In the state.
- [26:55]
A
And for those who don't know, what's the top 10% rule?
- [26:58]
B
The top 10% rule is that if you're in the top 10% of your high school graduating class in a public school in Texas, you are automatically admitted to public universities. In Texas, when it was implemented, it was all public universities. Now it's a slightly more stringent. It's a lot more stringent for UT Austin, but the same is true at all the other public universities.
- [27:18]
A
Jeff, grade your own barbecuing ability.
- [27:24]
B
It's probably gotten worse since I moved to Austin because I just go buy it all the time now, but I would give myself a B, probably a B.
- [27:30]
A
And what, what do you typically barbecue on? What is your equipment of choice now?
- [27:35]
B
You're gonna get me really in trouble in Texas. I, I use a, just a. A pellet grill, so. Which hardly counts as barbecue around here.
- [27:45]
A
Well, you're, you're in tough, tough territory there. Our understanding of the gender pay gap generally in America, I think we know
- [27:55]
B
a lot about it, so I'd give it a, you know, a B. I give a lot of B's, apparently.
- [28:02]
A
And, and how well do you think that Americans understand it? I mean, there's lots of sort of, you know, the, the research community may understand it one way. How well is it understood at our sort of politics and media?
- [28:16]
B
Oh, I don't know. Worse than a B, but maybe a C. I don't know. It's A. It's a very complicated, important topic. And that doesn't always translate to, you know, the broader media
- [28:30]
A
abolishing grades and replacing them with standardized test scores.
- [28:35]
B
I wouldn't do that. So I don't think that's a great idea. I think D. I think grades contain information for students and for teachers. And so, yeah, I, I don't like that idea.
- [28:48]
A
The nil's effects on the enjoyability of college football.
- [28:54]
B
I. Well, so I root for the University of Texas. University of Texas famously has a lot of nil money, so I think it's great. A. But I understand that that's maybe not a widely shared opinion.
- [29:10]
A
Austin A plus.
- [29:13]
B
Austin has great weather, great food, has a great university, lots of fun things to do. Yeah. Huge fan of Austin.
- [29:23]
A
The predictive power of elementary school class ranks.
- [29:29]
B
It is predictive, so I would say B or A. I mean, there are other things that are more predictive, so it depends on what you mean by that grade a little bit, but it's predictive of things later in your life.
- [29:42]
A
The idea of replacing human reviewers with AI in the peer review process,
- [29:51]
B
I can see you've seen my X account. Every AI that I've encountered is very positive. So I think that certainly relative to my academic reviewers, I think I would prefer the AI because they would tell me it was a great idea. So, you know, for me, a, we should totally do that.
- [30:11]
A
Well, there's also the arms race with many people producing more and more papers with AI. The peer review process is going to buckle under. Is it? If. If it's just free labor of peer reviewers unassisted.
- [30:26]
B
Yeah, I mean, we'll see. I. That's a whole. That's a whole interesting podcast by itself, probably.
- [30:32]
A
Okay, last one. Pell Grants.
- [30:37]
B
B Again, I think that there's a lot of really good, good things about Pell Grants. There are some potentially negative things too. So. But I think overall I like the idea. What I will say is that there's a lot of discussion about school vouchers right now, mostly in high school. And I always, when I teach my students, I always say, we have vouchers for school. It's just in, it's in college, not in, in high school. And I think that is a framing that many students haven't thought about. But I think that giving low income students money to go to college has been shown in the Pell Grant specifically, but also just generally to improve their outcomes.
- [31:19]
A
All right, Jeff, I want to get back to grade inflation because we have a whole set of results. We haven't talked about the passing grade inflation. So just to remind listeners, passing grade inflation isn't generic grade inflation because you've already controlled for that. It's even after you control for the regular grade inflation. Well, and tell me if I'm wrong on this, it's. Well, I just don't want my students to fail, so I'm going to sort of have a lower bound on the grades I'm going to give them. Is that right?
- [31:48]
B
That's right, yeah.
- [31:50]
A
And how much does passing grade inflation affect test scores?
- [31:57]
B
It does not have much of an effect on test scores, which is, well, maybe not terribly surprising. This is about, you know, this is mostly about a policy about whether you're going to get credit for the class. So that's why we focus on that margin.
- [32:14]
A
And this would only really matter for students who, at least as reflected in their test scores, are not learning as much as their peers who are higher on the grade scale, right?
- [32:25]
B
That's right.
- [32:27]
A
So passing grade inflation should in theory make it less likely that students are held back, right? I mean, if they're not passing classes, more likely to be held back in practice. Is that what you find?
- [32:39]
B
That's right. They're more likely to making, be making on time progression through school.
- [32:44]
A
So what effects does passing grade inflation have on high school graduation and college attendance? I mean, I could think, well, we're moving them along, but mainly on the goodwill of the teacher. That's probably not good for them or maybe we're just not holding them up unnecessarily. How does it play out?
- [33:03]
B
They're more likely to graduate from high school as a result of passing grade inflation. That is intuitive. I think that's, you know, you got credit for that algebra class, so now you can graduate. The effects on enrollment tend to be not, sorry, enrollment in college. There's not a huge effect on that. It changes overall. It changes a little bit where you go to college, but very not, I would say, not big effects. So not big effects on going to college from passing grade inflation.
- [33:41]
A
And then college completion.
- [33:45]
B
College completion. We also don't see very much from passing grade inflation.
- [33:50]
A
So on net there's not a ton of difference in college going and completion, but the kind of college that students go to and complete, there are some differences. Can you lay those out?
- [34:04]
B
Yes. So passing grade inflation makes you less likely to enroll in a bachelor's degree seeking program and more likely to enroll in associate's degree seeking program. And we see similar kinds of. But they, they're about the same size, so roughly it comes out to zero. And then we see similar things when we look at graduation where reduction in 4 year degrees increase in 2 year degrees, again roughly offsetting to be zero. So that's, I mean, roughly speaking, that's, that's right. So there's not a lot happening. There's some shifting about what's happening, but it's not really changing the number of degrees very much.
- [34:46]
A
So we don't see a whole lot on college going. We don't see it on college graduation. But again, just like with the mean grade inflation, you have the numbers on long term earnings and employment. Do we find any long term effects there?
- [35:04]
B
Depends a little bit what you mean by long term. We find some initially positive results on wages and on employment. So in the first couple years after high school, passing grade inflating, students are having higher wages, more likely to be employed, but those fade out, it looks like after three or four years roughly post high school. So, you know, as best we can tell, they don't seem to persist for a long time. And this may be because these students are more likely to have a high school diploma and so they're going to get jobs that require a high school diploma and they're getting higher wages. But then after a few years, kind of that doesn't, that signal of a high school diploma doesn't matter quite so much. And so there's not really this benefit to earnings in the long term. That's speculative. But the pattern is we find some benefits to earnings early, but those benefits don't seem to persist.
- [36:04]
A
Jeff, overall, how concerned are you about mean and passing grade inflation? I mean, in the scheme of things that are going on in high schools, how much should we focus on how strict a greater a teacher is?
- [36:23]
B
I think we should, we should definitely focus on at least a little. I think, you know, it depends on the high school, but we teachers have a lot of discretion in how they use and how they do grading. And I think it's worth us thinking about what's the best way to do grading. And so I think we should, we should have conversations about it. We should think about it. This has a lot of com. This paper and this approach has a lot of similarities to teacher value added, which some of your audience may be familiar with. The difference here is that this is something the teacher can choose and can change in kind of a concrete way where, you know, teacher value add is like, just go be a better teacher. That's like hard, right? This is something more concrete which is, you know, make sure your grades align more closely with kind of the how well your students are doing.
- [37:14]
A
And it's also something where you can just kind of nudge the joystick a direction. I mean, if there's a direction moving towards, towards being more lenient, well, that might have some somewhat continually negative effects on your students on average. So, you know, maybe if you're going to steer, steer a little bit in the other direction.
- [37:36]
B
That's how I think about it.
- [37:37]
A
That sounds fair.
- [37:38]
B
That sounds fair. Yeah.
- [37:41]
A
So you're an associate professor. You teach students. Before starting this paper, how liberal of a grader were you?
- [37:50]
B
Who are you asking? Me or my student?
- [37:52]
A
You.
- [37:52]
B
You're.
- [37:53]
A
You're the only one I. I have contact with.
- [37:55]
B
Okay. Yeah. I. So I have largely been in Economics departments where the Me like where there are standards for how much. What the grade distribution in the particular class will be. And so I follow those. I'm now not in an Economics department. I'm in a school of Public affairs and also in the College of Education, where there's not as strong of kind of directives. And so I'm still sorting out exactly what the. What the right thing to be is. I will. You know, it's. It's not really very fun to be the stringent grader, I will say, because students don't like it. I don't like being the bad guy. So anyway, I will say it's a. It's. I guess it's an evolving. It's an evolving thing for me.
- [38:45]
A
Well, when you think about it from the teacher perspective, I mean, to some degree, it is a collective action problem. I mean, if no one's giving Cs and if Ds are frowned upon by the administration, well, that kind of gives you a pretty good idea of which direction you should be headed. And this evidence does not suggest that that's a road to hell immediately. It just does suggest that that's not necessarily a safe thing to let the collective action put us, push us towards easier grading. That. That seems fair, even if it's my overwrought interpretation on it.
- [39:22]
B
I think that's right. I think. I mean, in. In college, I think high school has some different dynamics going on there. But I think in college, you know, my incentive as an instructor is if I give out A's, my students are very happy. They give me better ratings on my evaluations, which matter for promotion. You mentioned I'm an associate professor. I would like to be a full professor professor at some point, you know, they look at things like teacher ratings for that. And so one way to improve my teacher ratings is to give out higher grades. You know, my teach, I get less angry emails from students. And so when my department was like, hey, you have to have this grade distribution in your class, then I could be like, look, I'm not the bad guy. The department is. Right. And so I totally think your point about collective action, spot on in higher ed and probably has some resonance in K.12 as well. But I have less direct experience there.
- [40:10]
A
And if you were just going to take the results from your papers and apply it to sort of that idea, well, we should have some relative distribution. Not relative distribution, a distribution of grades in a given class. And we should expect a healthy number of Cs in that distribution, not just all Bs and As. Even if you were relatively lenient on failing grades in that distribution didn't require, you know, whatever, three kids to fail the class every time, that wouldn't be out of keeping with your suggestion. Right. The Economics Department is not necessarily totally off base.
- [40:56]
B
Yeah, I, I mean, I'm an economist, so I'm biased, but I think that that is the. I think that's a good solution to this problem. Now, I will say the Economics department, I've been in a couple. But many Economics departments behave this way. They're an outlier in many universities where they're having these kinds of standards that they're, you know, they're setting for grades across, across their faculty. And so they, they look very. Generally, Economics departments are very tough graders relative to other departments. And so that gets them in trouble, potentially with the administration. Like, hey, why are you giving so many low grades? It may be as tough if you want to attract students to your major. Like, hey, if I'm going to be an econ major, I'm going to have a way lower GPA than if I were going to be. I choose a different major. So this collective action thing is very,
- [41:42]
A
very important at the policy level. You know, you could see policy reactions to these kind of results that are overwrought. What would you say policymakers, district leaders should be aware of if they all of a sudden want to take charge on the great inflation front?
- [42:03]
B
Yeah. So I would say I hope that our paper leads people to have conversations about what do we think grades should be doing, how should we be giving them out, that sort of thing? We want them to. We want people to be intentional about them. And so to use this policy lever. That said, I don't think it would be right to take our results and say, okay, well, if we just made everyone, you know, follow this particular grading policy, we will take the numbers from this paper and we'll know exactly what happens to earnings. Lots of things change when you change the behavior of every teacher in a school or in a district or something like this. So I think you're. I liked your analogy or your, your example from earlier. Like, I think we should think about it, like, nudging the joystick. You know, like, maybe we should think about nudging towards having grades correspond more tightly to performance and, you know, kind of assess as we go and see what are the consequences of that and are there any unintended consequences from kind of everyone changing at once or something like this?
- [43:05]
A
What about different advice for teachers? I mean, if you were talking to teachers about their grading practices and around grade inflation, what message would you have for them?
- [43:16]
B
I would say, you know, you have a lot of discretion and think about how best to use that. You want to preserve incentives for students, especially top students, to. To study. At the same time, you don't want to unnecessarily fail people. So I think kind of, you know, our results broadly suggest that kind of making it hard to fail isn't so, so bad, but making it very easy to get an A might be not that great. So kind of, again, nudge that direction towards, you know, don't make it so easy to get an A.
- [43:52]
A
Thanks for listening to the Report Card with Nat Malkus. And special thanks to our guest, Jeff Denning. Also, I want to thank our producer, Ellie Lucas. He makes this podcast possible. That's all for this episode.
- [44:03]
B
I'm Nat Malkus.
- [44:06]
A
Sam.