← Harvard Data Science Review Podcast
Harvard Data Science Review Podcast
Polling for 2024 U.S. Election: What Should Voters Look for and Trust?
00:29:55
Loading summary
Transcript41 lines
- [00:02]
Liberty Vittert
Welcome to the Harvard Data Science Review Podcast. I'm Liberty Vittert, the feature editor of the Harvard Data Science Review, and joining me is my co host and editor in chief, Shaolin Meng. As America approaches another presidential election year, many of us are contemplating our beliefs, staying informed about election news and polls, and at times questioning the integrity of voting collection as a whole. Today we are joined by Kai Chen. Yo. Kai is a pollster and partner at Echelon Insights, a next generation opinion research, analytics and intelligence firm, and Scott Tranter, Head of Data Science at Decision Desk hq. Together we delve into the upcoming House, Senate and presidential elections. Where can we find reliable polls amidst an ocean of information? How are election outcomes determined? Which voter demographics might lead to surprising election results? Join us for a very insightful discussion on these topics and more on this month on the Harvard Data Science Review podcast.
- [01:10]
Shaolin Meng
So this election cycle has been exceptionally captivating so far. We've witnessed groundbreaking moments in the primaries, such as Nikki Haley becoming the first woman to win a Republican primary. So, looking ahead to November, what are really the distinctive data points that make this election stand out? Kai, are there new insights to consider, especially with the close Senate and House races and early indications pointing to a potential second Trump victory? What do you think, Kai?
- [01:40]
Kai Chen
So I would say that the biggest consistent factor throughout this election cycle isn't exactly a new insight. It might not even be that interesting to note, but it's been really undeniable how dramatically Biden's drop approval and favorability numbers have eroded in recent polls. And I'm not just talking consistently underwater job approval, but Biden favorability has dropped by 14 points since 2020. That puts his popularity numbers lower than Trump's. And Trump favorability numbers, on the other hand, have been largely static since the 2020 elections. I saw a statistic earlier today which I haven't fact checked yet, so this could just be something I saw on the Internet, but this could be the first time that a Republican candidate for president has been consistently ahead in state and national polls since 2004. I would say the other piece that's going to make this election really interesting and which has even more of a potential to affect some of those down ballot races are some of the shifts that we're seeing among the demographic groups which have traditionally broken strongly and reliably for Democrats. So specifically we're seeing very clear weaknesses when it comes to young voters, the youngest voters, approval of Joe Biden, and as well as preferences for left leaning positions really eroding among non white voters. I Think one of the most shocking data points that I've seen recently, which really ties all of this together, is that as of just a few weeks ago, Biden was only leading Trump among non white working class voters by six points. And that's down from a 45 point lead in 2020.
- [03:18]
Shaolin Meng
The approval rating stuff was so interesting because I remember right before the 2020 election, Trump was, his approval ratings were so low and everyone was like, this is the approval ratings we've ever seen. And now Biden's actually lower than Trump. So it's like a whole. I don't know whether it just means people are hating politicians more now. Who knows what it is? Scott, what do you think?
- [03:38]
Scott Tranter
Yeah, no, the new normal is you do not have to be well liked by a majority of the electorate to be elected president. I also look at turnout. One of my favorite stats is in 2016, Donald Trump got a little over 62 million votes nationwide. And in 2020, he got a little over 74 million votes nationwide. Now, obviously we don't elect our president by national vote, but he increase his vote share by 12 million. So no matter what you think of him as a president or as a candidate, there are about 12 million people. Some of them may not have been eligible to vote, but There are about 12 million people who saw the first four years and said, you know what, we're going to go ahead and vote for this guy again, along with a good chunk of the ones who were originally there. And so what we're looking at is both sides. Joe Biden increased his total vote count over Hillary Clinton in 2016 too. Well over 80 million votes is despite both candidates being at or underwater an approval rating. They're certainly able to get the vote up, the total turnout up, they're able to motivate voters to the poll. Maybe that's because they're motivated by who they're against, maybe not who they are. But we're seeing record numbers of turnout and that certainly makes it hard, at least from my point of view, from a forecast, because it does come down to turnout. It does come down to who shows up and doesn't show up. And that really makes some of these predictions hard.
- [04:49]
Unknown
Well, thank you, Scott. And before we go on with more questions that it will help our listeners, not everybody understand the organization Europe, where's the Decision Desk HQ does. So can you give a brief description of what Decision Desk does?
- [05:04]
Scott Tranter
Sure. So we're one of the three organizations in the United States that collects election results on election night, aggregates them up and then packages them and gives them to news organizations. So we're nonpartisan, we're not here with an opinion. We simply collect the data and we provide it to news organizations. So if you go to the New York Times and you look at some of their results, you're going to see data we collect. If you go watch Newsmax, you're going to see our data there. You're going to see a whole bunch of different news outlets. And that's why I like to highlight the non partisan factor. We also along that collect some of the statistics around absentee early voting, poll aggregation and we do a forecast model. We're not here as pundits, we're here to provide it for news organizations so they can provide the context that they want.
- [05:45]
Unknown
Thank you Scott, because this is a data science podcast. So I can't help ask you to elaborate on the data collection methods being employed by the student desk. Especially are there any innovative techniques that you use or similar firms are leveraging to enhance the accuracy of election prediction and streamline the election day process.
- [06:06]
Scott Tranter
So I wish I could say this is super high tech and we're using all these AI things and all that kind of stuff, but at the end of the day, the way the United States conducts its elections is there's 3,000 plus counties and they all have their own election vote counting apparatus. And so then we have to go out and and develop one on one relationships with every county, every voting district in America. And then we collect it one of three ways. The old fashioned way we send a person out to one of these counties on election night, they walk in, they have a decision desk HQ badge, they've had a relationship and they get the tally. Any United States citizen can do this. Once they get the tally, they enter it into a phone app and then we aggregate it up. The other way we do it is we have direct connection with some of these county websites. So we go ahead and scrape the data. And then in some of these northeastern states, like Massachusetts is my favorite one, or Connecticut or New Hampshire, they literally fax it to us on election night. So what we do is we are a primary source collector of this information. There's no modeling, there's no guessing, there's just straight from the county or local government. We get those numbers and we aggregate them up. Where there is some modeling and where there is some statistical forecasting applied is ahead of election night, we collect things on absentee and early voting patterns. We have voter files so we understand who's voted. We may conduct polls into those Universes of people who already voted. And then we also do a forecast model based on polling aggregates. What that helps us do is contextualize the race ahead of time so that we have some priors going into election night about what we think might happen. And then obviously, we use the actual data, not model data, the actual data from these local municipalities to inform our final prediction and our final analysis to these news organizations on election night.
- [07:44]
Shaolin Meng
Scott, I just have a follow up on that. When there's only X percentage of the vote in and you're calling the election, though, how does that work? Where does the concept of modeling versus not modeling come in there?
- [07:59]
Scott Tranter
So once we get a certain amount of vote, and it varies race by race, we're able to extrapolate out what the rest of the vote is going to be like. And that is where there is some statistical modeling coming into play. But it's based off of primary source data. We're not basing it off polling data. We're not basing it off some polling aggregate. We're basing it off because, hey, when we're ready to call Pennsylvania, we have enough actual real return votes from a number of different counties and regions to predict how it's gonna go. Which is why in some states like Idaho, once we get a little bit of vote in, we kind of know how it's gonna go if you're looking at, say, the GOP presidential primary coming up. Whereas a state like Wisconsin or a Michigan, we might need to see almost all the vote come in, including some of the overseas ballot and things like that, which is why some of those take a couple of days to project out.
- [08:45]
Shaolin Meng
Awesome. And, Kai, same question. What is Echelon Insights doing?
- [08:50]
Kai Chen
Yeah, absolutely. And this is something that we've always got a pretty close eye on, just because there is a constant drip of pieces, think pieces, criticizing or questioning polling accuracy. And what I always like to advise when it comes to this issue is to keep in mind that polling is still pretty reliable. Maybe a small plug for Echelon Insights, but we were ranked one of the top five most accurate pollsters by 538's recent analysis. And one thing that we're doing this year to keep our polling Results accurate through 2024 is we're building probabilistic models of the likely electorate in every state and figuring out what we think they're going to look like demographically, by race, by age, bipartisanship in the voter file. This is pretty similar to what Scott was talking about as far as what we can try to do in the lead up to the election. And besides that, we're just adhering to the same lessons that we have learned since 2016 as far as paying close attention to variables like education, race, age, as well as trying to always expand the way that we're sampling our survey audiences to make sure that we're getting as wide of a group as possible. We've diversified in the past year beyond just web links, phone dialing, and also sampling through text messaging, which we found pretty effective for response rates. So just keeping things broad, watching demographic trends as much as possible and making our projections to the best of our abilities based on that.
- [10:16]
Unknown
So I guess for any close elections, which probably 2024 just will be like before, and the one thing people pay a lot of attention is about whether there are some specific demographic groups changing their voting behaviors. They used to vote for one party, now they're shifting to another. Is there any evidence or any data that both of you see in terms of this shifting the voting behaviors for some specific demographic groups? Maybe Scott Star first sure.
- [10:46]
Scott Tranter
Well, what I like to say is one of the places I look for it is I'm a big fan of the Echelon omnibus poll and the crosstabs there. And that's where someone like me first looks for some of these shifts. And I'm sure Kai's gonna give us a good overview of why like crosstabs you have to take with a grain of salt sometimes just because of sample size. But as someone who likes to look for the canary in the Coal nine, I like to look at cross tabs. The other place I would say to look for this type of stuff is just straight voting patterns. I always like to reference Miami. Dade county, which has historically been a Democratic county in Florida, has switched to a Republican county recently with the Ron DeSantis getting reelected as governor. It's a large Hispanic county. We've seen it flip over to the Republican side. Can you extrapolate that out to California and Texas? Not exactly. It's not homogenous. It's not something like that. But I look at data points like that and different counties and different precincts to see if there is some actual shift in who some of these minority demographics support. There's been a lot of chatter about whether Hispanics are moving more towards Republicans, African Americans, Asian Americans. There's some cross tab data. There's a lot of fighting on Twitter or X as we talk about it now. But I like to look at some of these voting patterns and this might mean we don't have a definitive answer until November or December. But I do think there is some evidence here that there is some shifts among the Hispanic and African American voters. Whether that's enough to tilt the election towards the Republicans this fall is an open question. But there's certainly some data points out there that would infer that's possible.
- [12:13]
Shaolin Meng
Kai, what do you think?
- [12:14]
Kai Chen
So this is actually an issue that one of the founding partners here at Echelon Insights, Patrick Graffini, just published a great book on party of the people and it specifically explores this rightward shift among racial minority groups. And I think what we have ended up seeing repeated over the polling data is the fact that the Democratic Party's primary messaging right now, a lot of the issues that they're speaking most loudly on are not necessarily the issues which are going to win over these minority coalitions. We've seen through opinion polls throughout recent decades that those minority groups typically make up a more conservative wing of the Democratic Party than what some of the loudest messaging might be, especially when it comes to their views on social issues. And recent data is looking like that historic support for Democrats over the past few decades has been more driven by social norms than those deeply held beliefs. There was some pretty good analysis of recent PRRI data that showed that black and Latino voters who hold conservative views will support Democrats if their social groups are mainly also black or Latino. But once you introduce that element of diversity into their social groups, they are more likely to openly align with the right. So this is a type of kind of preference cascade. People who may have previously masked their true feelings to fit in start discovering that other people actually share their beliefs. And so the more that this kind of increased right leaning shift among minorities is covered and talked about in the media, the faster this could happen as this kind of discourse erodes some of those social expectations. And this isn't like an overnight snap your fingers switch where all of these minority long time Democratic voters are suddenly hard right partisans. I think instead what we're seeing is a movement more towards that swing voter territory, but really the path forward for the Democratic Party to regain less ground among minorities, especially if we see that lost ground in November be more significant than what we're expecting. That's really going to hinge on a shift in priorities and messaging away from the types of really strong left stances on social issues which we're seeing a lot of right now.
- [14:40]
Scott Tranter
I got a follow up question for Kai, if I can. Kai, do you think some of this shift is going to be new voters as well, people who wouldn't historically made it into your likely voter screens.
- [14:50]
Kai Chen
It certainly could be. And I think we're seeing some of those trends in terms of voter registration numbers. Things like the new number of black Republicans have risen from about 5% to 15%. So I think that we very well may see that likely voter model be different from what we're expecting after November.
- [15:09]
Scott Tranter
Kai, stat about 5% African American registration to 15%. I had seen some of that data there and obviously those are new voters and things like that. And one of the questions I have, and this is where I'll be looking for a lot of polling, is why are they registering? Right. And given that, you know, at least my take is this race is going to be, it feels like 50, 50 and it really is going to be a 5050 race. I think this cycle, you know, we're looking at 2, 3, 4, 5% shift among Democrats or Republicans. A certain way in a certain demographic can certainly mean the difference in a state like a Pennsylvania or Wisconsin. And it's always those small little demographic shifts that are hard to detect ahead of time. But yeah, that is kind of my, my favorite stat coming out of what Kai said is the voter registration because those people are the hardest to, hardest to predict.
- [15:56]
Unknown
I want to ask again because the data science nature of this podcast, I want to talk a little bit about all the technologies, particularly with the rise of AIs and all the deep fakes globally. Have these technologies significantly affected 2024 election or will they affect the 2024 elections? What's your takes for both of you? Maybe Kai, go first.
- [16:20]
Kai Chen
So my personal feeling on this is that the vast majority of voters aren't going to have their decision made up based on whether or not they viewed or did not view a potential deepfake. I mean people are deciding who to vote for based on very tangible quality of life issues. Things like the cost of groceries, cost of gas, concerns about immigration or crime. And the people who seem most likely to respond to you and debate any kind of deepfake or AI clips or images feel very much like the very noisy hyper online partisans. They probably already have their minds made up. Those type of swing voters who are thinking more about those stronger real world quality of life issues are probably not going to be swayed even if they see a video of their candidate of choice saying something really outlandish. And this is amplified even further given that voter skepticism is already pretty heightened and as well as just the fact that everyone's ability to deal with AI or deepfaked Imagery is already growing at a rapid clip, particularly if someone sees a candidate doing something that they feel is very out of the ordinary.
- [17:38]
Unknown
Scott, what's your take?
- [17:40]
Scott Tranter
I was going to say Kate Middleton gave us a good lesson on doctored photos and how the press reacts and all that kind of stuff. And we laugh, but that is something, at least from our point of view. Right. So we're a primary source provider of election results. And the first thing I'll tell folks is that when you're collecting results from 3,000 plus local government entities, sometimes there are mistakes. Sometimes someone, you know, records the wrong vote total for a candidate or has to roll it back or something like that. Like, it's just a statistical certainty that when you're dealing with this much counting and it's this disparate across the country, you're gonna have errors and it has nothing. It's not nefarious, it's not malicious. It's just simply how it works. And so we worry about all these different election interference type things, whether it's the ability to doctor a photo, a voice, a video, or something like that, that creates distrust among what we're trying to do, which is get the accurate results as quickly as possible to the public. You know, that's something we've been playing around with quite a bit and worrying about. I know that's something that our colleagues over the Associated Press who do this, as well as the NEP and the news networks worry about, because a picture is a very powerful thing. We saw that a little bit in 2020 with people doctoring up photos about vote totals and things like that or what candidates were saying or not saying. We try very hard to be very transparent with those things. As far as a technological sense, there's not a lot of technology you can do against it, other than just be very transparent and open with people about how we collect this stuff and when there's a mistake, why there's a mistake and why we corrected it. But, you know, this is a data science podcast. What I like to remind people is we are collecting results from 3,000 different counties around the country. These are local government workers. They're working very hard. But if you're moving that much paper through that many machines, there will be tallying errors and they will be corrected. People just need to expect those things.
- [19:28]
Shaolin Meng
You know, it's funny, I very anecdotally, I do this thing with my students before I start about my deep fake lecture and I show them a deep fake that was made of Obama and the first couple of years students were like, oh my God, I can't believe Obama said that. And then as the last year, like two years I've done it, the students immediately go, that's fake. So, like, the concept of things being fake is just way more in the presence for people. And I think people are really understanding what that is more and more. So I guess my question then comes, what should voters really be aware of when they're consuming news over the next coming months before the election in November? What should, what should they be looking for? What information should they be looking for?
- [20:13]
Liberty Vittert
What criteria?
- [20:14]
Shaolin Meng
What does it really seem like is going to change things?
- [20:18]
Scott Tranter
One thing is we get this question a lot, very specifically around polling and election predictions and things like that. And I'm sure Kai's got a pretty good opinion on this as well. But what I always like to tell people is there's plenty of data out there that will confirm your priors. If you want data or quote, unquote data that shows that Donald Trump is winning 90% of African Americans, there's probably some poll on some website out there that will confirm your priors and vice versa. And there will be a lot more of that with election forecasts and all those other things. And what I tell people is consume a lot and average it all out. And don't rely completely on the outliers. Look at the data and look at the articles that are more transparent in how they talk about their polling or their modeling or their analysis. And less about black box and less about the punditry. Go back to the critical thinking phases of your English and science classes where they said, look at all the data and average it out. That's my favorite thing. And I tell it to my mom. When my mom forwards me some poll that she got in 14 levels of chain emails, I'm like, mom, I'm not saying it's wrong. I'm just saying this doesn't align with the other 15 polls we're looking at. Just because it confirms your priors and what you think is going to happen doesn't necessarily mean it's good. And that's what I like to think. There's so much data out there and people are at least what's good is their condition to, hey, I need data to back up my opinion. The problem is there's some bad data out there and being able to recognize it is important.
- [21:41]
Kai Chen
Yeah, I think that's great advice. And I think that asking folks to apply some of that media and data literacy that they have built up over the past few years to what they're seeing reported in traditional media is important. I also think that unfortunately, I mean, we've all seen a decline in trust in traditional media. But what's more concerning than that is there's also been a pretty clear increased reliance on non traditional and less dependable sources. I can't tell you how many focus groups I've moderated recently where nearly everyone around the table says they simply can't stomach turning on the news, reading the news, and they're getting all of the information about current events through social media through what they see on Instagram and TikTok. So I think if there's anything that we can encourage, it's to still use the best known names in media as shorthand for honest and reliable information. If you're seeing both the New York Times and the Wall Street Journal both say the same thing, you that that's going to be something that you can depend on or at least to try to introduce some of the media literacy that people are using to evaluate traditional media sources. Use those same skills to evaluate what you come across on social media. And that's what I wish I could tell my focus group participants, but just keep it inside most of the time.
- [23:02]
Shaolin Meng
Just to follow up on that. When you talk about your focus group participants basically getting everything through social media, I see that with my students, but they're 18. Do you see that with across all age groups, are people 60 plus getting their information on social media or are they still looking at tv?
- [23:20]
Kai Chen
So I would say it's a little bit higher among our younger participants. But there was a pretty interesting anecdote from a recent group where we were just asking a general question about where people get their information and one of our younger participants said TikTok and I saw another participant roll his eyes very clearly, obviously a little rudely. But when we got around to this other individual who had rolled his eyes, he said that he gets most of his information from YouTube. So I think that there are certainly behavioral differences, but that decreased trust in traditional media is cutting across age groups.
- [23:58]
Scott Tranter
One thing to add, because I know the audience right. A lot of people who listen to this at least come from a stem background or deep in academia. When we've got a lot of polls. What I always like to tell people is you can get pretty deep in a poll. You can look at the questionnaire, you can look at the crosstabs, and if you can't find that stuff online, then that is your first red flag that maybe this isn't a good poll. And since we've got Echelon on here, they're very good if you want to go. If you don't like their omnibus crosstabs, you can go look at how they answer the question, you can go look at the splits and you can go look at it historically and you can argue online all you want, but they put that stuff out there. Some pollsters do that, some don't. Polling is in a lot of ways a science experiment done for the public. And good pollsters put their data out there and they put their information out there so that people can be critical about it and debate it. It takes an extra five minutes. It's a little bit longer than reading that sensational headline, but that is an important piece. Good pollsters put their data out there, put their questionnaire out there. I think a lot of this audience who listen to now you don't have to have a degree in polling to at least give it the smell test if you like or don't like the headline.
- [25:03]
Unknown
Well, thank you, Kai and Scott for this great advice and just want to echo what Scott said about, you know, it's so easy to find data to confirm your pride, belief and these days, particularly with issues we're all very emotional about, it's easy to do that. But I want to say that, you know, even you don't have much data and then this is known, half joke, half seriously. In statistics, we will say if you torture data enough, the data will confess. So you will be always getting what you wanted. And of course that doesn't necessarily help you, probably can even harm you. Well, we always end with a magical one question. And I know probably some audience want to say, well, the magical one I want to hear is what can I do to make sure, you know, my candidate will win. But that's not a question we're going to ask. What we're going to ask is to both of you, if you can get every American voters to truly understand one thing before casting their vote, what would it be? Let's start with Kai.
- [26:03]
Kai Chen
I don't know if I can narrow this down to one thing, but I think my general advice would just be to try to look past the noise, look at reputable numbers. I appreciate Scott, I appreciate the shout out as far as the quality of our numbers, but oftentimes sites and media that will post the full original sources of their statistics, you want to be able to see the full question text, you want to be able to see the crosstabs if possible and just really go to that original source data to form the basis of your beliefs. Definitely don't believe everything you see on social media. And if you are looking at Echelon's crosstabs, please don't pick fights with us on Twitter based on a subgroup with maybe 20 total responses in it. But definitely keep looking for reputable original source data and look past those partisan headlines.
- [26:56]
Unknown
Thank you, Scott.
- [26:58]
Scott Tranter
I guess my answer to that question is looking at some of the Pew polling and Gallup polling about whether people trust the elections process has been very disconcerting to me. You know, there are anywhere, depending on what poll you look at, 20 to 40% of Republicans who think there was some sort of irregularity or fraud happening in the 2020 elections. If we go back to the early 2000s, some Pew and Gallup polling thought that 20 to 30% of Democrats thought the similar thing about fraudulent or some malfeasance around elections in 2000, 2004. And what I do know about this, as someone who looks at the elections process very closely and works with these election officials, yes, there will be problems at polling locations, there will be mistakes, there will be issues there. But by and large, the election system works. It's very well run, it's very transparent, it's auditable. I would just tell people before they vote, trust in the system a little bit more than they do. And if you're worried about it, you can go talk to your local election elections official. You can observe these things. That's what I think you know, a lot of people miss is this all happens in a black box, all this stuff. These, these local election officials are working very hard to make sure your vote counts. And I think that's what everyone needs to remember as they go to the polls this fall.
- [28:09]
Unknown
Thank you again to both of you. I guess the point you raised is there's lots of communication needs to be done, right? As you, you know, what you said is incredibly important from someone who actually work with these systems, working with people, you see what's behind the scene because as you said, it's very easy to mistake statistical variations as something manipulative and vice versa. And this is the complexity of dealing with data or dealing with anything with variations. But on any hand, it's a good news for the data science community because of that data scientists always have a job. And in fact, there probably needs more data scientists who have the kind of training, understanding of the complexity of issues like voting, which obviously is not just a data science problem, but data science itself plays an incredible role there. And with that, I want to thank both you, Kai and Scott again. And I'm not going to ask you to predict who's going to be the winner, but I'm sure that at some point if we do it again somewhere else, but such question will come. I'm sure you'll get this question all the time and well, good luck with your answer. Thank you very much.
- [29:17]
Scott Tranter
Thanks for having me.
- [29:18]
Shaolin Meng
Thank you guys.
- [29:19]
Kai Chen
All right, thank you.
- [29:20]
Liberty Vittert
Thank you for listening to this week's episode of the Harvard Data Science Review podcast. To stay updated with all things HDSR, you can visit our website at HDSR, mitpress, mit.edu or follow us on Twitter and instagramhdsr. A special thanks to our executive producer Rebecca McLeod and producers Tina Toby Mack and Arianwood Frank. If you liked this episode, please leave us a review on Spotify, Apple or wherever you get your podcasts. This has been the Harvard Data Science Review, Everything Data Science and Data Science for everyone.