How the Polls Held Up

[music]

Brian Lehrer: Brian Lehrer on WNYC with our one-week-later post-election special, and now part two. Public opinion polls, what are they good for? The red wave that many people predicted early on based on polling trends did not happen. Is this another epic fail by the field of public opinion polling, or maybe the pollsters got things largely right and it was the media that reported them badly?

How good was polling and what's the best use of polls in our political lives? With us for this, we're happy to have back with us, Lee Miringoff, political science professor at Marist College and director of the Maris College Poll. Among other things, Maris does the NPR/PBS polls, and Jessica Taylor, Senate and governors editor for The Cook Political Report with Amy Walter. Hi, Lee. Hi, Jessica. Welcome back to WNYC.

Jessica Taylor: Hi, thank you.

Lee Miringoff: Good morning.

Brian Lehrer: Lee, how are you assessing your own performance here one week after Election Day? Did the Marist poll or the field in general overpredict a red wave or is it even wrong to conclude that you predicted one?

Lee Miringoff: Well, first of all, I have now exhaled. I don't usually exhale until, oh, I would say, at least a week until after the election. This morning, I was able to do that. Now, we had a really, really spot-on year. In fact, it was eerily so because the widest margin we had in any of our pre-election polls was two points. Most of them, we were one or right on. There were some traditional independent pollsters who had a very similar result, but there was a whole flood of, for lack of a better word, bogus polls that were in the business, I think, of spinning some of the averages like RealClearPolitics.

Because they weren't using the aggregators, the filters of quality control, transparency, track record, independence, all those kinds of things you want in a poll, you can get into some trouble. I think the polls had a terrific and a horrible year at the same time, and it depended on what the intent was. If the intent was inform and accuracy, those polls were good. If the intent was to have a partisan or ideological bias, those polls didn't do so hot, but that wasn't really their intent either.

Brian Lehrer: How would you explain then, Lee, that a lot of the mainstream news organizations that don't have political agendas wound up doing a lot of red-wave stories? Surely, they're sophisticated enough to tell a poll with an agenda apart from a poll that's just looking for information.

Lee Miringoff: I think your co-guest, Jessica, could probably speak to this better than I. I think people got caught up in it because the fact that it was going to be a so-called red wave really fit the preconceived notion that if you have inflation at a decade high and 9% and if you had a president at 44% and it was the midterm election, the first one for the party out of power, that's a real, real problem.

All of the fundamentals were suggesting that the Democrats are going to get wiped out. When some of these aggregators and maybe even the forecasters to some extent were reinforcing that preconception, I think it was bought hook, line, sinker. Even The New York Times very often had polls that were pretty darn good, and yet their interpretation of the election was, "Red wave, here it comes."

Brian Lehrer: Interesting. Jessica Taylor, let me throw that same question to you from your perch at The Cook Political Report. For people that don't know, one of the things Cook does is classify races during the campaign seasons as anywhere from a complete toss-up to leaning blue or red, or likely one way or the other, or solid for one in each case. What does Cook base those ratings on and how would you continue the conversation that we just started?

Jessica Taylor: Sure. Polls factor into this, but it's also just not the only factor that we look at. We talk with sources in the states. We talk with campaigns, committees, super PACs. We see a lot of private polling from both sides, and so that factors in that we're shown off-record, very deep background, and that factors into a lot of our analysis. Like we said, it really was a lot of the fundamentals that you would see.

We were seeing such a spike for Democrats over the summer in August and September. The thing I have to emphasize about polling is it's a snapshot in time. Then because we moved several races actually in Democrats' favor in August and September, including Arizona and Pennsylvania, to lean Democrat, and then we moved them back to toss-up. A lot of that was predicated on Democratic polling we were seeing as well that was showing this tightening.

We look back to 2020 and a lot of polls underestimated the Republican votes. I think you certainly had to have in the back of your mind that, okay, if some of this is underestimating Republicans again, what are these polls going to look like? Instead, it was largely that they were underestimating Democrats that it felt like that-- Again, we saw the spike. I think a lot of it was post-Roe, post-Dobbs, and that was clearly affecting things. We saw that being the driving factor.

Even Democratic pollsters I was talking to really felt like the economy was just usurping that. President Biden's approval ratings. That was one of the things I was tracking the closest to see because it's just very hard for Democratic incumbents to outrun where the President is, especially in a president's first midterm election. While Biden may have been at 44% or so nationally, at least the final pre-election polls in a lot of these states that we saw like Arizona, Georgia, Pennsylvania, we saw Biden below 40% in some of these cases.

Can a Democratic candidate outrun him by double digits? That's what we were facing and looking at. There was a lot of skepticism that that could happen. I noted when I was looking at the exit polling on election night that Biden's approvals in these states that did do the exit polling, it was much higher. It was closer to that, 42%, 43%, 44%, and that gives someone like a Mark Kelly, a Raphael Warnock, a John Fetterman, a Maggie Hassan that they can outrun them more.

Again, private polls were overestimating where they were. They thought that they had pretty sizable leads. Wisconsin, they ended up winning, but that was a much closer race. I can even say, in Pennsylvania, Fetterman's final-- This was in a Huffington Post story yesterday. I'd heard this from a source as well. His final poll had him down too. It wasn't just that some of these public polls were off. A lot of the private polls were too.

It was a lot of Democratic polling that I think was fearing the worst. I was texting with a Democratic pollster the week before the election. This was someone who's involved in Senate and governor's races, and I said, "How are you feeling about the environment?" They just texted me back a poop emoji. That sums up how a lot of these people were feeling. It was just very dour and Republicans were very confident. Clearly, that isn't what played out on election night.

Lee Miringoff: To be fair, Brian--

Brian Lehrer: Lee, if you--

Lee Miringoff: I was just going to say, to be fair, it did get closer. It's from the time Jessica was talking about. After Dobbs, the Democrats did have that lead and it did get closer. What I thought was really interesting in our pre-election polls, but also in the exit polls, is that you could somewhat disapprove of the job Joe Biden was doing as president and still vote Democratic.

That, I think, was the asterisk that explains an awful lot of this. I think part of it was because it was a referendum on both Joe Biden but also Donald Trump. We don't usually have the former president being so much part of the process. Obviously, you were talking earlier about the deniers. Those were a lot of people Trump bought into the arena to run for the Senate and they had a bad night.

Jessica Taylor: I think candidate quality has a lot to do with this too. Clearly, we saw. Republicans had very weak candidates in some of these places. The question to me on a cycle, it was candidate versus climate. We've seen just our Senate elections particularly become so parliamentary in nature that voters feel like they're voting for a party and not necessarily a person, but you had weak candidates cost Republican Senate seats back in 2010 and 2012.

Mitch McConnell certainly voiced that. That's what he worried about. I think Lee is also right on this too that a president's first midterm is supposed to be a referendum on him. When you have a former president that has inserted himself so much into the election, he's not going to fade into the background and go build habitat houses like Jimmy Carter or something. He's making this all about him.

We clearly saw that it's hard to measure polling when Trump is on the ballot. We saw that in 2016. We saw that in 2020, but I think almost there was this reverse effect that maybe we weren't picking up is that voters were judging Trump-based candidates differently in election-denying and different things. As Lee said, this tightened in our toss-up races, all of them at this point.

We're waiting on Georgia. Obviously, we see that they typically break one way or another heavily. So far, this is the first time that all of our toss-up races will have broken the same way so far. Usually, there is one that will break another way. Again, if we look at that, all of our ratings were correct. It was just that all of the toss-ups that we knew that were very, very close all broke in Democrats' way.

Brian Lehrer: Listeners, do you follow the horse race polls during election season? If so, why? 212-433-WNYC, or what question did you always want to ask a pollster or a journalist who reports on polls, but you never had one over to dinner? 212-433-WNYC, 212-433-9692, or tweet @BrianLehrer. Do you follow the horse race polls during election season? If so, why, or any question for our guests? 212-433-9692.

Our guests are Jessica Taylor from The Cook Political Report, and Lee Miringoff, director of the Marist poll from Marist College, here in part two of our one-week-later post-election special as we are assessing the polls and asking what they are for. I'd like to take one example of each of your articles and look back at those polling snapshots. I hear you Jessica when you remind people that a poll should be seen as a snapshot in time at that moment, not a prediction of what will happen on Election Day.

Lee, I've got your Marist poll article from October 13th on the New York State governor's race. I think this was your last poll in that race. You had Kathy Hochul up by 10 points, or Lee Zeldin at that point, about three weeks before Election Day. It wound up at about a five-point victory. Do you think Hochul actually lost some voters in the home stretch that she would have had three weeks earlier, or is that just as close as polling can get, or what would you say accounts for that difference?

Lee Miringoff: Well, I would just insert that among the people who we said would definitely vote, the margin was eight points, which gets you close to five. I think what happened in the last couple of weeks in a lot of campaigns is when the Republicans felt that the inflation issue alone didn't seem to be enough to make them feel comfortable that they inserted a lot of advertising about crime. I think for campaigning Hochul's disservice or discredit, they really didn't jump on that right away.

A week went by, I assume they saw their internal polls slipping. Then, all of a sudden, Hochul became the anti-crime candidate too. That was too late in an era of social media and everything else. Here's the bottom line. Building on the theme of polls, just being snapshots, but we do have a responsibility to tell the narrative, to tell the story of what's going on. When that poll came out, it was somewhat of an eye-opener because, in New York, where there are so many more Democrats and Republicans, the expectation was this would be double digits hands down.

I think that was a cautionary flag that this race might get a little close even if it is in the bluest of the blue states. There are a variety of reasons of that, not the least of which she was an upstate Democrat and he was a downstate Republican, which is the opposite of the way it normally is. That created some uncertainty, which I think was generally the case this time because we just had a lot of uncertainty because there were so many crosscurrents going on.

Brian Lehrer: Jessica, you had an article on The Cook Political Report on November 4th, just four days before Election Day, called In Final Push for Senate Control, Republican Momentum Grows. Well, we know that's not how that turned out with Democrats retaining control of the Senate. What were you looking at on November 4th that led to that article and what are the lessons learned with 2020 hindsight?

Jessica Taylor: I think, again, it was a lot of Democrats that were just fearing the worst when I was talking with strategists in campaigns. The committees were feeling a little bit better, but in their polling, all had things close. Republican polling that I was seeing had, again, not huge leads, but all the polls leaning in their direction. Again, when you looked at just the fundamentals of where things were, the economy, we were looking again at Biden's approvals. Very low.

Again, a lot of it was talking with sources. That's completely what my other colleagues, Amy Walter and David Wasserman, were hearing as well. David predicted that House gains would be, at least in the teams, in the double digits. We're not facing that obviously at this point. Republicans look like they'll hold on, but just by a very, very small margin. Again, a lot of this just did not capture some of that.

Maybe the momentum moved in the final week because a lot of these polls are conducted. You have to end it at least a week before. Sometimes those things can shift, but I think it is a real possibility also that some of this polling mist fired up younger people, women, and maybe overestimated the impact that some of the economic concerns we're having on swing voters, whereas threats to democracy, questions about legitimacy of the elections, things like that, and particularly abortion had.

Brian Lehrer: You sourced the Democratic strategists as the source for the premise of your article. You wrote, "The far better national environment for Republicans has many Democratic strategists we've talked to staring down a gloomy prospect." That was one line from that article, but other lines included, "Republicans enter the final days of the midterms with the political winds at their backs, and it appears Republicans could be peaking at the right time." I hear you that you sourced this from the actual parties themselves, but for a news consumer's edification, what good are any of these articles? I'm not singling you out at all, but from any news organization or from any polling organization, if they can be that wrong, that close.

Jessica Taylor: Well, we did emphasize again that all of these were close. When you looked at it, you expect, typically, these races fall one way. Again, talking with our sources and, as you said, we were far from the only ones that predicted that we're seeing this shift toward Republicans at the end. That failed to materialize. I think another thing and talking with poll sources over the past week, they expect a big push on Election Day, a heavy Republican turnout on Election Day that would swap these early voters.

We didn't really see that. I was listening a bit to your conversation with your first guest near the end as we logged on. I also think Republicans, they really missed a chance to build a lead in early voting the way that Trump has particularly questioned early voting and mail-in ballots. This is something that Republicans used to heavily emphasize. I think that hurt them as well.

That's something that they should look at going forward because, again, you had the perfect storm that was there for Republicans' taking. Clearly, also the polling again, I think it missed out. We were basing that on a lot of polling and private polling particularly. Again, the campaigns did not expect some of these races to break this way, at least by their own polling. I will say, it's still all within the margin of error as well.

Lee Miringoff: Yes, I will too because The Cook Political Report is one of my go-to places. Although they may have been saying that they thought that Republicans would carry double-digit houses of Congress, for example, they were calling, I think, 35 or 36 races. An unprecedented number is just close. They just felt it was going to go that way. If you read The Cook Political Report, your walkaway was this is a close election and that means single digits. That means a lot can happen.

Jessica mentioned women. What was interesting, I thought, also was we were hearing a lot about how young people didn't seem like they were going to come out in droves. If you did a crosstab and you looked at young women versus young men, there are two very different stories. Young women voted and they voted heavily Democratic. Young men, not so much, and they just voted slightly Democratic. The idea that young people were not in this, in the end, young people were probably the difference-maker, particularly young women.

Brian Lehrer: The perception that the abortion issue had faded turned out to be wrong or overemphasized or overestimated. This is WNYC-FM HD and AM New York, WNJT-FM 88.1 Trenton, WNJP 88.5 Sussex, WNJY 89.3 Netcong, and WNJO 90.3 Toms River. We are New York and New Jersey Public Radio and live streaming at wnyc.org in our one-week-later post-election special in this segment with Lee Miringoff, who runs the Marist poll, and Jessica Taylor, who reports on Senate and governors races for The Cook Political Report. Cindy on Staten Island, you're on WNYC. Hi, Cindy.

Cindy: Hi, good morning. Thank you for taking my call. I was wondering. We still have a landline. Around election time, we get a lot of calls, and before asking us for our opinion. How do the pollers reach people who only have a cell phone now?

Lee Miringoff: Well, first of all, I'm so happy to hear you have a landline because that's the easiest way we can reach people, and yet it's not the most common way people communicate. We talk to people with live interviewers on landline. We also talk to people on cell phones. Also, this year, we did a lot of experimenting over the summer to try to reach people in a mode that they were most comfortable with.

We were doing texting to web and we are also doing some online polling itself. That was, again, creating a little bit more uncertainty in what the numbers were, but people vote differently and in different ways. We have to accommodate that. It was getting very hard to reach people just on the telephone, whether it was landline or cell phones. Right now, we're using three different ways of collecting data as I said.

Our friend, Joe Lenski, who does the exit polls out of Edison Research, he always says, "No matter what you're looking at, you got to believe your numbers because you got nothing else to believe." We were looking at very, very close numbers and hearing about the red wave, but not seeing it in the numbers. We had to say, "Hey, look, this is what I got." Then you have a couple of days where your nails are bitten way, way down. Then, actually, you eat a lot of M&M's to be truthful. It's not the nail-biting. It's the fast food, but I hope that answers your question on the--

Brian Lehrer: The counts of red M&M's and brown M&M's and the purple M&M's and say, "Oh, that's the party that's going to win."

Lee Miringoff: That's right.

Brian Lehrer: Lee, what I think I hear you saying over a number of answers now is your biggest challenge is choosing a proper sample size for different groups.

Lee Miringoff: Yes, absolutely.

Brian Lehrer: Jessica brought this up in the 2016 and I think also 2020 presidential election. I think many people concluded that likely Republican voters were undersampled because there was this big hidden Trump vote that the polls didn't fully catch. It looked wrongly like Clinton in 2016, for example, had an 80% chance of winning, but she lost. This year, I've heard critiques that pollsters underestimated the turnout of young voters. You just acknowledged some of that, especially Gen Z women. Can you talk about how you adjust for the future to get as accurate a sample as you can, either based on hidden votes that emerged in past elections or with cell phone technology?

Lee Miringoff: Well, I think you mentioned one big word there. You said the forecasters had an 80% chance or whatever of somebody winning. The forecasters are a whole different world unto themselves. They take poles. They take a little bit of their special sauce and they model everything. They're the ones who were saying in 2016 that Hillary Clinton was going to have a 99% or whatever chance of winning. That was really, as I say, a misdirection of the narrative and of the story.

A lot of the national polls only, the polls only, had that race very differently, but what happened was like the spin got away from everybody. It was like, "I'll take your 90% chance of Clinton winning and raise it five like a poker game." The difference is the polling, the work at The Cook Political Report. These are analytic efforts. The forecasting is very quantitative, which relies on polls to some extent but goes at a whole different rate.

There was one thing I heard, which I have to repeat, which got me. It's just why I'm not a big forecaster fan. I believe it was at FiveThirtyEight. At one point, about a month out from the election, they said there's a 70% chance that the Democrats will hold the Senate and a 70% chance that the Republicans will take the House and a 60% chance that one of those two things is wrong.

[laughter]

Lee Miringoff: At which point, I said, "What does this do for me?"

[crosstalk]

[laughter]

Brian Lehrer: I think our last caller that we're going to have time for, Hank in the Jersey Shore, I think, wants to ask a question that goes to that point. Hank, you're on WNYC. Hello.

Hank: Hello, thank you. Good morning. My question is this, about polls. How much should the average public, let's say, really expect to get from polls? I heard that something like 6% of people answer polls, which would mean the vast, vast majority don't answer and we don't know what they think. Is that a huge problem and should we temper our expectations? Thank you.

Lee Miringoff: Yes, it's a problem to a degree that it costs a lot more money to do polls. One of the reasons we did move into a different realm of data collection was for that very reason that it was just going to be slower, harder, and more uncertainty, and more costly to do the polls. We found a way over time. I shouldn't say like it was instant, but it was most of the summer.

We were experimenting on the special elections and trying to get a sense of, "How many of each group do we need to have to make the recipe right?" To Brian, to your point earlier, a lot of it is in the sampling. It's who you pick and just waiting at the end and saying, "I don't have enough people without a college education." Well, if you don't have the right people without a college education and you count them more at the end, you're going to make your problems worse.

Brian Lehrer: To wrap this up, Jessica, I'll say with a little bit of pride that on this show, we try very consciously to talk very little about the horse race polls during an election season. Our usual series 30 Issues in 30 Days is specifically to keep us focused on the issues and not fall to the temptation of much of the media to make the lead political story, "Oh, this candidate is ahead by X percent, or that candidate is catching up, or whatever." I don't think we did one explicit polling segment on the show, the whole midterm campaign, but I want you to be able to articulate what you think the purpose of horse race poll reporting is during the height of an election season. What public purpose does it best serve when it's best serving a public purpose?

Jessica Taylor: Well, I think that's where it gets hard to differentiate between good polls and bad polls. We've had a lot of very bad polls that flooded this. Lee talked about this that I think skewed some of the averages on places like FiveThirtyEight and RealClearPolitics and Republican polls that were more biased and different things and that did not have as such sound methodology as people like Lee have at their polls.

We certainly look at polls differently. All polls are not the same. I think it can give you an idea of where things are going. Again, even when we're getting a poll late, we're not looking just at the horse race number. We're looking at those crosstabs and what is down in there. Again, I mentioned the Biden percentages, but I think one thing we had to look at this year is that these candidates mattered so much. For instance, in Pennsylvania, Dr. Mehmet Oz, he was just completely underwater the entire time.

That's just not a recipe for voters to vote for you versus in a place like Wisconsin, where Ron Johnson started down this cycle. His approvals were also underwater, but he moved up this cycle. I think that helped him in his race clearly against Mandela Barnes. I think, a lot of times, the media writ large can snap onto that horse race angle, but I think it's when you dig a little deeper where I think the real meat is. That tells you less about the horse race and more about what the electorate is looking for.

Brian Lehrer: Oh, that's so interesting. It's going to cause me, even though we're running late, to throw in one follow-up question to Lee. That is, and we have a couple of callers wanting to ask the same thing, by predicting a red wave or reporting on polls that showed Democrats were in danger of losing this seat or that seat, do you think that that has had the effect of motivating more Democrats to vote and just the reporting of the polls, the fact that there are other polls, actually affected the final outcome?

Lee Miringoff: Yes, let me take an odd answer to that. I just don't think we're quite as powerful as that. I'd like to say we are. I always tell people jokingly to vote for whoever the Marist poll has ahead, but that's a joke. [chuckles] Look, I think the bottom line in all this is there's plenty of polls showing all kinds of different races. It's like with political debates. If your candidate really flubs a debate, you don't just say, "Oh, now, I'm going for the other guy."

It's like you're upset. You lose your enthusiasm a little bit, or if your guy wins a debate, then you say, "Yes, this is great. It was really terrific." I think it tends to be reinforcing rather than persuasive. I don't think people wait around for the next Marist poll or Gallup poll or any other poll to decide how they're going to vote as much as, "Gee, isn't that great," or "Ugh, I got big problems this year." I think it's more of that kind of reaction.

It's a very good question because people think that the polls and the ones who are gaming some of these aggregators like RealClearPolitics with the bogus stuff, they think that this is having a major impact on affecting who votes and how they're going to vote. I'm not so sure that's the case. I think we should just stick to telling the narrative like they do at The Cook Political Report like we tried to do at Marist. Tell the story. What's going on? What's on people's minds? That's the best public service.

Brian Lehrer: Lee Miringoff, director of the Marist poll, and Jessica Taylor, Senate and governors editor for The Cook Political Report with Amy Walter. Thank you both so much for coming on and kicking this around with everybody.

Jessica Taylor: Thank you.

Lee Miringoff: Yes, my pleasure.

 

Copyright © 2022 New York Public Radio. All rights reserved. Visit our website terms of use at www.wnyc.org for further information.

New York Public Radio transcripts are created on a rush deadline, often by contractors. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of New York Public Radio’s programming is the audio record.