(To receive weekly emails of conversations with the world’s top CEOs and business decisionmakers, click here.)
As America heads into another election season, voters once again turn to pollsters for insights into how the results may turn out. After predictions that missed the mark during the 2016 and—to some extent—the 2020 presidential elections, people’s faith in polls has been shaken. So TIME asked The Harris Poll, which doesn’t do any political horse-race polling, to look at attitudes towards the polling industry. The news was mixed. About 60% of respondents believed that pollsters were right at least as often as they were wrong. And in a heavily politicized era, slightly fewer than that believed that poll questions were always fair and balanced. However, the vast majority of respondents believed that the media was more to blame for inaccurate interpretation of the data than pollsters were. And people who followed polls very closely tended to trust them more than those who used other media sources.
Will Johnson, CEO of The Harris Poll, which was founded by John F. Kennedy’s pollster Lou Harris, calls the elections in 2016 “a wake up call” for the polling industry, some of which is not keeping pace with technological changes, including bots, that are leading to the spread of disinformation. But he’s an optimist, both about polling, and about America. Having looked closely at the data, Johnson sees less division in the country than the fights on social media would suggest. He also offers several suggestions for how polling companies can improve their methodology and rebuild trust with the public.
This interview has been condensed and edited for clarity.
We live in an era at the moment where everything is politicized. How is polling dealing with politicization of facts?
The reason people get into this business is they’re curious about what people think. So we really make that the primary value. That comes in three places: Are you sure you’re measuring the people you’ve set out to measure with your poll? Are you asking the question in a way that is balanced and getting at the essence? And then—as you’re interpreting the statistics that come out of that result—are you able to provide a commentary that balances what you’re able to glean, as far as insights, and what it’s not able to tell you?
People are becoming very disenchanted with institutions—with media, government, law enforcement, scientific and academic institutions, and with health institutions and schools. How is that reflected in your data?
As everyone has access to more and more information across all those institutions, we’ve just got a way more sophisticated consumer. In general, people just have seen further behind the curtain. And that’s going to inherently bring more skepticism of things, which in some respects, I think is healthy, but I like to go back to local institutions. Someone may say they don’t like hospitals, but they love their local hospital. They love their doctor. When you go back to polling and look at the drivers, if you ask ‘Do you trust this institution or that,’ you may get a number. But if you go under and say, ‘Do you trust your local hospital? Do you trust your doctor? Do you like your kids’ teacher?’ Then all of a sudden it gets way more nuanced and complicated and I don’t see the level of erosion that some of these places that maybe benefit from creating fear see.
Have you noticed that people have been more distrustful of polls?
I think 2016 was clearly a miss for the industry in general as far as understanding the electorate—and 2020 to some extent. It was sort of a wake-up call in the sense that people were resting on their laurels as far as techniques. You can’t use the same techniques as things move faster and faster. Are you sure you’re getting the right respondent? Are you asking the right question? And then back to when you get the answer to those polls, are you really looking at the underlying drivers of the issues to [firstly] confirm that you got an accurate read, and [secondly] you know what is really happening, and not the headline. I push back on people who think that was inherent bias and nefarious. I think it was more about how rapidly people are changing and us having to be much better at how we actually get to the voter to ask them who they want to vote for.
Do you think that faith in political polls can be restored?
Yeah, I think that faith can be restored by getting it right. That’s kind of the bottom line. That’s the neat thing about polling, the results are out there and you can see how you did. So I think that’s number one, but number two is what exactly are you polling? What are you talking about? You know, is it just national horse race poll data, or are we talking about local polling about a particular issue? I don’t think that faith is lost in that. The whole horse race dynamic of how we cover elections seems to be sending us down a less optimal path.
So then what are you doing differently? Why should people trust polls?
I think it’s always good to make [the sample] as robust as you can and we are investing more and more in that. That changes the economics, but we live and die by the Harris brand, so [we will do] whatever it takes to make sure we maintain that integrity. The second and most important [thing] is making sure you’re getting the right people. There are bots, and the technology is changing so dramatically. We’re reaching people online or on their phones and you have to be vigilant. I’ve spent more time in the last couple of years talking to clients about who we’re measuring than what we’re measuring—that is no longer just a given. So being able to have a quality sample and really understanding if this is a real human who’s taking the survey and if it is a human, is it the human that we want to measure? And are they actually engaged in the survey? We’re putting a lot of investment into that, which I think, candidly, our industry probably took for granted previously, and as technology caught up, that’s a premium.
The third part is making sure that when you do get that data you have the right sample and you’re putting it out to media, fighting the temptation for what may be the salacious headline.
And having that discipline to say, ‘Let me explain and give you context behind this whole survey of what we’re looking at,’ whether it was polling about an issue or candidate or whatever. It may not be as dramatic as we’d all like, or as exciting. But it’s truth. The way people live and the way they consume has changed. And the way they take polls is just like everything else—it’s moving faster and faster and faster. So we just had to reprioritize.
In the poll you did for TIME, more than a third of the people you polled do not think poll questions are fair and balanced. Does that concern you?
There’s always gonna be a percentage whenever you ask a question that’s gonna think that. I think the way you frame a question can have a huge impact on the answer. So it just goes back to trying to get to the truth in the question. And are you designing it correctly? So, to answer your question, yes. We don’t take anything for granted.
The poll also showed that more than 70% of your survey participants say that the media misrepresents the results of polls. Is that also your view?
It’s kind of become a sport to bash the media on certain things. So you’re gonna have a population who do that, but back to a point I made earlier, I don’t think this is a bias issue. I think this is a business model issue. There’s that pressure to have the data point that is gonna get clicks. And that may cause media partners to publish something, or make something more prominent without telling a more nuanced story that’s actually, in my opinion, more interesting and more accurate—but may not get you the quick hit or the traffic that you need to survive.
One of the things that your polls have shown is that the popularity of businesses is affected by people’s political leanings, so that Chick-fil-A is much more popular with people who are conservative and Samsung is more popular among progressives. At the same time, businesses are getting more politically involved. Is this a helpful or an unhelpful trend?
As we’ve seen trust erode in big institutions, business has no choice but to get involved in these issues. Not only are their customers demanding it, but we see their employees—particularly this new kind of millennial workforce—say you can’t just stay out of the political fray. With that said, I think brands really need to be thoughtful about how they think about these particular issues. Because again, people are so much more complicated than snap polling or quick online media would give them credit for. For example, people may not be for defunding the police but they want more racial equity in how police forces work, right? So, as a company, you need to really move judiciously about how you take a stand on certain issues, and keep that in line with exactly what goods or services you’re trying to serve. Looking at a lot of data and polling, I’m an optimist. I think people are closer together than some would have you believe on a lot of issues. I think companies have to get involved. They can’t ignore these issues. It used to be that the CFO was the most important person after the CEO. It’s now the chief communications officer because you have to be involved, but you need to to think about it in a measured, nuanced approach, and understand that it’s not black and white.
Trader Joe’s is very popular on both sides of the aisle. So is Krogers. Why is that?
I think it goes back to being local. That’s where you’re getting your food; that’s in your community. I think those brands do a great job at keeping their brand promise and delivering for their consumers. People go in there and actually physically engage in most cases there. And so, while it’s national, to the average person, it’s the store down the block from them.
What are the trends that make you feel optimistic?
If you follow social media, you’d think that it’s just two worlds completely divided. But when you talk to neighbors and even go out into places that may be different from yours and talk to those constituencies, you see a great balance of the values are similar when you get deep underneath the whole [question of] whether you’re a Trump person or not.
Give me an example of these shared values. What would they be?
Well, family. People care about their families. People want to feel secure in their neighborhoods. Generally speaking people want other people to be happy and feel secure and feel like they have a good chance for a good life. I do polling for a local business publication in Chicago and we’re looking at health outcomes in Chicago. And you see that 87% of Caucasian people feel very good about their health and longevity. For minority populations, it’s 10 points less—significant. When you present that data in a non-polarized way, I find everyone says, ‘This is a problem we need to solve.’ That has nothing to do with are you a Republican or a Democrat. People see that data and it’s sort of arresting, and it’s like ‘How do we work for a solution?’
According to nearly all polls, most Americans believe that something should be done about assault rifles. But it doesn’t seem to get reflected at all in legislation. Are there times when you feel like polls are just not effective enough?
We do see in the data that an overwhelming bipartisan majority want to see a sensible action taken as it relates to that issue and I think the reason you’re not seeing more significant movement unfortunately has less to do with sort of general public sentiment and more the way that our electoral system is constructed, particularly as a relation to primaries.
Have you done some polls where a piece of data does not land in the way that you thought it would, where you thought, people are not seeing these trends in the way that they should?
Yes, COVID. So we ran a COVID tracker right after things started—we’ve run it over a hundred plus weeks—where we measure a bunch of different feelings about how people are thinking about COVID. Particularly this last year, older respondents were much more ready to get back out in the world and do things than younger respondents and Millennials. That runs counter to at least intuitively what you’d think from health, but they were sort of more, “Let’s get on. We want to get back out there. We want to go do things!” There was far more of a cautious attitude from the kind of Gen Z, young group, which was “Wait a minute…” I don’t think that young people are being disingenuous about some of the fear they had. But the older cohorts were much more. “We gotta keep moving forward.”