A poll recently conducted by NBC/WSJ from September 16-19 purportedly showing strong opposition to Brett Kavanaugh’s confirmation to the Supreme Court has several flaws and is yet another example of the news media trying to soften public opinion toward Kavanaugh’s accuser.
In essence, the poll is completely meaningless.
First, the poll surveys “registered voters.” As a matter of accepted practice, registered voters are interesting, but they are not a good gauge for what voting public thinks. No poll worth its salt would survey registered voters to get at actual public opinion of those who go to the polls and choose our elected representatives. In order to know what voters think, a survey should measure “likely voters.” This poll does not do that.
Second, the poll oversampled women. The country has 51 percent, not 52 percent women, as this poll surveyed. The single percentage point in their favor can skew a poll and generate a false result, or an inaccurate measurement. One percentage point difference in gender may skew a poll several points.
Third, the poll included 44 percent Democrats and only 36 percent self-identified Republicans. As a result, the poll was most certainly skewed in favor of the accuser over a Republican nominated justice. Interestingly, there may be more shenanigans at work in the poll as another 8 percent of the respondents did not identify either as Republican or Democrat, which leaves open the possibility that those surveyed in those categories are leftists, and hence predisposed to oppose whatever nominee comes from a Republican president. We do not know this, however, because this poll does not tell us.
Finally, the poll withholds not only many of the questions asked to those surveyed, but against proper social science procedure, it hides its methodology. Both of these things together should cast suspicion on a poll that claims to measure what the public is thinking.
Every reliable pollster knows that the type of questions asked, and the order they are asked, can have a profound impact on the outcome of a poll as the fabulous series “Yes, Prime Minister” brilliantly demonstrated. Therefore, it is troubling to say the least, that about half the questions of this poll are kept from the public view.
We should also note the time the poll was conducted as an additional measure of unreliability. The survey was conducted from Sunday to Wednesday. Usually weekends are awful measures of what the public thinks because the poll does not survey all it could capture randomly as people on the weekend are out and about taking care of family chores, or having a weekend holiday. But, in this instance, that amounts to one day.
The rest of the week could be a reliable measure barring anything major occurring like a weather event. In this case, there was a weather event—the aftermath of Hurricane Florence. Several states were preoccupied or otherwise unable to receive a phone call during this period. As a result, there were potentially hundreds of thousands of people left out of the chance of being surveyed in this poll. That alone can skew a poll in a direction that would be unreliable. In this case, several Republican states were unlikely to be surveyed.
There are very few polls that are accurate in the age of Trump. In fact, most polls in the last election got it wrong, save Trafalgar. Too often, polls are now meant to try to push voters in the direction the news media wants it to go. Reporters lazily build into their own stories polls that confirm their own political opinions. This is what we call confirmation bias. Just because there is a poll confirming our political ideology does not make the opinion, or the poll, true.
In 2016, we found that most every poll failed to capture the state of the electorate. This poll does nothing to restore confidence that we know anything about the public mind, save actual voting.