It’s election season again, which is why I know we’ll soon be inundated by polls.  Polls undoubtedly cover the exact wrong thing in politics—who is winning. The goal of a political process is not to “see who is winning” but to choose whom we would like to see in the office.  Every moment spent covering polls is a moment not spent covering what is really at issue — which candidate will make the best decisions for the country as a whole?

Why do journalists spend so much time reporting on polls?

Polls allow journalists to appear objective and scientific while doing the least amount of work.  It takes serious research and investigation to determine the actual short and long-term effects of different policy positions.  It takes time and effort to reconstruct the past — both the scandals and the successes.  Even harder to research are the nuanced successes where there isn’t a clear answer as to whether everyone was better off in the end.  All of these require both effort and risk on the part of the journalist.

With polls, however, the journalist can report seemingly objective facts with no fear of being contradicted.  The results of a poll are objective and factual, almost scientific.  As normally happens in real life, though, the things that are most objectively factual are usually fairly irrelevant.  The real meat of the issues are never as clear-cut. 

If someone asked me about how my family vacation went, the most objective thing I could tell them was the gas mileage for the trip. But I doubt that sharing that information with my friends would help them get a feel for what we encountered in our expedition.

The problem with polls

Unfortunately, polls not only replace real coverage of election candidates, they also dictate who is covered by the news.  If a candidate isn’t polling well enough, then the news doesn’t include their views in their coverage.  I can see this being valid in reverse — if a candidate is polling high enough, then the wider public deserves to know what their views are.  But using poor polling results as a reason not to consider certain viewpoints is a lazy way to choose coverage.

Additionally, the polls are never presented without comment.  The interpretation of polls is wildly subjective, yet that is usually the focus of the story.  Are we more interested in the current percentages or in their changes?  Are we going to compare the candidates directly to each other or to how they are doing compared to a specific landmark?  The degree of freedom in reporting means that a supposedly objective poll can be used to report the opinion or feeling that the reporter wants to create as if it were an objective fact.  That, coupled with human nature’s desire to be “on the winning team” means that polls are not only irrelevant to the political process, they actively distort it.

Could the polls be manipulated?

Because polls are so decisive in media reporting, one must ask:  Can a candidate who wants to make news force a bump in the polls which would generate a series of stories “moving up in the polls” stories? Former congressional staff investigator Rex Sparger probed that question in the 1980s in a little-known experiment. 

Few people today remember Rex Sparger.  In addition to writing a number of largely-forgotten books, he was at the center of a controversy involving the Nielsen rating service in the 1960s.  Originally part of a commission started by representative Oren Harris to better understand the impact of the ratings service on commerce, Sparger branched out to show how the ratings service could be manipulated. 

He wound up being too successful for his own good — the ratings bump he gave to the show An Evening with Carol Channing was so large that Nielsen recognized that there was a problem, and eventually slapped Sparger with a $1,500,000 lawsuit.

In his next escapade, he decided to see if he could rig the polls that the Los Angeles Times was doing.  He found out that, for as little as $300, he could take control of about 17 percentage points in some of the polls.  In his own experiment, he did this by hiring people to work at the pollster’s office.  Being a pollster is not an incredibly lucrative job, so having someone who pays you an additional salary to influence your results goes pretty far.  His system was to have his planted pollsters call one of his numbers first, and then he would give them the answers that he wanted. The degree to which he stressed his opinion determined the degree that his planted pollster would sway the results.

How polls can be manipulated

While this exact mechanism of poll rigging probably can’t be reproduced today, Sparger points out several other ways that polls could be rigged just by having inside information into the polling trade.

 Many polls which measure how public opinion changes do so by following up with the same interviewees later.  Because their names and addresses are known ahead of time, a marketing blitz could be targeted at them.  In-person polling can also be rigged if you know where the pollster is going to be.  Even just knowing the sample design for a survey can help a campaign statistically target people who are more likely to be a part of a poll.  More sophisticated tricks are possible, depending on just how far the manipulator is willing to go.  The goal, generally, is to make a change that is small enough to be believable, but large enough to get reported on and to get people excited or interested about the issue or the candidate.

We know that both political and corporate espionage occurs. Just in the past week, two separate indictments were handed down for foreign influence peddling in the US — one from Russia and one from China.  Sparger has shown just how little it takes plant people in polling operations to put a thumb on the scales to generate headlines.

The polls miss the big story

Polls have built-in limitations, which is precisely what allows us to analyze them numerically. They rarely investigate why people believe or behave the way that they do.  Of course pundits will comment on the why, but that isn’t actually part of the polling data.  You must know real-life individuals to get a sense for why things are shaking out the way that they are.

As we’ve previously reported, there is a growing divide in many areas of life between “wisdom” (having people in the know making decisions) and “the algorithm” (making decisions purely based on data and its processing).  The problem, as we pointed out, is that data has many limitations that we rarely think about. 

Data tends to give a good first-order model of reality and provides short-term predictive capability.  However, the predictive ability of such first-order models tends to tank on a longer timescale when second-order effects come into play.

There could not be a more obvious instance of this than in political polling and commentary.  Love him or hate him, it is an undeniable fact that Donald Trump has been the center of American political interest for almost a decade now.  But the question is, was the rise of Trump predicted by “wisdom” or “the algorithm”?

In the modern era, politics are absolutely dominated by “algorithmic” and data-oriented thinking.  Old-style “wisdom” thinking has been out of favor for some time.  At the top of the game for the algorithmic political punditry is Nate Silver, founder of FiveThirtyEight and the Silver Bulletin.  To find someone more in the “wisdom” column, we have to go back to a bygone era, and find Dick Morris, who was the campaign manager and political advisor for the Clintons.

How did the two fare in predicting the next wave of presidential politics?  While most people have forgotten this incident, in 2012, Dick Morris predicted that Trump would enter presidential politics and win.  He was highly derided for this at the time, with some calling him the “world’s worst political pundit”.

So what did the “algorithm” side of the fence predict?

In a tweet on May 16, 2011, Nate Silver called Morris’s prediction a fail.  He even reiterated the point in his book, The Signal and the Noise (Penguin 2015), where he called it “horribly wrong.”  It was true that Trump didn’t run for president in 2012, but I would say that mis-predicting the timing of Trump’s entrance onto the political scene is a much better call than missing it entirely.

Again, for first-order effects, the “algorithm” view tends to be the way to go.  But the algorithm misses second-order effects almost entirely.  Not because it is wrong about them, but because it is completely blind to them.  What poll could Nate Silver have run in 2011 to predict Trump’s entrance into politics?  By being beholden entirely to the algorithm, Silver missed the larger undercurrents of what was happening.

What to do about polling?

So what should we do about polling?  At the end of the day, we need to end our societal obsession with polling data.  There are some good uses of polling data, but they are mainly for politicians measuring their impact.  Basing news coverage on polls, or changes in polls, is likely to lead to either manipulation by candidates or opinions on polls masquerading as fact. We need to get back to the business of politics—looking at people, issues, policies, and results.  Until we do that, all we will be doing is hearing the distorted echo of our own voices.