Polling's Failures Have Led to Improved Techniques

(Bloomberg View) -- Back in late 2016, following a year of political upsets, I argued here that political polling and forecasting wasn’t dead but needed to adapt. Fast forward a little over a year, and change is already apparent.

Polls in the U.K. badly missed in the 2015 general election, showing a tied result when then Conservative Prime Minister David Cameron’s party emerged with a six-and-a-half-point margin of victory. They wrongly predicted a Remain victory in the 2016 referendum on European Union membership, and were wildly off-target again in the 2017 general election. In the U.S., state polls — particularly in states where whites without college degrees are numerous — missed key swing-state victories for Donald Trump in 2016. 

Although the magnitude of polling misses hasn’t changed much over time, the number of close, high-stakes contests has increased the risk of a bad call on a major vote. This matters especially in the U.K., because in a winner-takes-all electoral system (as opposed to one that uses proportional representation) even modest errors in the popular vote translate into much larger errors in the share of seats won.

It’s against this backdrop that Britain’s legislative upper chamber, the House of Lords, convened a committee on polling and digital media, which delivers its report Tuesday. The Lords report follows an industry-led inquiry that was initiated after the 2015 polling debacle. Its main conclusions were that poll samples were unrepresentative and contained too many politically engaged respondents. The report’s recommendations — which cover both methodology and the way polling is communicated — have yet to be implemented in full.

Given the stakes, it’s not that surprising that London is at the forefront of the evolution in polling. New players with more sophisticated analytical and polling methods have led changes in the industry (Number Cruncher Analytics Ltd, of which I’m owner and founder, is among them). A common theme across the board is a desire to address the central failing of unrepresentative polling samples identified in the industry report.

Between those of us chasing the perfect poll, solutions to the problem vary. One relatively new entrant, Politech Ltd, uses instant messaging platforms such as Facebook Messenger rather than email to communicate with its respondents, and aims to incentivize them with polls on topics that interest them, and also shows them how their views sit with the wider population, which is a way of rewarding them for their participation.

Another new entrant, Deltapoll Ltd, uses "implicit response testing" — a way of measuring emotional connection with policies and issues. The focus is not simply what the voter thinks, but what they feel and how strongly they feel it. This is of particular interest when head and heart appear to conflict with one another.

Number Cruncher, which is already established as a polling consultancy to financials, corporates and governments, has begun fielding its own polls in-house. Its techniques include both non-political survey panels and more direct approaches, such as polling visitors to non-news-related partner sites at random and recruitment over social media, to reach a more representative audience — instead of simply polling those who follow the news closely or join political polling panels to share their opinions.

Some of the initial results are promising. Number Cruncher’s first published poll asked whether and how people voted in the 2017 U.K. election and also in the 2016 Brexit referendum. The results closely matched those of the academic surveys — the type that require months of door-knocking and a budget in the hundreds of thousands of dollars following votes — including among the notoriously elusive and politically disillusioned under-25 age group.

Achieving more representative samples has obvious benefits for political polling. But since politically attentive people also tend to consume more news, sampling them in the correct proportions can benefit other types of research, too. A corporate tracking its brand reputation — or for that matter, a competitor or equity analyst tracking it — needs the confidence that they’re hearing from a reliable cross-section of the market, not one that overrepresents those likely to have followed the latest scandal or product launch in the news.

But it’s political polling that remains the highest-stakes venture for researchers, because it’s one of the few areas where their work can be measured against an observed outcome — an election result. On that score, the industry has a ways to go yet. The real test of would-be disruptors like my company is whether we achieve increased accuracy in future elections.

This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.

Matt Singh runs Number Cruncher Politics, a nonpartisan polling and elections site that predicted the 2015 U.K. election polling failure.

To contact the author of this story: Matt Singh at matt@ncpolitics.uk.

To contact the editor responsible for this story: Therese Raphael at traphael4@bloomberg.net.

For more columns from Bloomberg View, visit http://www.bloomberg.com/view.

©2018 Bloomberg L.P.