BES reveals longstanding flaw in opinion polls

Flaws which made the opinion polls wrong in 2015 also explain errors in 2010, new research from the British Election Study (BES) reveals.

BES researchers Dr Chris Prosser from The University of Manchester and Dr Jon Mellon from The University of Oxford last year revealed that sampling was the reason for the 2015 polling miss, confirmed in Tuesday's Independent Inquiry preliminary report.

Now they show the same mechanism of over-representing groups such as young people who were less likely to vote, is also likely to have been at least partially responsible for the 2010 over-estimation of the Liberal Democrats' vote share.

The flaw, they say, was also likely to have been present in the 2001 and 2005 general elections, but the electoral maths meant the polls correctly forecast which party would form the Government, despite overestimating Labour's share.

Dr Mellon said, "We have shown the 2015 polls were fundamentally unrepresentative of the electorate and that Labour leaning respondents were the most over-represented as voters in the sample.

"Our new research shows that this problem is a longstanding issue: in 2010 these over-represented respondents were Liberal Democrat voters, which could potentially explain the 2010 polls' over-estimation."

In 2010, the polls over-estimated the Liberal Democrats by around 4.0 percentage points, but no reason was found for the error.

The BES findings now suggest the error was down to sampling voters from demographic groups with low turnout demographics, who tended to support the Liberal Democrats.

Dr Chris Prosser said: "The surprising surge of support for the Liberal Democrats among low-turnout demographic groups, such as students, may in fact have saved the pollsters from a 2015 polling miss in 2010."

The academics have devised a way of correcting the flaw, so pollsters can in future create more accurate forecasts by factoring in additional weighting to account for expected proportions of non-voters.

By adding an additional weighting factor of the expected proportion of non-voters, respondents from demographics which vote less would be more represented in the non-voting section of the general population sample.

That means they will be less represented as voters – improving accuracy.

"While we do not know exactly what the turnout will be in a given election, we do know that it is likely to be reasonably close to the previous election's turnout," said Dr Mellon.

"But further work is needed to determine whether this method will be sufficient."

The BES face-to-face survey, carried out by GfK between May 2015 and September 2015, was a random probability sample of 2,987 people.

It showed a closer match to the 2015 general election Conservative lead over Labour than the pre-election polls, which failed to forecast the scale of the Conservative victory.

Analysis of the data by the BES team suggests that this closer match was made possible by using probability-based sampling methods and successful targeting of hard-to-reach groups, including non-voters.

Dr Mellon said: "We believe these findings have serious implications for pollsters both in the UK and in other countries.

"Weighting to population targets and then removing non-voters is by far the most dominant election polling approach, and any pollster that uses it risks the same kind of polling miss that Britain saw in 2015.

"Given the increasing number of polling misses throughout the world, we may already be seeing the results of this methodological choice."