16	 POLITICAL INSIGHT • APRIL 2016
I
n the months running up to last May's
General Election, opinion polls predicted
a tight finish, with both Labour and
Conservatives falling far short of an overall
majority. But on election night, the exit poll told
a very different story: Conservatives had won,
Labour was trounced. In the aftermath, the
British Polling Council initiated an inquiry into
this polling failure with the aim of discovering
what went wrong.
Table 1 shows that eight different survey
agencies understated the Conservative vote
by an average of 4.2 per cent and over-stated
the Labour vote by 2.4 per cent in their final
pre-election surveys. These discrepancies in
the polls can be explained by a wide variety
of different factors, including problems with
research designs, sampling, survey modes,
inaccurate voter registration, and also
difficulties in the recording of survey answers.
In this article I will concentrate on one factor
which may help to explain the problem; the
fact that some respondents gave misleading
answers to interviewers in the pre-election
surveys about their participation in the election.
This is described in the literature as‘overreporting’.
One of the great advantages of an
exit poll is that everyone in the survey actually
voted, since they are approached by researchers
outside the polling station.This is not true for
respondents in pre-election polls. If pollsters
interviewed a number of individuals who
claimed they were going to vote but in the end
did not, that could contribute to inaccurate polls.
Over-reporting Electoral Participation
There is a long history of survey methodologists
worrying about whether people tell the truth
For months ahead of last May’s UK General Election, polls predicted
that the outcome was too close to call. On the day, the result was
clear-cut. How did the polls get it so wrong? Paul Whiteley examines
one reason: over-reporting.
in polls. An early paper on this issue published
by Herbert Hyman in 1944 was titled simply Do
they tell the truth? and this question has been
repeatedly asked ever since2
.This is thought to
be a particular problem in surveys asking about
sensitive topics such alcohol consumption, drug
use and sexual behaviour. But it has also been a
longstanding issue about whether people vote
in elections, which arguably is not an overly
delicate subject. Despite this, there is ample
evidence to show that individuals do overstate
their electoral participation. In a report written
in 1968, which compared survey data from the
American National Election Study with electoral
records, Clausen concluded that:‘Estimates
of turnout have consistently exceeded the
population figures by about 12 to 13 per cent’1
.
In the case of the UK we can use the 2010
face-to-face British Election Study survey data
to investigate this issue3
.This was a probabilitybased
panel survey which interviewed the same
people before and after the election, and asked
about voting intentions and, subsequently,
voting behaviour. In addition, fieldworkers
verified if their respondents actually did vote
Why Do Voters Lie
to the Pollsters?
Political Insight April 2016.indd 16 04/02/2016 10:54
APRIL 2016 • POLITICAL INSIGHT	 17
in the election using records held by local
authorities.These records do not identify how
people vote, but they do show whether they
actually vote.This makes it possible to compare
what people say they will do, or have done, with
what they actually did on polling day.
The standard comparison made in studies
of over-reporting is between self-reported
turnout in the post-election survey and the
validated turnout measure. Figure 1 shows that
in 2010 about 71 per cent of the respondents
truthfully reported that they voted in the
election, while just over 17 per cent also
truthfully reported that they did not vote. In
addition, about 10 per cent claimed to have
voted but did not, and 1.5 per cent claimed
not to have voted when in fact they did. These
figures show that over-reporting is a significant
problem in surveys of UK general elections.
Likelihood to Vote
A second way of measuring over-reporting
is to examine people’s predictions of the
likelihood that they will vote before the
election compared with their validated vote
measured after the election.This is arguably
more important than the standard measure
because it aims to identify over-reporters
before the election takes place.This opens up
the possibility of doing something about the
problem in pre-election surveys. In the first
wave of the 2010 BES survey, respondents were
asked to predict the likelihood that they would
vote using an eleven-point scale (0 – very
unlikely, 10 – very likely). Clearly, if respondents
scored themselves zero on this scale and did
not subsequently vote they were not overreporting.
But if they gave themselves a higher
score than zero this makes them over-reporters.
A score of one on the likelihood of voting scale
for a non-voter represents a very modest level of
over-reporting, whilst a score of ten represents
extreme over-reporting.
Some 81 per cent of respondents who
scored themselves zero on the likelihood of
voting scale subsequently did not vote, so their
intentions were in line with their behaviour. But
the remaining 19 per cent assigned themselves
Agency Con % Lab % LD % UKIP % Green % Others % Mode N May
Opinium 35 34 8 12 6 5 Online 2960 4-5
Survation 31 31 10 16 5 7 Online 4088 4-6
Ipsos MORI 36 35 8 11 5 5 Phone 1186 5-6
ICM 34 35 9 11 4 7 Phone 2023 3-6
ComRes 35 34 9 12 4 6 Phone 2015 3-5
Populus 33 33 10 14 5 6 Online 3917 5-6
YouGov 34 34 10 12 4 6 Online 10307 4-6
Panelbase 31 33 8 16 5 7 Online 3019 4-6
Average 33.6 33.6 6.9 13 4.8 6.1
Result 37.8 31.2 8.1 12.9 3.8 6.3
Difference -4.2 2.4 0.9 0.1 1 -0.2
Table 1: Eve of Poll Forecasts for Eight Polling Agencies in
the 2015 General Election
Figure 1: The Relationship
between Actual Voting and SelfReported
Voting in 2010
Source: BES face-to-face panel survey, 2010.
Source: British Polling Council http://www.britishpollingcouncil.org/general-election-7-may-2015/.
Voters who claimed
to have voted %
Voters who claimed
not to have voted
Non-voters who
claimed not to
have voted
Non-voters who
claimed to
have voted
70.8
1.5
17.4
10.4
a score greater than zero and subsequently did
not vote, which makes them over-reporters.
Figure 2 shows that 38 per cent of these overreporters
scored ten on the scale, making up 7
per cent of all respondents. Clearly, there were
quite a few people in the 2010 election who
were really determined to vote but in the end
failed to do so.
Political Insight April 2016.indd 17 04/02/2016 10:54
18	 POLITICAL INSIGHT • APRIL 2016
There is an important distinction between
the two ways of over-reporting.The standard
measure captures the people, who for whatever
reason misled the interviewer when they
reported their voting behaviour.This second
measure does not necessarily do this, since
respondents could have legitimately changed
their minds between the first wave of the
survey and polling day. It is possible to take into
account the effect of people changing their
minds in any analysis by constructing another
scale, based on the difference between voting
intentions prior to the election and self-reported
turnout after the election.
Individuals who thought they were very
likely to vote before the election and then
subsequently reported not voting, are clearly
not trying to mislead the interviewer, but have
merely changed their minds. But individuals
who claimed both that they were going to
vote beforehand, and also that they actually
did vote, but according to the validation
exercise did not actually participate, are
clearly trying to mislead. This scale can be
used in any modelling to take into account
legitimate changes of mind between the preelection
surveys and polling day.
Why do people over-report?
There are a number of explanations for why
people over-report their electoral participation.
One is that non-voters recognise that voting
is a socially desirable thing to do and so they
claim to have participated in order to impress
the interviewer. In some cases they may
even have forgotten that they abstained and
convinced themselves that they did in fact
vote.This can happen since memory failure is
evident in the case of the 1.5 per cent in figure
1 who had forgotten that they voted. One of
the implications of this argument is that highstatus,
well-educated, and politically engaged
individuals are more likely to over-report than
low-status, uneducated and the politically
disengaged, because they are more likely to
think that voting is socially desirable.
A second explanation is that a‘spiral of silence’
may exist for supporters of some political parties.
If these supporters perceive that their party is
unpopular, this might make them reluctant to
admit to interviewers that they are going to
vote for it.The pollsters got it wrong in the 1992
election much as they did in 2015, and at that
time this was partly attributed to‘shyTories’, that
is, interviewees who were Conservatives but
who told interviewers that they did not vote.
As a consequence the Conservative vote was
underestimated in 1992, much as it was in 2015.
A similar process can occur if there is a‘spiral
of demobilisation’among supporters of a party.
In this case the party is not seen as particularly
unpopular, but some of its supporters are
unenthusiastic about voting for it.When asked
they claim to have voted for the party, but in the
end they don’t go to the polls.This may have
happened to Labour in 2015, a phenomenon
known as the‘lazy Labour’problem and if so,
it would have inflated the Labour vote in preelection
surveys.
Pressure
Another longstanding explanation of overreporting
was first mooted in the US by
Bernstein and his collaborators.They observed:
‘People who are under the most pressure to
vote are the ones most likely to misrepresent
their behaviour when they fail to do so’4
.This
idea suggests that over-reporters feel pressured
to vote by friends, family and others during
the election campaign, but at the end of the
day they do not vote.This makes them lie to
interviewers because they feel that they have
let down the people who pressured them in
the first place.This is an example of what the
psychologists Dan Arierly calls the‘fudge factor,’
where people make excuses for themselves
about low level dishonesty. US research found
that blacks were more likely to over-report than
whites, because they felt guilty about letting
down their fellow ethnic group.
There is an alternative take on the issue
of pressure.We know that the mobilising
activities of political parties can encourage
people to vote and also to change their
party in an election. By implication, such
campaigning could transform over-reporters
into actual voters and thereby reduce the
problem of identifying who is telling the truth.
This is not necessarily inconsistent with the
pressure argument, since the latter is linked
to friends, family and group loyalties, whereas
campaigning is associated with the mobilising
activities of candidates and political parties.
Some pressure works, but when it does not
people will often tell lies.
40
30
20
10
0
1 2 3 4 5 6 7 8 9 10
Scale of Over-reporting
%
Figure 2: The Likelihood of
Voting Scale Measured Before
the Election for People who
Failed to Vote in the 2010
Source: BES face-to-face panel survey, 2010.
Political Insight April 2016.indd 18 04/02/2016 10:54
APRIL 2016 • POLITICAL INSIGHT	 19
Finally, we know that if an individual believes
that a‘good citizen’should vote, this has a strong
influence on their participation in elections. If
they believe this but for whatever reason fail to
vote, they are more likely to be over-reporters.
Going against one’s normative beliefs create
pressures for dishonesty.Young people in
particular have a weaker sense of the civic duty
to vote than their older counterparts, and so
they are more willing to admit that they did not
vote. A similar point can be made about people
who perceive that big differences exist between
the political parties. If they feel this is true but
end up not voting then they are more likely to
over-report as a consequence.
A multivariate analysis of over-reporting in the
2010 General Election using the scale in figure
2 showed that there was not much support for
the social desirability bias, that is the shyTories
and lazy Labour explanations of over-reporting.
But the pressure theory, measured by an
indicator of an individual’s sense of guilt if they
failed to vote, did stimulate over-reporting.
Similarly, the perception that a good citizen
has a duty to vote was also important. In
addition, if people were contacted by the
political parties during the election campaign,
they were less likely to be over-reporters and
more likely to be voters.
Correcting Polling Errors
The findings from this analysis suggest a
methodological strategy which can be used
by pollsters to try to correct for over-reporting
in pre-election surveys in the future. Surveys
should include a likelihood of voting scale which
can be used to weight the responses. In the
2010 survey, actual voters scored an average
of 8.36 on the scale and non-voters scored
3.87, with the over-reporters scoring 7.91.This
shows that over-reporters are closer to being
voters than they are to being non-voters.This
information can be used to weight the data in
order to reduce the impact of over-reporting.
Some of the pollsters already do this, but there is
a case for trying to improve the questions used
to identify over-reporters in the surveys.
A similar point can be made about individuals
who were contacted by political parties during
the election campaign.They were less likely
to be over-reporters in 2010, indicating that
campaigning turns over-reporters into actual
voters. Again, this information can be used
to weight the data, in order to reduce overreporting.This
issue needs to be addressed
because there is clear evidence that lying to
interviewers is becoming more frequent, which
explains, in part, why the polls in 2015 were less
reliable than in earlier elections.
References
1.Hyman,Herbert.1944.“DoTheyTelltheTruth?”PublicOpinion
Quarterly8:557–59.
2.Clausen,AageR.1968-1969.Responsevalidity:votereport.Public
OpinionQuarterly32(Winter1968-1969):588-606..
3.http://bes2009-10.org.
4.Bernstein, Robert, Anita Chadha, and Robert Montjoy. 2001.
“Over-reporting Voting: Why It Happens and Why It Matters.”
Public Opinion Quarterly 65:22–44.
PSA to hold Annual
Assembly at Brighton Conference
The PSA is changing…
We’re opening up and looking to involve all our members in how we’re run.
Our first ever Annual Assembly is an opportunity for members to feed into the work of
the PSA and drive the future of the Association. Whether you have concerns, ideas, or
just thoughts in general about the current state of YOUR professional association, we
encourage you to come along and participate in this unique open forum.
The PSA is your professional body, so join us for a lively discussion…
The event will take place on 22nd
of March 2016 from 5pm – 6:30pm at the
Hilton Brighton Metropole where drinks and nibbles will be served.
Paul Whiteley is professor of politics in
the Department of Government at the
University of Essex.
Political Insight April 2016.indd 19 04/02/2016 10:54