An Overview &
Introduction to Polling
& Public Opinion Research
MU MVZ 449
Spring 2010
Dave McCuan
Masaryk University
Class Meetings, Rest of the Semester
* Here's the schedule for the rest of the term:
­ 07 April: Public Opinion Measurement & Polling
­ 14 April: Who Votes, How They Vote, What They Want
­ 14 April: Release of Midterm Exam. Due TWO WEEKS
LATER.
­ 21 April: Media Effects & Media Role in Elections
­ NO CLASS: Wednesday, 28 April 2010
­ MIDTERM DUE by 4 pm, Department Office Mailbox
­ 05 May: Media, continued & Changing Demographics of
Elections
­ 12 May: Campaign Finance Systems
­ 19 May: Wrap-up, Campaign Effects Thesis / Final Exam
released
Table 1. Presidential approval ratings: Predictors & Effects, 19Table 1. Presidential approval ratings: Predictors & Effects, 19Table 1. Presidential approval ratings: Predictors & Effects, 19Table 1. Presidential approval ratings: Predictors & Effects, 1959595959----2008200820082008
N=579N=579N=579N=579
What We're Seeing With Polls Today Source: UVA, Center for
Politics, April 2010
Gallup Poll, Released 06 April 2010
What We Cover:
* Measuring change & opinion among agents of socialization
* Why is measuring public opinion more important?
* Who are involved in measuring public opinion?
* How do we measure public opinion & Sampling Theory?
­ Telephone polls
­ In-person interviews
­ Focus groups
­ Internet polls
­ Television and radio monitoring systems
* How do we design samples for public opinion measurement?
­ Sample designs
­ Weighting of data for non-response
­ Special populations
What We Should Consider Here:
* How do we design poll questions?
­ Election polls
­ Issues polls
* How do we interpret poll results?
* How do we respond to the controversies in polling
­ Accuracy
­ Inappropriate use
­ Misrepresentation of results
­ Ethical concerns about collection of data
* Special issues?
­ Polling internationally
­ Polling on race
­ The media's use of polls
Why Election Polling is More Important?
* Much larger and more diverse voting population
* Much less loyalty to political parties
* More interest in issues
* Decline of interest in election campaigns
* Growth of "floating" voters
* Possibility in some countries of government "stealing" the
elections
Why Issues Polling is More Important?
* Much larger and more diverse population
* Decline in trust in political leaders
* Decline in trust in government decision makers
* Decline in trust in experts and authority figures
Final Public Opinion Polls: Pre-Election 2000
Candidate
Final result
Bush
48%
Gore
49%
Nader
3%
Buchanan
0%
Lead
-1%
Reuters/MSNBC 46% 48% 5% 1% -2%
CBS 47% 48% 4% 1% -1%
Fox 48% 48% 3% 1% 0%
Harris 47% 47% 5% 0% 0%
CNN/USA Today/(Gallup) 48% 46% 4% 1% 2%
Washington Post 49% 46% 3% 1% 3%
Newsweek 49% 46% 5% 0% 3%
ABC 49% 46% 3% 1% 3%
NBC/WSJ 49% 46% 3% 2% 3%
Sources: AEI, PollingReport.com, WP
3%2%3%46%49%NBC/WSJ
3%1%3%46%49%ABC
3%0%5%46%49%Newsweek
3%1%3%46%49%Washington Post
2%1%4%46%48%CNN/USA Today/(Gallup)
0%0%5%47%47%Harris
0%1%3%48%48%Fox
-1%1%4%48%47%CBS
-2%1%5%48%46%Reuters/MSNBC
Lead
-1%
Buchanan
0%
Nader
3%
Gore
49%
Bush
48%
Candidate
Final result
Butterfly Ballot Example ­ Palm
Beach County, FL (2000)
Recall Our Agents of Political Socialization
* SES determinants
* The Family - monopolizes early years, evidence suggests
parents influence children's beliefs.
* Mass Media - provides political information.
* Schools - Civics courses, history, this Academic Dept, student
government.
* Friends and Colleagues - Who we surround ourselves with
influence our political beliefs.
* Church - Church goers are likely influenced by others in church
and politically relevant beliefs in church.
What is Political Socialization?
* The process through which an individual acquires his
or her particular political orientations­ his knowledge,
feelings, and evaluations regarding his or her political
world.
* Often includes "socio-political" attitudes, opinions
related to "culture wars" have come under study
relative to political socialization.
Public Opinion
* Democracy: A method of selecting policymakers
and of organizing government so that policy
represents and responds to citizen preferences.
* Given this definition, why is public opinion important
in the United States?
* How well does American democracy respond to
citizen preferences? Depends on the institution, but
most scholarships suggests that the US government
is quite responsive.
* Electoral forces (shifts in who governs and
anticipated responses) tend to be the key to this.
Contradictory Expectations of the Public
* We want the president to be a leader vs. a responsive
president.
* Flexibility and willingness to bargain vs. Decisive, firm
and consistent principled stands.
* An open administration vs. a president in control.
* An average person that can identify with typical person
vs. Someone who's a cut above, different, virtuous.
Problems with Polls
* Are politicians leading or following the polls? Evidence suggests
both. Government hardly insulated from public opinion, as
Founders designed.
* Many polls aren't scientific. Many are, however.
* Findings may change due to question wording & question
ordering.
* People draw conclusions from polls that aren't really in the poll.
The Public Opinion Poll
* Polls are used to identify the distribution of the population's
beliefs about politics and policy.
* Developed by George Gallup and others in the 1930s.
* Scientific polling uses Samples, which include a small proportion
of the population selected to represent the population as a
whole.
* Important polling concepts: Random sampling and Sampling.
The World of Polling: Focus
Public
Demographic
Groups
Culture
Public
Readers and
Viewers
Elites
Media
Public
Likely
Voters
Election
Predictions
Actual
Voters
Politics
Public
Elites
Interest
Groups
Public Policy
Polling Community
Key Historical Events in Polling
* (1935) First national newspaper policy poll/Gallup/Washington Post
* (1936) Literary Digest fiasco
* (1936) Gallup/Roper/Crossley victory
* (1940) First presidential use of polls for policy / FDR
* (1948) Truman polling disaster
* (1960) First presidential candidate use of pollsters
* (1962) First election projections/CBS/Harris
* (1990) First unified network exit poll
* (2000) Polling calls, Bush vs. Gore, General Election
The Nature of American Public Opinion
* The Public's knowledge on issues is extremely low. Most
people are dumb when it comes to policies and issues.
* Individuals rarely hold stable opinions.
* Public opinions often contradict and can be difficult to interpret.
* Policies are complex; people don't spend the time to
understand them.
* Collective opinion IS rational fluctuating predictably with events
and the economy.
How People Form Opinions
* Opinions are often based on our own core values, but fluctuate
(w/ new information) much more than our values.
* People use heuristics (or short-cuts) to help form their opinions.
Often take the form of environmental cues including:
­ political party (Party ID ­ PID ­ often influences our opinion)
­ What's salient in the media?
­ Conversations with friends, colleagues, etc.
Political Ideologies - Caveats
* Usually discussed on a liberal to conservative scale.
* The scale only captures one dimension. Ideology has more than
one dimension.
* Conservative, Liberal, Libertarian, Communitarian
* Types of voters based on ideology (The American Voter):
­ Ideologues (about 10%)
­ Group Benefit Voters (about 40%)
­ Nature of Times voters (about 25%)
­ No issue content voters (about 25%)
Political Ideologies
* Ideology is a difficult subject, because most Americans do
not fit on the typical ideology scale, of left to right.
* Ideology defined: A consistent relation between individual
opinions.
* Liberals: Believe in a strong role for government in
regulating equality. A weak role for government in regulating
order.
* Conservatives: Believe in limited government in regulating
equality, strong gov't in regulating order.
The Left-Right Continuum, on size of
government
Liberal Conservative
Moderates
Typology of Ideologies
Equality
Freedom Order
ConservativeLibertarian
Liberal Communitarian
The Distinct American Ideology
* Based on the conservative belief of Limited
Government. Americans are suspicious of
government, thus they limit government's capacity
to govern.
* Two elements of this ideology:
­ Individualism: liberty or freedom for autonomous
individuals. Freedom from government intrusion.
­ Equality: specifically, equality of opportunity as opposed
to equality of results.
Intro to Survey Sampling Theory
The intuition behind the science
Question: How can the views of 1,000 individuals
chosen at random accurately depict the views of
280,000,000 Americans?
George Gallup's answer (paraphrased): You don't
have to eat an entire stockpot of soup to know if it
you like it ... you only need to have a taste.
Survey sampling is the "science of tasting."
From population to sample
Non-probability sampling
* Accidental, haphazard, or convenience
* Purposive
* Volunteer subjects
* Quota
* Snowball
Probability sampling
* Simple random
* Stratified random
* Systematic
* Cluster (also area, multi-stage)
Simple random sampling Systematic sampling
Stratified random sampling
Exit poll methodology (CBS)
* State-by-state sample selection
* 2-stage probability sampling
* Selection made proportionate to size, stratified
by geography and past vote
* Past race used for size and party vote
* Used in 2000 election with mixed results
(from Frankovic, 2/9/02)
Interpreting survey results
Question: How can the views of 1,000
individuals chosen at random accurately depict
the views of 285,000,000 Americans?
Sampling theory
N = ?
m = ?
= ?
n
s
X
Population
Sample
The key question is: how can we make an inference back to the
population from the sample? We are not interested in the
characteristics of the sample, only the characteristics of the population.
sampling properties
?
Statistical Inference Statistical Inference
Sampling Error Sampling Error
How large a sample do I need?
3,4573841380.1
3,4573841380.9
6,1476832460.8
8,0678963230.7
9,22010243690.6
9,60410673840.5
9,22010243690.4
8,0678963230.3
6,1476832460.2
0.010.030.05proportion
Desired Margin of ErrorPopulation
Diminishing returns to sample size
What else determines sample size?
Interpreting polls correctly
4%600
7%185
16%40
0.4%52,000
1.4%5000
3%1000
margin of errorsample size
3559Latinos
1188blacks
4651Women
5245Men
54
42
44
50
49
48
Dem
4645-59
4930-44
4818-29
42Asians
55whites
5460+
Rep
Americans Following the Results
of Polls in the Media
* Cantril, Gallup
24% 31%
16% 25%
19% 9%
0% 20% 40% 60% 80% 100%
1996
1985
1944
Occasionally Regularly
28%28%28%28%
41%41%41%41%
55%55%55%55%
How Much Attention Should Elected and
Government Officials Pay to Public Opinion
Polls
2%
5%
18%
39%
36%
Don't know
None at all
Not too much
A fair amount
Great deal
Brodie et al. Public Perspective (July/August 2001)
Do Polls Accurately Reflect What the
Public Thinks?
3%
11%
53%
28%
5%
Don't know
Hardly ever
Only some of the time
Most of the time
Just about always
Brodie et al. Public Perspective (July/August 2001)
Some Cultural Differences between Polls
and Traditional Surveys (1)
Purpose
* To measure public response to current events or political
issues than can change quicklychange quicklychange quicklychange quickly
* To provide data for predicting election outcomes
Some Cultural Differences between Polls
and Traditional Surveys (2)
Operational Issues
* Polls conducted over shorter periods of time
* Frequent measures seen as very important
* Concerns about fairness in design and presentation of
paramount importance
* Costs of surveys a significant issue
* Response rates are a greater problem
* More difficult to "validate" the results
Summing Up
* Survey research is an excellent example of descriptive inference
in the social sciences
­ Random sampling helps convince us that the cases we are looking at are
similar to the whole universe that we care about
­ Research on the psychology of survey response has helped us develop
rules that help us write questions that will actually measure our concepts
­ Whenever you run into an example of descriptive inference in the social
sciences, compare it with survey research to see what its strengths and
weaknesses are
* Experimentation is the best example of causal inference in the
social sciences
­ Random assignment to treatment and control groups rules out confounding
variables and helps ensure internal validity
­ Field experimentation can help ensure external validity as well
­ When you see a causal inference in the social sciences, compare the
methods behind it with experimentation to see if it is credible
­ ALWAYS visualize your data & inform your analysis based on "the real
world."
Part II
Questionnaire Design in Surveys & Polls
MU MVZ 449
Spring 2010
Dave McCuan
Masaryk University
Why Care about Survey Questions?
Need to be a critical consumer of polling data
Many Americans trust no polls, but many others
trust them far too much
Separate the good from the bad
Information must be reported to the consumer
Sources of Error
1. Measurement error (bias) ­ instrument
2. Sampling error ­ finite sample
3. Sampling bias ­ unrepresentative sample
Error: random missing, but right on average
Bias: wrong on average
Error usually known, bias usually unknown
Questionnaire Design
Questions themselves contribute to
measurement error
Wording, order, coding of responses, context
Considerations: time is money, population is
diverse, interviewers are part-time workers
Ask Precisely
Do you agree or disagree that people have a
constitutional right to own guns and the
government should not regulate gun
ownership?
Problem: Double-barreled (and confusing)
Priming
Do you think the United States should let Communist newspaper
reporters from other countries come in here and send back to
their papers the news as they see it?
Do you think a Communist country like China should let American
newspaper reporters come in and send back to America the
news as they see it?
Problem: Question order causes priming
(percentages go up by 20 points when American question #2) is
asked first)
Careful with Comparisons
Please tell me whether you agree or disagree with this
statement:
Individuals are more to blame than social conditions for
crime and lawlessness in this country.
Social conditions are more to blame than individuals for
crime and lawlessness in this country.
Question Order Matters
57%40%Conditionsmore to
blame?
43%60%Individuals
Who is
Conditions
First
Individuals
First
Which in your opinion is more to blame for crime and lawlessness
in this country ­ individuals or social conditions?
Multiple Parts
Taken altogether, how would you say things are
these days: would you say that you are happy,
pretty happy, or not too happy?
Taking things all together, how would you
describe your marriage: would you say you
marriage is very happy, pretty happy, or not too
happy?
Part-Whole Effects from Question
Order
38%52%"Very Happy"
in general
63%70%"Very Happy"
marriage
Marriage
First
General
First
Other Question Order Effects
* Political interest (self-reported)
* Party identification
* Presidential approval
Purposes: Control versus Realism
Which of these is the most important problem
facing the country at present?
What do you think is the most important problem
facing the country at present?
Open-Ended v. Closed-Ended
0%3%Welfare
13%13%Inflation
16%35%Crime/violence
Closed-endedOpen-ended
Don't Know? as an Out!
Do you think that quite a few of the people running
the government are crooked, not very many are,
or do you think hardly any of them are crooked?
Problem: No filter to screen out "nonattitudes"
4% DK as is ­ "or do you not have an opinion on
that" increases DK to 19%
Defining Response Categories
Should divorce in this country be easier or more
difficult to obtain than it is now?
Problem: No middle position available
Should divorce in this country be easier to
obtain, more difficult to obtain, or stay as it is
now?
Importance of a Middle Position
4%5%Don't know
40%22% (vol)Stay as is
33%45%More difficult
23%29%Easier
OfferedOmitted
Balance
Are you in favor of the death penalty for persons
convicted of murder?
Do you think that smut-peddlers should be free to
sell their filthy pornography to our children?
Problems: Leading, unbalanced
Compare "welfare" and "assistance to the poor"
Limits
The Supreme Court recently ruled that
congressional district boundaries may not be
drawn solely on the basis of racial
considerations. Do you agree or disagree with
this decision?
Problems: Complexity, acquiescence to authority
figure
Touchy Subjects
Did you vote in the 19xx or 20xx presidential
election(s)?
Problem: Social desirability
In talking to people about the election we often find
that a lot of people weren't able to vote because
they weren't registered or they were sick or they
just didn't have time. How about you, did you
vote in the elections this November?
Can you ask this?
How many times did you use heroin last week:
not at all, once or twice, three to five times, or
more than three times?
When did you stop beating your wife?
Conclusion
At a minimum, respondents should know the wording, response
categories, and coding rules used or at least have some
understanding of what is being asked.
FYI: http://wikileaks.org/
Document dated, "26 March 2010" and found at:
http://file.wikileaks.org/file/cia-afghanistan.pdf