~I
8 INTRODUCTION
office staff to the social environment in a village or on a reservation. Related,
virtual forms of exchange may limit the presence of interviewer effects, generated
when the researcher's personal characteristics influence the type ofinformation
gathered. Tills could be either a help or a hindrance to the research process.
Second, it is more difficult for the researcher to establish rapport with the
subject from afar, and this can limit the depth and accuracy ofthe information
offered. Third, the ethical considerations that arise during in-person interviews
certainly are present for those conducted electronically. In some ways,
the risk to an informant may be greater ifher responses are recorded and can be
forwarded (as in an e-mail exchange), or ifthere is a possibility that her phone
or Internet connection is not secure. Given these considerations, political scientists
continue to rely largely on in-person interactions; when virtual modes
are employed, they can be most useful in the context of an initial interaction (a
preview of an in-person interview), or a follow-up interview (for which context
and rapport have been established already). This distinguishes political
scientists from journalists, who routinely rely on virtual means of communication
for interacting with informants. For journalists, the practical demands
of much shorter time-horizons increase the appeal of new technologies. For
political scientists, new technologies sometimes are useful, but they come with
some important drawbacks. While we certainly recommend attention to ethical
and research design issues when nontraditional modes of interviewing are
employed, we retain a focus on fieldwork that generally involves travel to the
research site.
Furthermore, changes in technology may reinforce the importance of interviews
to answering research questions. Technology renders many other types of
evidence, such as transcripts of hearings or records of campaign contributions,
more easily available. Yet this increased transparency also may lead those involved
in the political process to move their activities out of the limelight. For
instance, in their discussion oflegislative politics, Beckmann and Hall argue that
interviews have been rendered more important as a research tool by the increase
in information availability that is a hallmark of the Web 2.0 era. As records of
formal legislative activity have become more readily available to journalists and
the public, legislators and their aides have moved more oftheir efforts behind the
scenes. Interviews may be the only means of gathering data on the informal behaviors
that lead to political outcomes. Technological change notwithstanding,
then, "talking to people" remains a central means ofproducing outstanding and
innovative political science scholarship.
INTRODUCTION 9
The Political Scientist as Interviewer
The unique features ofinterviews offer many opportunities to political scientists,
but they also require that we carefully address certain issues. Interviews are used
in a wide variety offields, including public health, sociology, economic and social
geography, psychology, history, and anthropology. Indeed, in the absence of research
method guidance that is specific to political science, graduate students and
faculty often rely on advice based in other academic disciplines. But the features
of political science interviewing-both in terms of the epistemological orientation
ofthe discipline andthe interviewer's relationship to interviewees-combine
to create specific considerations for scholars of political science. I discuss these
two issues in turn.
Epistemological Considerations
One can classify scholars of political science along an epistemological continuum
that ranges from positivist to interpretivist. The positivist ;view acknowledges
that while the subjects of social science research are perhaps messier than
those of natural science research, the social researcher should aim to identify
patterns of cause and effect (Steinmetz 2005). Based on these patterns and on
theoretical reasoning, the researcher should develop falsifiable hypotheses and
test these hypotheses empirically. While admitting the possibility of some slippage
between theoretical concepts and their empirical operationalization, a positivist
orientation assumes that qualitative-as well as quantitative-methods
can facilitate the discovery oftruths.
An interpretivist viewpoint, On the ~ther hand, treats the world as socially
made; knowledge is impossible to separate from historical context and power
relationships. While some interpretivist social scientists aim to make generalizations
or to generate causal explanations (Wedeen 2010), many focus instead on
causal understanding-on developing knowledge about how subjects understand
their own actions and circumstances, and on how this understanding is conditioned
by power and social relations. When interpretivist scholars employ ethnographic
methods (as many do), they are sensitive to the difficulty of separating
the collection and processing of interview data from the individual researcher's
circumstances and knowledge. Indeed, political ethnography-while itself encompassing
a diverse set of approaches and subjects-is marked by the use of
participant-observation, an attempt to un~erstand interactions from the perspective
of an insider, and a desire to develop a "sensibility" about the context in
which one is immersed (Schatz 2009b). Kuhn (1970, 113) also advances such a
claim: "what a man sees depends upon both what he looks at (observations)
10 INTRODUCTION
and also upon what his previous visual-conceptual experience has taught him
to see.''5
Those who work from a largely positivist tradition treat interviews as a
means of generating objective knowledge, either to generate or test falsifiable
hypotheses. For these scholars, interviews serve to identify the causal processes
that generate specific outcomes, and-when used for theory testing rather than
for theory development-to allow one to differentiate between alternative hypotheses.
Although positivist scholars are sensitive to the existence of "interviewer
effects"-in that their individual characteristics, and how these are perceived by
their interviewees, may influence the information that is provided-their focus
is more on interview data as a product, often collected over a relatively short period
oftime.
Scholars operating from an interpretivist stance-currently more common
in anthropology or social geography, but also represented by some contributions
to this volume
6
-doubt the extent to which a purely objective social science is
possible. Therefore, while these scholars may employ interviews in service of
broad social science aims (including testing falsifiable claims), they highlight the
need for attention not only to information itself, but also to how, and by whom,
the information is generated and gathered. The researcher brings subjective elements
to the knowledge-gathering process; these are an asset to the research process,
but they make truth claims impossible to achieve. Interpretivists tend to see
interviews as a process, rather than a product: they ask how interviewees them..
selves make sense of the world, and why the interview data take the form that
they do. Seen this way, interviews are a useful way for an individual researcher to
develop knowledge regarding a certain community or issue; but replication of
this knowledge by others may be difficult, and interview-based knowledge may
not offer definitive tests of a given set ofpropositions.7
The discipline ofpolitical science currently is centered at the positivist end of
the spectrum; this is particularly true for research-oriented universities in the
United States. The content of this volume, in terms of the balance between
positivist-oriented and interpretivist-oriented approaches, mirrors the current
state ofthe discipline.' Although there are many scholars who would place themselves
in the middle of the continuum or at the interpretivist end of the spectrum,
and some scholars whose placement on the continuum shifts over the
course of their careers, much of the profession remains centered on positivism.
Given that the main objective of this volume is to help scholars in political science
use interviews systematically and well, many of the contnbutors approach
interviews from a positivist perspective. Such a perspective also allows us to
achieve another objective, which is to complement the vast array of extant work
on interviews from an interpretivist or ethnographic perspective.9 Indeed, despite
INTRODUCTION 11
the fact that mixed methodologies are encouraged in most graduate programs,
there is very little formal training available for those who want to deploy mixedmethod
approaches. This volume fills part of this gap by illustrating how interview
techniques connect to positivist political science, and how they can be used
(and are being used) by political science scholars.
Often, rather than living or working among interview participants, as would
an ethnographer or participant-observer, political science graduate students
choose to include two or three case studies in their dissertations, spending a few
months-or even just several weeks-at each research site. Or a faculty member
might conduct interviews with policymakers during a semester-long research fellowship,
aiming to speak with forty or fifty interview subjects during that time
frame. The information gathered from such interviews could offer greater external
validity than a longer-lasting, more narrowly defined ethnography. But such
interview data has its limitations: it does not allow for immersion, nor for the
"insider" perspective that is a hallmark of ethnographic approaches (see Schatz
2009b). This challenge to the internal validity of political science interview data
renders the appropriate design ofinterview studies-asking the right questions of
the right people-particularly important. I return to these challenges in part 3.
Two qualifications to the generally positivist perspective of this volume are
in order. First, the dividing line between positivist and interpretivist approaches
in political science is sometimes blurred. Researchers may be simultaneously
thinking about how to address sources ofbias in interviews (something more in
a positivist tradition) and also about why interview subjects answer questions
in the ways that they do. Moreover, an individual scholar's placement on the
interpretivist-positivist continuum is not necessarily fixed: it may vary with the
particular research project being'Undertaken. Within this volume, some contributors
represent approaches that are self-consciously interpretivist; for instance,
Lauren MacLean (chapter 3) and Lee Ann Fujii (chapter 7) are centrally concerned
with positionality and power relationships. Other contributors, including Bleich
and Pekkanen, Gallagher, and Martin, work in a positivist manner, but with an
awareness that converting interview transcripts and answers into more discrete
concepts and categories always involves some type ofinterpretive work Throughout
this volume, therefore, we highlight the areas ofoverlap between interpretivist
and positivist interview research.
Second, our volume offers many lessons that are useful to all political scientists
who use interviews. For instance, we discuss how to navigate the IRB process,
whether and how to use an interpreter,_ or how to report a sufficient amount
of information about one's interview study. Such practical matters confront all
researchers who use interviews, regardless of subject matter or epistemological
outlook Our volume intends to underscore the similarities across, as well as the
12 INTRODUCTION
differences between, broad approaches to knowledge. We acknowledge that
interview studies can be used to address a range of substantive questions at a
variety ofstages in the research process, and as the core empirical tool or as one
part of a mixed-method approach. While we certainly are aware of broader debates
regarding research design within the field,10 our volume is intended to appeal
to a wide audience within the discipline.
Identity and Interview Effects
Scholars from across the epistemological spectrum recognize that their individual
traits can affect the interview research process. A young woman conducting
interviews with (almost entirely male) investment bankers may find the gender
dynamics that prevail in the financial industry more generally (McDowelll997)
also color the interview process. A scholar who is perceived as an "expert," given
his university affiliation, age, or class, may receive a different set of answers from
one who is viewed as naive or uninformed. And a scholar who is assumed to hold
certain political views may have difficulty gaining access to some communities:
Woliver (2002) notes, for instance, that she faced greater hurdles in attempting
to interview pro-life activists (as compared with pro-choice activists), because
they often assumed that she did not agree with their views.
Scholars working in the positivist tradition usually label these as "interviewer
effects"; they are important to the analysis and interpretation of interview data,
because they may affect the (non)response to individual interview questions,
as well as the tone and amount of information given in response to questions.
Within the interpretivist tradition, these considerations closely relate to the concept
of"positionality," which refers to the researcher's awareness of her position
in the world relative to her informants (Ortbals and Rincker 2009a).11 Interviewees
and potential interviewees use various social, physical, linguistic, and
cultural markers {ranging from eating habits and dress to accent and hair type)
to make sense of a given researcher. Many of these features may be obvious to
informants, while others, such as religion, sexual orientation, or previous research
site experiences, may not. And informants may ascribe incorrectly certain qualities
to a researcher, especially at the stage of arranging interviews (assuming, for
instance, that American scholars conducting research in southern Africa will be
white; see Henderson 2009).
Whether scholars think about this phenomenon as "interview effects" or as
"positionality," it is quite possible that different researchers using very similar research
designs will wind up with different sets ofinterview data. Part ofthis difference
could stem from variation in access (which makes providing information
about how the sample was conducted important; see chapter 4); another portion
INTRODUCTION 13
ofthis variation would be due to differences in information provided during the
actual interview process. Yet another piece ofthe variation comes at the interpretation
stage: how a scholar understands evidence from an interview may depend
on her own experiences and worldview (see, for instance, chapter 3).
Scholars vary in their concerns about the extent to which interviewer effects
or positionality affects the nature ofthe evidence gathered in interviews. For the
strictest of interpretivists, positionality cannot be overcome: it should be acknowledged
and studied, but it is unavoidable that interview data (and all data)
are somewhat subjective and contextual. A different researcher--one who is
older, male, and African American, for instance-may well receive different responses
to his questions and understand the same responses in a different way.
Positivist scholars often acknowledge interviewer effects (see chapters 9 and 5,
for example), but they are not viewed as limiting the objective knowledge that
can be gleaned from interviews; rather, positivists view interview effects more as
a source of (quantifiable) bias or measurement error (also see the discussion of
reliability in part 3). MacLean (chapter 3) approaches this issue from an interpretivist
point ofview: she traces work on the topic in other disciplines, paying particular
attention to how positionality relates to the power ofthe interviewer vis-il-vis
the interview subjects. She suggests that a more collaborative relationship between
the researcher and her interlocutors not only improves the researcher's access,
but also can enhance the theoretical quality ofthe work itself.
Related to interviewer effects and positionality are concerns about access. In
some situations, a researcher's individual qualities improve his access: local politicians
maybe more willing to share their views with a foreigner affiliated with a
major research university than with a local scholar. Or a woman may be more
willing to speak with another wotnan, than with a male researcher, about the use
of sexual violence in the context of civil wars. Conversely, in male-dominated
societies, young women may have difficulty gaining access to, or gathering
sufficient information from, older male political leaders. In politically closed
societies, informants may worry that U.S.-based researchers are, in fact, spies
(Reinhardt 2009).
In chapter 6, Cammett explores how, in conducting research in Lebanon
as well as elsewhere in the Middle East, her outsider status limits her capacity
to effectively carry out interview-based work. Cammett offers the strategy of
"matched, proxy" interviewing to address these limitations. With this technique,
the researcher relies on carefully trained local proxies to carry out interviews. The
proxies are matched with the respondents according to various features (including
religion, sect, age, and socioeconomic status), with the notion being that respondents
will be more forthcoming when speaking with someone who appears more
similar. In describing her work on the provision of social services in Lebanon,
14 INTRODUCTION
Cammett details the recruitment, training, and supervision ofhired interviewers.
And she considers the tradeoff between community access (where hired interviewers
should look most like the intended interview subjects) and research skills
(where hired interviewers should have training in social science methodologies,
but might be of a higher socioeconomic class than their interviewees). Similarly,
Fujii's (chapter 7) advocacy of using an interpreter to carry out interviews is
based, in large part, on considerations related to the researcher's identity compared
with those ofher interlocutors.
Note, however, that not all contributors to this volume view differences between
the researcher and her interviewees as impediments to access. MacLean,
for example, suggests that outsider status and social differences smoothed her
access to village residents in Ghana and C6te d'Ivoire. Similarly, some contributors
to the Ortbals and Rincker (2009b) symposium suggest that outsider status
can facilitate the research process.
Challenges: Ethics, Sampling, Validity,
and Reliability
This discussion ofaccess, as it relates to identity and interviewer effects, exemplifies
some ofthe challenges associated with conducting interview research. While
this volume aims to encourage the use of interviews in a wide array of political
science research, we are very aware of the challenges associated with interview
evidence. These involve not only the practical elements ofgaining access to interview
subjects, but also the theoretical elements of sampling the right set of respondents,
convincing others ofthe reliability and validity ofinterview data, and
ensuring that research is conducted ethically. Addressing these challenges allows
one to reap the benefits ofinterviews, which often are-alone or in combination
with other research methods-an incredibly useful means of measuring key
variables and assessing central causal connections. In this section, I discuss four
challenges facing interview researchers: ethics, sampling, validity, and reliability.
Ethics
Social scientists routinely confront ethical issues in the course of designing and
conducting research. General standards of ethical research dictate that scholars
do not harm participants in a given study (Woliver 2002).12 If a study provides
direct benefits to participants-for instance, a medication that can stop the
growth ofcancerous tumors, or a more successful early intervention for children
with autism-then exposing them to some degree of risk (such as side effects
INTRODUCTION 15
from medication) may be ethically acceptable. But where research provides little
in the way of direct benefit to participants-as is usually the case with social
scientific studies-then the risks to participants also must be minimized.
Although one could argue that many political science research projects do
not expose participants to significant levels of individual risk, it would be a mistake
to assume that ethical considerations are absent from the interview process.
In extreme cases (for instance, comments against a repressive political regime),
breaches of an interviewee's confidentiality or anonymity could lead to imprisonment
or violence. In other instances, interview participants may find that the
interview leads them to discuss traumatic individual or collective experiences,
such as sexual violence or civil war. More routinely, the disclosure by researchers
of interview data with identifying information intact could have negative professional
or social consequences. One scholar's breach ofprofessional ethics also
can diminish the willingness of participants to participate in future studies,
harming the broader research enterprise.
Minimal risk to participants is not only what-professional ethics demands; it
also is required by university IRBs, which authorize research involving human
subjects. For participants in interview studies, risk often relates to concerns
about confidentiality. Interviews are generally aimed at collecting information
that is specific to an individual or a class of respondents: the researcher wants to
know how a policymaker came to a decision about a given issue, or how a rebel
fighter decided whether to join a local insurgent group, or how connected an
individual of Afro-Caribbean descent feels with others who share her ethnic
identity. While the interviewer certainly can use the "people like you" phrase to
depersonalize the interview,13
the fact remains that the interview subject is being
asked to provide information that' may be private and sensitive. These concerns
are particularly acute when the behaviors in question include illicit behavior,
such as the payment ofbribes or the participation in insurgent movements.14
In chapter 2, Sarah Brooks investigates the role of ethics in the interview
research process. She explores the origins of IRBs, as well as the expansion of
their purviews to social science research and their concerns with minimizing risk.
She offers practical suggestions for political scientists to navigate the universitylevel
IRE process. Brooks points out that, given its origins in medical studies,
the IRB process does not always easily accommodate the types ofwork done by
political scientists. For example, Reno (chapter 8) notes that scholars who study
civil conflicts may face specific difficulties with institutional review boards:
their standard categories do not normally include options for "rebel fighters."
More generally, the IRB process can be focUsed more on protecting universities
and their personnel from risk (or liability) than on protecting research
participants.15
il I
16 INTRODUCTION
Central to the process of IRB approval, as well as to the conduct of ethical
research generally, is the granting of informed consent by the interview participant.
Informed consent establishes that informants realize the purpose of the
research being conducted, as well as the risks (and potentially the benefits) to
them from participation. Participants also are made aware that their participation
is voluntary, and that they will not suffer harm (e.g., not have access to a
given service) ifthey decline to participate. As Brooks discusses, the exact procedures
for obtaining consent vary with the research setting; for example, semiliterate
informants often give oral, rather than written, consent. And where informants
mayworry more about the confidentiality oftheir comments, oral (versus
written) consent can increase participation rates (Wood 2006).
Most informed-consent procedures involve guaranteeing the confidentiality
of participants, both in terms of their identities and participation in the study,
and in terms of what they reveal in the interview. Policymakers in democratic
nations may be willing to be interviewed "on the record" and to have their responses
cited in academic publications.16 But many other types of participants
may not. And as informants' actual and perceived risk increases, the researcher
must think more deeply about the limits that should be placed on the research
effort-who should be contacted, and how should they be interviewed?
In the short run, guaranteeing confidentiality includes keeping field notes
separate from identifying information (so that one's notes refer to interviews by
number, but the master list ofthese numbers is stored separately, in a passwordprotected
or encrypted document).17
In the longer run, guaranteeing confidentiality
means not naming informants in publications that result from interviews.
Again, one rnigbt use a list ofinterviewees arranged by number (and perhaps by
broad location and interview date). The scholar is ethically obliged to trade the
protection of subjects against the transparency of the research process. In the
case of non-elite informants, this is likely sufficient to satisfy academic reviewers.
If one's informants are elite policymakers in democratic nations-who are
perhaps less at risk from breaches of confidentiality, and about whom revealing
more information is important to the use ofinterview evidence-then interview
citations might offer more detail. For instance, a «senior finance ministry official"
or a "mid-level central banker who works on regulatory policy" might be
cited.18
When informants are particularly exposed to risk-for instance, when they
are members of an opposition movement in a politically closed society-the
"interview appendix" (see chapter 4) should reveal less about the sample. In special
circumstances, such as conflict and post-conflict environments, even greater
considerations of confidentiality may be necessary (see chapter 8, for instance).
There may be locations and situations in which the ethical pursuit ofresearch is
INTRODUCTION 17
impossible or nearly so (also see Wood 2006). At the extreme, then, ethical considerations
could include embargoing publication or circulation of a document
for some period of time. This can create professional challenges for graduate
students or untenured scholars, and it suggests that some research questions, or
some field locations, may need to be excluded, at least for a time.
Communicating the risks of one's study, and the ways in which respondents
will be protected from risks-how confidentiality of responses is assured, for
instance-often requires going beyond what one's university IRB mandates.
Rather, the discussion of consent needs to be put in a context that potential informants
understand, one that seeks to remove coercive elements from the process.
The lead scholar also needs to ensure that all additional members of the
research team-graduate students, research assistants, interpreters-are aware
of the informed-consent principle and of its implications for behavior.19
Last, while some scholars would argue that minimizing risk to participants
in interview studies, as university IRBs require, is all that is needed to fulfill the
political science researcher's ethical obligations, others would argue that ethical
considerations go far beyond IRB approval and informed-consent procedures.
In chapter 3, MacLean notes that many entities other than universities-such as
American Indian tribes-have IREs. These entities can encourage the researcher
to think -about his or her relationship with the subject community. While most
political science scholars do not treat their interview subjects as active participants
in the research process-for instance, offering them the opportunity to
collaborate in the interpretation ofinterview :findings-MacLean posits that political
scientists should consider moving in a more participatory direction (also
see Hertel, Singer, and VanCott 2009). This rnigbt include sharing the results of
research-giving a public talk, or "Sending copies of publications-with participants
(Woliver 2002). Or it may include the researcher offering (to the local community,
not only to those individuals who consent to participate directly) other
services, such as tutoring children in English or providing information about
available government health care programs.20
More generally, the interviewer needs to consider her ethical obligations,
as well as her professional responsibilities vis-cl-vis her home academic institution.
These obligations often demand a careful consideration of whom to
interview, how to protect the data that are gathered, and how to interact with
informants before, during, and after the interview process. Finally, note that
"risk" also can extend to the researcher. Many political scientists are interested
in phenomena-insurgency, human traffic~ing, corruption-that can be dangerous
to study. Other scholars find themselves in research locations that have
higb levels of personal crime; that do not guarantee personal liberties; or that
are undergoingmomentous social change. In such instances, individual researchers
I
li
18 INTRODUCTION
must weigh the benefits of conducting interviews against the risks they present
(Johnson 2009).
Sampling
Another central issue that a scholar using interviews confronts is whom to
interview: How should the population of potential interviewees be defined? Are
groups of individuals, such as investment bankers or Lebanese citizens, the subject
of interest? Leech et al. (chapter 11) note that individuals or organizations
are not always the unit ofanalysis; their project sought to draw comparisons across
issues, which were selected in a largely random fashion. But the i~formants on a
given issue were selected in a non-random fashion; researchers made sure to
sample lobbying groups on both (or all) sides ofa given issue. This level ofselection
was purposive rather than random. Once the unit of analysis is identified,
the bounds ofthe population must be specified; for instance, is the sample drawn
from all Lebanese citizens, only those in certain geographic areas, or only those
belonging to certain religious and ethnic groups?
Next, the interviewer must decide how to sample interviewees from within the
population. Researchers should always think carefully about how the sample is
drawn. But, as julia Lynch (chapter 1), Martin (chapter 5), and Leech et al. (chapter
11) discuss, sampling decisions often depend on the stage and purpose ofthe
research (theory building or hypothesis testing). If the purpose of the interviews
is to test a theory, and the researcher is therefore concerned with generating a
representative sample, then a random sample may be most appropriate. Random
sampling facilitates better causal inferences from the sample to the population
and, as such, is the "gold standard" for observational research, including survey
research.21 A random sample helps to ensure the external validity-from the
sample to the population-of one's findings. Interview researchers who pursue
random sampling strategies often aim to generate "dataset observations" (DSOs),
which can be thought of as values (qualitative or quantitative) for a set of variables
on a single observation (Brady and Collier 2010). For instance, ifthe unit
of analysis is a member ofparliament, the variables might include the amount of
time she spends on constituency service, the three most important issues on
which she has worked in the last year, and the three most important sources
of information for her. These DSOs could be analyzed qualitatively or, as in the
case ofBeckmann and Hall's (chapter 10) and Leech et al.'s (chapter 11) projects,
statistically. Scholars who employ random sampling strategies will need to pursue
various strategies to ensure access to a representative sample-for instance,
following up repeatedly on requests for interviews, or making multiple visits to
the same research site.
INTRODUCTION 19
Many interview researchers, however, employ a non-random sampling strategy.
They may do so for theoretical or practical reasons. On the theoretical side,
when the purpose of interviews is to develop causal explanations or to generate
theories, the researcher may be interested more in specific cases than in a representative
sample. These specific cases may be "most likely" or "least likely" instances,
and they may be chosen because of the insights they provide regarding
the causal circumstances that generate particular outcomes. Deliberately choosing
cases or interview subjects that are outliers or exemplars in important ways
("least likely" or "most likely" instances) can offer "smoking gun" evidence that
facilitates theory development. On the basis of one's theory, for instance, one
might employ purposive (quota) sampling, selecting individuals on the basis of
certain characteristics (Danish firms in specific industrial sectors, Chinese workers
in specific types of firms). Used in this way, interview data are likely to take
the form of"causal process observations," defined as "an insight or piece of data
that provides information about context, process or mechanism, and that contributes
distinctive leverage in causal inference'' (Brady and Collier 2010 277-
278; also see Beck 2009; Seawright 2002).22
Within this volume, both Cammett's
and Fujii's projects rely on non-random samples to generate causal process
observations.
Practical considerations also can lead researchers to employ a non-random
sampling strategy. Resource and time constraints, an unwillingness of individuals
to participate in the study, or the sensitivity of the subject matter can render
random sampling impossible. For example, in describing her research in El Salvador,
Wood (2006) notes that, in the context ofa civil conflict with multiple insurgent
groups and low levels of societal trust, constructing representative samples
oflocal respondents was impossible. Rather, she attempted to gain access to a wide
variety of individuals, but with the awareness that bias was unavoidable. Reno
(chapter 8) expresses similar concerns about, and implements a similar solution,
gaining access to insurgents and rebel fighters.23
In other contexts, activists may
worry about granting access to a stranger, even one with academic credentials:
Woliver (2002) notes that abortion activists worry about physical harm, thereby
rendering them reluctant to be interviewed. And elite interview participants,
such as former or current government officials, may be reluctant to be inter-
viewed.24
Non-random samples of interviewees can limit the researcher's capacity to
generalize, especially in the area oftheory testing. 25 But non-randomly sampled
interviewees can still provide the researcher :-"ith significant inferential leverage,
as Lynch discusses in the next chapter, and as Martin describes {with respect to a
snowball sampling strategy) in chapter 5. Moreover, non-random samples can
be very useful for purposes other than theory testing. As both Gallagher and
20 INTRODUCTION
Rogers (chapters 9 and 12) describe, non-random samples of interviewees can
be used to assess the extent to which larger-n surveys capture the concepts that
scholars aim to measure. Additionally, Cammett (chapter 6) posits that, while
necessarily non-random in their selection (given the research context and the
sensitivity ofher subject), her interviews with ordinary citizens related to social
service provision by sectarian parties in Lebanon were an important part of her
overall empirical strategy. In her case, this strategy included a mass survey, geographic
information systems (GIS) data, elite interviews, and government documents.
Each saropling strategy, from convenience saropling (talking with whoever
happens to be available) to random sampling, has its benefits as well as its drawbacks;
these also vary with the type of interview data collected and the purpose
for which the data are used. Regardless of the strategy one ultimately employs,
an awareness of these tradeoffs is central to designing and conducting an inter-
viewstudy.
Furthermore, the author should make her sampling choices clear-whatever
they are-in reporting interview findings. Scholars who use quantitative data are
now expected to make their datasets, variable descriptions, and statistical software
code available once their manuscripts are published. This not only allows
others to understand more completely the results presented; it also allows others
to replicate the analysis (King 1995). While confidentiality constraints may prohibit
the sharing of interview transcripts (Golden 1995), information about the
sampling process certainly should be shared. Indeed, one could envision two
different types ofreplication ofinterview studies: one in which the collected data
are provided and then reanalyzed (the way that "replication" usually is practiced
in statistical studies), and another in which the entire data collection processdefining
a population, identifying a sample, and conducting interviews with
those in the sample-is replicated.26
The latter is facilitated by scholars' transparency
regarding sampling decisions. And even ifother scholars do not attempt
to replicate one's research, greater transparency about the interview process will
increase confidence in the researcher's conclusions. In this spirit, Bleich and
Pekkanen (chapter 4) suggest that researchers provide a wide range of information
about how their samples were constructed-who was chosen for interviews,
how many potential interviewees declined ~o participate, and what "type" (in a
quota sampling sense) each interviewee is.
Validity
Information about the sampling process pertains to a third issue related to interview
research: the extent to which one's measuring instrument (in this case, the
interview) actually gauges the properties it is supposed to measure. In the context
INTRODUCTION 21
ofthe interview, concerns about validity revolve around whether the researcher is
asking the right questions, or asking questions in the right way, as well as whether
the interviewparticipant is offering truthful answers (and, ifshe is not, whether the
researcher is able to detect this).
For the interview researcher, asking the right questions often requires not
only substantive knowledge of the issues at hand (for instance, what extant accounts
from the media or by other scholars indicate about a given policymaking
episode, or what other interview subjects have reported with respect to how violence
unfolded in a particular region or village), but often some experience in
conducting interviews on a given subject. For example, in my interviews ofinvestment
professionals, it was important to gain a sense of how they talked about
politics, so that I could assess how political events entered their decision-making
calculus. "Government partisanship" meant very little to most professional investors;
but "left-leaning government" or "the Labour Party" or "the Swedish Social
Democrats" did.
Earlyinterviewsthat allowthe researcher to discover howbestto ask questionsnot
to get the answers she wants, but to get at the right underlying phenomenacan
therefore be very important. Indeed, this is the rationale for many predissertation
fellowship programs, as well as for the "learning by doing" model
with which this chapter begins. Additionally, in many contexts, interview questions
are more useful when they ask for information about actual behavior (what
happened in a specific instance), rather than for interlocutors' explanations of
why things happen (see Beckmann and Hall, and Leech eta!., in this volume). In
other contexts, gaining a sense of how informants understand their reality (the
"why" view) may be a central aim of the research project (see Fujii, MacLean,
this volume).
The validity ofthe interview instrument also hinges on the accuracy ofinformation
provided by the informant. Even ifthe researcher asks the right questions,
she may not receive answers that are accurate or truthful; she must guard against
the possibility that the interviewee is-deliberately or inadvertently-"playing
her." Decision makers, when asked to reflect upon past events, may strategically
misremember or revise their accounts, and likely in a way that is favorable to
them. For example, in chapter 8, Reno describes how, in recent years, members of
rebel groups in Africa shifted from describing themselves as perpetrators ofviolence
to focusing on their roles as victims ofviolence. Moreover, even ifinterview
subjects do not intend to deceive researchers, they may not remember accurately.
Decades ofresearch in psychology, much ofit focused on the accuracy ofwitness
testimony, suggest that eyewitness accounts are.often unreliable (Loftus 1979; Wells
and Olson 2003). Especially when interviewed about disturbing or chronologically
distant events, interlocutors may make errors without intending to do so.
II
I
~I
22 INTRODUCTION
The researcher can guard against this threat to validity by considering a given
interview in the context of other information-something that becomes easier
to do as the research project progresses. Researchers can use what they have
learned in previous interviews to check the validity offuture interviews. Indeed,
one of the benefits of interviews is that the researcher is aware of the context in
which they are conducted, and ofhow informants might attempt to frame their
answers or evade certain questions. The interview can use this metadata to assess
validity; one ofthe advantages ofinterview data is that the researcher usually has
a sense of the internal consistency of the interviewees' answers, the biases revealed
by the interviewee, and the points ofhesitation during the interview. This
is much more information than users of quantitative indicators, especially
those from other sources, usually have. And this interview metadata can be useful
not only for thinking about validity ofthe answers, but also for understanding
the social context in which answers are offered (Fujii 2010). The researcher
might therefore ask herselfwhether one former cabinet minister's account comports
with another cabinet minister's account. Ifit does not, she might consider
why their answers to specific questions would differ, or how the context in which
they were interviewed could condition their responses. Note, however, that
guarantees of confidentiality-and professional ethics-usually preclude making
this strategy obvious to informants («that's not what Mr. Jones told me
about what your party promised!"). Of course, the fact that there are multiple
accounts and views of the same issue or story may be part of what is interesting
to the researcher (Fujii 2010). In such cases, ensuring that all points ofview are
captured in the interviews is important. In their study of a range of U.S. public
policy issues, Leech and her colleagues (chapter 11) aimed to ensure that the interviews
represented every side of the issue. Often, an issue had only two sides,
so that interviews with three individuals provided sufficient coverage of the
issue. In other cases, though, the number ofinterviews required to cover the case
fully was greater-up to fifteen, in one instance. This reminds us that the "right
number" of interviews depends on the specific task and subject (see chapters
4 and 5).
The validity of interview evidence also depends upon the scholar's use, synthesis,
and interpretation of interview material: to what extent do the facts and
viewpoints revealed in interviews correspond to the researcher's theoretical
constructs? 27
Is the researcher operationalizing and measuring key concepts appropriately
(Gerring 2012; King, Keohane, and Verba 1994)? Once the interviews
are completed, what conclusions should researchers draw from them? Here, the
researcher must guard against hearing what she wants to hear. A good means of
doing this is to employ various triangulation strategies, all ofwhich evaluate interview
data in light ofother empirical material (see Gallagher, chapter 9, for an
INTRODUCTION 23
expanded discussion). One could consider how well interview participants' accounts
fit with journalistic accounts of the same episode (this also is useful for
addressing concerns about deception, as· above). Or the researcher might consider
howwell the interviews line up with scholarly studies ofthe same or similar
phenomena. For example, the researcher could compare what lobbyists say about
which groups supported gun control legislation with the campaign contributions
made by gun control advocates to members of Congress. If lobbyists' reports
differ from what the contributions data suggest, then one might question whether
other elements of their accounts are accurate. Or one could compare the International
Monetary Fund's archival materials regarding its lending decisions with
IMF staffmembers' accounts ofsuch decisions.28
Such strategies are particularly
important when interviews are the sole form of evidence on which a research
project relies. This concern is not unique to interview-based work; those using
archival sources must confront the fact that the preservation ofsome materialsand
the destruction of others-can generate a bias in the historical record. And
an individual researcher's decisions about which archival materials to use, and
how to interpret these materials, also can generate debate and criticism (Lieberman
2010).29
Within multi-method studies, the researcher can compare interview evidence
with other types of data. She might evaluate how well the interview data line up
with comparative statics from a formal model, results from regression analysis,
or findings from archival research. In my research on financial markets' evaluations
of government policies, I found that professional investment managers pay
little attention to government ideology (whether a left- or right-leaning government
is in office) in wealthy democracies. I used statistical analyses to assess
whether this pattern was borne out in the sovereign bond market's aggregate
pricing decisions. Using cross-sectional time series regressions, I found that leftleaning
governments paid borrowing costs that, all else equal, were not very different
from those paid by their centrist or right-leaning counterparts (Mosley
2003a).
Of course, if differences across types of evidence exist, these do not necessarily
mean that interview information is invalid. Indeed, they could indicate that
interviews are capturing a process more accurately than other forms of analysis.
But such disparities do suggest to the researcher that she consider whether
and how her interpretations might be biased. Furthermore, in order to make a
compelling case that the assessment of the interview data is a valid one, the
~cholar should provide sufficient information for readers to understand the ways
in which interview data are interpreted, as well as the context in which interview
data are cited. Bleich and Pekkanen (chapter 4) suggest providing information
about the sample frame, as well as about the broader data set from which reported
24 INTRODUCTION
interview evidence is drawn. If only some interviews or portions of other interviews
are employed or quoted, whywere these interviews chosen? When quoting
or reporting from interviews, scholars also should provide context: To what extent
is a specific view that is reported from one interview consistent with the
overall tone of that interview and, more important, with the overall picture that
emerges from the entire sample ofinterviews?
The object here is to guard against the selective use of "outliers" from the interviews.
Quantifying or coding interview data-not necessarily for use in statistical
analysis, but often for summary statistics-can be a useful means ofproviding
a sense of the entire interview landscape: for instance, a researcher may note
that "85 percent of informants mentioned that someone they knew had been
asked to pay a bribe in order to obtain a business license." The best way to address
concerns about one's interpretation of interview data might be to make all data
public-the analogy to sharing one's quantitative data set and codebook, so that
others can reevaluate a scholar's coding and modeling decisions. But doing so,
which would mean sharing full interview transcripts, often will conflict with ethical
and lRB considerations, specifically with guarantees of confidentiality (but
also see Bleich and Pekkanen, this volume, as well as Aldrich 2009).
Reliability
A final, and related, concern for interview researchers is reliability. Reliability is
about the confidence we can place in a given instrument of measurement. To
what extent is the information collected in an interview accurate, and how much
confidence do we have that, were the interview to be repeated again, the same
information would be generated? Just as users of statistical data would do well to
ask how indicators ofinterstate disputes, unemployment, or legislative effort are
measured, and how these measurements might vary across units ofanalysis, users
of interview data should be vigilant for threats to reliability.
Accurately capturing the information offered by informants requires the researcher
to have an effective means ofrecording data from the interview. Researchers
vary in their practices: some record each interview, asking for permission
to record after collecting informed consent and guaranteeing confidentiality.
Beckmann and Hall (chapter I0) report thattheir requests to record were almost
always answered in the affirmative, and they do not judge their informants to have
been inhibited by the presence of a recording device. Leech and her coauthors
(chapter 11) also recorded almost every interview. When interviews are recorded,
it is advisable to transcribe the interview as soon as possible, so that any ambiguities
or gaps are fresh in the researcher's mind. The recorded interview also
can be consulted in the future, as a means of addressing reliability (and validity)
INTRODUCTION 25
concerns. And for those who use an interpreter to help conduct interviews, a
recorded session allows one to go over the interview later, along with the interpreter,
to address any vagaries in translation. (Furthermore, as Fujii argues in
chapter 7, using an interpreter can enhance reliability by allowing interviewees
to answer questions conversationally and in their native language, rather than
slowing the pace oftheir responses to match the researcher's level of fluency).
30
Many other scholars, however, choose not to record interviews. This choice
often relates to concerns about power; democracy campaigners in authoritarian
societies, for instance, may worry that a recording would fall into the wrong
hands. Or government officials may worry that their "off the record" comments
will not be kept confidential (also see Woliver 2002). Often, then, the interviewer
takes notes during the interview meeting and then more fully fills in these notes
after the meeting (also see Aldrich 2009; Hertel, Singer, and Van Colt 2009; Reinhardt
2009). If this strategy is employed, the potential for measurement error is
reduced by writing full notes as soon as possible after the interview concludesperhaps
in the closest coffee shop or park. In practical terms, then, it is best not to
schedule immediately back-to-hack interviews, but rather to allow time between
meetings. Bleich and Pekkanen (chapter 4) suggest that, no matter what method
of recording data one uses, information about the overall tone and context ofthe
interview-the interview metadata-also should be noted.
Concerns about reliability also relate to interviewer effects and positionality,
as discussed earlier in this chapter. First, different scholars may receive different
answers (as a result of the qualities oftheirs that are obvious to their interviewees).
Scholars debate the seriousness of this challenge to reliability; it may be
impossible to eliminate entirely> but one could use a triangulation strategy to
compare the answers received by some types of interviewers (political scientists
versus journalists, men versus women, graduate students versus full professors)
versus others. This would provide a sense of the size and direction of interviewer
effects. Second> individual scholars may receive the same answers but perceive
them differently. The individual researcher's familiarity with a given research site,
as well as her past experiences more generally, may lead her to privilege some
details over others (or, related to validity, to privilege some interviewees' perspectives
over others; see Allina-Pisano 2009). Reinterviews ofthe same individuals by
different scholars may again be a means of assessing the extent of this phenomenon.
Where reinterviewing does not occur, the individual researcher should
nonetheless be aware ofthe potential for measurement error.
A final check on reliability addresses whether interviewees are consistent over
time in the data they report: Does the same i~dividual, interviewed by the same
scholar, but at different points in time, offer similar responses? If she does not,
this could indicate problems of reliability; or it could indicate that exogenous
I
~I
26 INTRODUCTION
changes-things that occur between the two interviews-have affected respondents'
attitudes and behavior. When reinterviews happen very soon after initial
interviews, the former is more likely; when more time elapses before the reinterview,
the latter may explain differences. Multiple interviews of the same individual
also may be useful as a practical matter: in many cases, the researcher's
questions evolve during the process offieldwork.
Returning to previous interview subjects, with an eye toward asking additional
questions, also can enhance consistency (in terms ofquestions asked) across interview
subjects. For instance, in my research on professional investors' assessments
of sovereign borrowers (Mosley 2003a), I returned to tbe field approximately
sixteen months after my initial research trip. On this second trip, I conducted
follow-up interviews with professional investors.31
These interviews allowed me
both to gauge the consistency of investors' views over time and to inquire about
how changes in tbe global financial climate had led to changes in their asset allocation
decisions: in the time between my first round of interviews (January-May
1997) and my second round (October 1998), the Asian financial crisis, as well as
Russia's near-default, had intervened. Gauging the contrast between the same
actors' views in a period of global financial optimism and a time of global pessimism
allowed me to consider the dynamic nature offinancial market-government
relations. In this instance, what was interesting was that professional investors'
views often were not consistent over time; explaining this dynamism became a
part of my work on the subject. Of course, using repeat interviews with the same
informants to increase reliability presents the researcher with a tradeoff between
intenselyinterviewing a smaller group ofinformants (perhaps about a narrow set of
subjects) and interviewing a larger group ofinformants in a less extensive manner.
Again, the researcher will want to strike a balance-where the right balance
depends on the research project at hand-between tbe internal validity that is enhanced
by deep and repeated interviews and the external validity tbat is enhanced
by a wider population ofpotential interview subjects.
Interviews often are, and frequently should be, an important component of political
science research. For many projects, interview-based evidence is a central
component of a mixed-method strategy. In other instances, interviews are the
only viable source ofempirical information with which to evaluate a theory or
set of hypotheses. In still other situations, interviews allow the researcher to
evaluate whether or not her other tools-such as surveys-measure what she
thinks tbey do.
This volume offers advice, both practical and theoretical, about how to effectively
use interviews as part ofthe research process. The contributors come from
INTRODUCTION 27
a variety of subfields in political science, and we have worked in a range oflocations
and with a variety of subject populations. Part 1 of this volume addresses
the issues that will first confront the interview researcher: how to choose a sampling
strategy, interact with interview participants, and collect and report interview
data. In part 2, the contributors investigate specific challenges faced by researchers,
includinglinking interviews with causal claims, using proxyinterviews
or an interpreter to improve access, and conducting research in challenging field
locations. Part 3 offers examples of the varying ways in which scholars from
across political science use interview data. Throughout the volume, the contributors
provide insights from their own field experiences, as well as from those of
others in the profession. The appendix contains various materials related to interviews
and meant to serve as examples-of consent documents, semi-structured
interview questions, and interview protocols, among other things.
We hope to fill the noticeable gap in the guidance offered regarding interviewing
in political science. Today's graduate students are under greater pressure to
complete graduate school more quickly and yet to acquire a broader range of
methodological skills. They also may have difficulty obtaining funding for long
periods of field research. Furthermore, although the rise of cross-country (and,
sometimes, cross-regional) comparative projects serves to increase the external
validity of many research findings, it also shortens the time spent at each field
location. Taken together, these trends heighten tbe importance of "hitting tbe
ground running" (Hertel, Singer, and VanCott 2009). Iftbere is less time to "do"
one's research, there also will be less time for «learning by doing." A researcher
can never anticipate all issues that will arise during his or her study, but an
awareness of the challenges and complications greatly increases the probability
of executing the study successfully:......uJtimately, of returning from the field with
useful data.
Although PhD programs tend to offer less training in qualitative methods to
graduate students, this does not mean that qualitative methods are easy or obvious,
while econometric and formal ones are difficult and sophisticated. This may
be particularly true when interviewing requires knowledge of foreign languages
(although Lee Ann Fujii notes that this may be overcome with use of an interpreter)
and cultures.32 In some cases, then, our advice is similar to that provided
in fieldwork "how to" guides that draw from several disciplines (Barbour 2008;
Barrett and Cason 2010).33 In many others instances, however, our advice is specific
to the political science profession.34 The contributors also acknowledge that
the methodology of political science is conti~ually evolving. Graduate students
and-assistant professors today face a different set of challenges, and possess a different
set ofmethodological tools, than did scholars who were trained in previous
decades. Indeed, most of the contributors to this volume completed their first
28 INTRODUCTION
research projects in a different era, one in which requirements for working with
human subjects were less stringent, and in which potential interviewees perhaps
were contacted by letter rather than by e-mail. "Recording" interviews may have
involved cassette tapes rather than a tiny hard drive or a smartphone. And preserving
the confidentiality and integrity ofinterview notes did not necessarily involve
making sure that one's laptop and USB drive were password protected.
While many of the lessons offered in this book are enduring-for instance, the
tradeoffs regarding various sampling strategies--others will necessarily evolve.
While this volume does not provide a specific "how to" for every element of
interview research, it offers ample food for thought as one constructs one's own
interview-based project. We hope that this volume will encourage political scientists
from all subfields, and at all stages oftheir careers, to "just tallr to people"but
to do so in a way that is as rigorous, transparent, and ethical as possible.
Part 1
GENERAL
CONSIDERATIONS
Research Design, Ethics,
and the Role of the Researcher