Research Design and the Research Process 39
Chapter 2
Research Design and the Research Process
The purpose of this chapter is to introduce researchers to the idea and importance of research design and also to describe the process of research in political science. Every research project has an implicit or explicit logic that provides the framework for the research and guides the research strategy. The research design will set out the priorities of the research: for example, describing the hypotheses to be tested, listing the research questions and, most important of all, specifying the evidence needed to provide a convincing test for the research hypotheses and to provide the data needed to answer the research questions. This data must be valid and reliable (see Box 2.1). These priorities will determine whether the evidence should be predominantly qualitative or quantitative and how it should be collected and analysed. This chapter will thus discuss some basic issues that arise in research design, the meaning of research design, its functions, and the nature of the research process. Five different types of research design will be considered, namely experimental design, cross-sectional design, longitudinal design, case study design and, finally, comparative design.
It must be emphasized that the practice of research can be very different from the ideal model of the research process. Researchers often give the impression that their work is carefully thought out, well designed and skilfully executed. In reality research can be a messy and fraught business. The process often starts with a grant application, which takes time to prepare, and to be successful must promise substantial results at reasonable cost within a specified period of time. Achievement of this assumes a smooth research process, but problems can arise at any stage. Research staff must be recruited and trained and work well together; access must be negotiated with target groups and institutions; research strategies must be developed and agreed. Inevitably changes have to be made in the research design to overcome difficulties or to
seize opportunities that may arise such as gaining access to groups unexpectedly. Limited resources and time constraints are constant pressures. These factors mean that research is often far from a smooth process of formulation, execution and publication and more often a process of crisis management. The needs of sponsors, employers and subjects have to be balanced with the need to produce good research which will convince a sceptical audience that worthwhile results have been achieved. This chapter will focus on the ideal processes of research and research design while recognizing that the reality is more complicated and more difficult.
box 2.1 Reliability and validity
These are fundamental concepts in social research. 'A reliable: measure is one that, gives consistent, results' (Mclntyre, 2005, p. 67). The measure, itself must be capable, of being used inotHet studies, even : though it may yield different results, because the conditions, or timing are different; A reliable measure is,: however^ not necessarily a valid one and 'Reliability ... cannot compensate fór low validity'(Penriitigs, Kemari and Kleinnijenhuis, 2006, p. 67). 'A valid measure is one that. is actually measuring what you think you are measuring''(Mclntyre,. 2005,;p.: 66),.The research design has an. important role in ensuring : that valid measures are used.
The meaning of research design
Research design is the logical structure of the research inquiry that the political scientist is engaged upon. It is the plan, the structure and the strategy of the investigation, so conceived as to obtain answers to research questions or problems (Kerlinger, 1986). Research design thus provides the framework for the generation and analysis of data according to the priorities set by the researcher (Bryman, 2001). Hakim argues that 'research design is the point where questions raised in theoretical or policy debates are converted into feasible research projects and research programmes that provide answers to these questions' (Hakim, 2000). The researcher observes a phenomenon that he or she feels is interesting, puzzling, neglected or difficult to understand, and then speculates about its possible causes. Naturally the researcher's training and culture help that person to develop a number of
38
40  Research Methods in Politics
possible explanations, but it is the role of all scientists to question most of the explanations of phenomena in their field. It is part of their training to subject these explanations to rigorous testing and research. The aim of the research is to generate new knowledge about the phenomenon and to apply, test and refine theories to explain its occurrence and operation.
In deciding how to research a phenomenon, the political scientist, like any social scientist, is confronted with a large number of possible research strategies and methods. The key question is, which research method will provide the best evidence to test the research hypothesis or answer the research questions? In practice, a combination of methods may be used, such as observation, a survey or comparative analysis. The use of such a combination of methods may provide complementary data which can strengthen the findings. This strategy of cross-checking data by using a variety of research methods is known as 'triangulation'.
A distinction is often made between qualitative and quantitative research. Qualitative research is very attractive in that it involves collecting information in depth but from a relatively small number of cases. Examples might be a detailed case study of an election, or in-depth interviews with political leaders or members of international organizations, or participant observation of a new social movement. Qualitative research's emphasis on knowledge in depth is at the expense of being able to make generalizations about the phenomenon as a whole. Thus a branch of a political party studied in depth cannot be taken as typical of all branches: it may be completely unrepresentative. This is also true for a participant observation study or focus group research. Statistical analysis, based perhaps on a survey of a large number of branches, or more likely on a random sample of branches, would be needed to provide data from which generalizations could be made about all such organizations. However, surveys are expensive and often provide relatively limited information. The analysis of survey information also requires specific statistical skills. The best strategy is therefore one that provides the best evidence to test the research hypotheses and one that the researcher is competent to undertake. The research questions and hypotheses will provide an excellent guide to the methods needed to collect the appropriate data. Political scientists also need to be aware of statistical pitfalls such as the ecological and individualistic fallacies (see Box 2.2). Ackoff argues that the ideal research design is 'the optimum procedure that could be followed where there is no practical restriction' (Ackoff, 1953).
Research Design and the Research Process 41
box 2.2 Ecological and individualistic fallacies
There are two fallacies that political science risks committing in carrying out research: the ecological fallacy and the individualistic fallacy. Political science has a particular interest in collective decision-making. The ecological fallacy entails 'inferring Without ' investigation that relationships among collectivities. are the same as those for. individuals. Only under very specific circumstances ...are such inferences from ecological data valid; otherwise the observer has committed the ecological fallacy' (Alker, 1965, p. 102). Tn other words the ecological fallacy involves the identification of statistical relationships at the aggregate level that do not accurately reflect1 the corresponding relationship at the individual data level. 'Anyone : who draws a conclusion about individuals based on evidence about
■ groups commits what is known as the: 'Ycologkui l.i'.lacy'" iMclntxrr, 2005, p. 42). For example, 'the relationship between marginality and turnout, while suggestive, may be merely an example of the ecological fallacy" (Mclntyre, 2005). Political scientists have to be particularly careful when they are handling aggregate level data that cannot be
■ disaggregated, for example census data.
'It is likewise a logical error lo draw conclusions about groups based on data gathered with the individual as the unit of analysis' (Mclntyre,
■} 2005; p. 43). This is known as.the individualistic fallacy;or sometimes L asthefedactionist fallacy. 'The individu
.of the ecological fallacy .....social scientists arc ... i-kely ro generalize from individual behaviour to aggregative relationships' (Alker, 1965, p. 103.). An example of the disconnect between individual level data and societal level observations is the 'assumption mat overt support for democracy at the individual level is a.reliable measure ;>i democratic institutions at the societal1 level, is.mistaken ~;and; represents--an example of the individualistic fallacy': (Inglehart and; Welzel,: :2003, p. 19). Particular care has to be taken in handling case studies where. ■the N.may be one, an issue discussed more: fully in Chapter.5. .;,
The problem is, of course, that there are always practical considerations. If, for example, a researcher wished to discover why people join political parties, as Seyd, Whiteley and Richardson have done in their studies of the British Labour and Conservative parties (Seyd and Whiteley, 1992; Whiteley, Seyd and Richardson, 1994; Seyd, Whiteley and Parry, 1996), then a possible obvious research design would be to interview the whole membership or,
S
42  Research Methods in Politics
Research Design and the Research Process 43
alternatively and more cheaply, a random sample of the membei-ship. Generalizations could then be made about their reasons for joining (providing that they gave honest and frank answers to the interview questions). However, the practical problems involved in setting up the research project would be substantial. The partit., might not have a clearly defined membership, or they might define membership in different ways by, for example, including or excluc ■ ing members of affiliated organizations. They might well be relut- \ tant to cooperate with a social scientist, regarding him or her ; likely to be critical of the party and its procedures, although the \ ! might also feel that the research might benefit the party, providing useful information, for example about recruitment strategies, that < they did not already have. Even if the leaders agreed to cooper- i ate with the research, ordinary members might refuse to do so. u ; permission is refused, what should the researcher do? Change the i research project? Or adopt a different strategy, perhaps by joining ;; the organization and collecting the data through covert participart -observation even though this would violate the ethical principle ■,, that it is essential to gain the informed consent of those beiiu i researched? (See Chapter 11.)
Whatever the research strategy adopted, whether it is a sample | survey, comparative analysis, a case study or participant obse*"- : vation, the political scientist has to consider what is the most % appropriate and logical structure for the research project about 'k to be started. Whatever the practical considerations, it would b-: 5 disastrous to be forced to adopt an inappropriate research design, | A content analysis, for example, of party documents would W i unlikely to shed much light on why people joined the party. Inter- j views with party members, particularly those who had recentk | joined, would provide the most valuable evidence. Many researchers feel that it is essential to use several methods to collect data sr ? that material collected by a survey may be supplemented by obser- J vations and in-depth interviews to check the accuracy of the data 1 and to verify that people behave in the ways that they say they do.
There are thus two functions of research design: first, to develop I or conceptualize an operational plan; and second, to ensure th. ■ the procedures adopted within the plan are adequate to provide valid, objective and accurate solutions to the research problems. It 3 is this second function that is stressed most frequently. As D.mJ de Vaus argues, 'the function of research design is to ensure that the evidence collected enables the researcher to answer the initial research question and test the hypotheses that have been formu-
lated, as unambiguously as possible' (de Vaus, 2001). The choice of research design is thus the result of the researcher's decisions about convincing his or her audience that the hypothesis has been reli-: gjjiy tested, and that accurate explanations have been proposed.
The first function, however, is equally important. The researcher must develop the research questions, transform them into hypotheses, and organize these in a logical and consistent way so that they form a theoretical framework for the research. Key concepts will have to be defined. The objective is to develop a clear and logical framework for the research project. In addition, there are a number of practical considerations that play a major role in the decision about which research strategy to adopt. It is very rare for a researcher to have the luxury of being able to choose the ideal research design. Usually the researcher has limited funds and limited time to do the research. For example, a political scientist might be commissioned by a government to assess the political impact of reducing the voting age from 18 to 16 years, and in particular to assess whether these young people wished to have the vote and would use it if it were granted to them. The brief might be to present the results to the relevant government minister in three months' time. It would be impossible and far too expensive and too time-consuming to interview all 16- to 18-year-olds, and so a sample would have to be selected. As the timescale is short random sampling would not be possible, so focus group interviews with small numbers of young people in different parts of the country might be the most practical methodology. The focus group findings might not be representative of the age group as a whole, but they would be qualitative and indicative of the feelings of young people, especially if care has been taken to ensure that the focus groups included the appropriate proportions of young men and young women, ethnic minorities and those from comprehensive and independent schools and sixth-form colleges. In the final analysis the researcher has to balance the wish to provide the most convincing evidence possible with the time and resources available to carry out the project.
When timescales are short, sponsors often ask researchers to collect and analyse data that is already available. The collection and analysis of secondary data, as this is known, is a cost-effective way of discovering what research has already been done on the topic and what evidence is available. The analysis of secondary data can reveal what the likely answers to the research questions will be and highlight areas where new research needs to be done. An example of secondary analysis is provided in Box 2.3.
44  Research Methods in Politics
Research Design and the Research Process 45
box 2.3
Example of secondary analysis
The aim of Norris's book, Democratic Phoenix: Reinventing Political Activism (2002), was to challenge the widespread view that there has been a long-term decline in political activism in many countries since the Second World War. Norris discusses the problems of defining political participation and measuring long-term trends in a consistent and reliable way. To test her hypotheses about political activism she focuses on electoral turnout, activism in political parties, and participation in a wide range of civic associations. Norris deploys data on 193 different countries from a wide variety of sources, such as survey data from the World Values Study, Eurobarorneter, International Social Survey Program, the International Labour Organization and the International Institute for Democratic and Electoral Assistance.
Norris was able to show that there Was no consistent fall in electoral turnouts. Countries emerging from poverty and lack of education experienced substantial growth in electoral turnouts. In affluent countries, the costs of registration, the choice of parties and. the political impact of the vote affected turnout. Young people were least likely to vote, but the political culture,, especially membership of voluntary associations such as unions and churches, was important. In analysing post-modern trends, Norris suggests that new social movements, internet activism and transnational policy networks may be harbingers of new forms of civic activism.
Source: Norris (2002). . '
A major practical consideration in choosing a research design is provided by the research training and skills of the researcher. It is natural for a behavioural political scientist trained in statistical analysis to formulate research problems and research designs amenable to survey research, because this methodology is closely associated with voting behaviour and opinion polling, which are central areas of the behavioural approach. Similarly, an economist may design mathematical models to test and illuminate the research problems, while an anthropologist may prefer participant observation and a historian may be most attracted to a project based on documentary analysis. Political science as a 'junction' discipline is not associated with a particular research method, and so its practitioners use a wide variety of methods and research strategies. This makes research
design even more important as a guide for selecting appropriate methods for research.
The overall research plan or research design thus provides the framework for the research project. It involves spelling out the research questions, defining key terms and developing hypotheses. In the example of a project to explain why people are attracted to political parties, key terms to be defined would include the party itself and the concept of membership. Pilot interviews could be carried out to develop hypotheses about why people were attracted into membership: whether, for example, they were attracted by its policies, its literature, friendships with party members, its social events, frustration at government failures, or particular incidents which galvanized them into political activity.
Research design thus specifies the kind of evidence needed to answer the research question, test the hypotheses and evaluate the issues that may arise in the course of the research. It determines the research methods and techniques used. It describes how the research will be conducted and carried out. It should do this in a way that will convince a sceptical audience that the researcher has adopted an appropriate methodology and one that will provide convincing data. The choice of research design is thus closely linked to decisions about the appropriate ways to collect the evidence needed to test the hypotheses and to provide answers to the research questions.
The research process
There are two main ways of viewing the process of research. The first is the linear model, which assumes that the process is relatively clear and straightforward and can be broken down into various stages or steps that all research projects go through. The second model is the research labyrinth, which describes a more complicated process that includes false starts, confusion, re-evaluations, and replications of the research as new findings, contradictions and fresh insights reinvigorate the research process. In practice these two models describe an idealized version of the research process and the more haphazard and erratic process that often occurs in practice. The linear model has the virtue of clarity while the research labyrinth is closer to the research process in operation.
An ideal typical description of the research process as a linear series of steps would be as shown in Figure 2.1.
46  Research Methods in Politics
Theory								"--
■ specification		Data,						
,. and the	—*»-			Data,	n	Data		Publication ■
development		specification		'. collection		analysis		
of hypotheses								
Figure 2.1 A linear model of the research process
Theory specification
The first stage of the linear model consists of deciding on the topic of the research project, specifying its scope, developing hypotheses to explain its working, and constructing a conceptual framework or model showing the relationships between the different hypotheses and variables the researcher wishes to investigate. This stage has been described as 'theory specification', but it includes ail the thinking and theorizing that have to be done before the empirical work can begin. In the example of research on the lowering of the voting age to 16 years, a key hypothesis might be that young people in the relevant age groups are uninterested in politics and therefore would not bother to use the right to vote if it were given to them. This could be formulated as 'Lowering the voting age to 16 years will lead to lower turnouts in elections'. Questions could be designed to discover the political interest and knowledge of young people and their opinions about having and exercising the vote. The researchers might wish to investigate the impact of citizenship studies in schools, the importance of gender, the impact of political discussions in the family and the attitudes of parents and teachers to the proposed lowering of the voting age.
Data specification
The second stage consists of deciding what kinds of data are appropriate for answering the research questions, testing the hypotheses or investigating the accuracy of the model. This will partly depend on the nature of the research problem but also on the kinds of data that are available and that can be generated and collected. Researchers need to consider if it is possible to gain access to the relevant documents, to interview key officials, or to obtain permission to draw a sample of the relevant group. As usual, the expertise
Research Design and the Research Process 47
c i P researchers and the resources they have at their disposal will iuence decisions about the kind of data they consider appropri-
ate and collectable
and coiieciauit.
In the example of why people join political parties, we would probably decide, as did the actual researchers, that interviewing a random sample of members would be the most appropriate method. The members themselves would know best what had stimulated them to join the parties. If permission were obtained for the research, it would be a straightforward process to draw a random sample of members from a central membership list, providing the parties kept such a list. The lack of a central membership list would greatly complicate the selection of a representative sample (White-ley, Seyd and Richardson, 1994). A questionnaire schedule could then be drawn up and tested on a small pilot sample of members and new hypotheses developed if unexpected reasons for joining were suggested. The final sample would then be interviewed and explanations developed to explain why people joined and what the profile of membership was. The study of British Labour party and Conservative party members found that the social characteristics of members of the two parties differed significantly and that their political views differed in significant respects from those of the party leaders. Members claimed to join the parties for altruistic and inspirational reasons, but social reasons and political ambition also played a significant role in the decisions of a minority of party members (Seyd and Whiteley, 1992; Whkeley, Seyd and Richardson, 1994).
Design of the data collection instrument
The third stage of the research process is to organize the ways in which data is to be collected. This may involve the design of a data collection instrument. In a social survey, for example, this will involve designing a questionnaire, a process which includes the framing of questions and the development of attitude scales. If people are to be interviewed in a more informal way, then interview schedules must be drawn up. These may take the form of a fairly detailed list of questions or just topic headings to guide the interviewer in a more relaxed discussion. This may be particularly appropriate when the interviewee is the person with the detailed knowledge of the policy area or the organization that the researcher is investigating.
48  Research Methods in Politics
Research Design and the Research Process 49
If it has been decided to carry out a sample survey, then this stage will include the planning and design of the survey. A number of decisions have to be made, such as defining the population to be surveyed, locating an accurate sampling frame, and determining the sampling design. Depending on the nature of the research project, the population might be, for example, the electorate, taxpayers, policymakers, civil servants, party members or MPs.
If the researcher is carrying out a random sampling procedure, then a complete list (or sampling frame) of the relevant population will be required. This could be the electoral register for voters, a housing register for households or another population list. The list may have to be checked for accuracy. In the USA, electoral registers are rarely used for surveys as many people either are not registered or leave their registration to the last minute before the election. A better procedure is thus to conduct a telephone survey or to take a sample of houses and then randomly choose a member of each household.
There are several types of sample design that can be used (see Chapter 4). The most representative are based on simple random sampling procedures, but these are often not used because they are relatively expensive. Other types of sample design include quota sampling and snowball sampling, but these procedures are less rigorous and therefore less representative. Quota samples are often used by market research firms because they are quick and relatively inexpensive. Academic researchers prefer samples based on simple random sampling, as this enables the data to be subjected to tests of statistical significance. However, quota samples are a popular means of generating information on public opinion, and market research companies claim that if quota selection procedures are carefully followed they can provide accurate and reliable results,
Pilot study
An important step in any research project is the pilot study (see Box 2.4). This is often omitted in 'ideal' type models of the research process, as in the one above, because in practice this step is often left out due to time constraints. This illustrates the difficult choices that often confront the researcher, since in any research project a trial run has considerable advantages. In particular, the data collection instruments such as the questionnaire and the sample design need to be tested.
box 2.4 The value of pilot studies
Pilot studies enable the researcher to do the following:
1, Test the questionnaire
{a) Reveal ambiguous, meaningless or embarrassing questions.
(b) Convert open-ended questions into closed questions if only, a ■ limited range of answers is given.
(c) Discover whether new issues are raised during the pilot test and ■     new questions need to be developed.
2. Rehearse the actual survey
(a) Provide training for interviewers. . ■ (b) Alert them to difficulties that were unforeseen.
(c) Give them an opportunity to discover how the respondents will react to the survey and thus estimate the level of non-response..
(d) Test the accuracy oi the sampling frame.
It is essential that the pilot study is carried out on a sample of the actual population that the researcher will investigate in the mam survey. Pilot tests on other groups may not uncover the problems that the actual population will have with the survey.
Once the pilot studies have been carried out, the design of the final versions of the data collection instruments can proceed. It is impossible to eliminate every flaw in the questionnaire, so the decision must be made when to go ahead with the survey, that is, when the questionnaire is as good as the researcher can reasonably make it. The time quickly comes when further refinements bring diminishing returns and additional complications. The researcher will also have to adhere to the timetable for the research agreed with the funding organization.
While the questionnaire is being finalized, the researcher will be deciding on the final sample design, such as whether to use a simple random sample, a multi-stage sample or a non-random design such as quota sample. The pilot study will have provided a check on the accuracy of the sampling frame and the success of the sampling procedures.
Data collection
This stage is the actual process of collecting the data. In a documentary study, this might involve long periods in libraries obtaining copies of significant documents such as minutes of meetings
50  Research Methods in Politics
and files from a particular organization. Many organizations such as governments, political parties and politically prominent families place restrictions on access - for example, a 30-year rule - so that researchers are refused access to more recent documents (see Chapter 7).
In the case of opinion polls and sample surveys, the collection of data is crucial to the success of the project. It will involve the greatest investment of resources, especially if a commercial market research firm such as Market and Opinion Research International (MORI) has been used to interview the final sample and administer the questionnaire. If a postal sample is being used, then a telephone helpline may help to resolve some of the concerns of members of the sample and thereby increase the response rate. This was a successful strategy used by the researchers undertaking the postal survey of Conservative party members (Whiteley, Seyd and Richardson, 1994). One issue that often arises is how often an interviewer should call back if people are out. The general rule is three times if the overall response rate is to be 50 per cent or higher. Market research firms will carry out random checks to ensure that the interviews have actually been carried out. Butler and Stokes, in their classic study of electoral change in Britain (Butler and Stokes, 1969), had a small part of their sample reinterviewed by experienced interviewers to check that the data collected was reliable and consistent.
Coding
Once the data has been collected, it needs to be coded so that analysis of the results can take place more efficiently. Many questions in a survey can be closed if the distribution of the responses is known. They can thus be coded while the interview is taking place by the interviewer ticking the appropriate box.
The real challenge at the coding stage is how to code open-ended questions. This can be a laborious and difficult process, but is also interesting and stimulating, as it gives the respondents scope to embellish their answers. It also enables the researcher to get a better understanding of how the respondents actually feel about the issues being investigated. Interpretation can be difficult, however, and there is a serious danger that early answers may contaminate (that is, influence the coding of) later answers. After reading the answers to the first set of responses, the coder will
Research Design and the Research Process 51
gain a general impression of the answers the respondent is giving. This may mean that ambiguous answers given to later questions will be coded to confirm this general impression and the answers of the respondent may become more consistent than is actually warranted by the data.
Data analysis
The analysis of data is never as straightforward as might be thought. This is because all researchers will have expectations about the kinds of results the project is likely to generate and this may influence their analysis. Even when analysing quantitative data where there are clear conventions that the researcher can follow, the interpretation of the data and the drawing of conclusions can be influenced by the values and disciplinary training of the analyst. Thus we would not be surprised if a psychologist and a sociologist emphasized different findings and drew different conclusions from the same study. The strength of quantitative research is that since the methods of data analysis are well known and open it is easy for other researchers to follow the same procedures, check the statistical tests and, if those have been carefully done, come up with the same results. The interpretations and conclusions may differ but perhaps not by very much.
In qualitative data analysis the challenges of conducting rigorous and objective data analysis are far greater (Punch, 1998, pp, 198-238). How are interview transcripts or field notes to be classified and analysed, and conclusions drawn? If confidentiality has been promised to interviewees, then how can other researchers check that the classifying, editing and summarizing of the data have been well done, and that the analysis conducted and conclusions drawn are appropriate?
Analytic induction is often used by qualitative researchers in their efforts to generalize about social behaviour. Concepts are developed inductively from the data, and are then defined, refined and their implications deduced. In a study of international migrants this might involve defining those who can be described as transnational and those who are settlers and drawing conclusions about their present and future behaviour: for example, how likely they are to send remittances to relations in their country of origin, how likely they are to learn a new language and how likely they are
I
52  Research Methods in Politics
to migrate further to a new country. The most important feature of data analysis is that the researcher should enable the reader to follow the analytical procedures and present sufficient evidence to show that the conclusions are strongly supported.
Publication
The final stage of research is the publication of the results. This usually occurs in the form of a report, academic papers or a book. In general, academic researchers have freedom to publish the results of their research providing they have the informed consent of those who have provided them with the research data. If the research has been funded by a foundation, a research council or a university, publication is strongly encouraged. If the research is done as part of a contract with a government department or a company, then permission to publish will have to be obtained from the funder. Normally this is negotiated at the beginning of the project when the contract is drawn up, otherwise the researchers may find their publishing ambitions thwarted after the research has been done. However, it is usually researchers directly employed by funders as part of company or government research departments who are the most strictly controlled. Researchers must also ensure that their manuscript complies with relevant legislation such as the laws of libel and data protection.
The linear model: summary
As the above discussion shows, the simple linear model outlined above (see Figure 2.1) can easily be amended to include more stages, especially if the research project involves a sample survey, A more complete outline of the linear model of the research process as described above is shown in Figure 2.2.
The linear model has the great advantage of clarity. It specifies the various steps or stages in the research process in a logical and coherent way, but scientific research rarely involves logical sequences. Research rarely goes according to plan, although this is not an argument against having a plan!
Research Design and the Research Process
53
Design of final data collection ■ instrument.
/Data collection
K
Coding and checking
Data, analysis
Publication
Design of final sample
Figure 2.2 The main stages progression
of the research process as a linear
The research labyrinth
An alternative model of the research process would be to see it as a confusing and haphazard process involving frustrations and setbacks as well as opportunities for inspiration and success. The smooth logical progression of the linear model is fine in theory and may seem realistic when the process of research goes well. However, in practice there are many possibilities for taking wrong turnings, coming to dead ends and needing to retrace one's steps - just as in a labyrinth.
The generation of new ideas and hypotheses does not come easily. It involves the creation of new models and theories, which then have to be tested by practical research designs. This can result in setbacks as mistakes are discovered or the hypotheses are refuted by the evidence. The search for new ideas and hypotheses may have to begin again. These frustrations and setbacks are well described by Watson (1968) in his account of the processes and events that led to the discovery of the structure of DNA (see Box 2.5).
box 2 5 The discovery of the structure of DMA
54  Research Methods in Politics
Research Design and the Research Process 55
might assume. The process is often a messy one depending on luck, guesswork, access to ideas and data, competition with other scientists, and setbacks as well as lucky breaks.
Several scientists in Britain and America were researching the molecular structure-of-DNA, in particular Maurice Wilkins and Rosalind Franklin at King's College London, James Watson and Francis Crick at Cambridge, and Linus Pauling at the California Institute of ; Technology. Wilkins and Franklin were using x-ray diffraction but for a long time they failed to realize their x-ray data was consistent with a helical structure for DNA. Access to their x-ray pictures and other data helped Watson and Crick to develop their ideas. Wilkins and Franklin were also slow to appreciate the importance of model construction. Competition with Linus Pauling also stimulated Watson and Crick, especially when Pauling proposed a new structure for DNA.However, Pauling's proposed structure was flawed and (his allowed Watson and -i Crick the time, after some mistakes, to develop their double helix? theory. Watson admits that any of these scientists could have won the race to discover the structure of DNA.
Source: Watson (1968).
Another area of difficulty is the business of raising finance to fund the research. Resources are scarce and competition may be fierce. Major flinders of academic research have to justify the expenditure of money and ensure that a rigorous procedure is followed, so applicants often have to prepare detailed proposals, which take time and effort. There may be a long and frustrating wait before a final decision is made, and proposals may be unsuccessful. Furthermore, the review process may well include peer evaluation of the grant applications by reviewers who are researchers working in a related area of research, and although this often works well, relations with colleagues can sometimes be difficult if they are also competitors.
In large research projects there are a host of potential problems involved in recruiting and training staff and working together as a team. This can be further complicated if some of the researchers are doing their own doctoral research as part of the project as well as research for the overall project. In the late 1960s Margaret Stacey and Colin Bell began a second community study of the town of Banbury in Oxfordshire, England, investigating how the town had changed in 20 years. The research team for the Banbury
Restudy consisted of four researchers, and Bell later recounted how a number of problems arose relating to the organization of the project and allocation of authority, which caused bad personal relationships to develop among team members. Disputes arose over who had ownership of the data collected by team members and who had the right to publish research findings arising from the
study (Bell, 1977).
Even preparing the final report for publication can be difficult. Bell found that the laws of libel limited what he could publish about the Banbury Restudy, and Wallis found the same when writing up his research on the Church of Scientology (Wallis, 1977).
Giulianotti shows how researchers sometimes have to do deals with 'gatekeepers' or potential subjects in order to gain access to the data they need to complete their research. In his case he agreed to pass on information he had gained from one group of football hooligans, the Aberdeen Casuals, to another such group, the Hibernian Casuals, in order to carry out research on them (Giulianotti, 1995).
Research can thus be a haphazard and complicated process involving false starts, negotiations with gatekeepers, competition with colleagues, ethical dilemmas and pressure from funders and publishers. The project and methods may change in the course of the research. It can also produce inspiration, breakthroughs and the discovery of new and unexpected findings. The process is complex, uncertain and exciting. If it is rarely as straightforward as the linear model, it is seldom as impenetrable as a labyrinth.
Types of research design
There are five major types of research design, namely experimental design, cross-sectional design, longitudinal design, case study design and comparative design.
Experimental design
Experiments are widely considered to be the best way of determining cause and effect in scientific research. In the laboratory, the scientist is able to control all the variables and, in particular, to manipulate the independent variable and measure its effects, if there are any, on the dependent variable. The effect of heat, light and pressure, for example, on chemical compounds can be measured
56 Research Methods in Politics
Research Design and the Research Process 57
very accurately and their effects recorded with complete accuracy. In the social sciences it is difficult to carry out experiments, first because it is extremely difficult to control all the variables, and second because of ethical difficulties involved in experimenting on human beings. However, because of their simplicity and the confidence with which causation can be determined, they are becoming more popular.
The experimental method is thus very straightforward. It argues that when x (the independent variable) occurs, then y (the dependent variable) will also occur. When x is absent, then y will also be absent. Thus we might hypothesize that frustration leads to aggression, and devise laboratory situations which test this. In laboratory conditions there is a high degree of internal validity in determining cause and effect because the researchers have complete control over the variables. They can manipulate them and determine the extent and direction of causation. However, the extent to which laboratory conditions can be transferable to real-life situations will vary and so the ability to generalize to non-laboratory situations may be limited, thus reducing the external validity of the results. So experiments on frustration and aggression in laboratories may or may not explain incidents of road rage. We shall see that this problem of eternal validity is an issue in generalizing from the Milgram experiments.
A common form of experimental design is to create two identical groups and expose one to a stimulus (the independent variable) and not the other. The difference, if any, between the two groups is then attributed to the stimulus. In practice it is hard to create identical groups but random selection of the groups should control most variables (Moses and Knutsen, 2007).
In the early development of sociology, experimental designs were used by researchers interested in the interactions between individuals in small groups. In particular, they were interested in how different leadership styles affected group cohesion and problem-solving capabilities. Groups would be randomly selected to ensure that factors such as age, gender and ethnicity would not influence the results, and the leaders were told to act in democratic or dictatorial styles and given tasks to perform. The behaviour of the groups would be observed through two-way mirrors by the researchers in an adjoining room. The impact of these different leadership styles would then be analysed. The smallness of the groups and the use of two-way mirrors enabled the experiments to be conducted, more or less, under laboratory conditions
(Bales, 1951). This kind of research is relatively expensive and time-consuming to establish and the results were rather limited.
A famous example of experimental design was an experiment carried out by Milgram to test how far people can be induced to inflict extreme pain and harm on others, when they are ordered to do this (Milgram, 1974). How much pain would they be prepared to inflict before refusing to obey the orders? Milgram recruited volunteers to act out the role of teachers who were instructed to punish learners when they gave incorrect responses to questions. The punishment was in the form of electric shocks that were not, of course, real. The learners were instructed by the researchers on how to act in response to the supposed intensity of the shocks (for example by howling in simulated pain). The level of the electric shocks was increased until the teacher volunteers refused to administer more shocks. One of the researchers remained in the room with the volunteer teachers encouraging them to continue to administer shocks, arguing that this was part of the experiment and suggesting that the pain was not as bad as the howls from the volunteer learners suggested. To the surprise of the experimenters, some volunteers were willing to continue administering electric shocks even when the learners were exhibiting extreme pain and some even administered (supposedly) lethal doses.
This experiment highlights some of the problems of experimental design in the social sciences. First, how far can this experiment be used to explain 'real life' examples of brutality such as occurred in Nazi concentration camps or in situations such as Rwanda and Bosnia? In other words, how much external validity did this experiment have? Second, did the volunteers who participated in the experiment really believe that they were inflicting extreme pain on their supposed victims or did they believe, perhaps subconsciously, that the researchers would not allow the experiment to go so far? This raises the issue of whether experiments involving people can illuminate real-life situations. Third, is it ethical to induce people to inflict pain on others even when this is simulated and in the interests of scientific research?
Events may occur in the real world, as opposed to the laboratory, where something close to an experimental situation happens. In the mid-1990s, television was introduced to the isolated island of St Helena in the South Atlantic. This was seen by some researchers as an ideal opportunity to measure the impact of television (the independent variable) on the local community (the depen-
58  Research Methods in Politics
dent variable) and in particular to assess the impact of television violence on the behaviour of children and young people. A key research question was whether television causes anti-social behaviour. The initial results from the research programme suggest that it does not (Charlton, Gunter and Coles, 1998; Charlton, Coles-', and Hannan, 1999). However, even in this unique situation it is ' impossible for the researchers to control all the variables and so ; measuring the impact of television will be very difficult. It is, for i example, impossible to know how the community would have :; developed without television.
An excellent piece of research was carried out by Daniel into 1 racial discrimination in Britain in the 1960s. Daniel employed African-Caribbean, Hungarian and English actors, all claim- Is ing the same qualifications and references, to apply for jobs and accommodation. He found that frequently the Caribbean actor 4 was told that the job or accommodation was taken when it was later offered to one of the other candidates. By controlling ethnic-.21 it.y, Daniel was able to prove without any doubt that high levels '2 of racial discrimination existed in the employment and housing 4 markets (Daniel, 1968).
Research Design and the Research Process 59
box 2.6 The Portuguese In North America a quasi-experimental design
Bloemraad attempted to measure tbe! integration of Portuguese immigrants in Canada and the USA. Integration was measured in : terms of the Willingness.to acquire Canadian or American citizenship. The ideal research design id evaluate individual level explanations of integration; outcomes would involve experimental manipulation,:that is, to randomly assign;,a group, of individuals;to migrate to Canada and the USA and to compare integration outcomes. Random selection ; would eliminate state .selection or self-selection as factors explaining different rates of naturalization. If it was then found that there were significantly different outcomes we could be sure that the results were due n.i experiences of livim- in one country or rhe other.. Bloemraad was not able to use the ideal research design,.but she used ■-. : existing, data on Portuguese migrants to Canada and rhe USA in the late 1960s and 1570s .to ensure that she had comparable groups.. She argues that the Portuguese groups that migrated to the USA and Canada at this time were broadly similar in education and background and therefore the much higher naturalization rate for Portuguese:
immigrants to Canada compared with those who went to the USA is due to their different experiences in the countries where they settled. She argues that Canada is more proactive in providing support for ethnic organizations and in promoting community leadership than the USA. The positive encouragement given in Canada to multicultural" integration compared with the laissez-faire approach to integration in the USA results in much higher rates of naturalization by Portuguese ■ -in Canada,
Source: Bloemraad (2006). ■
Cross-sectional design
As experiments are so difficult to carry out in real life, social scientists have adopted cross-sectional, design as one of their most popular research designs. Cross-sectional design involves the collection of information on a large number of cases at a single point in time, in order to accumulate a body of quantitative or qualitative data in relation to a number of variables in order to discover patterns of association [Bryman, 2001).
Cross-sectional design is strongly associated with quantitative analysts due to the large number of cases on which data is collected and the ease of carrying out statistical tests on these. The data is collected over a short period of time and then analysed for patterns of association between the variables. The variables are not controlled and manipulated by the researcher. This means that it can be very difficult to prove the direction of causality. It is well known, for example, that people upwardly mobile from the working class are more likely to vote Conservative than those who are not upwardly mobile, but is this Conservative voting a result of upward mobility or a precondition in the sense that working-class people with conservative predispositions arc more likely to strive for upward mobility?
In political science, social surveys are used with great frequency ro explain voting behaviour. In America, Australia and European countries, research funders provide considerable finance for surveys that attempt to explain the reasons why voters have supported particular parties in general elections. The reasons may be short term, such as their policy priorities, their assessments of the patty leader, their impression of the economic competence of the party,
60  Research Methods in Politics
Research Design and the Research Process 61
or manifesto promises. Longer-term factors such as the social environment of the voter - for example working in the public or private sectors, their class identification and housing tenure - may also be important. Sophisticated statistical analysis is necessary to determine the relative importance of each of these variables.
In Australia in the late 1990s there was considerable concern at the rapid rise and then sharp decline of Pauline Hanson's One Nation party. In 1998, the One Nation party gained nearly a quarter of the votes in the Queensland state elections and in the federal elections in October won almost 10 per cent of the votes. Gibson and her colleagues (Gibson, McAllister and Swenson, 2002) used data from the Australian Election Study survey to try to explain the bases of support for the One Nation party. This survey was based on a random sample of the electorate and was conducted by means of a self-completion questionnaire, filled in and returned shortly after the federal elections in October 1998. The response rate was 58 per cent.
The One Nation party's platform was based on socio-economic populism combining economic protection, subsidies for farmers and small businessmen with opposition to Asian immigration, subsidies for Aborigines and gun control. The researchers wished to determine whether racial prejudice formed the basis of support for the One Nation party, or whether it was based on economic insecurity and fear of unemployment. The data from the 1998 Australian Election Study showed clearly that strong dissatisfaction with Australian democracy and resentment against immigrants and Aborigines made voters more likely to support the One Nation party. The findings indicated that a voter who was dissatisfied with Australian democracy, viewed immigration as an important issue, resided in the most rural areas and was very hostile to immigrants and Aborigines had a 78 per cent probability of voting for the One Nation party in the 1998 election for the Australian House of Representatives (Gibson, McAllister and Swenson, 2002, pp. 836-7). The authors concluded that race and immigration issues have the potential to mobilize voters and that the decline of the One Nation party after 1998 did not mean that this issue would not recur in Australian politics.
A major limitation of this kind of cross-sectional design is that the information is collected at a particular point in time and so quickly becomes out of date and is overtaken by new events. In order to overcome the time limitations of cross-sectional design, social scientists often use longitudinal design.
Longitudinal design
The major limitation of cross-sectional design is that it is unable to explain change over time. Social surveys provide a snapshot of facts and opinion at the time the survey was carried out. It is thus ^possible to make generalizations over time and to explain the changes that may have taken place. Longitudinal design is one way of overcoming this difficulty. Usually longitudinal design is an extension of survey research. A sample is selected and interviewed at a particular time, tl, and the identical sample, as far as possible, is reinterviewed at subsequent intervals, perhaps a year later, t2, and then a year after that, t3. The researcher can then compare the data from different periods and discover what changes have taken place in, for example, the attitudes and opinions of the sample. He or she will then attempt to explain why the changes have taken place.
Two main types of longitudinal design are often used. The first is the panel study, which has already been described. A sample js drawn from a population and interviewed on a topic or series of topics, and then at a later time the identical sample is reinterviewed. It is difficult in practice to reinterview the whole sample as some may have moved, may refuse to be interviewed a second or third time, or may be ill. Those most interested in the research topic are most likely to stay in the panel and these may become more knowledgeable on the subject of the research as a result of the interviews.
The second method is to select a group that forms a 'cohort': that is, they are people with a similar characteristic or experience. They may have all been born on the same day, or belong to the same class at school, or belong to the same church, or have been married on the same day. The researcher may wish to interview the cohort about, for example, their education experiences, their health over the years, their career aspirations, or their experience of family life. This data may be related to their parents' background, the area they come from, and other factors.
A good example of longitudinal analysis in political science was the panel study carried out by David Butler and Donald Stokes called Political Change in Britain (Butler and Stokes, 1969). Butler and Stokes wished to explain electoral change in Britain and, in particular, why voters switched between the major parties from election to election. They examined some long-term factors influencing electoral change, such as demographic factors, the decline
62 Research Methods in Politics
Research Design and the Research Process 63
HI H2
H3
H4
H5
of religion, and immigration, but they were mainly concerned about short-term factors such as changes in attitudes and opinion. They assumed that a national random sample of the electorate reinterviewed several times would provide the best data to explain short-term electoral changes.
Butler and Stokes chose a sample size of 2,560, of whom 2,009 I were interviewed in 1963. These people in the panel were reinter-viewed in 1964 and 1966. A wealth of detailed information was collected on attitudes to policies, parties, the media, class affilia-1 tion, trade union membership and the influence of the local politi- I cal environment. They were able to begin a popular article on the book with five hypotheses:
People tend to become more conservative as they grow older, j Trade union membership and activity increase the disposition i to vote Labour.
The number of people voting on class lines has been decreas- ~ ing.
The terms 'left' and 'right' are familiar to most of the electorate.
Within the lifetime of a single Parliament, deaths and com-ing-of-age among the electorate have only a negligible effect on party fortunes compared with all the switches due to political moods and events.
They were able to claim on the basis of their data that all these hypotheses were false (Butler, 1969).
When it was published, Political Change in Britain provided by far the most detailed analysis of voting behaviour in Britain. The authors made some notable findings on the volatility of political opinion, the lack of political sophistication of the electorate and the importance of local environmental effects. They were criticized for having a narrow definition of electoral change that focused on the two main parties, and neglected both turnout decline and increasing support for minor parties. However, the book greatly increased the sophistication of research on British voting behaviour.
Several disadvantages of longitudinal design were evident in the study. First, people in the panel proved increasingly hard to rein-terview and so the panel's size declined over time. Second, those most interested in politics stayed in the panel so, for example, the proportion who knew the name of their MP rose from 50.3 per
cent in 1963 to 56.3 per cent in 1966. This could partly have been Hue to the fact that the interviews had an educative effect.
The timing of the interviews can have a big impact on cross-sectional design, but this is also true of longitudinal design. The timing of the interviews in 1963, 1964 and 1966 coincided with times of Labour party ascendancy in the opinion polls. If the timing had been different, the Conservative party would have done much better.
In the USA the major example of a longitudinal study is the Panel Study of Income Dynamics which is carried out by the Institute for Social Research at the University of Michigan. The study began in the 1960s with a representative sample of over 5 000 families who are interviewed each year about their income, employment, health, changes in family composition and standard of living. When family members leave to create new households and families, these are included in the panel to compensate for families that drop out of the study. Care is taken to ensure that the panel is as representative of American families as possible. The result of this research is a rich source of data on the employment, income, health and mobility of American families (Hakim, 2000, pp. 114-15).
Cohort studies are frequently based on age cohorts: that is, groups of people born at around the same time, A major example is the National Child Development Study in Britain, which focuses on all people born in a single week in March 1958. Information has been collected on this cohort at birth, then at the ages of 7, 11, 16, 23, 33 and 42 years. The data collected has been used to research child development, health, educational progress and attainment, the transition from school to work, family formation, participation in the housing market, and income. Important comparative research has been done on the impact of type of school attended, of an unemployed father, of physical disabilitv and of unequal pay (Hakim, 2000).
Case study design
Case studies are an extremely popular form of research design and are widely used throughout the social sciences. Case studies enable researchers to focus on a single individual, group, community, event, policy area or institution, and study it in depth, perhaps over an extended period of time. The researcher will have a number
64  Research Methods in Politics
r
Research Design and the Research Process 65
of research questions or hypotheses to give focus to the research and organize rhe data collection, analysis and presentation of the material. This approach is closely associated with historical study and with anthropology, especially the study of tribal groups, each of which is assumed to be unique, although some of the rituals and milestone events - such as the transition from adolescence to adulthood - may provide interesting comparisons with similar events in other communities.
While both quantitative and qualitative data can be generated by case study design, the approach has more of a qualitative feel to it as it generates a wealth of data relating to one specific case. The data cannot be used to generalize about the population as a whole as the case study is unique and not a representative sample of a tribal group, institution or policy area. Some researchers therefore argue that case studies should be used only to generate hypotheses and theories which then require testing by generating data
through other forms of research design which then may lead to.....
wider generalizations.
Despite the limitations of case study design, this approach has had considerable influence in social science research. Whyte's study of a street corner gang in a Boston slum (Whyte, 1993) generated considerable insight into the life of the community and fierce debates over the appropriateness of his methodology (Boelen, 1992). Similarly, Pettigrew's study of Imperial Chemical Industries (Pettigrew, 1985) and Pryce's study of African-Canbbeans in Bristol (Pryce, 1979) both generated considerable discussion about their wider importance and applicability. In political science, the theoretical debates that surrounded community power studies in the 1960s were illuminated by case studies of the organization and operation of power in such local communities as New Haven (Dahl, 1961) and Baltimore (Bachrach and Baratz, 1970).
In order to have a wider impact than that of merely being a detailed account of a unique case, a strong theoretical dimension is often incorporated into case study design. A good example of this approach is the critical case study. Here the researcher has a clearly defined hypothesis or theory to test and the case study is designed so that wider generalizations can be drawn. A good example of a critical case study is the research by Goldthorpe and his colleagues, The Affluent Worker: Political Attitudes and Behaviour (Goldthorpe et al., 1968). In the 1960s the theory developed that rising living standards and higher pay were causing some manual workers to adopt middle-class values and lifestyles, includ-
ing changing their political allegiances from Labour to Conservative. This became known as the 'embourgeoisement thesis'.
Goldthorpe and his colleagues were highly sceptical of this thesis and decided to test it using a critical case study design, -phey selected a town and three groups of affluent workers whom they thought fitted the affluent worker thesis. They thus gave the embourgeoisement thesis the most favourable conditions for being proved. They then interviewed random samples of these affluent workers about their lifestyles, voting behaviour and political attitudes. They found no evidence to support the embourgeoisement thesis but they did argue that traditional forms of working-class solidarity were being replaced by more instrumental forms of behaviour. They then argued that as the embourgeoisement thesis was disproved under these most favourable conditions, the thesis was unlikely to exist anywhere else. It was widely agreed that Goldthorpe and his colleagues had convincingly supported their case. As a result of the study, the embourgeoisement thesis was discredited and new theories had to be developed to explain Conservative voting among manual workers.
Subsequent research challenged some of the conclusions of the affluent worker case study, in particular the argument that more instrumental forms of lifestyle were being adopted. Devine suggested that lifestyles and attitudes had not changed as much as Goldthorpe and his colleagues had supposed (Devine, 1992).
Even in descriptive case studies there must be a focus for the research so that it does not become a haphazard collection of material about the selected case study. In Seyd and Whiteley's study of British Labour party membership, the focus is on the importance of ordinary party members, their functions in the party and reasons for joining. Seyd and Whiteley then develop a theory of selective incentives to explain why people join the party (Seyd and Whiteley, 1992).
A case study design can be based on single or multiple cases. Carefully selected multiple cases will provide a much more robust test of a theory and can specify the conditions under which hypotheses and theories may or may not hold. Thus Seyd and White-ley followed up their study of Labour party membership with a similar study of Conservative party membership to see whether their theory of party membership could be confirmed or modified (Whiteley, Seyd and Richardson, 1994).
The attractiveness of case studies is that data on a wide range of variables can be collected on a single group, institution or policy
66 Research Methods in Politics
area. A relatively complete account of the phenomenon can thus be achieved. This enables the researcher to argue convincingly about the relationships between the variables and present causal explanations for events and processes. These explanations and generalizations are limited to the particular case study at the actual time of the investigation so a wealth of detailed information is collected which is specific to the particular case study. It may be possible to replicate the research at a later date but it may be impossible to know whether changes in an institution, for example, are due to changes in personnel or external developments such as new government policies.
Comparative design
Comparative design is one of the most important research designs in political science. If we wish to understand the conditions under which democracies develop and thrive, or the conditions under which revolutions occur or under which wars break out, there is no alternative but to compare these events. Other forms of research design would be impracticable or less appropriate. The more cases or examples that are studied, the more likely that common causes can be found and generalizations made. Political events and processes are often clarified and illuminated by comparison with similar events and processes in other contexts. Thus it is impossible to judge how successful British democracy is without a benchmark for comparison. The benchmark may be an idealized model of a modern democratic system or existing democratic systems that are felt to make fair and appropriate comparators, perhaps because they have similar levels of education, wealth and industrialization. Thus Butler and Stokes argue that 'Comparisons of political systems can extend our understanding of British politics and lead to still more general formulations of the process of change' (Butler and Stokes, 1969, p. 533).
The major difficulty with comparative design is in finding comparable cases: that is, examples which are similar in a large number of respects to the case which the researchers want to treat as constant, but dissimilar in the variables that they wish to compare with each other. The researchers can never be certain that the two or more political systems being compared agree or differ in all respects save the ones under investigation (Lijphart, 1971).
Research Design and the Research Process 67
Comparative design has thus grown out of the desire both to understand better one's own political system by comparing it with others, and also to make generalizations about political practices and processes, such as what are the conditions under which territorial minorities prefer integration, or autonomy in a federal state, or complete independence? Does proportional representation ensure greater parliamentary representation for minorities and women?
A classic study using comparative design in political science was The Civic Culture by Almond and Verba (Almond and Verba, 1963). They wished to investigate the political cultures of western democracies in order to understand the basis on which democracy was sustained. They decided to interview random samples of the electorates in five democracies, namely Britain, the USA, Germany, Italy and Mexico. They endeavoured to build up a picture of the political culture of each country by interviewing voters about their knowledge of the political system and its operation, their feeling about political roles and institutions and their judgements about the effectiveness of government, the wider political system and the role of democracy. They were particularly interested in what voters felt about their own rights and duties as citizens and their own political role in the democratic process. Almond and Verba developed three typologies of political culture, namely the parochial, subject and participant political cultures. They then analysed the effectiveness of democracy in each of these cultural contexts and classified each of the countries they had investigated. They concluded that a mixed political culture, which they called the Civic Culture, showed the best degree of congruence between democratic political institutions and types of political culture.
Armer argues that a major methodological task in comparative research is to devise and select theoretical problems, conceptual schemes, samples and measurement and analysis strategies that are comparable or equivalent across the societies involved in a particular study {Armer, 1973, p. 51). The Civic Culture was a hugely ambitious comparative study, which generated considerable information and stimulating ideas about political culture, but failed some of Armer's tests. The countries were too diverse, particularly Mexico and the USA, to make appropriate comparators and it proved impossible to standardize the samples and questionnaires. There thus developed considerable scepticism about the concept of political culture used by Almond and Verba and the value of some of their findings.
68  Research Methods in Politics
Comparative design thus presents the researcher with considerable challenges, especially when different countries are bi i: — compared. The researcher must select a theoretical problem that is best illuminated by comparative research: for example, why women are included in the political elite in Denmark but excluded in Britain and France (Siim, 2000). Relevant and equivalent data should then be collected and hypotheses tested, such as the impact of the electoral system or diverse traditions of citizenship in the various countries, and appropriate conclusions drawn. Comparative analysis sharpens our understanding of the context in which theoretical problems occur and enables causal inferences to be drawn. However, as comparative analysis usually involves only a relatively limited number of cases, caution has to be maintained about the levels of generalization that can be made.
Conclusion
The planning and execution of a research project are critical to its success. Planning involves determining the objectives of the research, developing research questions, transforming these questions into hypotheses and deciding on the appropriate research design to test the hypotheses and to convince a sceptical audience that the evidence is valid and reliable and that the conclusions drawn from the analysis are accurate. The choice of research design is fundamental to this process as it will determine how the evidence will be generated and analysed. The better the design, the clearer will be the evidence of cause and effect and the more likely that the findings and explanations will be accepted.
This chapter has also discussed the process of research and suggested that there are two ways of describing this, namely the linear model and the research labyrinth. The linear mode! suggests that the research process is smooth and ordered, moving from theoretical speculations to the collection of data, the analysis of the findings and finally the publication of the results. The research labyrinth emphasizes the complexity and pitfalls involved in research, especially the false starts, the need for inspiration, the difficult negotiations with funders and subjects, ethical dilemmas and competition and conflict with colleagues.
The issues raised in this chapter are discussed in greatet detail in the following chapters, beginning with the next chapter on the comparative method.
OiaPKr 3
comparative Methods
To make comparisons is a natural way of putting information in a context where it can be assessed and interpreted. This is especially true when we encounter new information about some issue and begin to integrate it with previous knowledge. For example, we might know that in Germany, Denmark and the Netherlands the political executive is based in the legislature and can only survive with support from the legislature; this is what makes these countries so-called parliamentary political systems. Then, turning to France, it is striking that its political executive is linked to the legislature in quite a different way: the French president is a directly elected head of government (and head of state) whose survival does not depend on the support of the legislature. To understand fully the significant political consequences of the French semi-presidential political system, it is quite instinctive to compare it with the already familiar parliamentary systems, and in this way to integrate old and new knowledge. By comparing parliamentary and presidential systems, it is possible to acquire a greater understanding of each; as Rudyard Kipling said in his poem, 'The English Flag': 'what should they know of England who only England know?' Research from virtually all political science research traditions and sub-fields of study can (and does) fit under the label 'comparative'. There are examples of quantitative and qualitative comparative research, large-w, small-n and even single-w comparative research, and inductive and deductive comparative research, spanning every conceivable substantive topic (see Rogowski, 1993, for an extensive survey of comparative research). The goal of comparative research, as set out in an influential book from 1970, is to be able to remove proper names, and to reason instead in terms of variables (Przeworski and Teune, 1970), meaning that ultimately the uniqueness of each case itself (such as US states Florida, Texas, Louisiana, Kentucky, the Carolinas ...) is less important than the case understood as a combination of values on a number of specific variables when it comes to generating general theories of politics.
69