CHAPTER 4
Research Design
CHAPTER OVERVIEW
Here you'll see the wide variety of research designs available to social researchers as well as how to design a study—that is, specifying exactly who or what is to be studied when, how, and for what purpose.
Introduction
Three Purposes of Research
Exploration Description Explanation
Idiographic Explanation
Nomothetic Explanation
Criteria for Nomothetic
Causality Nomothetic Causal Analysis
and Hypothesis Testing False Criteria for Nomothetic
Causality
Necessary and Sufficient Causes
Units of Analysis
Individuals
Groups
Organizations
Social Interactions
Social Artifacts
Units of Analysis in Review
Faulty Reasoning about Units of Analysis: The Ecological Fallacy and Reductionism
V
The Time Dimension
Cross-Sectional Studies Longitudinal Studies Approximating Longitudinal
Studies Examples of Research
Strategies
Mixed Modes
How to Design a Research Project
Getting Started
C onceptualization
Choice of Research Method
Operationalization
Population and Sampling
Observations
Data Processing
Analysis
Application
Research Design in Review
The Research Proposal
Elements of a Research Proposal
Introduction
Science is an enterprise dedicated to "finding out." No matter what you want to find out, though, there will likely be a great many ways of doing it. That's true in life generally. Suppose, for example, that you want to find out whether a particular automobile—say, the new Turbo Tiger—would be a good car for you. You could, of course, buy one and find out that way. Or you could talk to a lot of Turbo Tiger owners or to people who considered buying one but didn't. You might check the classified ads to see if there are a lot of Turbo Tigers being sold cheap. You could read a consumer magazine evaluation of Turbo Tigers. A similar situation occurs in scientific inquiry.
Ultimately, scientific inquiry comes down to making observations and interpreting what you've observed, the subjects of Parts 3 and 4 of this book. Before you can observe and analyze, however, you need a plan. You need to determine what you're going to observe and analyze: why and how. That's what research design is all about.
Although the details vary according to what you wish to study, you face two major tasks in any research design. First, you must specify as clearly as possible what you want to find out. Second, you must determine the best way to do it. Interestingly, if you can handle the first consideration fully, you'll probably handle the second in the same process. As mathematicians say, a properly framed question contains the answer.
Let's say you're interested in conducting social research on terrorism. When Jeffrey Ross (2004) addressed this issue, he found the existing studies used a variety of qualitative and quantitative approaches. Qualitative researchers, for example, generated original data through
Autobiographies Incident reports and accounts Hostages' experiences with terrorists Firsthand accounts of implementing policies
Ross goes on to discuss some of the secondary materials used by qualitative researchers:
"biographies of terrorists, case studies of terrorist organizations, case studies on types of terrorism, case studies on particular terrorist incidents, and case studies of terrorism in selected regions and countries" (2004: 27). Quantitative researchers, on the other hand, have addressed terrorism in a variety of ways, including analyses of media coverage, statistical modeling of terrorist events, and the use of various databases relevant to the topic. As you'll see in this chapter, any research topic can be approached from many different directions. Each of the topics we'll examine is relevant to both qualitative and quantitative studies, though some topics may be more relevant to one than to the other approach.
This chapter provides a general introduction to research design, whereas the other chapters in Part 2 elaborate on specific aspects of it. In practice, all aspects of research design are interrelated. As you read through Part 2, the interrelationships among parts will become clearer.
We'll start by briefly examining the main purposes of social research. Then, we'll consider units of analysis—the what or whom you want to study. Next we'll consider ways of handling time in social research, or how to study a moving target that changes over time.
With these ideas in hand, we'll turn to how to design a research project. This overview of the research process serves two purposes: Besides describing how you might go about designing a study, it provides a map of the remainder of this book.
Next, we'll look at the elements of research proposals. Often, you'll need to detail your intentions before you actually conduct your research; this might be required in order to obtain funding for a major project or perhaps to get your instructor's approval for a class project. You'll see that the research proposal provides an excellent opportunity for you to consider all aspects of your research in advance. Also, this section should help you with the end-of-chapter exercise concerning the research proposal, if you are doing that. Finally, the last section of this chapter focuses on the ethical dimension of research design.
90 ■ Chapter 4: Research Design
Three Purposes of Research
Sodal research can serve many purposes. Three of the most common and useful purposes are exploration, description, and explanation. Although a given study can have more than one of these purposes—and most do—examining them separately is useful because each has different implications for other aspects of research design.
Exploration
Much of social research is conducted to explore a topic, that is, to start to familiarize a researcher with that topic. This approach typically occurs when a researcher examines a new interest or when the subject of study itself is relatively new.
As an example, let's suppose that widespread taxpayer dissatisfaction with the government erupts into a taxpayers' revolt. People begin refusing to pay their taxes, and they organize themselves around that issue. You might like to learn more about the movement: How widespread is it? What levels and degrees of support are there within the community? How is the movement organized? What kinds of people are active in it? An exploratory study could help you find at least approximate answers to some of these questions. You might check figures with tax-collecting officials, collect and study the literature of the movement, attend meetings, and interview leaders.
Exploratory studies are also appropriate for more-persistent phenomena. Suppose you're unhappy with your college's graduation requirements and want to help change them. You might study the history of such requirements at the college and meet with college officials to learn the reasons for the current standards. You could talk to several students to get a rough idea of their sentiments on the subject. Though this last activity would not necessarily yield an accurate picture of student opinion, it could suggest what the results of a more extensive study might be.
Sometimes exploratory research is pursued through the use of focus groups, or guided small-group discussions. This technique is frequently used in market research; we'll examine it further in Chapter 10.
Exploratory studies are most typically done for three purposes: (1) to satisfy the researcher's
curiosity and desire for better understanding, (2) to test the feasibility of undertaking a more extensive study, and (3) to develop the methods to be employed in any subsequent study.
A while back, for example, I became aware of the growing popularity of something called "channeling," in which a person known as a channel or medium enters a trance state and begins speaking with a voice that claims it originates outside the channel. Some of the voices say they come from a spirit world of the dead, some say they are from other planets, and still others say they exist in dimensions of reality difficult to explain in ordinary human terms.
The channeled voices, often referred to as "entities," sometimes use the metaphor of radio or television for the phenomenon they represent. "When you watch the news," one told me in the course of an interview, "you don't believe the network news anchor is really inside the television set. The same is true of me. I use this medium's body the way the reporter uses your television set."
The idea of channeling interested me from several perspectives, not the least of which was the methodological question of how to study scientifically something that violates so much of what we take for granted, including scientific staples such as space, time, causation, and individuality.
Lacking any rigorous theory or precise expectations, I merely set out to learn more. Using some of the techniques of qualitative field research we will discuss in Chapter 10,1 began amassing information and forming categories for making sense of what I observed. I read books and articles about the phenomenon and talked to people who had attended channeling sessions. I then attended channeling sessions myself, observing those who attended as well as the channel and entity. Next, I conducted personal interviews with numerous channels and entities.
In most interviews, I began by asking the human channels questions about how they first began channeling, what it was like, and why they continued, as well as standard biographical questions. The channel would then go into a trance, whereby the interview continued with the entity speaking. "Who are you?" I might ask. "Where do you come from?" "Why are you doing this?" "How can I tell if you are real or
04945_ch04_ptg01.indd  90 8/21/14  11:32 AM
a fake?" Although I went into these interview sessions with several questions prepared in advance, each of the interviews followed whatever course seemed appropriate in light of the answers given.
This example of exploration illustrates where social research often begins. Whereas researchers working from deductive theories have the key variables laid out in advance, one of my first tasks was to identify some of the possibly relevant variables. For example, I noted a channel's gender, age, education, religious background, regional origins, and previous participation in things metaphysical. I chose most of these variables because they commonly affect behavior.
I also noted differences in the circumstances of channeling sessions. Some channels said they must go into deep trances, some use light trances, and others remain conscious. Most sit down while channeling, but others stand and walk about. Some channels operate under pretty ordinary conditions; others seem to require props such as dim lights, incense, and chanting. Many of these differences became apparent to me only in the course of my initial observations.
Regarding the entities, I have been interested in classifying where they say they come from. Over the course of my interviews, I've developed a set of questions about specific aspects of "reality," attempting to classify the answers they give. Similarly, I ask each to speak about future events.
Over the duration of this research, my examination of specific topics has become increasingly focused as I've identified variables that seem worth pursuing: gender, education, and religion, for example. Note, however, that I began with a reasonably blank slate.
Exploratory studies are quite valuable in social science research. They're essential whenever a researcher is breaking new ground, and they almost always yield new insights into a topic for research. Exploratory studies are also a source of grounded theory, as discussed in Chapter 2.
The chief shortcoming of exploratory studies is that they seldom provide satisfactory answers to research questions, though they can hint at the answers and can suggest which research methods could provide definitive ones. The
Three Purposes of Research ■ 91
reason exploratory studies are seldom definitive in themselves has to do with representativeness; that is, the people you study in your exploratory research may not be typical of the larger population that interests you. Once you understand representativeness, you'll be able to know whether a given exploratory study actually answered its research problem or only pointed the way toward an answer. (Representativeness is discussed at length in Chapter 7.)
Description
A major purpose of many social science studies is to describe situations and events. The researcher observes and then describes what was observed. Because scientific observation is careful and deliberate, however, scientific descriptions are typically more accurate and precise than casual ones are.
The U.S. Census is an excellent example of descriptive social research. The goal of the census is to describe accurately and precisely a wide variety of characteristics of the U.S. population, as well as the populations of smaller areas such as states and counties. Other examples of descriptive studies are the computation of age-gender profiles of populations done by demographers, the computation of crime rates for different cities, and a product-marketing survey that describes the people who use, or would use, a particular product. A researcher who carefully chronicles the events that take place on a labor union picket line has, or at least serves, a descriptive purpose. A researcher who computes and reports the number of times individual legislators voted for or against organized labor also fulfills a descriptive purpose.
Many qualitative studies aim primarily at description. An anthropological ethnography, for example, may try to detail the particular culture of some preliterate society. At the same time, such studies are seldom limited to a merely descriptive purpose. Researchers usually go on to examine why the observed patterns exist and what they imply.
Explanation
The third general purpose of social science research is to explain things. Descriptive studies answer questions of what, where, when, and how;
04945_ch04_ptg01.indd  91 8/21/14  11:32 AM
92 ■ Chapter 4: Research Design
Research in Real Life
Putting Social Research to Work
Jacob Perry is a graduate student at the Clinton School of Public Service in Little Rock, Arkansas, and he chose to do his semester of field work abroad, in North Africa. He quickly discovered that the research techniques he had mastered in his studies did not necessarily fit into the research situation. Here's how he described it:
We Americans have our quite fixed ideas about what research is: intense preparation, thorough literature review, ample discussions to outline details, timeline of events to take place, schedule of responsibilities and activities to perform, etc. These simply must happen in order to perform reputable research. And everything must be agreed upon before the research begins. That is the American way (granted, it is also somewhat of an internationally accepted way as well) and it is in many ways a reflection of our organized, prompt, accountable, obligation and time-oriented culture. However, I am in Morocco performing a research project, and Moroccan culture has quite different views on time, responsibilities, and planning. Time here is neither rigid nor fixed. It is not linear but rather cyclical, meaning time is not lost—it is simple recovered later. Life is now; it is present; it is mostly unplanned.
Jacob found ways to develop rapport with his fellow researchers in order to put his research training to good use.
spent the first two weeks building trust, familiarity, comfort— relations! This has been vital to the project's progress, as our team is able to discuss openly, honestly, and sometimes aggressively to arrive at an agreement. I also speak the local language, which has allowed my Moroccan partners to remain in their linguistic comfort zone, remain in control of conversations concerning project work, and not feel they are being neo-colonized by a foreigner here to "save"their country.
Ultimately, Jacob felt the project had been successful, and it is obvious that it was a powerful learning experience for the young social researcher.
am so pleased with this project in Morocco because it is a partnership between myself and Moroccans who have already established the goals, needs, and approach to developing their society. This project depends entirely on local expertise and initiative. I am here because of a mutually expressed interest by myself and locals who want to improve their city. And the project has progressed because of continuous discussions in which all team members offer their perspectives. In short, learning, considering, and respecting local culture is necessary for international development work to be successful, and I am seeing a great example of effective development work on the ground in Morocco.
Source: Private communication with author, June 24,2013.
explanatory questions, of why. So when William Sanders (1994) set about describing the varieties of gang violence, he also wanted to reconstruct the process that brought about violent episodes among the gangs of different ethnic groups.
Reporting the voting intentions of an electorate is descriptive, but reporting why some people plan to vote for Candidate A and others for Candidate B is explanatory. Identifying variables that explain why some cities have higher crime rates than others involves explanation. A researcher who sets out to know why an antiabortion demonstration ended in a violent confrontation with police, as opposed to simply describing what happened, has an explanatory purpose.
Let's look at a specific case. Recent years have seen a radical shift in American attitudes toward marijuana. At first, support for the use of medical marijuana has increased in many states and, dramatically, Washington and Colorado legalized
its recreational use in 2012 and 2013, respectively. (The list may be longer by the time you read this.)
What factors do you suppose might shape people's attitudes toward the legalization of marijuana? To answer this, you might first consider whether men and women differ in their opinions. An explanatory analysis of the 2012 General Social Survey (GSS) data indicates that 53 percent of men and 42 percent of women said marijuana should be legalized.
What about political orientation? The GSS data show that 59 percent of liberals said marijuana should be legalized, compared with 51 percent of moderates and 37 percent of conservatives. Further, 50 percent of Democrats, compared with 52 percent of Independents and 38 percent of Republicans, supported legalization.
Given these statistics, you might begin to develop an explanation for attitudes toward marijuana legalization. Further study of gender and
04945_ch04_ptg01.indd  92 8/21/14  11:32 AM
political orientation might then lead to a deeper explanation of these attitudes.
In Chapter 1, we noted there were two different approaches to explanation in social research (and in everyday life). Let's return to those now.
In the remainder of this chapter, we'll examine some general approaches to research and the elements of research design you can choose among. As you do this, keep in mind that the advanced planning for research may not perfectly fit the situations you will confront in the field. The Research in Real Life box, "Putting Social Research to Work," reports on a graduate student's experience in the field.
Idiographic Explanation
As you will recall from Chapter 1, idiographic explanation seeks an exhaustive understanding of the causes producing events and situations in a single or limited number of cases. If you wished to understand why a student protest broke out on a particular college campus, you would seek to root out everything that contributed to that result. You would consider the history of the college, its organizational structure, the nature of the student body, the actions of influential individuals (administrators, faculty, students, others), the context of student activities nationally, triggering events (e.g., shutting down a student organization, arresting a student), and so forth. You'll know your analysis is complete when the explanatory factors you have assembled made the protest inevitable and when the absence of any of those factors might have kept it from happening.
There is no statistical test that can tell you when you have achieved this analytical success, however. This conclusion rests on the "art" of social research, which is achieved primarily through experience: by reading the analyses of others and by conducting your own. Here are a few techniques to consider.
•   Pay attention to the explanations offered by the people living the social processes you are studying. It is important that you not believe everything you are told, of course, but don't make the opposite mistake of thinking you understand the situation better than those living there. (Social researchers have sometimes been accused of a
Nomothetic Explanation ■ 93
certain degree of arrogance in this respect.) If there is wide agreement as to the importance of a certain factor, that should increase your confidence that it was a cause of the event under study. This would be more so if participants with very different points of view agree on that point. In the case of the student protest, administrators and students are likely to have very different opinions about what happened, but if they all agree that the arrest of a student activist was a triggering event, then it probably was an important cause. •   Comparisons with similar situations, either in different places or at different times in the same place, can be insightful. Perhaps the campus in question has had previous protests or perhaps there was a time when a protest almost occurred but didn't. Knowledge of such instances can provide useful comparisons and contrasts to the case under study. Similarly, protests or nonprotests at other campuses can offer useful comparisons.
Nomothetic Explanation
Earlier in this chapter, the examination of what factors might cause attitudes about legalizing marijuana illustrates nomothetic explanation. Recall that in this model, we try to find a few factors (independent variables) that can account for much of the variation in a given phenomenon. Thus, we saw, men were more likely than women to support legalization; liberals more likely than conservatives, and so on. This explanatory model stands in contrast to the idiographic model, in which we seek a complete, in-depth understanding of a single case.
In our example, an idiographic approach would suggest all the reasons that one person was opposed to legalization—involving what her parents, teachers, and clergy told her about it; any bad experiences experimenting with it; and so forth. When we understand something idiographically, we feel we really understand it. When we know all the reasons why someone opposed legalizing marijuana, we couldn't imagine that person having any other attitude.
In contrast, a nomothetic approach might suggest that overall political orientations account for much of the difference of opinion
04945_ch04_ptg01.indd  93 8/21/14  11:32 AM
94 ■ Chapter 4: Research Design
about legalizing marijuana. Because this model is inherently probabilistic, it is more open to misunderstanding and misinterpretation than the idiographic model is. Let's examine what social researchers mean when they say one variable (nomothetically) causes another. Then, we'll look at what they don't mean.
Criteria for Nomothetic Causality
There are three main criteria for nomothetic causal relationships in social research: (1) the variables must be correlated, (2) the cause takes place before the effect, and (3) the variables are nonspurious.
Correlation
Unless some actual relationship—a statistical correlation—is found between two variables, we can't say that a causal relationship exists. Our analysis of GSS data suggested that political orientation was a cause of attitudes about legalizing marijuana. Had the same percentage of liberals and conservatives supported legalization, we could hardly say that political orientations caused the attitude. Though this criterion is obvious, it emphasizes the need to base social research assertions on actual observations rather than assumptions.
Time Order
Next, we can't say a causal relationship exists unless the cause precedes the effect in time. Notice that it makes more sense to say that most children's religious affiliations are caused by those of their parents than to say that parents' affiliations are caused by those of their children— even though it would be possible for you to change your religion and for your parents to
correlation An empirical relationship between two variables such that (1) changes in one are associated with changes in the other, or (2) particular attributes of one variable are associated with particular attributes of the other. Correlation in and of itself does not constitute a causal relationship between the two variables, but it is one crite rion of causality.
spurious relationship A coincidental statistical correlation between two variables, shown to be caused by some third variable.
follow suit. Remember, nomothetic explanation deals with general patterns but not all cases.
In our marijuana example, it would make sense to say that gender causes, to some extent, attitudes toward legalization, whereas it would make no sense to say that opinions about marijuana determine a person's gender. Notice, however, that the time order connecting political orientations and attitudes about legalization is less clear, though we sometimes reason that general orientations cause specific opinions. And sometimes our analyses involve two or more independent variables that were established at the same time: looking at the effects of gender and race on voting behavior, for example. As we'll see in the next chapter, the issue of time order can be a complex matter.
Nonspuriousness
The third requirement for a causal relationship is that the effect cannot be explained in terms of some third variable. For example, there is a correlation between ice-cream sales and deaths due to drowning: the more ice cream sold, the more drownings, and vice versa. There is, however, no direct link between ice cream and drowning. The third variable at work here is season or temperature. Most drowning deaths occur during summer—the peak period for ice-cream sales.
Here are a couple of other examples of spurious relationships, or ones that aren't genuine. There is a negative relationship between the number of mules and the number of Ph.D.'s in towns and cities: the more mules, the fewer Ph.D.'s and vice versa. Perhaps you can think of another variable that would explain this apparent relationship. The answer is rural versus urban settings: There are more mules (and fewer Ph.D.'s) in rural areas, whereas the opposite is true in cities.
Or, consider the positive correlation between shoe size and math ability among schoolchildren. Here, the third variable that explains the puzzling relationship is age. Older children have bigger feet and more highly developed math skills, on average, than younger children do. See Figure 4-1 for an illustration of this spurious relationship. Notice that observed associations go in both directions. That is, as one variable occurs or changes, so does the other.
04945_ch04_ptg01.indd  94 8/21/14  11:32 AM
Nomothetic Explanation ■ 95
Observed Correlation Positive (direct) correlation
Shoe size   4 Math skill
Bigger shoe size is associated with greater math skill, and vice versa.
Spurious causal relationships Actual causal relationships
Neither shoe size nor math skill is a cause The underlying variable of age causes both
of the other. bigger shoe size and greater math skill,
thus explaining the observed correlation.
FIGURE 4-1
An Example of a Spurious Causal Relationship. Finding an empirical correlation between two variables does not necessarily establish a causal relationship. Sometimes the observed correlation is the incidental result of other causal relationships, involving other variables.
© Cengage Learning®
At the top of the diagram, we see the observed correlation between shoe size and math ability. The double-headed arrow indicates that we don't know which variable might cause the other. To the bottom left of the diagram, we suggest that thinking math ability causes shoe size or that shoe size causes math ability are spurious, not real causal relationships. The bottom right of the diagram indicates what is actually at work. A third variable, age, causes (1) shoe size, in that older kids have bigger feet and (2) older kids know more math.
The list goes on. Areas with many storks have high birthrates. Those with few storks have low birthrates. Do storks really deliver babies? Birthrates are higher in the country than in the city; more storks live in the country than the city. The third variable here is urban/rural areas.
Finally, the more fire trucks that put out a fire, the more damage to the structure. Can you guess what the third variable is? In this case, it's the size of the fire.
Thus, when social researchers say there is a causal relationship between, say, education and racial tolerance, they mean (1) there is a statistical correlation between the two variables, (2) a
person's educational level occurred before their current level of tolerance or prejudice, and (3) there is no third variable that can explain away the observed correlation as spurious.
Nomothetic Causal Analysis and Hypothesis Testing
The nomothetic model of causal analysis lends itself to hypothesis testing (see Chapter 1), although hypotheses are not required in nomothetical research. To test a hypothesis, you would carefully specify the variables you think are causally related, as well as specifying the manner in which you will measure them. (These steps will be discussed in detail in the following chapter under the terms conceptualization and operationalization.)
In addition to hypothesizing that two variables will be correlated, you may specify the strength of the relationship you expect within the study design you are using. Often this specification will take the form of a level of statistical significance: the chance you are willing to take that a given relationship might have been caused by chance in the selection of subjects for study.
04945_ch04_ptg01.indd  95 8/21/14  11:32 AM
96 ■ Chapter 4: Research Design
(This will be discussed further in Chapter 7, "The Logic of Sampling.")
Finally, you may specify the tests for spuri-ousness that any observed relationship must survive. Not only will you hypothesize, for example, that increased education will reduce levels of prejudice, but you will specify further that the hypothesized relationship will not be the product of, say, political orientations.
False Criteria for Nomothetic Causality
Because notions of cause and effect are well entrenched in everyday language and logic, it's important to specify some of the things social researchers do not mean when they speak of causal relationships. When they say that one variable causes another, they do not necessarily mean to suggest complete causation, to account for exceptional cases, or to claim that the causation exists in a majority of cases.
Complete Causation
Whereas an idiographic explanation of causation is relatively complete, a nomothetic explanation is probabilistic and usually incomplete. As we've seen, social researchers may say that political orientations cause attitudes toward legalizing marijuana even though not all liberals approve nor all conservatives disapprove. Thus, we say that political orientation is one of the causes of the attitude, but not the only one.
Exceptional Cases
In nomothetic explanations, exceptions do not disprove a causal relationship. For example, it is consistently found that women are more religious than men in the United States. Thus, gender may be a cause of religiosity, even if your uncle is a religious zealot or you know a woman who is an avowed atheist. Those exceptional cases do not disprove the overall, causal pattern.
Majority of Cases
Causal relationships can be true even if they don't apply in a majority of cases. For example, we say that children who are not supervised after school are more likely to become delinquent than are those who are supervised; hence,
lack of supervision is a cause of delinquency. This causal relationship holds true even if only a small percentage of those not supervised become delinquent. As long as they are more likely to be delinquent than are those who are supervised, we say there is a causal relationship.
The social science view of causation may vary from what you are accustomed to, because people commonly use the term cause to mean something that completely causes another thing. The somewhat different standard used by social researchers can be seen more clearly in terms of necessary and sufficient causes.
Necessary and Sufficient Causes
A necessary cause represents a condition that must be present for the effect to follow. For example, it is necessary for you to take college courses in order to get a degree. Take away the courses, and the degree never follows. However, simply taking the courses is not a sufficient cause of getting a degree. You need to take the right ones and pass them. Similarly, being female is a necessary condition of becoming pregnant, but it is not a sufficient cause. Otherwise, all women would get pregnant. Figure 4-2 illustrates this relationship between the variables of sex and pregnancy as a matrix showing the possible outcomes of combining these variables.
A sufficient cause, on the other hand, represents a condition that, if it is present, guarantees the effect in question. This is not to say that a sufficient cause is the only possible cause of a particular effect. For example, skipping an exam in this course would be a sufficient cause for failing it, though students could fail it other ways as well. Thus, a cause can be sufficient, but not necessary. Figure 4-3 illustrates the relationship between taking or not taking the exam and either passing or failing it.
The discovery of a cause that is both necessary and sufficient is, of course, the most satisfying outcome in research. If juvenile delinquency were the effect under examination, it would be nice to discover a single condition that (1) must be present for delinquency to develop and (2) always results in delinquency. In such a case, you would surely feel that you knew precisely what caused juvenile delinquency.
04945_ch04_ptg01.indd  96 8/21/14  11:32 AM
Units of Analysis ■ 97
Female
t t
f.f.f.f.f.f f I t f I
^   ^   ^   ^ ^
FIGURE 4-2
Necessary Cause. Being female is a necessary cause of pregnancy; that is, you can't get pregnant unless you are female.
© Cengage Learning®
Took the exam
Didn't take the exam
you have determined the sufficient causes for a particular result. (Anyone with all the same details of your genetic inheritance, upbringing, and subsequent experiences would have ended up going to college.) At the same time, there could always be other causal paths to the same result. Thus, the idiographic causes are sufficient but not necessary.
Units of Analysis
FIGURE 4-3
Sufficient Cause. Not taking the exam is a sufficient cause of failing it, even though there are other ways of failing (such as answering randomly).
© Cengage Learning®
Unfortunately, we never discover single causes that are absolutely necessary and absolutely sufficient when analyzing the nomothetic relationships among variables. It is not uncommon, however, to find causal factors that are either 100 percent necessary (you must be female to become pregnant) or 100 percent sufficient (skipping an exam will inevitably cause you to fail it).
In the idiographic analysis of single cases, you may reach a depth of explanation from which it is reasonable to assume that things could not have turned out differently, suggesting
In social research, there is virtually no limit to what or who can be studied, or the units of analysis. This topic is relevant to all forms of social research, although its implications are clearest in the case of nomothetic, quantitative studies.
The idea for units of analysis may seem slippery at first, because research—especially nomothetic research—often studies large collections of people or things, or aggregates. It's important to distinguish between the unit of analysis and the aggregates that we generalize about. For instance, a researcher may study a class of people, such as Democrats, college undergraduates, African American women under 30, or some
units of analysis The what or who being studied. In social science research, the most typical units of analysis are individual people.
04945_ch04_ptg01.indd  97 8/21/14  11:32 AM
98 ■ Chapter 4: Research Design
other collection. But if the researcher is interested in exploring, describing, or explaining how different groups of individuals behave as individuals, the unit of analysis is the individual, not the group. This is true even though the researcher uses the information about individuals to generalize about aggregates of individuals, as in saying that more Democrats than Republicans favor legalizing marijuana. Think of it this way: Having an attitude about marijuana is something that can only be an attribute of an individual, not a group; that is, there is no one group "mind" that can have an attitude. So even when we generalize about Democrats, we're generalizing about an attribute they possess as individuals.
In contrast, we may sometimes want to study groups, considered as individual "actors" or entities that have attributes as groups. For instance, we might want to compare the characteristics of different types of street gangs. In that case our unit of analysis would be gangs (not members of gangs), and we might proceed to make generalizations about different types of gangs. For example, we might conclude that male gangs are more violent than female gangs. Each gang (unit of analysis) would be described in terms of two variables: (1) What gender are the members? and (2) How violent are its activities? So we might study 52 gangs, reporting that 40 were male and 12 were female, and so forth. The "gang" would be the unit of analysis, even though some of the characteristics were drawn from the components (members) of the gangs.
Social researchers tend to choose individual people as their units of analysis. You may note the characteristics of individual people—gender, age, region of birth, attitudes, and so forth. You can then combine these descriptions to provide a composite picture of the group the individuals represent, whether a street-corner gang or a whole society.
For example, you may note the age and gender of each student enrolled in Political Science 110 and then characterize the group of students as being 53 percent men and 47 percent women and as having a mean age of 18.6 years. Although the final description would be of the class as a whole, the description is based on characteristics that members of the class have as individuals.
The same distinction between units of analysis and aggregates occurs in explanatory studies.
Suppose you wished to discover whether students with good study habits received better grades in Political Science 110 than students with poor study habits did. You would operational-ize the variable study habits and measure this variable, perhaps in terms of hours of study per week. You might then aggregate students with good study habits and those with poor study habits and see which group received the best grades in the course. The purpose of the study would be to explain why some groups of students do better in the course than others do, but the unit of analysis is still individual students.
Units of analysis in a study are usually also the units of observation. Thus, to study success in a political science course, we would observe individual students. Sometimes, however, we "observe" our units of analysis indirectly. For example, suppose we want to find out whether disagreements about the death penalty tend to cause divorce. In this case, we might "observe" individual husbands and wives by asking them about their attitudes toward capital punishment, in order to distinguish couples who agree and disagree on this issue. In this case, our units of observation are individual wives and husbands, but our units of analysis (the things we want to study) are couples.
Units of analysis, then, are those things we examine in order to create summary descriptions of all such units and to explain differences among them. In most research projects, the unit of analysis will probably be clear to you. When the unit of analysis is not clear, however, it's essential to determine what it is; otherwise, you cannot determine what observations are to be made about whom or what.
Some studies try to describe or explain more than one unit of analysis. In these cases, the researcher must anticipate what conclusions she or he wishes to draw with regard to which units of analysis. For example, we may want to discover what kinds of college students (individuals) are most successful in their careers; we may also want to learn what kinds of colleges (organizations) produce the most-successful graduates.
Here's an example that illustrates the complexity of units of analysis. Murder is a fairly personal matter: One individual kills another individual. However, when Charis Kubrin and
04945_ch04_ptg01.indd  98 8/21/14  11:32 AM
Ronald Weitzer (2003: 157) ask, "Why do these neighborhoods generate high homicide rates?" the unit of analysis in that phrase is neighborhood. You can probably imagine some kinds of neighborhoods (e.g., poor, urban) that would have high homicide rates and some (e.g., wealthy, suburban) that would have low rates. In this particular conversation, the unit of analysis (neighborhood) would be categorized in terms of variables such as economic level, locale, and homicide rate.
In their analysis, however, Kubrin and Weitzer were also interested in different types of homicide: in particular, those that occurred in retaliation for some earlier event, such as an assault or insult. Can you identify the unit of analysis common to all of the following excerpts?
1. The sample of killings . . .
2. The coding instrument includes over 80 items related to the homicide.
3. Of the 2,161 homicides that occurred from 1985 [to] 1995 . . .
4. Of those with an identified motive, 19.5 percent (n = 337) are retaliatory.
(Kubrin and Weitzer 2003:163)
In each of these excerpts, the unit of analysis is homicide (also called killing or murder). Sometimes you can identify the unit of analysis in the description of the sampling methods, as in the first excerpt. A discussion of classification methods might also identify the unit of analysis, as in the second excerpt (80 ways to code the homicides). Often, numerical summaries point the way: 2,161 homicides; 19.5 percent (of the homicides). With a little practice you'll be able to identify the units of analysis in most social research reports, even when more than one is used in a given analysis.
To explore this topic in more depth, let's consider several common units of analysis in social research.
Individuals
As mentioned, individual human beings are perhaps the most typical units of analysis for social research. Social researchers tend to describe and explain social groups and interactions by aggregating and manipulating the descriptions of individuals.
Units of Analysis ■ 99
Any type of individual may be the unit of analysis for social research. This point is more important than it may seem at first. The norm of generalized understanding in social research should suggest that scientific findings are most valuable when they apply to all kinds of people. In practice, however, social researchers seldom study all kinds of people. At the very least, their studies are typically limited to the people living in a single country, though some comparative studies stretch across national boundaries. Often, though, studies are quite circumscribed.
Examples of classes of individuals that might be chosen for study include students, gays and lesbians, autoworkers, voters, single parents, and faculty members. Note that each of these terms implies some population of individuals. Descriptive studies with individuals as their units of analysis typically aim to describe the population that comprises those individuals, whereas explanatory studies aim to discover the social dynamics operating within that population.
As the units of analysis, individuals may be characterized in terms of their membership in social groupings. Thus, an individual may be described as belonging to a rich family or to a poor one, or a person may be described as having a college-educated mother or not. We might examine in a research project whether people with college-educated mothers are more likely to attend college than are those with non-college-educated mothers or whether high school graduates in rich families are more likely than those in poor families to attend college. In each case, the unit of analysis—the "thing" whose characteristics we are seeking to describe or explain—is the individual. We then aggregate these individuals and make generalizations about the population they belong to.
Groups
Social groups can also be units of analysis in social research. That is, we may be interested in characteristics that belong to one group, considered as a single entity. If you were to study the members of a criminal gang to learn about criminals, the individual (criminal) would be the unit of analysis; but if you studied all the gangs in a city to learn the differences, say, between big
04945_ch04_ptg01.indd  99 8/21/14  11:32 AM
100 ■ Chapter 4: Research Design
gangs and small ones, between "uptown" and "downtown" gangs, and so forth, you would be interested in gangs rather than their individual members. In this case, the unit of analysis would be the gang, a social group.
Here's another example. Suppose you were interested in the question of access to computers in different segments of society. You might describe families in terms of total annual income and according to whether or not they had computers. You could then aggregate families and describe the mean income of families and the percentage with computers. You would then be in a position to determine whether families with higher incomes were more likely to have computers than were those with lower incomes. In this case, the unit of analysis would be families.
As with other units of analysis, we can derive the characteristics of social groups from those of their individual members. Thus, we might describe a family in terms of the age, race, or education of its head. In a descriptive study, we might find the percentage of all families that have a college-educated head of family. In an explanatory study, we might determine whether such families have, on average, more or fewer children than do families headed by people who have not graduated from college. In each of these examples, the family is the unit of analysis. In contrast, had we asked whether college-educated individuals have more or fewer children than do their less-educated counterparts, then the individual would have been the unit of analysis.
Other units of analysis at the group level could be friendship cliques, married couples, census blocks, cities, or geographic regions. As with individuals, each of these terms implies some population. Street gangs implies some population that includes all street gangs, perhaps in a given city. You might then describe this population by generalizing from your findings about individual gangs. For instance, you might describe the geographic distribution of gangs throughout a city. In an explanatory study of street gangs, you might discover whether large gangs are more
social artifact Any product of social beings or their behavior. Can be a unit of analysis.
likely than small ones to engage in intergang warfare. Thus, you would arrive at conclusions about the population of gangs by using individual groups as your unit of analysis.
Organizations
Formal social organizations may also be the units of analysis in social research. For example, a researcher might study corporations, by which he or she implies a population of all corporations. Individual corporations might be characterized in terms of their number of employees, net annual profits, gross assets, number of defense contracts, percentage of employees from racial or ethnic minority groups, and so forth. We might determine whether large corporations hire a larger or smaller percentage of minority group employees than small corporations do. Other examples of formal social organizations suitable as units of analysis include church congregations, colleges, army divisions, academic departments, and supermarkets.
Figure 4-4 provides a graphic illustration of some different units of analysis and the statements that might be made about them.
Social Interactions
Sometimes social interactions are the relevant units of analysis. Instead of individual humans, you can study what goes on between them: telephone calls, kisses, dancing, arguments, fistfights, e-mail exchanges, chat-room discussions, and so forth. As you saw in Chapter 2, social interaction is the basis for one of the primary theoretical paradigms in the social sciences, and the number of units of analysis that social interactions provide is nearly infinite.
Even though individuals are usually the actors in social interactions, there is a difference between (1) comparing the kinds of people who subscribe to different Internet service providers (individuals being the units of analysis) and (2) comparing the length of chat-room interactions on those same providers (interactions being the units of analysis).
Social Artifacts
Another unit of analysis is the social artifact, or
any product of social beings or their behavior. One class of artifacts includes concrete objects
04945_ch04_ptg01.indd   100 8/21/14  11:32 AM
Units of Analysis ■ 101
Units of Analysis
**   ♦* H
*t ** ♦ ♦
Sample Statements
60% of the sample are women
10% of the sample are wearing an eye patch
10% of the sample have pigtails
20% of the families have a single parent
50% of the families have two children 20% of the famillies have no children
The mean number of children per family is 1.3
20% of the households are occupied by more than one family
30% of the households have holes in their roofs
10% of the households are occupied by aliens
Notice also that 33% of the families live in multiple-family households with family as the unit of analysis
FIGURE 4-4
Illustrations of Units of Analysis. Units of analysis in social research can be individuals, groups, or even nonhuman entities.
© Cengage Learning®
04945_ch04_ptg01.indd 101
8/21/14  11:32 AM
102 ■ Chapter 4: Research Design
Tips and Tools
Identifying the Unit of Analysis
The unit of analysis is an important element in research design, and later, in data analysis. However, students sometimes find identifying it elusive. The easiest way to identify the unit of analysis is to examine a statement regarding the variables under study.
Consider the following: "The average household income was $40,000." Income is the variable of interest, but who or what has income? Households, in this instance. We would arrive at the given statement by examining the incomes of several households. To calculate the mean (average) income, we would add up all the household incomes and divide by the number of households. Household is the unit of analysis. It is the unit being analyzed in terms of the variable, income.
Consider another statement: "Italian movies show more nudity than do American movies." The variable here is the extent to which nudity is shown, but who or what stas nudity? Movies. Movies are the units of analysis.
One way of identifying the unit of analysis is to imagine the process that would result in the conclusion reached.
Consider this research conclusion:"Twenty-four percent of the families have more than one adult earning at least $30,000 a year."To be sure, adults are earning the income, but the statement is about whether families have such adults. To make this statement, we would study several families. For each, we would ask whether they had more than two adults earning in excess of $30,000; each family would be scored as"yes" or"no"in that respect. Finally, we would calculate the percentage of families scored as"yes."The family, therefore, is the unit of analysis.
such as books, poems, paintings, automobiles, buildings, songs, pottery, jokes, student excuses for missing exams, and scientific discoveries.
For example, Lenore Weitzman and her associates (1972) were interested in learning how gender roles are taught. They chose children's picture books as their unit of analysis. Specifically, they examined books that had received the Caldecott Medal. Their results were as follows:
We found that females were underrepre-sented in the titles, central roles, pictures, and stories of every sample of books we examined. Most children's books are about boys, men, male animals, and deal exclusively with male adventures. Most pictures show men singly or in groups. Even when women can be found in the books, they often play insignificant roles, remaining both inconspicuous and nameless.
(Weitzman etal. 1972:1128)
In a follow-up study, Roger Clark, Rachel Lennon, and Leana Morris (1993) concluded that male and female characters were portrayed less stereotypically than before, observing a clear progress toward portraying men and women in nontraditional roles. However, they did not find total equality between the genders.
As this example suggests, just as people or social groups imply populations, each social object implies a set of all objects of the same class:
all books, all novels, all biographies, all introductory sociology textbooks, all cookbooks, all press conferences. In a study using books as the units of analysis, an individual book might be characterized by its size, weight, length, price, content, number of pictures, number sold, or description of the author. Then the population of all books or of a particular kind of book could be analyzed for the purpose of description or explanation: what kinds of books sell best and why, for example.
Similarly, a social researcher could analyze whether paintings by Russian, Chinese, or U.S. artists showed the greatest degree of working-class consciousness, taking paintings as the units of analysis and describing each, in part, by the nationality of its creator. Or you might examine a newspaper's editorials regarding a local university, for the purpose of describing, or perhaps explaining, changes in the newspaper's editorial position on the university over time. In this example, individual editorials would be the units of analysis. See the Tips and Tools box, "Identifying the Unit of Analysis," for more.
Units of Analysis in Review
The examples in this section should suggest the nearly infinite variety of possible units of analysis in social research. Although individual human beings are typical objects of study, many research questions can be answered more appropriately
102
through the examination of other units of analysis. Indeed, social researchers can study just about anything that bears on social life.
Moreover, the types of units of analysis named in this section do not begin to exhaust the possibilities. This has been a topic of discussion and elaboration for some time. Morris Rosenberg (1968: 234-48), for example, speaks of individual, group, organizational, institutional, spatial, cultural, and societal units of analysis. John Lofland and his associates (2006: 122-32) speak of practices, episodes, encounters, roles and social types, social and personal relationships, groups and cliques, organizations, settlements and habitats, subcultures, and lifestyles as suitable units of study. The important thing here is to grasp the logic of units of analysis. Once you do, the possibilities for fruitful research are limited only by your imagination.
Categorizing possible units of analysis might make the concept seem more complicated than it needs to be. What you call a given unit of analysis—a group, a formal organization, or a social artifact—is irrelevant. The key is to be clear about what your unit of analysis is. When you embark on a research project, you must decide whether you're studying marriages or marriage partners, crimes or criminals, corporations or corporate executives. Otherwise, you run the risk of drawing invalid conclusions because your assertions about one unit of analysis are actually based on the examination of another. We'll see an example of this issue in the next section as we look at the ecological fallacy.
Faulty Reasoning about Units of Analysis: The Ecological Fallacy and Reductionism
At this point, it's appropriate to introduce two types of faulty reasoning that you should be aware of: the ecological fallacy and reductionism. Each represents a potential pitfall regarding units of analysis, and either can occur in doing research and drawing conclusions from the results.
The Ecological Fallacy
In this context, "ecological" refers to groups or sets or systems: something larger than individuals. The ecological fallacy is the assumption that something learned about an ecological unit says something
Units of Analysis ■ 103
about the individuals making up that unit. Let's consider a hypothetical illustration of this fallacy.
Suppose we're interested in learning something about the nature of electoral support received by a female political candidate in a recent citywide election. Let's assume we have the vote tally for each precinct so we can tell which precincts gave her the greatest support and which the least. Assume also that we have census data describing some characteristics of these precincts. Our analysis of such data might show that precincts with relatively young voters gave the female candidate a greater proportion of their votes than precincts with older voters did. We might be tempted to conclude from these findings that younger voters are more likely to vote for female candidates than older voters are—in other words, that age affects support for the woman. In reaching such a conclusion, we run the risk of committing the ecological fallacy because it may have been the older voters in those "young" precincts who voted for the woman. Our problem is that we have examined precincts as our units of analysis but wish to draw conclusions about voters.
The same problem would arise if we discovered that crime rates were higher in cities having large African American populations than in those with few African Americans. We would not know if the crimes were actually committed by African Americans. Or, if we found suicide rates higher in Protestant countries than in Catholic ones, we still could not know for sure that more Protestants than Catholics committed suicide.
In spite of these hazards, social researchers often have little choice but to address a particular research question through an ecological analysis. Perhaps the most appropriate data are simply not available. For example, the precinct vote tallies and the precinct characteristics mentioned in our initial example may be easy to obtain, but we may not have the resources to conduct a postelection survey of individual voters. In such cases, we may reach a tentative conclusion, recognizing and noting the risk of an ecological fallacy.
Although you should be careful not to commit the ecological fallacy, don't let these warnings
ecological fallacy Erroneously drawing conclusions about individuals solely from the observation of groups.
04945_ch04_ptg01.indd   103 8/21/14  11:32 AM
104 ■ Chapter 4: Research Design
Research in Real Life
Red Families and Blue Families
During recent American political campaigns, concern for "family values" has often been featured as a hot-button issue. Typically, conservatives and Republicans have warned of the decline of such traditional values, citing divorce rates, teen pregnancies, same-sex marriage, and such. This is, however, a more complex matter than would fit on a bumper sticker.
In their analysis of conservative "red families"and liberal "blue families," Naomi Cahn and June Carbone report:
Red family champions correctly point out that growing numbers of single-parent families threaten the well-being of the next generation, and they accurately observe that greater male fidelity and female"virtue" strengthen relationships. Yet red regions of the country have higher teen pregnancy rates, more shotgun marriages, and lower average ages at marriage and first birth.
(2010:2)
Reviewing the Cahn-Carbone study, Jonathan Rauch headlines the question/'Do'Family Values'Weaken Families?"and summarizes the data thusly:
Six of the seven states with the lowest divorce rates in 2007, and all seven with the lowest teen birthrates in 2006, voted blue in both elections. Six of the seven states with the highest divorce rates in 2007, and five of the seven with the highest teen birthrates, voted red. It's as if family strictures undermine family structures.
(Rauch 2010)
Assuming that young people are going to have sex, Cahn and Carbone argue that the "traditional family values"that oppose sex education, contraception, and abortion will result in unplanned births that will typically be dealt with by forcing the young parents to marry. This, in turn, may interrupt their educations, limit their employment opportunities, lead to poverty, and result in unstable marriages that may not survive. This interpretation of the data may be completely valid, but can you recognize a methodological issue that might be raised?Think about the ecological fallacy.
The units of analysis used in these analyses are the 50 states of the union.The variables correlated are (1) overall voting patterns ofthe states and (2) family-problem rates in the states. States voting Republican overall have more problems than those voting Democratic overall. However, the data do not guarantee that Republican families or teenagers in Republican families have more problems than their Democratic counterparts. The ecological data suggest that's the case, but it is possible that Democrats in Republican states have the most family problems and Republicans in Democratic states have the least. It is unlikely but it is possible.
To be more confident about the conclusions drawn above, we would need to do a study in which the family or the individual was the unit of analysis.
Sources: Jonathan Rauch. 2010."Do'Family Values'Weaken families!" National Journal May 6; Naomi Cahn and June Carbone. 20]0. Red Families v. Blue Families: Legal Polarization and the Creation of Culture. (New York: Oxford University Press).
lead you into committing what we might call the "individualistic fallacy." Some people who approach social research for the first time have trouble reconciling general patterns of attitudes and actions with individual exceptions. But generalizations and probabilistic statements are not invalidated by individual exceptions. Your knowing
reductionism A fault of some researchers: a strict limitation (reduction) of the kinds of concepts to be considered relevant to the phenomenon under study.
The ecological fallacy deals with something else altogether—confusing units of analysis in such a way that we draw conclusions about individuals solely from the observation of groups. Although the patterns observed between variables at the level of groups may be genuine, the danger lies in reasoning from the observed attributes of groups to the attributes of the individuals who made up those groups, even though we have not actually observed individuals. The Research in Real Life box, "Red Families and Blue Families," illustrates some of the complexities presented by different units of analysis.
Reductionism
A second type of faulty reasoning related to units of analysis is reductionism. Reductionism
involves attempts to explain a particular phenomenon in terms of limited and/or lower-order
a rich Democrat, for example, doesn't deny the fact that most rich people vote Republican—as a general pattern. Similarly, if you know someone who has gotten rich without any formal education, that doesn't deny the general pattern of higher education relating to higher income.
04945_ch04_ptg01.indd   104 8/21/14  11:32 AM
The Time Dimension "105
concepts. The reductionist explanation is not altogether wrong; it is simply too limited. Thus, you might attempt to predict this year's winners and losers in the National Basketball Association by focusing on the abilities of the individual players on each team. This is certainly not stupid or irrelevant, but the success or failure of teams involves more than just the individuals in them; it involves coaching, teamwork, strategies, finances, facilities, fan loyalty, and so forth. To understand why some teams do better than others, you would make team the unit of analysis, and the quality of players would be one variable you would probably want to use in describing and classifying the teams.
Further, different academic disciplines approach the same phenomenon quite differently. Sociologists tend to consider sociological variables (such as values, norms, and roles), economists ponder economic variables (such as supply and demand and marginal value), and psychologists examine psychological variables (such as personality types and traumas). Explaining all or most human behavior in terms of economic factors is called economic reductionism, explaining it in terms of psychological factors is called psychological reductionism, and so forth. Notice how this issue relates to the discussion of theoretical paradigms in Chapter 2.
For many social scientists, the field of sociobiology is a prime example of reductionism, suggesting that all social phenomena can be explained in terms of biological factors. Thus, for example, Edward O. Wilson, sometimes referred to as the father of sociobiology (1975), sought to explain altruistic behavior in human beings in terms of genetic makeup. In his neo-Darwinian view, Wilson suggests that humans have evolved in such a way that individuals sometimes need to sacrifice themselves for the benefit of the whole species. Some people might explain such sacrifice in terms of ideals or warm feelings between humans. However, genes are the essential unit in Wilson's paradigm, producing his famous dictum that human beings are "only DNAs way of making more DNA."
Reductionism of any type tends to suggest that particular units of analysis or variables are more relevant than others. Suppose we ask what caused the American Revolution. Was it a shared commitment to the value of individual liberty? The
economic plight of the colonies in relation to Britain? The megalomania of the Founders? As soon as we inquire about the single cause, we run the risk of reductionism. If we were to regard shared values as the cause of the American Revolution, our unit of analysis would be the individual colonist. An economist, though, might choose the 13 colonies as units of analysis and examine the economic organizations and conditions of each. A psychologist might choose individual leaders as the units of analysis for purposes of examining their personalities. Of course, there's nothing wrong in choosing these units of analysis as part of an explanation of the American Revolution, but I think you can see how each alone would not produce a complete answer.
Like the ecological fallacy, reductionism can occur when we use inappropriate units of analysis. The appropriate unit of analysis for a given research question, however, is not always clear. Social researchers, especially across disciplinary boundaries, often debate this issue.
The Time Dimension
So far in this chapter, we've regarded research design as a process for deciding what aspects we'll observe, of whom, and for what purpose. Now we must consider a set of time-related options that cuts across each of these earlier considerations. We can choose to make observations more or less at one time or over a long period.
Time plays many roles in the design and execution of research, quite aside from the time it takes to do research. Earlier we noted that the time sequence of events and situations is critical to determining causation (a point we'll return to in Part 4). Time also affects the generalizability of research findings. Do the descriptions and explanations resulting from a particular study accurately represent the situation of ten years ago, ten years from now, or only the present? Researchers have two principal options available to deal with the issue of time in the design of their research: cross-sectional studies and longitudinal studies.
sociobiology A paradigm based on the view that social behavior can be explained solely in terms of genetic characteristics and behavior.
04945_ch04_ptg01.indd   105 8/21/14  11:32 AM
106 ■ Chapter 4: Research Design
Cross-Sectional Studies
A cross-sectional study involves observations of a sample, or cross section, of a population or phenomenon that are made at one point in time. Exploratory and descriptive studies are often cross-sectional. A single U.S. Census, for instance, is a study aimed at describing the U.S. population at a given time.
Many explanatory studies are also cross-sectional. A researcher conducting a large-scale national survey to examine the sources of racial and religious prejudice would, in all likelihood, be dealing with a single time frame—taking a snapshot, so to speak, of the sources of prejudice at a particular point in history.
Explanatory cross-sectional studies have an inherent problem. Although their conclusions are based on observations made at only one time, typically they aim at understanding causal processes that occur over time. This problem is somewhat akin to that of determining the speed of a moving object on the basis of a high-speed, still photograph that freezes the movement of the object.
Yanjie Bian, for example, conducted a survey of workers in Tianjin, China, for the purpose of studying stratification in contemporary, urban Chinese society. In undertaking the survey in 1988, however, he was conscious of the important changes brought about by a series of national campaigns, such as the Great Proletarian Cultural Revolution, dating from the Chinese Revolution in 1949 (which brought the Chinese Communists into power) and continuing into the present.
These campaigns altered political atmospheres and affected people's work and nonwork activities. Because of these campaigns, it is difficult to draw conclusions from a cross-sectional social survey, such as the one presented in this book, about general patterns of Chinese workplaces and their effects on workers. Such conclusions may be limited to one period of time and are subject to further tests based on data collected at other times.
(1994:19)
cross-sectional study A study based on observations representing a single point in time.
longitudinal study A study design involving the collection of data at different points in time.
The problem of generalizations about social life from a "snapshot" is one this book repeatedly addresses. One solution is suggested by Bian's final comment—about data collected "at other times": Social research often involves revisiting phenomena and building on the results of earlier research.
Longitudinal Studies
In contrast to cross-sectional studies, a longitudinal study is designed to permit observations of the same phenomenon over an extended period. For example, a researcher can participate in and observe the activities of a UFO cult from its inception to its demise. Other longitudinal studies use records or artifacts to study changes over time. In analyses of newspaper editorials or Supreme Court decisions over time, for example, the studies are longitudinal whether the researcher's actual observations and analyses were made at one time or over the course of the actual events under study.
Many field research projects, involving direct observation and perhaps in-depth interviews, are naturally longitudinal. Thus, for example, when Ramona Asher and Gary Fine (1991) studied the life experiences of the wives of alcoholic men, they were in a position to examine the evolution of troubled marital relationships over time, sometimes even including the reactions of the subjects to the research itself.
In the classic study When Prophecy Fails (1956), Leon Festinger, Henry Reicker, and Stanley Schachter were specifically interested in learning what happened to a flying saucer cult when their predictions of an alien encounter failed to come true. Would the cult members close down the group, or would they become all the more committed to their beliefs? A longitudinal study was required to provide an answer. (The cult redoubled their efforts to get new members.)
Longitudinal studies can be more difficult for quantitative studies such as large-scale surveys. Nonetheless, they are often the best way to study changes over time. There are three special types of longitudinal studies that you should know about: trend studies, cohort studies, and panel studies.
04945_ch04_ptg01.indd   106 8/21/14  11:32 AM
Trend Studies
A trend study is a type of longitudinal study that examines changes within a population over time. A simple example is a comparison of U.S. Censuses over a period of decades, showing shifts in the makeup of the national population. A similar use of archival data was made by Michael Carpini and Scott Keeter (1991), who wanted to know whether contemporary U.S. citizens were better or more poorly informed about politics than citizens of an earlier generation were. To find out, they compared the results of several Gallup Polls conducted during the 1940s and 1950s with a 1989 survey that asked several of the same questions tapping political knowledge.
Overall, the analysis suggested that contemporary citizens were slightly better informed than earlier generations were. In 1989, 74 percent of the sample could name the vice president of the United States, compared with 67 percent in 1952. Substantially higher percentages of people in 1989 than in 1947 could explain presidential vetoes and congressional overrides of vetoes. On the other hand, more of the 1947 sample could identify their U.S. representative (38 percent) than the 1989 sample (29 percent) could.
An in-depth analysis, however, indicates that the slight increase in political knowledge resulted from the fact that the people in the 1989 sample were more highly educated than those from earlier samples were. When educational levels were taken into account, the researchers concluded that political knowledge has actually declined within specific educational groups.
Cohort Studies
In a cohort study, a researcher examines specific subpopulations, or cohorts, as they change over time. Typically, a cohort is an age group, such as people born during the 1950s, but it can also be some other time grouping, such as people born during the Vietnam War, people who got married in 1994, and so forth. An example of a cohort study would be a series of national surveys, conducted perhaps every 20 years, to study the attitudes of the cohort born during World War II toward U.S. involvement in global affairs. A sample of people 15-20 years old might be surveyed in 1960, another sample of those 35-40 years old in 1980, and another sample of those 55-60 years old
The Time Dimension "107
in 2000. Although the specific set of people studied in each survey would differ, each sample would represent the cohort born between 1940 and 1945.
Figure 4- 5 offers a graphic illustration of a cohort design. In the example, three studies are being compared: one was conducted in 1990, another in 2000, and the third in 2010. Those who were 20 years old in the 1990 study are compared with those who were 30 in the 2000 study and those who were 40 in the 2010 study. Although the subjects being described in each of the three groups are different, each set of subjects represents the same cohort: those who were born in 1970.
James Davis (1992) turned to a cohort analysis in an attempt to understand shifting political orientations during the 1970s and 1980s in the United States. Overall, he found a liberal trend on issues such as race, sex, religion, politics, crime, and free speech. But did this trend represent people in general getting a bit more liberal, or did it merely reflect liberal younger generations replacing the conservative older ones?
To answer this question, Davis examined national surveys (from the General Social Survey, of which he is a founder) conducted in four time periods, five years apart. In each survey, he grouped the respondents into age groups, also five years apart. This strategy allowed him to compare different age groups at any given point in time as well as to follow the political development of each age group over time.
One of the questions he examined was whether a person who admitted to being a Communist should be allowed to speak in the respondents' communities. Consistently, the younger respondents in each time period were
trend study A type of longitudinal study in which a given characteristic of some population is monitored over time. An example would be a series of Gallup Polls showing the electorate's preferences for political candidates over the course of a campaign, even though different samples were interviewed at each point.
cohort study A study in which some specific subpopulation, or cohort, is studied over time, although data may be collected from different members in each set of observations. For example, a study of the occupational history of the class of 2000 in which questionnaires were sent every five years would be a cohort study.
04945_ch04_ptg01.indd   107 8/21/14  11:32 AM
108 ■ Chapter 4: Research Design
40
30
<
20
_L
1990
2000 Year
2010
FIGURE 4-5
A Cohort Study Design. Each of the three groups shown here is a sample representing people who were born in 1970.
© Cengage Learning®
more willing to let the Communist speak than the older ones were. Among those aged 20-40 in the first set of the survey, for example, 72 percent took this liberal position, contrasted with 27 percent among respondents 80 and older. What Davis found when he examined the youngest cohort over time is shown in Table 4-1. This pattern of a slight, conservative shift in the 1970s, followed by a liberal rebound in the 1980s, typifies the several cohorts Davis analyzed (J. Davis 1992: 269). Thus we see that this generational cohort did not change their overall level of political liberalism over time. We do not know, of course, whether individuals became more or less liberalism, only that such changes balanced out overall. The panel study design, which we will examine next, would reveal that possibility.
In another study, Eric Plutzer and Michael Berkman (2005) used a cohort design to completely reverse a prior conclusion regarding aging and support for education. Logically, as people grow well beyond the child-rearing years, we might expect them to reduce their commitment to educational funding. Moreover, cross-sectional data support that expectation. The researchers present several data sets showing those over 65 voicing less support for educational funding than those under 65 did.
panel study A type of longitudinal study, in which data are collected from the same set of people (the sample or panel) at several points in time.
Such simplistic analyses, however, leave out an important variable: increasing support for educational funding in U.S. society over time in general. The researchers add to this the concept of "generational replacement," meaning that the older respondents in a survey grew up during a time when there was less support for education in general, whereas the younger respondents grew up during a time of greater overall support.
A cohort analysis allowed the researchers to determine what happened to the attitudes of specific cohorts over time. Here, for example, are the percentages of Americans born during the 1940s who felt educational spending was too low, when members of that cohort were interviewed over time (Plutzer and Berkman 2005: 76):
Year Interviewed
1970s
1980s
1990s
2000s
Percent Who Say Educational Funding Is Too Low
58
66
74
79
As these data indicate, those who were born during the 1940s have steadily increased their support for educational funding as they have passed through and beyond the child-rearing years.
Panel Studies
Though similar to trend and cohort studies, a panel study examines the same set of people
04945_ch04_ptg01.indd 108
8/21/14  11:32 AM
The Time Dimension "109
TABLE 4-1
Age and Political Liberalism
Survey dates 1972 to 1974 1977 to 1980      1982 to 1984      1987 to 1989
Age of cohort 20-24 25-29 30-34 35-39
Percent who would let the
Communist speak 72 68 73 73
© Cengage Learning®
each time. For example, we could interview the same sample of voters every month during an election campaign, asking for whom they intended to vote. Though such a study would allow us to analyze overall trends in voter preferences for different candidates, it would also show the precise patterns of persistence and change in intentions. For example, a trend study that showed that Candidates A and B each had exactly half of the voters on September 1 and on October 1 as well could indicate that none of the electorate had changed voting plans, that all of the voters had changed their intentions, or something in-between. A panel study would eliminate this confusion by showing what kinds of voters switched from A to B and what kinds switched from B to A, as well as other facts.
Joseph Veroff, Shirley Hatchett, and Elizabeth Douvan (1992) wanted to learn about marital adjustment among newlyweds, specifically regarding differences between white and African American couples. To get subjects for study, they selected a sample of couples who applied for marriage licenses in Wayne County, Michigan, April through June 1986.
Concerned about the possible impact their research might have on the couples' marital adjustment, the researchers divided their sample in half at random: an experimental group and a control group (concepts we'll explore further in Chapter 8). Couples in the former group were intensively interviewed over a four-year period, whereas the latter group was contacted only briefly each year.
By studying the same couples over time, the researchers could follow the specific problems that arose and the way the couples dealt with them. As a by-product of their research, they found that those studied the most intensely seemed to achieve a somewhat better marital adjustment. The researchers felt that the interviews
could have forced couples to discuss matters they might have otherwise buried.
Panel mortality is a fundamental problem in panel studies: subjects dropping out of the study. Over the years, researchers have developed many techniques for tracking down the missing subjects. Bryan Rhodes and Ellen Marks (2011) used Facebook as a vehicle for tracking down members of a longitudinal study who had been unreachable by telephone or mail. They were successful in locating a third of the subjects.
Comparing the Three Types of Longitudinal Studies
To reinforce the distinctions among trend, cohort, and panel studies, let's contrast the three study designs in terms of the same variable: religious affiliation. A trend study might look at shifts in U.S. religious affiliations over time, as the Gallup Poll does on a regular basis. A cohort study might follow shifts in religious affiliations among "the 9/11 generation," specifically, say, people who were 10-20 years old on September 11, 2001. We could study a sample of people 20-30 years old in 2011, a new sample of people aged 30-40 in 2021, and so forth throughout their life span. A panel study could start with a sample of the whole population or of some special subset and study those specific individuals over time. Notice that only the panel study would give a full picture of the shifts among the various categories of affiliations, including "none." Cohort and trend studies would uncover only net changes.
Longitudinal studies have an obvious advantage over cross-sectional ones in providing information describing processes over time. But
panel mortality The failure of some panel subjects to continue participating in the study.
04945_ch04_ptg01.indd   109 8/21/14  11:32 AM
110 ■ Chapter 4: Research Design
Cross-Sectional Longitudinal
Trend      Cohort Panel
Snapshot in time X Measurements
XXX
across time
Follow age group across time
Study same people ^ over time
FIGURE 4-6
Comparing Types of Study Design
© 2016 Cengage Learning®
this advantage often comes at a heavy cost in both time and money, especially in a large-scale survey. Observations may have to be made at the time events are occurring, and the method of observation may require many research workers.
Panel studies, which offer the most comprehensive data on changes over time, face a special problem: panel mortality. Some of the respondents studied in the first wave of the survey might not participate in later waves. (This is comparable to the problem of experimental mortality, which will be discussed in Chapter 8.) The danger is that those who drop out of the study may be atypical, thereby distorting the results of the study. Thus, when Carol Aneshensel and her colleagues conducted a panel study of adolescent girls (comparing Latinas and non-Latinas), they looked for and found differences in characteristics of survey dropouts among Latinas born in the United States and those born in Mexico. These differences needed to be taken into account to avoid misleading conclusions about differences between Latinas and non-Latinas (Aneshensel et al. 1989).
Roger Tourangeau and Cong Ye (2009) were curious about ways of decreasing panel mortality. Specifically, they considered positive and negative inducements for subjects to continue. To find out, they randomly divided their panel survey sample in half and gave the two groups different pleas to continue. In one subsample, they stressed the benefits to be gained if everyone continued with the study. In the other sub-sample, they stressed how the study would be hurt by people dropping out. The latter, negative,
message increased continued participation by ten percentage points.
Figure 4-6 provides a schematic comparison of the several study types we have been discussing.
Approximating Longitudinal Studies
Longitudinal studies do not always provide a feasible or practical means of studying processes that take place over time. Fortunately, researchers often can draw approximate conclusions about such processes even when only cross-sectional data are available. Here are some ways to do that.
Sometimes cross-sectional data imply processes over time on the basis of simple logic. For example, in the study of student drug use conducted at the University of Hawaii (Chapter 2), students were asked to report whether they had ever tried each of several illegal drugs. The study found that some students had tried both marijuana and LSD, some had tried only one, and others had tried neither. Because these data were collected at one time, and because some students presumably would experiment with drugs later on, it would appear that such a study could not tell whether students were more likely to try marijuana or LSD first.
A closer examination of the data showed, however, that although some students reported having tried marijuana but not LSD, there were no students in the study who had tried only LSD. From this finding it was inferred—as common sense suggested—that marijuana use preceded
LSD use. If the process of drug experimentation occurred in the opposite time order, then a study at a given time should have found some students who had tried LSD but not marijuana, and it should have found no students who had tried only marijuana.
Researchers can also make logical inferences whenever the time order of variables is clear. If we discovered in a cross-sectional study of college students that those educated in private high schools received better college grades than those educated in public high schools did, we would conclude that the type of high school attended affected college grades, not the other way around. Thus, even though we made our observations at only one time, we would feel justified in drawing conclusions about processes taking place across time.
Very often, age differences discovered in a cross-sectional study form the basis for inferring processes across time. Suppose you're interested in the pattern of worsening health over the course of the typical life cycle. You might study the results of annual checkups in a large hospital. You could group health records according to the ages of those examined and rate each age group in terms of several health conditions—sight, hearing, blood pressure, and so forth. By reading across the age-group ratings for each health condition, you would have something approximating the health history of individuals. Thus, you might conclude that the average person develops vision problems before hearing problems. You would need to be cautious in this assumption, however, because the differences might reflect society-wide trends. Perhaps improved hearing examinations instituted in the schools had affected only the young people in your study.
Asking people to recall their pasts is another common way of approximating observations over time. Researchers use that method when they ask people where they were born or when they graduated from high school or whom they voted for in 1988. Qualitative researchers often conduct in-depth "life history" interviews. For example, C. Lynn Carr (1998) used this technique in a study of "tomboyism." Her respondents, aged 25-40, were asked to reconstruct aspects of their lives from childhood on, including experiences of identifying themselves as tomboys.
The Time Dimension "111
The danger in this technique is evident. Sometimes people have faulty memories; sometimes they lie. When people are asked in postelection polls whom they voted for, the results inevitably show more people voting for the winner than actually did so on election day. As part of a series of in-depth interviews, such a report can be validated in the context of other reported details; however, results based on a single question in a survey must be regarded with caution.
Cohorts can also be used to infer processes over time from cross-sectional data. For example, when Prem Saxena and his colleagues (2004) wanted to examine whether wartime conditions would affect the age at which people married, he used cross-sectional data from a survey of Lebanese women. During the Lebanese Civil War from 1975 to 1990, many young men migrated to other countries. By noting the year in which the survey respondents first married, he could determine that the average age-at-first-marriage increased with the onset of the war.
For a more in-depth and comprehensive analysis of longitudinal methodologies, you might consider Peter Lynn's 2009 edited volume. The several authors cover more aspects of this subject than would be feasible in this introductory textbook.
This discussion of the ways that time figures into social research suggests several questions you should confront in your own research projects. In designing any study, be sure to look at both the explicit and implicit assumptions you're making about time. Are you interested in describing some process that occurs over time, or are you simply going to describe what exists now? If you want to describe a process occurring over time, will you be able to make observations at different points in the process, or will you have to approximate such observations by drawing logical inferences from what you can observe now? If you opt for a longitudinal design, which method best serves your research purposes?
Examples of Research Strategies
As the preceding discussions have implied, social research follows many paths. The following short excerpts from a variety of completed studies further illustrate this point. As you read each
04945_ch04_ptg01.indd   111 8/21/14  11:32 AM
112 ■ Chapter 4: Research Design
excerpt, note both the content of each study and the method used to study the chosen topic. Does the study seem to be exploring, describing, or explaining (or some combination of these)? What are the sources of data in each study? Can you identify the unit of analysis? Is the dimension of time relevant? If so, how will it be handled?
• This case study of unobtrusive mobilizing by Southern California Rape Crisis Center uses archival, observational, and interview data to explore how a feminist organization worked to change police, schools, prosecutors, and some state and national organizations from 1974 to 1994. (Schmitt and Martin 1999: 364)
• By drawing on interviews with activists in the former Estonian Soviet Socialist Republic, we specify the conditions by which accommodative and oppositional subcultures exist and are successfully transformed into social movements. (Johnston and Snow 1998: 473)
• Using interviews obtained during fieldwork in Palestine in 1992, 1993, and 1994, and employing historical and archival records,
I argue that Palestinian feminist discourses were shaped and influenced by the sociopolitical context in which Palestinian women acted and with which they interacted. (Abdulhadi 1998: 649)
• I collected data [on White Separatist rhetoric] from several media of public discourse, including periodicals, books, pamphlets, transcripts from radio and television talk shows, and newspaper and magazine accounts. (Berbrier 1998: 435)
• In the analysis that follows, racial and gender inequality in employment and retirement will be analyzed, using a national sample of persons who began receiving Social Security Old Age benefits in 1980-81. (Hogan and Perrucci 1998: 528)
Mixed Modes
In this chapter, I have mentioned a number of ways to conduct social research: experiments, survey research (telephone, in person, online), field research, and so forth. In my observation over time, researchers have often spoken of the value of using more than one approach to
understanding a social phenomenon. But, as researchers find techniques they are comfortable with and adept at, the support for multiple techniques has been talked about more than practiced. However, this may be changing. Partly in response to the growing problems faced by survey researchers, perhaps, a review of the literature will produce increasing numbers of studies actually using mixed modes.
The U.S. Energy Information Administration's Residential Energy Consumption Survey (RECS), for example, seeks to get detailed information on Americas' energy use. Household interviews provide much of the needed information, but the researchers discovered that household subjects usually could not provide special details of their energy consumption. So the agency collects billing data from the energy suppliers and matches those data to the households interviewed (Worthy and Mayclin, 2013).
On the other side of the globe, Peggy Koopman-Boyden and Margaret Richardson (2013) used diaries and focus groups to study the activities of New Zealand seniors in a three-year study that aimed "to analyse the experiences and perceptions of elders and organizational representatives with whom they interact in everyday encounters" (2013: 392). Participating seniors were asked to maintain logs of their interactions, and they were invited to participate in periodic discussions of their experiences. In addition to providing researchers with a greater depth, breadth, and richness of data, they report that the participants often indicated they had benefited from the combination of methods, as in the focus-group discussions of the experience of keeping a research diary.
Moving north from New Zealand to India, Prem Saxena and Dhirendra Kumar (1997) examined the risk of mortality among seniors after retirement, focusing on the importance of work for defining social position. In the context of a number of social psychology studies of problems regarding retirement, the researchers found a data source in the Office of the Accountant General to add to the previous studies, allowing them to examine overall patterns of mortality after retirement. Ultimately, they concluded that "In developing countries few people look forward to retirement, while the majority dread it. However, the way pensioners react depends mainly upon
04945_ch04_ptg01.indd   112 8/21/14 11:32AM
their social liability and their unmet needs at the time of retirement" (1997: 122).
While social researchers have been using mixed modes of inquiry for a long time, this approach has begun attracting more attention and, more important, more actual use in recent years. I think you can expect to see more mixed modes in the future and may utilize this approach yourself.
How to Design a Research Project
You've now seen some of the options available to social researchers in designing projects. I know there are a lot of components, and the relationships among them may not be totally clear, so here's a way of pulling them together. Let's assume you were to undertake research. Where would you start? Then, where would you go?
Although research design occurs at the beginning of a research project, it involves all the steps of the subsequent project. This discussion, then, provides both guidance on how to start a research project and an overview of the topics that follow in later chapters of this book.
Figure 4-7 presents a schematic view of the traditional image of research design. I present this view reluctantly, because it may suggest more of a step-by-step order to research than actual practice bears out. Nonetheless, this idealized overview of the process provides a context for the specific details of particular components of social research. Essentially, it is another and more detailed picture of the scientific process presented in Chapter 2.
At the top of the diagram are interests, ideas, and theories, the possible beginning points for a line of research. The letters (A, B, X, Y, and so forth) represent variables or concepts such as prejudice or alienation. Thus, you might have a general interest in finding out what causes some people to be more prejudiced than others, or you might want to know some of the consequences of alienation. Alternatively, your inquiry might begin with a specific idea about the way things are. For example, you might have the idea that working on an assembly line causes alienation. The question marks in the diagram indicate that you aren't sure things are the way you suspect they are—that's why you're doing the research. Notice that a theory is represented as a set of complex relationships among several variables.
How to Design a Research Project "113
Or you might want to consider this question: How is leadership established in a juvenile gang? You may wonder how much age, strength, family and friendship ties, intelligence, or other variables figure into the determination of who runs things. We don't always begin with a clear theory about the causal relationships at play.
The double arrows between "interest," "idea," and "theory" suggest that a movement back and forth across these several possible beginnings often takes place. An initial interest may lead to the formulation of an idea, which may be fit into a larger theory, and the theory may produce new ideas and create new interests.
Any or all of these three may suggest the need for empirical research. The purpose of such research can be to explore an interest, test a specific idea, or validate a complex theory. Whatever the purpose, the researcher needs to make a variety of decisions, as indicated in the remainder of the diagram.
To make this discussion more concrete, let's take a specific research example. Suppose you're concerned with the issue of abortion and have a special interest in learning why some college students support abortion rights and others oppose them. Going a step further, let's say you've formed the impression that students in the humanities and social sciences seem generally more inclined to support the idea of abortion rights than those in the natural sciences do. (That kind of thinking often leads people to design and conduct social research.)
So, where do you start? You have an idea you want to pursue, one that involves abortion attitudes and choice of college major. In terms of the options we've discussed in this chapter, you probably have both descriptive and explanatory interests, but you might decide you only want to explore the issue. You might wonder what sorts of attitudes students with different majors have about abortion (exploratory), what percentage of the student body supports a woman's right to an abortion (descriptive), or what causes some to support it and others to oppose it (explanation). The units of analysis in this case would be individuals: college students. But we're jumping the gun. As you can see, even before we've "started," we've started. The reciprocal processes described in Figure 4-7 begin even before you've made a commitment to a project. Let's look more
04945_ch04_ptg01.indd   113 8/21/14 11:32AM