Back Stage at Social Problems: An Analysis of the Editorial Decision Process, 1993-1996 JOANN MILLER, Purdue University ROBERT PERRUCCI, Purdue University The publication of scholarly papers in professional journals certifies new contributions to knowledge, as well as the skills of the authors who have subjected their work to a process of critical review. The peer review process, designed to attract quality research with the use of objective practices, cannot avoid the infusion of particularistic judgments. This study examines how more than 600 manuscripts submitted to Social Problems were processed by its editor and associate editor. It traces the influences of manuscript, author, and reviewer characteristics as papers are judged, initially by the editor, subsequently by expert reviewers, and, once again, by the editor. The reviewers' recommendations, although often reflecting disagreement, outweigh all the other measured factors that may affect the editor's decisions. The editors, however, remain obligated to formulate credible accounts of the decision process, especially when a inal disposition departs from the recommendation made by expert reviewers. Scholarly journals carry out two important activities. First, they certify new contributions to knowledge, thereby providing a continuity and transmission of knowledge in the scholarly community. Second, publication validates the skills of the authors who have applied the theory and methods of their discipline or field to produce publishable papers. Many parties— authors, editors, and reviewers—share an interest in creating and maintaining a publication process that is committed to one objective: identifying the highest quality papers for publication. The method for identifying high quality knowledge is peer review, an organized process of formal and informal practices that guide a manuscript from the hands of an author, to expert reviewers, and to the desk of a journal editor for a final decision. The peer review process is a self-regulating system that protects the professional autonomy of a community of scholars as they certify knowledge, and it is guided by the values of objectivity and fairness (Chubin and Hackett 1990). However, it is possible that other values may intervene to distort the alleged universalism of the peer review process and the search for the highest quality work. Editors may be partial to research that is aligned with their own theoretical and methodological preferences, or their judgments may be influenced by the prestige of authors or their institutional afiliations. In this paper, we examine the manuscript review and decision making process in one social science journal, Social Problems. A paper published in Social Problems is certified for the correctness of its procedures and findings, and it is academic capital that can assist a scholar in the acquisition of tenure or a merit increase from a research university (Gomez-Meija and Balkin 1992). We examine how 673 manuscripts were processed by the Social Problems editors Direct correspondence to: JoAnn Miller or Robert Perrucci, Department of Sociology, Purdue University, West Lafayette, IN 47906. E-mail: millerj@soc.purdue.edu or perruccir@soc.purdue.edu. SOCIAL PROBLEMS, Vol. 48, No. 1, pages 93-110. ISSN: 0037-7791 © 2001 by Society for the Study of Social Problems, Inc. All rights reserved. Send requests for permission to reprint to: Rights and Permissions, University of California Press, Journals Division, 2000 Center St., Ste. 303, Berkeley, CA 94704-1223. 94 MILLER/PERRUCCI between 1993 and 1996,1 resulting in 78 publications, or academic capital for 120 authors and disappointments for approximately 897 authors.2 The responsibility of getting quality issues of Social Problems to its readers included moments of high drama, when page proofs were lost in Africa, a dreary rainy noon hour when the associate editor unloaded from a tractor trailer all the copies of an issue shipped in error, the agony of explaining to authors why their papers would not be published, and the task of extensively editing a small number of manuscripts that were first rate studies in need of a great deal of clarification. What does, or should, an editor of Social Problems do when deciding, first, whether or not to seek reviews for manuscripts and, subsequently, to decide which manuscripts will be published? When talking with a recent editor of the journal, he recalled how optimistic he was when preparing to begin work on the journal. He would shape it in certain directions; make it an even better journal than it was. He was certain he could reduce the "deflection rate" to less than one-half of its current rate. He quickly shifted the conversation to focus on why it was impossible to reduce the deflection rate, and how he was altogether humbled by the editor's job, one that depends on a large number of unknown or virtual colleagues, representing a score of specializations, to submit their best work, and to get useful, critical reviews and recommendations for making publication decisions. At the beginning of that editor's tenure, like the beginning of the 1993-1996 term, the commitment to publishing only the highest quality papers was strong, but unrealistic. Candidly, and with a dose of humility, we acknowledge a small number of papers, perhaps as many as three or four, should have received revise and resubmit decisions rather than the rejection decisions they did receive, and perhaps another one or two should not have been published. Professional journal editors in the social and behavioral sciences, and in most other fields, promote the communication of research findings, theoretical and methodological advances, and discussions of relevant, professional controversies. Editors serve a gate-keeping role, as well (Fyfe and Simon 1994). In the worst case, the gate-keeping role is deliberately used as a political tool, to promote or repress perspectives. In the best case, the editor recognizes the gate-keeping role and proceeds to make decisions carefully and critically that reflect the broad interests of the journal, its readers, and its contributors. Although Gilliland and Beckstein (1996) studied authors' perceptions of distributive justice, that is, decision outcomes, no objective measure of an editor's distributive justice can be operationalized or used to study the fairness of an editor's decisions in the social and behavioral sciences. In an ideal world, the communication of powerful insights and ideas would be the only decision-making determinant for the journal's editor. In the academic world, the journal business is connected to a reward structure, especially in research universities (Wolfe 1990). The publish or perish dictum is well known by first-year graduate students. Moreover, for sociology and other disciplines, in which no single paradigm dominates the ield, all publication and rejection decisions are arguably political decisions. Thus, understanding the editor's decision process is important; yet, it has been, more often, the subject of opinion essays (see, e.g., Fyfe and Simon 1994) than the subject of systematic study in sociology. Bakanic, et al. (1987) acknowledge that Garfield (1986) and other sociologists express concern over the disparity between the large number of published opinion essays without a corresponding number of research studies of the editorial decision-making process. They also recognize, as Zuckerman 1. The authors of this paper were the Associate Editor and Editor for Social Problems volumes published in the years 1994, 1995, and 1996. They had independent responsibility for all manuscripts in their special areas of expertise, and joint responsibility on all other manuscripts. Work for the 1994 Volume began in 1993. 2. Of the 673 papers processed, 78 resulted in publication and 595 papers were not published. The published papers were authored by a total of 120 persons. The non-published papers were authored by a total of 897 individuals. This number indeterminably exaggerates or underestimates the actual number of "rejected authors" because, in some cases, an author or authors submitted more than one paper. In three cases, authors who submitted multiple papers had one or two of their papers published between 1994 and 1996. Back Stage at Social Problems 95 and Merton (1971) did earlier, that publication in a peer reviewed proiessional journal certifies the social researcher's claims, procedures, and evidence. For disciplines like sociology, in which replication studies are rarely encouraged or rewarded (Wilson, et al. 1973), research certified with a publication decision can remain unchallenged, especially ii the paper is publicized, even ii the research findings result in socially harmiul outcomes (see Lempert 1989 ior a cogent discussion oi this problem). A Research Literature on Editors' Decisions Pre- and postdating the Bakanic et al. and Garfield publications,3 a research literature on the biases and potential biases in the editor's decision-making process is available to guide this study. However, its net is cast widely across the various science and social science disciplines. In the public health field, Susser and Yankauer (1993) report a high rate oi repeated or duplicated publications, oiten published as iragments oi the results irom a single study. Gariunkle and Ulshen (1994) find that articles, or what are known as major manuscripts, submitted to the Journal of Pediatrics are no more or less likely to be published as a iunction oi the prestige oi the author's employer. However, employer prestige is directly and strongly related to the likelihood oi research reports being published. In the social sciences, Presser (1980) reports that collaborations apparently improve the quality oi research, and thereiore, the likelihood oi publication in the journal Social Psychology Quarterly. In mass communication, Cherry (1991) reports a somewhat strong bias against publishing articles written by or about Airican Americans or Blacks, arguing that articles are most typically written irom the standpoint oi a white man's world. Cherry's claims are deduced, in part, irom the Grainey, et al. (1984) study oi news coverage oi a mayoral contest, and the Dates and Gandy study (1985) oi media coverage oi Jesse Jackson's campaign ior the Democratic presidential nomination. In both cases, more news coverage was iocused on the candidate's race—their Black race—than on their political campaigns. Epstein (1990), whose colleagues in social work question the ethics oi his research methods, tested and empirically supported the premise that studies confirming the eificacy oi social work programs are more likely to be published in some oi the sociology and social work journals. Aiter submitting a conirming or non-conirming version oi a counterieit article to several journals that vary in prestige, measured by high rejection rates or reputation, he reports that the most prestigious journals show no conirmatory bias to publish. It was the middle range journals that were most likely to reject the version oi the counterieit article that iailed to confirm that a social work program works as designed. Those journals also tended to accept a program-confirming paper, in spite oi analytic flaws deliberately included in both versions (conirming and non-conirming) oi the counterieit article. Most oi these studies examine a number oi concepts and characteristics in search oi uncovering empirical relationships between or among variables. Beyer (1978) 4 and, more recently, Gilliland and Cortina (1997) attempt to oiier explanations ior the editor's decision-making process that simultaneously examine general or multiple iactors that relect bias or preierences by journal editors or by the reviewers selected to evaluate manuscripts. Gilliland and Cortina oiier a conceptual iramework that can be adapted ior this particular study. They examine how iactors such as social particularism, e.g., an author's ethnicity or gender, content particularism, and an author's advantage, indicated by experience or seniority, inluence the editor's decision to publish certain manuscripts. We modiiy their model slightly by examining 3. In sociology, many oi the studies and essays on editors' decisions are embedded in the sociology oi knowledge field. 4. Beyer (1978) reports that, in science fields with established paradigms, reviewers and editors use more univer-salistic criteria, resulting in judgment consensus. In social science ields with competing paradigms, reviewers and editors use more particularistic criteria. 96 MILLER/PERRUCCI the gate-keeping role of the editor that is exercised at two points in time: first, when a decision is made to deflect a manuscript from the review process, and second, when the editor decides to publish a paper, invites author revisions, or rejects a paper. We also examine how reviewers use particularism in their evaluations and recommendations. Like the Bakanic, et al. (1987) study of a sample of American Sociological Review articles that found small empirical associations among reviewers' recommendations to the editor, we examine the issue of reviewer disagreement in their assessments and the degree to which the editor's publication decisions are structured by the reviewers' evaluations and recommendations. If the editor's decisions are not structured by the reviewers' evaluations, it is likely that the editor's inal publication decisions represent some form of professional or personal discretion. The framework we use to examine manuscript processing for Social Problems between 1993-1996 is summarized in Figure 1. It is a simple combination of the Bakanic, et al. (1987) and the Gilliland and Cortina (1997) work. It also reflects the flow of processing manuscripts from the initial submission stage through the eventual publication decision. Research Methods The 673 manuscripts analyzed in this study were numerically coded along many of the dimensions used by Bakanic and his colleagues in their study of a sample of the papers published or rejected by the American Sociological Review during the years Rita Simon edited the journal. All papers in this study had numerical codes assigned to symbolize: The editor's decision (deflect, reject, revise and resubmit, or accept for publication) The number of authors (ranging from one to five) Author nationality (from U.S. or from another nation) Each author's gender, race, rank, seniority, and employer The paper's substantive area, coded numerically, to represent SSSP5 divisions The paper's data collection and data analysis methods The number of reviewers for each paper The recommendations received from each reviewer The authors' ranks are coded non-academic, with or without advanced degrees; and for academics, from graduate student to full professor. Seniority is the number of years since the author completed the highest degree. All employers are coded on the following seven-point scale, intended to represent employer prestige: Research I University (7), Elite Liberal Arts College (6), State College and Other Liberal Arts College (4.5), Regional Campus (3), Community College (2), and Non-academic firm (1). The outcome variables (the dependent variables) are coded with dichotomous accept (1), or not accept (0), values to represent the editor's final decisions, and similar to the Bakanic, et al. study, on a three-point, ordinal level scale: 1 = reject/not publish, 2 = revise and resubmit, 3 = accept. The three-point scale is also used to code the reviewers' recommendation scores.6 5. The Society for the Study of Social Problems (SSSP) is the professional association that sponsors the publication of Social Problems. 6. Two graduate students in sociology coded the papers. They were instructed to use missing values whenever they were unsure about any code. All records with missing values were later inspected and the missing codes replaced, when possible, with appropriate codes by one of this paper's authors, who also selected a 10% sample of the papers to code for the purpose of establishing coding reliability. The only differences found when comparing graduate student and author codes, were in the values assigned initially for the "primary" and "secondary" substantive areas of the papers. For example, a paper on a drug prevention program used in schools was initially coded "educational problems" and "drinking and drugs" as the primary and secondary areas. As a consequence, all analysis is based on "primary" or "secondary" substantive area of all papers to avoid misattributions. Back Stage at Social Problems 97 (A) Papers Submitted to Journal Editorial Office (B) Editor's First Gate-keeping Decision: Particularism based on Content, Methods, Author's Status, and Employer's Prestige Editor Deflects Paper from Review Process or: Editor Seeks Peer/Expert Reviews (C) Reviewers' Recommendations: Reject, R&R, Accept Particularism based on Content, Methods, Author's Status, and Employer's Prestige (D) Editor's Second Gate-keeping Decision: Reject, Invite R&R, or Accept Manuscript for Publicatior Figure 1 • The Editor's Gate-keeping Decisions: Discretion Structured by Reviewers' Recommendations Our analysis begins with a description and discussion of the papers deflected from the review process, the first gate-keeping decision made by the editor, and a hotly contested issue by some editors and authors. We, then, discuss the papers sent out for external reviews. Because reviewers did not always agree with each other, we explore this issue, and how it is associated with the editor's decisions. In that section of the paper, we examine many of the problems discussed in the Bakanic, et al. study, as well elements of the more general conceptual model we modify for this inquiry. We cross tabulate data, examine bi-variate correlations, and specify an ordinary least squares regression model7 to analyze the data. Unlike the more traditional sociological research study, we attempt to describe and explain our decisions and experiences throughout the various sections of the paper. Our intention is not to relect on three years of our professional lives. Instead, we attempt to ill in the gaps or chasms left by the empirical analysis, or what some would call a quantitative study. We refer to information gathered through conversations with reviewers, editorial board members, authors, and three other editors of Social Problems. We trust no research ethics are 7. In addition, we specified a logistic regression model to explain a dichotomous, publish or not publish, outcome variable. The results are not reported here because they merely reflect the OLS findings. 98 MILLER/PERRUCCI breached because no records of conversations were taped or created, and we keep all identities anonymous. The Papers Deflected or Reviewed for Publication bySocial Problems: 1993-1996 Deflections Unlike many peer reviewed sociology journals, Social Problems, perhaps explained by its title, attracts an extensive assortment of manuscripts, including letters from prison and polemical essays, ranging from two to fifty pages long, that fall beyond the parameters of any theory or empirical work in the social and behavioral sciences. Ironically, it also attracts a number of research papers that are not about social problems. It is a journal that publishes only articles, excluding the review essays, research notes, and debates that are found in other major journals, such as The American Sociological Review or the American Journal of Sociology. As a consequence, Social Problems editors deflect from the review process a substantial percentage of its submissions.8 During 1993-1996, 35% of the work, or a total of 235 manuscripts that were submitted to Social Problems were not sent out to external reviewers for evaluation and publication recommendations.9 The purpose for deflecting manuscripts is to return papers as quickly as possible to enable the authors to seek a more appropriate outlet for their work. Generally, papers deflected were received and returned by the Social Problems editorial office within two weeks. When we began work on the journal, we discussed the delection rate, like earlier editors, vowed to lower it (a vow broken well before the first issue went into production), and decided, at the outset, that any paper sent out for external reviews would (a) be focused on a social problem, (b) make at least a modest contribution to a theoretical perspective, and (c) not be exclusively an opinion essay. We decided not to publish, or send out for review, papers written on topics such as the correlates of personal happiness; and, not to seek reviews for papers that were literature reviews, or summaries of descriptive studies. Many of the papers were worthy of publication, but in journals better suited to their subject. Some of the authors who submitted their work to Social Problems, however, perceived various motives for the deflection process. One author, a well-known sociologist whose work is qualitative, wrote a letter to the editor claiming a strong bias against qualitative work. The irony is Social Problems' long standing predilection for qualitative research—obviously retained during the 1993-1996 period when less than 25% of the papers published were quantitative, that is, centered on inferential statistical analysis. Another author, whose work was deflected, subsequently refused to review a manuscript because his work had been delected. Since this refusal to review occurred soon after we began our work on the journal, we wondered how many reviewers, if any, would participate in the process under such circumstances. We reasoned that virtually every author who has ever published a manuscript in a peer reviewed professional journal has had a paper delected 8. Other professional journals deflect manuscripts but use a method that makes the deflection less obvious than Social Problems' method. For example, an editorial board member will advise the editor to deflect a paper from the full review process. With the comments received from the editorial board member, the editor sends a letter to the paper's author that is what we could call a 'rejection letter.' The letter is likely to focus on the 'not suitable for this journal' issue. The American Sociological Association identifies two types of rejections: papers screened by the editor and rejected outright, and papers reviewed and rejected outright. 9. The deflection rates for Social Problems in 1991 and 1992 were 20% and 32%. In 1997 and 1998, the deflection rates were 34% and 27%. Thus, the rates are fairly similar across three different editors, suggesting that the rates are more a function of the kind of manuscripts submitted to the journal. Bakanic reports a deflection rate of approximately 5% for manuscripts submitted to The American Sociological Review between 1977 and 1981. Back Stage at Social Problems 99 or even worse—rejected! Thus, if our "deflected, won't review" problem was widespread, we would face a severe problem. We are pleased to acknowledge that our early fears were quickly assuaged. Most authors whose papers were deflected or rejected did review manuscripts, indicating at least implicitly that their participation in the peer review process was perceived by them to be important for identifying the highest quality papers for publication in Social Problems. On those occasions when we violated our own deflection guidelines and sent papers out for review, our decisions tended to result in one of two types of problems. Some reviewers would return the paper without making a publication recommendation, but include a note, generally asking "whatever made you think this should be reviewed?" The other problem was receiving extremely positive reviews, yet, the editor or associate editor would reject the paper, thus wasting two or three months of the author's time in trying to get a piece of work published. Joseph Schneider (1990) addresses this particular problem in his reflection of his tenure as the editor of Social Problems. On one occasion, not only did all four reviewers return recommendations, they unanimously recommended publication of an excellent empirical, but atheoretical paper. The author, after reading four very positive reviews from experts in the field along with a rejection letter, called the associate editor in search of a better explanation than what the rejection letter contained.10 All told, the deflection problem is one infrequently discussed, though used by most journals in the social sciences, including the American Sociological Review, for decades. It is like a well-guarded secret that needs to be disclosed. A journal's deflection process, or its deflection rate, provides useful information to authors who submit their work to peer reviewed journals. Over time, as more authors submit more and more manuscripts, delection rates increase. Our position is a truth in advertising one: materials printed on journal covers that provide directions for submitting manuscripts should include general expectations for publication and information on the possibility of a manuscript being delected from the external review process. Reviewed Manuscripts and the Reviewers Manuscripts submitted to Social Problems are subjected to a double blind review. The author's and reviewer's identities are, in principle, unknown to each other. Nonetheless, in a small, but unknown number of cases, reviewers accurately surmised the author's identity and communicated that information to the editor. Most often, the reviewer inquired if she or he should review the manuscript under such circumstances. In all but one case, the reviewers evaluated the manuscripts, perceiving a fair and unbiased assessment was possible. The editorial staff of Social Problems, at least over the recent ifteen years, inherits a reviewer ile. It is a card ile, alphabetically arranged, containing the names, addresses, areas of expertise, and the manuscript number that each person in the ile was asked to review. It also contains editors' notations about especially strong or weak reviews that were received. Because we did not initiate the reviewer file, we cannot detail its rationale or how others used it. We can only report that, although we used it as a resource to identify appropriate reviewers, we relied as heavily on two additional types of resources, i.e., our own knowledge of networks of individuals in a particular ield, and the bibliographies of the submitted papers. The Social Problems editor selects a board of 25 advisory editors that is approved by the Board of Directors of the SSSP. The advisory editors are chosen for their expertise, and to represent diversity in sex and race or ethnicity, and the diversity of Social Problems' authors and their work. When approached to serve as an advisory editor, each individual was told by the 10. This particular paper, focused on drug use, was published within one year in a prestigious specialty journal that is ideally suited for the work. 100 MILLER/PERRUCCI editor not to expect more than four to six papers to review per year. We found that the workload of the advisory editors was very uneven, influenced, largely, by the manuscripts submitted to the journal. A small number of advisory editors completed a disproportionate share of the advisory editors' workload. Not only did they review more than their numerical share of papers, their reviews tended to include comprehensive and constructive suggestions to the author for revisions. Three of the advisory editors always included either a bibliography or complete citations for any work they referenced in their reviews. Each of the 438 reviewed papers in this study was read and evaluated by an average of 2.86 individuals, selected by the editor or the associate editor. Ideally, the reviewer's work was similar to the primary substantive area, the theory, and the research methods that characterized the paper. Each paper was mailed to four individuals, with the anticipation that not all those approached would be able to review the manuscript within the time limits specified by the editorial office. In unusual cases (only 11 of the 438 papers), no initial reviewers returned recommendations. For 117 papers, all four of the persons approached, initially, reviewed the papers and made recommendations to the editor. Perhaps in an earlier generation of sociology, when one paradigm nearly dominated work in the discipline (Friedrichs 1972), reviewer gender, race, sexual preference, or field of specialization was not important. Except for the unusually brave, most sociologists who attempted to publish their work in mainstream peer reviewed journals worked from doctrinal theory and a delimited set of preferred methods for data collection and analysis. The introduction of critical race theory, feminist perspectives, queer theory, and the consequences of Affirmative Action have added diversity to some disciplines, including sociology (see e.g., Collins 1999; Ferree and Hall 1996; Ferree, Lorber, and Hess 1999; Fox and Ferri 1992; Gamson 1995; Hartsock 1998). The all-white-male reviewer panel, like the all-white-male criminal trial jury, is no longer an acceptable approach for seeking fair evaluations. Sociology is an example of an academic discipline that has achieved near gender parity. With the exception of gender, however, tokenism might best describe the representation of social and cultural diversity in sociology, as well as many other social and behavioral science disciplines and academic departments (Bellas 1994; National Science Foundation 1997). The typical manuscript sent out for review was evaluated by at least one woman, and two or more women reviewed 28% of the papers. Although not coded and analyzed in this study, papers authored by those who are "doing [other] difference[s]" (West and Fenstermaker 1995), for example, studies of race and ethnicity or the politics of sexual preferences were also deliberately sent out to at least one peer reviewer who works in a similar area. Characteristics of the Reviewed Manuscripts Slightly more than one-half of the reviewed manuscripts are first-authored or sole-authored by a male assistant professor, with the remaining 42% irst-authored by women. Only a minority (5%) of the manuscripts sent out for external reviews was submitted by international authors. The journal's sponsor (SSSP) maintains seventeen11 special divisions or ields of specializations. We used numerical codes to represent the special divisions for analysis. Considering all the papers in this study (including the submitted, but deflected manuscripts), Crime and Juvenile Delinquency (12.8%), Conflict, Social Action, and Social Change (10.8%), Family (10.8%), and Social Problems Theory (10.8%) papers dominate the distribution of specializations. The three specializations that dominate the distribution of manuscripts sent out for review between 1993 and 1996 are: Conflict, Social Action, and Change (18.3%), Poverty, 11. A new division, teaching social problems, was approved in 1997 by the Society for the Study of Social Problems. No papers representing the new teaching division are included here. Back Stage at Social Problems 101 Class, and Inequality (16.6%), and Social Problems Theory (11.6%). The difference in the two distributions is that Crime and Delinquency and Family papers dominated the papers submitted, but not the papers reviewed. Papers were also coded for the method of data collection and the method of analysis (both are generic concepts, i.e., a direct observation is a unit of datum). The total sample includes: 12.3% direct observation studies, 13.7% open interview studies, 20.1% survey studies, 14.0% document studies (using newspapers, magazines, historical documents, or television scripts), 5.9% census studies, with the remaining papers based on no data or classified as opinion essays. The sample of reviewed manuscripts, relative to the total number of manuscripts submitted, included more direct observation studies (16%), more open interview studies (22.4%), nearly the same percentage of survey studies, and many more document studies (44.7%). The distributions differ in ways that are not newsworthy to Social Problems readers. Larger numbers of manuscripts sent out for review, compared to the total number of manuscripts received, relied on qualitative data and documents. Two categories of data collection, studies using census data and opinion essays, are not represented among the papers reviewed. We do, indeed, acknowledge that one-fifth of the papers reviewed are survey studies, and we also recognize that four-fifths were not the survey studies that dominate other prestigious journals in sociology. Social Problems continues to deserve its reputation for attracting and reviewing more qualitative than quantitative research, regardless of the meaning of those terms for epistemologists.12 When comparing the deflected and reviewed papers along the dimension of data analysis, we find that 26.7% of the deflected papers were based on inferential statistics or hypothesis testing. That percentage increases to 30.6% for the manuscripts sent out to reviewers. The majority (69.4%) of the manuscripts reviewed depended on no numerical values or descriptive data only, such as the percentage of respondents who met some criterion relevant to the researcher. Qualitative researchers should be assured that no trend toward reviewing or publishing an increasing number of manuscripts that showcase multivariate analysis or hypothesis testing is apparent in Social Problems. Reviewer (Dis)agreement and Publishing Decisions Bakanic, et al. (1987) report little agreement between the first and the second reviewers' recommendations to the editor (r = .16). Gilliland and Cortina (1997) reviewed the available studies on this particular issue and generally report correlations among reviewers' recommendations to be lower than .30 (see especially, Fogg and Fiske 1990 and Gottfredson 1978). A number of possible explanations are reasonable: subjectivity or a negative bias in a reviewer's evaluations, the use of inconsistent criteria across reviewers to judge manuscripts, or a variation of a confirming bias (Epstein 1990), i.e., the reviewer makes a positive, but biased, recommendation when the paper conirms the reviewer's own work. Capricious reviews are also possible, but the Social Problems editor did not receive a substantial number of reviews that could be classiied as capricious. Admittedly, with a few exceptions, the reviewers' evaluations of manuscripts were careful, thoughtful, and relected a level of expertise that corresponded to at least one dimension of the manuscript—its theory, research methods, or substantive area. It is our conclusion that a focus on a manuscript's theory, or its research methods, or its contribution to a ield of specialization is what accounts for disparate reviewer evaluations. It 12. Getting ahead of our story for a moment, we acknowledge, immodestly, that excellent "quantitative analysis" papers were published in Social Problems between 1994 and 1996. Nearly 25% of the published papers are "quantitative" papers, i.e., using inferential statistics, and 75% are "qualitative" papers in which the only numbers included are descriptive summaries. A deliberate decision was made to publish the best papers and not to promote any particular perspective or research method. 102 MILLER/PERRUCCI Table 1 • Reviewer Agreement (N = 93 Papers) and Editor's Decisions (a) Reviewer's (b) Editor's Recommendation Decision N (%) N (%) Reject the manuscript 50 (53.8) 68 (73.1) Revise and resubmit 21 (22.6) 7 (7.5) Publish the manuscript 22 (23.6) 18 (19.4) ra,b = .717 p < .001 was not uncommon for a reviewer to include the caveat "I'm not an expert on . . ." when returning a manuscript evaluation to the editor. The lack of agreement between or among reviewers is viewed by some who study the issue to be a problematic indication of inconsistency or an open invitation to the editor to rely upon his or her own preferences in making publication decisions. In this study, we also ind a low level of agreement between Reviewers A and B recommendations (r = .134, p < .05).13 The level of agreement between the editor's decision and either the recommendation from Reviewers A (r = .456, p < .001) or from Reviewers B (r = .477, p < .001) represents a moderate level of agreement, implying the editor's decisions could be partly a function of discretion and partly a representation of judgments structured by a single reviewer's recommendation. When the average recommendations (x of the recommendation scores) from the two reviewers are correlated with the editor's decision, however, agreement increases (r = .598, p < .001). To pursue these empirical puzzles further, we identified 93 manuscripts for which Reviewers A and Reviewers B agreed. When the reviewers agree, the editor makes a publication decision that corresponds highly to their joint recommendation (r = .717, p < .001). The distributions of the reviewers' recommendations and the editor's decisions for the 93 reviewer agreement papers are shown in Table 1. The editor, compared to the reviewers, made more rejection decisions, and fewer revise and resubmit or acceptance decisions. The differences in the distributions of decisions and recommendations, however, are small, although statistically signiicant.14 This analysis helps to illustrate the notion that the editor's reliance on agreement between or among reviewers does not substantially increase the likelihood of an acceptance decision for a manuscript. When we examine the total 438 papers reviewed, we find that the reviewer, and not the editor, is the harsher critic. Reviewers A recommended publication for 11.2% of all the papers (N = 365) they evaluated, and Reviewers B, even less accepting, would publish 10.9% of the total manuscripts (N = 357) they evaluated. The editor accepted 17.8% of the papers reviewed, usually after substantial revisions and a second round of reviews were received. To make the editor's acceptance rate as low as the reviewers' acceptance recommendations, we would need to change the denominator from the 438 papers sent out for external reviews to the total of 673 manuscripts submitted. Then, and only then, is the acceptance rate 11.6%. The lower acceptance rate for Social Problems includes the papers delected by the editor or associate editor at the irst stage of the gate-keeping process. Journal editors take one of at least three positions to resolve the supposed reviewer disagreement problem. One type of editor routinely seeks some higher level of reviewer agreement by sending out a manuscript that elicits mixed reviews to more reviewers (a process, by 13. Reviewers A evaluated 365 of the 438 manuscripts reviewed. Reviewers B reviewed fewer papers, 357. Like Bakanic, et al., we rely on two reviewers only in this analysis because depending on 3 reviews would leave too few decisions to analyze. 14. The statistical significance of the difference in the distributions was determined with ANOVA. There are no reviewer recommendations for an acceptance that resulted in a revise and resubmit. There are also no recommendations for a rejection that resulted in an acceptance or publication decision. Back Stage at Social Problems 103 deinition, that increases the likelihood oi reviewer disagreement), thus, keeping the author waiting ior a protracted time ior a review and decision. The second type writes a letter to the author whose paper has received only one review that says: . . even ii I had received another review . . . I would not accept your manuscript ior publication because . . ." (an actual example). The third type iollows a practice used ior 1993-1996 Social Problems papers that received mixed reviews. The editor and the associate editor engaged in three related tasks. First (and most commonly), they attempted to discern irom among the mixed reviews, the more insightiul review, and depend on it more heavily than on the other recommendations. Second, ii their own reading oi a manuscript leit some questions raised by any oi the reviewers unanswered, they sought advice irom their colleagues within or outside their own academic department who had the necessary expertise to provide iniormation in order to achieve a more iniormed decision. Third, ii either the editor or the associate editor perceived any potential conlict oi interest, an advisory editor was approached to evaluate the manuscript. Only in the most unusual case was a manuscript that received mixed reviews sent out ior additional recommendations. What is the bottom line ior authors? When reviewers agree and recommend a paper ior publication to the editor, the chances ior publication improve, but not by much. However, reviewers rarely agree (only 21.2% oi the time), and reviewers' recommendations, in toto, tend to advise iewer acceptance decisions, 11.2% or 10.9%, compared to the editor, who accepted 17.8% oi 438 manuscripts that were reviewed between 1993-1996. Ii reviewer disagreement occurs, the author should hope ior one strong, thorough, and constructive review that is accompanied by a iair editor's decision, although an indeterminable outcome ior any ield oi study that relies on probability, rather than certainty, to convey its perspectives. This may, indeed, represent gate-keeping, but it is a decision-making process that presents an outcome to the author in a timely iashion. It poses no more oi a reviewer disagreement problem than does the paradoxical process oi seeking more reviewers whenever the irst group oi reviewers disagrees. Particularism: How It Affects Deflecting, Reviewing, and Publishing Manuscripts Table 2 summarizes the characteristics oi manuscripts and their authors that may, according to earlier studies, inluence either the editor's decisions or the reviewers' recommendations to the editor, as shown in Figure 1. In the irst column, the iactors that aiiect the editor's decision to delect the paper, coded 1, or send it out to reviewers, coded 0, are examined. Quite clearly, the deflection decisions made by the editor or the associate editor are influenced by a manuscript's content and methods and several author characteristics. Content particularism is statistically associated with the decision to deflect a manuscript, that is, to return it to the author, rather than sending it out ior external reviews and recommendations. Oi those papers delected, nearly hali represent two ields oi specialization— crime and delinquency, and iamily. The large number oi polemical essays on such topics as the demise oi the American iamily, or the problem oi drug use among younger segments oi the general population, permits a plausible explanation ior the content particularism associated with delections. This explanation is supported by another indicator oi particularism—the signiicant association between the method oi data collection and the decision to delect a manuscript irom the review process. An inspection oi the distribution oi deflections (not included here) shows that more than hali oi the delected papers are opinion essays. The reviewers' recommendations show no empirical evidence oi content or method particularism. Although the numerical summaries provided by the reviewers by no means begin to capture the content or helpiulness oi their reviews, the numbers suggest two signiicant indicators oi another type oi particularism, i.e, the author's social characteristics. Reviewers A appear to be inluenced by whether the author is irom the U.S. or is an international scholar. Reviewers B appear to be inluenced by whether the author wrote a paper in the area oi race 104 MILLER/PERRUCCI Table 2 • Manuscript and Author Influences on Deflection/Review Decision, Reviewers' Recommendations, and Decisions to Publish Papers in Social Problems, 1993-1996 (1) (2) (3) (4) X2 Analysis (Manuscript Editor's Decision Reviewer A Reviewer B Editor's Decision and Author's Status to Deflect (1) or (Reject, R&R, (Reject, R&R, to Publish (1) or Characteristics are Review (0) Publish) Publish) Not Publish (0) Nominal Level Data) (N = 673) (N = 341) (N = 324) (N = 438) Manuscript content Social Problems divisions+ 212.509*** 19.843 34.294 18.214 (df = 16) (df = 32) (df = 32) (df = 16) Manuscript in editor's specializations 1.484 — — 2.126 (df = 1) (df = 1) Manuscript in associate editor's specializations 1.848 — — 2.944 (df = 1) (df = 1) Research methods Data collection 157.728*** 8.233 2.156 2.297 (df = 3) (df = 6) (df = 6) (df = 3) Data analysis 2.570 4.327 1.830 4.998 (df = 2) (df = 4) (df = 4) (df = 2) Author's social status Author gender .114 3.775 .256 .795 (df = 1) (df = 2) (df = 2) (df = 1) Race/ethnicity paper .033 1.110 9.307** .538 (df = 1) (df = 2) (df = 2) (df = 1) Author is international scholar 14.152*** 8.260* 1.622 1.322 (df = 1) (df = 2) (df = 2) (df = 1) (1) (2) (3) (4) Editor's Decision Reviewer A Reviewer B Editor's Decision Pearson Correlation Coefficients to Delect (1) or (Reject, R&R, (Reject, R&R, to Publish (1) or (Author Academic Status are Review (0) Publish) Publish) Not Publish (0) Ordinal or Interval Level Data) (N = 673) (N = 341) (N = 324) (N = 438) Author's academic status First author's rank+ + .114** .085 .060 .147** First author's seniority - .101 .129* .086 .037 First author's employer's prestige (1-7 scale)++ + - .232** .113* .013 .120* Second author's rank .063 - .044 - .066 .021 Second author's seniority .052 .076 .050 .065 Second author's employer's prestige - .324** .112* .061 .065 *p < .05 **p < .01 *** p < .001 + SSSP recognized 17 special divisions during the 1994-1996 period of publication. The editor's deflection and publication decisions are coded 1/0. Cross tabulations and %2 statistics are used to examine associations between outcome variables and particularism indicators that are measured as nominal level variables. Correlation coefficients are used to indicate the degree of association between outcome variables and particularism indicators that are measured at the ordinal or interval level. + + Author's rank codes range from 1 (non-academic) to 7 (full professor). + + + Prestige of employer is coded on a seven-point scale, ranging from non-academic organizations to Research I Universities. Back Stage at Social Problems 105 and ethnicity. How can a reviewer, in a double blind review, know the author is not from the U.S.? The size or the type of paper on which the manuscript is reproduced usually indicates which manuscripts international authors submit. What does the race and ethnicity inding suggest? Not much. Race papers are used here as a proxy for the author's race, premised on the assumption that a reviewer of a race or ethnicity paper attributes the same race or ethnicity to the author as the subject of the paper. Likewise, we make a similar assumption about the reviewers' attribution of the author's sexual preference to a manuscript written about gay and lesbian social movements. We acknowledge a tremendous amount of error in these assumptions, but we do ind it necessary to report that, in terms of the author's social characteristics coded for this study, only the irst author's gender has no signiicant association with the recommendations made to the editor by the reviewers. Rank and Seniority: How Do They Affect the Decision to Deflect, Reviewers' Recommendations, and Publication Decisions? The more experienced author, whether measured by rank or seniority, and the author who works in a more prestigious organization, such as a research university, gets a head start in the publication competition. We examine measures of these types of advantage, also indicators of particularism, to see how they affect the editor's decision to delect or publish a paper and how they influence reviewers' recommendations. (Consider the seniority message that is communicated in a double blind review when an author strings references to his or her own work that literally span decades.) Two measures of this type of particularism are signiicantly associated with the editor's delection and publication decisions, but not always as expected. The more senior the irst author is, the more likely the paper was to be deflected. Papers authored by academics employed by more prestigious universities, however, were less likely to be deflected from the review process. The editor's decisions to publish manuscripts are somewhat inluenced by this type of particularism. If the higher ranked author's manuscript is reviewed (and not deflected), it is more likely to be published than the manuscript written by the person with a lower academic rank. The irst author, who works for a more prestigious academic organization (coded on a seven point scale, ranging from non-academic firms to a Carnegie Research I University), is less likely to have a manuscript delected from the review process. If external reviews are sought for the paper, moreover, the author employed by the more prestigious type of university is more likely to attain a publication decision. These indings may indicate an editor's bias. However, they may also indicate that submissions to Social Problems from authors working in research universities are more sophisticated or more polished. Authors who work in academic settings with tangible support for the production of scholarship, in principle, have more resources to produce higher quality scholarship. Although the reviewer data, to some readers, should be dismissed because of the double blind review process, we note that Reviewers A are more likely to recommend publication for a more experienced first author and for papers written by authors who are employed by more prestigious organizations. These relationships modestly support our earlier argument. More sophisticated or more polished papers are more likely to be positively reviewed and, therefore, accepted for publication in Social Problems. Putting the Pieces of the Puzzle Together: What Explains the Editor's Decisions? Table 3 summarizes the results of an ordinary least squares regression of the editor's decision (reject, revise, or accept) on the characteristics of the manuscript; the author's social status, academic status, and the number of authors; and, the number of reviewers and the reviewers' 106 MILLER/PERRUCCI Table 3 • Manuscript and Author Influences: OLS Model to Explain Variance in Editor's Decisions to Accept, Request Revisions, or Reject Manuscripts for Publication in Social Problems, 1993-1996+ Editor's Decision b (s.e.) Manuscript content Manuscript in editor's specializations Manuscript in associate editor's specializations Number of reviewers Method of data analysis* + Qualitative analysis Quantitative analysis Author's social status Author gender Race/ethnicity paper Author is international scholar Number of authors Author's academic status First author's rank First author's employer's prestige Second author's rank Second author's employer's prestige Reviewer recommendation Reviewer A's recommendation Reviewer B's recommendation Intercept N Adj. R2 (F Value) .164* (.080) -.078 (.084) -.195*** (.039) -.031 -.122 (.089) (.094) -.011 (.140) .011 (.140) .056 (.189) .075 (.063) .031 (.018) .034 (.021) .020 (.023) .015 (.020) .346*** (.046) .379*** (.048) .326 (.336) 250 .449*** (F = 14.530) *p < .05 **p < .01 ***p < .001 + The outcome variable is coded on a three-point scale (1 — Reject, 2 — R&R, 3 — Publish or Accept after Revisions). The total number of papers reviewed is 438. The papers deflected from the review process are not included in the analysis. + + Method of data analysis (not data collection) is "qualitative" or "quantitative." The omitted category is "no analysis," i.e., the paper used no data. "Quantitative" analysis means hypothesis testing or the use of multivariate, inferential modeling was included in the paper. recommendations. An examination of how characteristics of the manuscript and the author might inluence editorial decisions permits a determination of whether particularism might be involved in the inal decision. The inclusion of the number of authors and the number of the reviewers in the regression is designed to answer two questions. First, do more reviews improve the likelihood for publication or rejection? This question stems from our earlier discussion on reviewer disagreement and the editor's decision. Second, does multiple authorship increase the chances for publication? Some researchers argue that collaboration brings greater expertise to a project and, thereby, improves the quality of the paper (Presser 1980). The final difference between the multivariate model in Table 3 and the earlier analysis is the inclusion of the method of data analysis (qualitative or quantitative), rather than the method of data collection. Multicollinearity precludes the inclusion of both dimensions of research methods. The partial regression coeficients reported in Table 3 indicate that reviewers' recommendations are the most important inluences for the editor's decisions, apparently encouraging Back Stage at Social Problems 107 the editor to invite a revision and resubmission. We also see that manuscripts reviewed by a larger number of reviewers are less likely to be published. Since the very unusual decision to seek additional reviewers is probably because the initial reviewers are sharply divided in their recommendations, and since reviewers, in general, are more likely to recommend rejection than the editor is to make a rejection decision, the advice of additional reviewers is more likely to be negative than positive. Finally, we note that a manuscript in the editor's areas of specialization has a small but positive effect on making a publication or revise and resubmit decision. The effect of an editor's area of specialization is likely indirect, in that an editor with research expertise in the subject of the manuscript, is more likely to be knowledgeable about reviewers' theoretical and methodological preferences. When such reviewers are assigned to a manuscript, they are more likely to agree in their recommendations regarding publication. Moreover, researchers work in substantive areas and seek out journals with editors who represent their expertise as preferred publication outlets. Thus, the indirect effect of the editor's area of expertise on editorial decisions is due, partly, to the reviewers assigned by the editors, and partly to the authors seeking the most appropriate journal for their work. The remaining coeficients in the regression model do not provide any additional information to help explain the editor's decision-making process. Regardless of how the data are analyzed and interpreted, reviewers' recommendations obviously inluence the editor's decisions, controlling for the many characteristics of the manuscripts and the authors that are examined throughout this study. Discussion We examined the review and decision process for 673 manuscripts submitted for publication to Social Problems between 1993 and 1996 to determine how the disposition of manuscripts is inluenced by the editor's role in the review process, by the recommendations of reviewers, and by the characteristics of the authors and their manuscripts. Our analysis is concerned with assessing the relative importance of the double blind review process, often perceived as evidence of objective judgments, yet, a process that includes the potential intrusion of particularistic judgments that can distort the search for the highest quality papers for publication. Manuscripts submitted to Social Problems undergo a two-stage review process. In the irst stage, the editorial ofice is the sole agent that decides if a manuscript should be reviewed or returned to the author. This decision is partly technical or mechanical: "We do not publish opinion essays or literature reviews." It is also partly substantive: "We do not publish work that lacks theoretical grounding," or "In our judgment, this paper is not appropriate for this particular journal." Moreover, a decision to delect a paper is based on a presupposition that not all manuscripts submitted are worthy of publication and that, as editors, we are willing to make the error of delecting a worthy paper. Scholars of the journal review process have conirmed some of the differences between journals in the physical and the social sciences in their presuppositions about manuscripts submitted for publication and how these differences result in different review processes and different rejection rates (Zuckerman and Merton 1971; Hargens 1988). Although the statistical evidence on the factors inluencing the delection decision indicates that there may be some particularism at work—for example, manuscripts by authors from more prestigious institutions are less likely to be delected—as the editors who are behind that statistical evidence, we are not inclined to accept that interpretation. We believe that delection decisions were based primarily on the quality of manuscripts and their appropriateness for Social Problems. We do not think that we ever made a conscious decision to privilege manuscripts representing certain substantive areas or those authored by scholars from prestigious universities. If we did so, then we were very inconsistent in our application of particularism, because authors from prestigious universities, ceteris paribus, were not more likely than others to have their papers accepted in the inal decision. 108 MILLER/PERRUCCI The second stage of the process involved only those manuscripts that were sent out for peer review. The empirical evidence indicates that the recommendations of reviewers have the greatest influence on the editor's final decisions. The recommendations of external reviewers were most likely to influence the editor's decision to reject a paper, ask for a revision, or accept the paper for publication. Even when reviewers disagreed in their recommendations, they still influenced the final decision because reviewer disagreement often resulted in a decreased likelihood for publishing a manuscript. The social characteristics of the authors, their academic ranks, and the prestige of their employing institutions have virtually no inluence on the inal decisions made by the editor. Although we believe that the peer review was the most important factor shaping the editorial decision-making process, it would be a mistake to see the process as routine or a mechanical tally of the reviewers' recommendations. When editors select the reviewers for a manuscript, they begin a social process that requires editors to evaluate the quality of the reviews they receive; at the same time they attempt to evaluate the quality of the manuscript. The editor's inal decision must, simultaneously, be a credible account to the reviewers, as well as to the authors. If an editor chooses not to follow reviewer recommendations, there is an obligation to justify that decision by disputing the substance of the reviewer's account. This is a delicate matter and it is probably what is responsible for the positive relationship between reviewer recommendations and editorial decisions. Thus, the certiication of new knowledge is, to some extent, socially negotiated and socially produced—a, not altogether, surprising conclusion for editors of a sociological journal. References Bakanic, Von, Clark McPhail, and Rita J. Simon 1987 "The manuscript review and decision-making process." American Sociological Review 52:631642. Bellas, Marcia Louise 1994 "Comparable worth in academia: The effects of faculty salaries of the sex composition and labor market conditions of academic disciplines." American Sociological Review 59:807-82 1. Beyer, Janet 1978 "Editorial policies and practices among leading journals in four scientific fields." Sociological Ouarterlv 19:68-88. Cherry, Dianne Lynne 1991 "Attention to black people's P.O.W.E.R. in the major metropolitan press." Journal of Black Studies 21:387-397. Chubin, Daryl E., and Edward J. Hackett 1990 Peerless Science: Peer Review and U.S. Science Policy. Albany: State University of New York Press. Collins, Patricia Hill 1999 "Moving beyond gender: Intersectionality and scientific knowledge." In Revisioning Gender, Myra Marx Ferree, Judith Lorber, and Beth B. Hess, eds., 261-284. Thousand Oaks, CA: Sage Publications. Dates, J., and O. Gandy 1985 "Ideological constraints affected coverage of the Jesse Jackson campaign." Journalism Quarterly 62:595-625. Epstein, William M. 1990 "The sleeping sentinels of social work." Academic Questions 90:40^8. Ferree, Myra Marx, and Elaine J. Hall 1996 "Rethinking stratification from a feminist perspective: Gender, race, and class in mainstream textbooks." American Sociological Review 61:929-950. Back Stage at Social Problems 109 Ferree, Myra Marx, Judith Lorber, and Beth B. Hess 1999 "Introduction." In Revisioning Gender, Myra Marx Feree, Judith Lorber, and Beth B. Hess, eds., xv-xxxvi. Thousand Oaks, CA: Sage Publications. Fogg, Louis, and Donald W. Fiske 1990 "Foretelling the judgments of reviewers and editors." American Psychologist 48:293-295. Fox, Mary Frank, and Vincent C. Ferri 1992 "Women, men, and their attributions for success in academe." Social Psychological Quarterly 55:257-271. Friedrichs, Robert W. 1972 A Sociology of Sociology. New York: Free Press. Fyfe, James J., and Rita J. Simon (eds.) 1994 Editors as Gatekeepers: Getting Published in the Social Sciences. New York: Rowman and Littlefield. Gamson, Joshua 1995 "Must identity movements self-destruct? A queer dilemma." Social Problems 42:390407. Garield, Eugene 1986 "Refereeing and peer review, part 2. Research on refereeing and alternatives to the present system." Current Contents 32:3-9. Garfunkle, Joseph M., and Martin H. Ulshen 1994 "Effect of institutional prestige on reviewers' recommendations and editorial decisions." Journal of American MedicalAssociation 272:137-139. Gilliland, Stephan W., and Brendan A. Beckstein 1996 "Procedural and distributive justice in the editorial review process." Personnel Psychology 49:669-691. Gilliland, Stephen W., and Jose M. Cortina 1997 "Reviewer and editor decision making in the journal review process." Personnel Psychology 50:427-452. Gottfredson, S. 1978 "Evaluating psychological research reports: Dimensions, reliability and correlates of quality judgments." American Psychologist 33:920-934. Gomez-Meija, Luis R., and D. B. Balkin 1992 "Determinants of faculty pay: An agency theory perspective." Academy of Management Journal 35:921-95 5. Grainey, T., D. Pollack, and L. Kusmierek 1984 "How three Chicago newspapers covered the Washington-Epton campaign." Journalism Quarterly 61:352-363. Hargens, Lowell L. 1988 "Scholarly consensus and journal rejection rates." American Sociological Review 53:139151. Hartsock, Nancy C. M. 1998 The Feminist StandpointRevisited and Other Essays. Boulder, CO: Westview Press. Lempert, Richard 1989 "Humility is a virtue: On the publicization of policy-relevant research." Law and Society Review 23:146-161. National Science Foundation 1997 Women, Minorities, and Persons with Disabilities in Science and Engineering: 1996. Washington, D.C.: National Science Foundation. Presser, Stanley 1980 "Collaboration and the quality of research." Social Studies of Science 10:95-101. Schneider, Joseph W. 1990 "The case of the 'unfair' review: Ethical issues from an editor's file." The American Sociologist, Spring, 88-95. Susser, Mervyn, and Alfred Yankauer 1993 "Prior, duplicate, repetitive, fragmented, and redundant publications and editorial decisions." American Journal of Public Health 83:792-794. 110 MILLER/PERRUCCI West, Candace, and Sarah Fenstermaker 1995 "Reply: (Re)doing difference." Gender and Society 9:8-37. Wilson, Franklin D., Gale L. Smoke, and David J. Martin 1973 "The replication problem in sociology: A report and a suggestion." Sociological Inquiry 43:141-149. Wolfe, Alan 1990 "Books vs. articles: Two ways of publishing sociology." Sociological Forum 5:477-489. Zuckerman, Harriet, and Robert K. Merton 1971 "Patterns of evaluation in science." Minerva 9:96-100.