80 Fundamental Considerations in Language Testing that need to be made in educational programs and rank them from most important to least important. What factors did you consider in this ranking (for example, number of students affected, impact on students' careers, impact on teacher's career)? 5. How is the relative importance of the decisions to be made related to the qualities of reliability and validity? 4 Communicative language ability Introduction Performance on language tests is affected by a wide variety of factors, and an understanding of these factors and how they affect test scores is fundamental to the development and use of language tests. Although language testing specialists have probably always recognized the need to base the development and use of language tests on a theory of language proficiency {for example, Carroll 1961a, 1968; Lado 1961), recently they have called for the incorporation of a theoretical framework of what language proficiency is with the methods and technology involved in measuring it (Upshur 1979; Henning 1984; Bachman and Clark 1987). The frameworks presented in this chapter and the next constitute an initial response to this call, and reflect my conviction that if we are to develop and use language tests appropriately, for the purposes for which they are intended, we must base them on clear definitions of both the abilities we wish to measure and the means by which we observe and measure these abilities. In this chapter I describe communicative language ability in a way that I believe provides a broad basis for both the development and use of language tests, and language testing research. This description is consistent with earlier work in communicative competence (for example, Hymes 1972b, 1973; Munby 1978; Canale and Swain 1980; Savignon 1983; Canale 1983), in that it recognizes that the ability to use language communicatively involves both knowledge of or competence in the language, and the capacity for implementing, or using this competence (Widdowson 1983; Candlin 1986).l At the same time, I believe the framework presented here extends earlier models, in that it attempts to characterize the processes by which the various components interact with each other and with the context in which language use occurs. I do not presume to present this framework as a complete theory of language abilities; books, indeed entire libraries have been written on 82 Fundamental Considerations in Language Testing specific aspects of this. 1 expect that those who become interested in measuring specific language abilities will also become familiar with the relevant research literature, and that, as test development and use proceed, will themselves contribute to that research. This framework is, however, presented as a guide, a pointer, if you will, to chart directions for research and development in language testing. As research progresses, it is likely that changes will be made in the framework itself to reflect our growing knowledge. And while this framework is based largely on research in linguistics and applied linguistics, it has evolved through empirical research in language testing (Bachman and Palmer 1982a). The model presented here is thus a result of refinement on the basis of empirical evidence, illustrating, I believe, its utility for guiding and informing empirical research in language testing. Language proficiency and communicative competence An earlier framework for describing the measurement of language proficiency was that incorporated in skills and components models such as those proposed in the early 1960s by Lado (1961) and Carroll (1961b, 1968). These models distinguished skills (listening, speaking, reading, and writing) from components of knowledge (grammar, vocabulary, phonology/ graphology), but did not indicate how skills and knowledge are related. It was not clear whether the skills were simply manifestations of the knowledge components in different modalities and channels, or whether they were qualitatively different in other ways.2 For example, does reading differ from writing only in that it involves interpretation rather than expression? If that were so, how can we account for the fact that although few of us can write with the sophistication and elegance of T. S. Eliot or William Faulkner, we can read and comprehend such writers? A more serious limitation of the skills/components model was its failure to recognize the full context of language use — the contexts of discourse and situation. Halliday's (1976) description of language functions, both textual and illocutionary, and van Dijk's (1977) delineation of the relationship between text and context, clearly recognize the context of discourse. Hymes (1972b, 1973, 1982) further recognizes the sociocultural factors in the speech situation. What has emerged from these ideas is an expanded conception of language proficiency whose distinguishing characteristic is its recognition of the importance of context beyond the sentence to the appropriate use of language. This context includes both the Communicative language ability 83 discourse, of which individual utterances and sentences are part, and the sociolinguistic situation which governs, to a large extent, the nature of that discourse, in both form and function. Along with this recognition of the context in which language use takes place has come a recognition of the dynamic interaction between that context and the discourse itself, and an expanded view of communication as something more than the simple transfer of information. Thus, Hymes (1972b) describes language use as follows: the performance of a person is not identical with a behavioral record. ... It takes into account the interaction between competence (knowledge, ability for use), the competence of others, and the cybernetic and emergent properties of events themselves. (emphasis added) (Hymes 1972b:283) Similarly, Savignon (1983) characterizes communication as: dynamic rather than .. . static. ... It depends on the negotiation of meaning between two or more persons. ... [It] is context specific. Communication takes place in an infinite variety of situations, and success in a particular role depends on one's understanding of the context and on prior experience of a similar kind. (Savignon 1983:8-9) Kramsch's (1986) discussion of communicative interaction echoes these notions: Interaction always entails negotiating intended meanings, i. e., adjusting one's speech to the effect one intends to have on the listener. It entails anticipating the listener's response and possible misunderstandings, clarifying one's own and the other's intentions and arriving at the closest possible match between intended, perceived, and anticipated meanings. (Kramsch 1986:367) Recent formulations of communicative competence thus provide a much more inclusive description of the knowledge required to use language than did the earlier skills and components models, in that they include, in addition to the knowledge of grammatical rules, the knowledge of how language is used to achieve particular communicative goals, and the recognition of language use as a dynamic process. 84 Fundamental Considerations in Language Testing A theoretical framework of communicative language ability Communicative language ability (CLA) can be described as consisting of both knowledge, or competence, and the capacity for implementing, or executing that competence in appropriate, con-textualized communicative language use. This is essentially how Candlin (1986) has described communicative competence: the ability to create meanings by exploring the potential inherent in any language for continual modification in response to change, negotiating the value of convention rather than conforming to established principle. In sum, ... a coming together of organized knowledge structures with a set of procedures for adapting this knowledge to solve new problems of communication that do not have ready-made and tailored solutions. (Candlin 1986:40) The framework of CLA I propose includes three components: language competence, strategic competence, and psychophysiological mechanisms. Language competence comprises, essentially, a set of specific knowledge components that are utilized in communication via language. Strategic competence is the term I will use to characterize the mental capacity for implementing the components of language competence in contextualized communicative language use. Strategic competence thus provides the means for relating language competencies to features of the context of situation in which language use takes place and to the language user's knowledge structures (sociocultural knowledge, 'real-world' knowledge). Psychophysiological mechanisms refer to the neurological and psychological processes involved in the actual execution of language as a physical phenomenon (sound, light). The interactions of these components of CLA with the language use context and language user's knowledge structures are illustrated in Figure 4.1. Language competence Recent frameworks of communicative competence have included several different components associated with what I will call language competence. In describing a theoretical framework for specifying an individual's communicative competence in a second language, Munby (1978) includes 'linguistic encoding' (the realization of language use as verbal forms), 'sociocultural orientation' (contextual appropriacy and communicative needs), 'sociosemantic Communicative language ability 85 Figure 4,1 Components of communicative language ability in communicative language use basis of linguistic knowledge', and 'discourse level of operation'. Canale and Swain (1980), examining the theoretical bases of language teaching and language testing, distinguish 'grammatical competence', which includes lexis, morphology, sentence-grammar semantics, and phonology, from 'sociolinguistic competence', which consists of sociocultural rules and rules of discourse, while Canale (1983) makes a further distinction between sociolinguistic competence (sociocultural rules) and 'discourse competence' {cohesion and coherence). Finally, Hymes (1982), in a far-reaching description of 'linguistic competence', includes 'resource grammar' (features that are part of the formal code), 'discourse grammar' (features typically associated with style, such as informality and politeness), and 86 Fundamental Considerations in Language Testing 'performance style' (idiosyncratic features of individual language use). Attempts to empirically validate these various components have not been conclusive. Allen et. al. (1983), for example, developed measures of grammatical competence (morphology and syntax), discourse competence (cohesion and coherence), and sociolinguistic competence (sensitivity to register). The factor analysis of their test scores failed ro support the factorial distinctness of these particular componenrs. Bachman and Palmer (1982a), on the other hand, found some support for the distinctness of components of what they called 'communicative proficiency'. They developed a battery of language tests that included grammatical competence (morphology and syntax), pragmatic competence (vocabulary, cohesion, and organization), and sociolinguistic competence (sensitivity to register, naturalness, and cultural references). The results of their study suggest that the components of what they called grammatical and pragmatic competence are closely associated with each other, while the components they described as sociolinguistic competence are distinct. The description of language competence presented here builds upon these empirical findings by grouping morphology, syntax, vocabulary, cohesion, and organizarion under one component, organizational competence. 'Pragmatic competence' is redefined ro include not only elements of Bachman and Palmer's sociolinguistic competence, but also those abilities related to the functions that are performed through language use. Language competencies can thus be classified into two types: organizational competence and pragmatic competence. Each of these, in turn, consists of several categories. The components of language competence are illustrated in Figure 4.2. This 'tree' diagram is intended as a visual metaphor and not as a theoretical model, and as with any metaphor, it captures certain features at the expense of others. In this case, this diagram represents the hierarchical relationships among the components of language competence, at the expense of making them appear as if they are separate and independent of each other. However, in language use these components all interact with each other and with features of the language use situation. Indeed, it is this very interaction between the various competencies and the language use context that characterizes communicative language use. In the last part of this chapter, a model of how these competencies may interact in language use is presented in the discussion of strategic competence. Communicative language ability 87 LANGUAGE COMPETENCE ORGANIZATIONAL COMPETENCE PRAGMATIC COMPETENCE GRAMMATICAL TEXTUAL ILLOCUTIONARY SOCIOLINGUISTIC COMPETENCE COMPETENCE COMPETENCE COMPETENCE Voc Moipti Synt. PhortGraeh. Coles. Rtet Ideal Manlp. Heur. Imag. Senat. Sens«. Sans*. Cultural Org. Funds.Funds. Funds. Funds, lo Dial, lo Reg to Nat. Rels.4 or Variety Figs oi Speech Figure 4.2 Components of language competence Organizational competence Organizational competence comprises those abilities involved in controlling the formal structure of language for producing or recognizing grammatically correct sentences, comprehending their propositional content, and ordering them to form texts. These abilities are of two types: grammatical and textual. Grammatical competence Grammatical competence includes those competencies involved in language usage, as described by Widdowson (1978). These consist of a number of relatively independent competencies such as the knowledge of vocabulary, morphology, syntax, and phonology/ graphology. These govern the choice of words to express specific significations, their forms, their arrangement in utterances to express propositions, and their physical realizations, either as sounds or as written symbols. Suppose, for example, a test taker is shown a picture of two people, a boy and a taller girl, and is asked to describe it. In so doing, the test taker demonstrates her lexical competence by choosing words with appropriate significations {boy, girl, tall) to refer to the contents of the picture. She demonstrates her knowledge of morphology by affixing the inflectional morpheme (-er) to 'tall'. She demonstrates her knowledge of syntactic rules by purring the words in the proper order, to compose the sentence 'The girl is taller 88 Fundamental Considerations in Language Testing than the boy'. When produced using the phonological rules of English, the resulting utterance is a linguistically accurate representation of the information in the picture. Textual competence Textual competence includes the knowledge of the conventions for joining utterances together to form a text, which is essentially a unit of language - spoken or written - consisting of two or more utterances or sentences that are structured according to rules of cohesion and rhetorical organization. Cohesion comprises ways of explicitly marking semantic relationships such as reference, sub-situation, ellipsis, conjunction, and lexical cohesion (Halliday and Hasan 1976), as well as conventions such as those governing the ordering of old and new information in discourse. Rhetorical organization pertains to the overall conceptual structure of a text, and is related to the effect of the text on the language user (van Dijk 1977:4). Conventions of rhetorical organization include common methods of development such as narration, description, comparison, classification, and process analysis (McCrimman 1984). We teach some of these organizational conventions formally in expository writing classes when we show students how to order information in a paragraph: topic sentence, first primary support sentence, secondary support sentences, second primary support sentence ... conclusion, or transition sentence. Other conventions for organizing discourse may not be taught formally at all, however, either because they are not fully understood or because they are simply too complex to teach. Textual competence is also involved in conversational language use. Indeed, much of the work in discourse analysis that takes conversation as its primary point of departure deals with components of textual competence (for example, Sinclair and Coulthard 1975; Coulthard 1977; Hatch 1978; Larsen-Freeman 1980; Richards and Schmidt 1983b). Conventions involved in establishing, maintaining, and terminating conversations have been discussed in terms of 'maxims' (Grice 1975), 'conversation rules' (Hatch and Long 1980), 'conversational routines' (Coulmas 1981b), and 'conversational competence' (Richards and Sukwiwat 1983). These conventions, such as attention getting, topic nomination, topic development and conversation maintenance (Hatch 1978) appear to be ways in which interlocutors organize and perform the turns in conversational discourse, and may be analogous to the rhetorical patterns that have been observed in written discourse. Communicative language ability 89 What this work on the analysis of conversational language use reveals, I believe, is a rich variety of devices for marking cohesive relationships in oral discourse, and for organizing such discourse in ways that are maximally appropriate to the language use context and maximally efficient in achieving the communicative goals of the interlocutors. And while many of these conventions have analogues in written discourse, it is obvious that conversational interchange, by its interactive or reciprocal nature, gives rise to or necessitates devices for organizing discourse that are unique to this genre of discourse. Nevertheless, rather than considering these conventions as a separate component of communicative language ability, I believe they can be best described in terms of the abilities associated with textual competence. Pragmatic competence The abilities discussed thus far pertain to the organization of the linguistic signals that are used in communication, and how these signals are used to refer to persons, objects, ideas, and feelings. That is, they concern the relationships among signs and their referents. Equally important, in communicative language use, are the relationships between these signs and referents on the one hand, and the language users and the context of communication, on the other. The description of these latter relationships constitutes the domain of pragmatics, which van Dijk (1977) has described as follows: pragmatics must be assigned an empirical domain consisting of CONVENTIONAL RULES of language and manifestations of these in the production and interpretation of utterances. In particular, it should make an independent contribution to the analysis of the conditions that make utterances ACCEPTABLE in some situation for speakers of the language, (emphasis added) (van Dijk 1977:189-90) Van Dijk distinguishes two aspects of pragmatics: (1) the examination of the 'pragmatic conditions' that determine whether or not a given utterance is acceptable to other users of the language as an act, or the performance of an intended function; and (2) the characterization of the conditions that determine 'which utterances are successful in which situations' (p. 190). Pragmatics is thus concerned with the relationships between utterances and the acts or functions that speakers (or writers) intend to perform through these utterances, 90 Fundamental Considerations in Language Testing which can be called the illocutionary force of utterances, and the characteristics of the context of language use that determine the appropriateness of utterances. The notion of pragmatic competence presented here thus includes illocutionary competence, or the knowledge of the pragmatic conventions for performing acceptable language functions, and sociolinguistic competence, or knowledge of the sociolinguistic conventions for performing language functions appropriately in a given context. Illocutionary competence Speech acts The notion of illocutionary competence can be introduced by reference to the theory of speech acts. A sentence such as 'It's cold in here', for example, may function as an assertion about the physical atmosphere in a room, as a warning not to bring the baby in, or as a request to turn on the heater. Each of these is a different speech act. Searle (1969) distinguishes three types of speech act: utterance acts, propositional acts, and illocutionary acts. An utterance act is simply the act of saying something. A propositional act involves referring to something or expressing a predication about something. An illocutionary act is the function (assertion, warning, request) performed in saying something. The meaning of an utterance can thus be described in terms of its propositional content (reference and predication) and its illocutionary force (intended illocutionary act). Austin (1962) and Searle (1969) also include, in their discussions of meaning, perlocutionary acts, or the effect of a given illocutionary act on the hearer.3 To illustrate these different speech acts, imagine a context in which I wish to get someone to leave. To accomplish this, I use my illocutionary competence, which indicates that a simple statement can function as a request. (I will also use my sociolinguistic competence, discussed below, to determine which of several possible statements is the most appropriate in this specific context.) If I say, 'I would like you to leave', I am performing a propositional act in producing a sentence which is both grammatically well-formed and has propositional content, or signification. My ability to perform this propositional act derives from my grammatical competence. If the person I am addressing understands the signification of the utterance, interprets the illocutionary force of the act as a request (as it was intended to be interpreted), and carries out the request, the performance of the speech act has the consequence, or perlocutionary effect, of his leaving. This perlocutionary effect is, of course, Communicative language ability 91 dependent upon the grammatical and illocutionary competencies of both me and my interlocutor, but it also depends upon non-language competency factors, such as the other person's willingness and ability to comply. There are a number of general strategies by which a speaker can signal his intent in performing an illocutionary act. He can signal his intent directly by announcing its illocutionary force ('I request that you leave now'). He may also signal his intent by using an appropriate syntactic form, such as the imperative in 'Leave!' In this case, the general intention of the speaker is clear (the act is a directive of some sort), but the specific illocutionary force is not, for the imperative could be interpreted by a listener as an order, a command, or a warning. Another strategy available to the speaker is to be less direct. This consists of using a sentence type whose form is not generally associated with the given illocutionary act, and whose interpretation depends very heavily on the circumstances under which the act is performed. For example, the speaker could use a declarative sentence to state why the hearer should act: 'I can hardly stand your company any more.' Or he could be even less direct, and simply state, 'It's nearly midnight.' The less direct the speaker is in signaling the illocutionary force he intends, the more dependent its interpretation will be on the way it is said, and the context in which it is said. (The choice from among several alternative utterances of differing degrees of directness will thus be a function of both the speaker's illocutionary competence, and his sensitivity to the characteristics of the specific context, which is part of sociolinguistic competence, discussed below.) Fräser and Nolan (1981) have described eighteen strategies for requesting, illustrating the wide range of directness in requests that is possible in English. Sixteen of these strategies do not entail the use of the word 'request'. The following five examples from their list provide some indication of the complexity of language use: (1) By announcing the intent to perform the act ('I request that you help me.') (2) By using an imperative sentence, which conveys the intent ('Please help me.') (3) By expressing a consequence of the hearer's acting ('If you help me, I'll buy you a new comic book.') (4) By asking if the hearer has the ability to act ('Can/could/canVcouldn't you . . . help me?') 92 Fundamental Considerations in Language Testing (5) By asking if the hearer has a reason for (not) acting ('Why are (aren't) you helping me?') (Fräser and Nolan 1981: 101) Illocutionary competence is used both in expressing language to be taken with certain illocutionary force and in interpreting the illocutionary force of language. An extension of the example above illustrates what is intended and interpreted when one uses language over and above what is understood when one employs only the organizational competencies associated with usage. A: It's nearly midnight! B: It's raining cats and dogs. A: Thanks a lot! Employing grammatical competence, one could determine that the forms of these three utterances are grammatically accurate and that their propositional meanings match a mental image: two people, one of whom appears upset, in a room, a violent thunderstorm outside. Employing illocutionary competence, one interprets the three sentences by assigning to each an illocutionary force (recognizing each as a particular illocutionary act or collection of acts). The illocutionary forces of the three utterances (request, refusal, sarcastic rebuttal) are clarified by the addition of the words in parentheses. A: It's nearly midnight! (Please leave.) B: (No, I won't leave because) it's raining cats and dogs. A: Thanks a lot (for nothing)! Language functions The previous section has introduced the distinction between form and function in language use through the discussion of speech acts. However, to account for this distinction as it relates to both the expression of language (speech, writing) and its interpretation (listening, reading), we need to consider a broader framework of functions that we can accomplish through language use. The description of language functions provided here is drawn, to a large extent, from Halliday (1973, 1976), although it extends to adult language use several of the functions he has described in the context of child language acquisition. Furthermore, the functions described here are grouped into four macro-functions: ideational, manipulative, heuristic, and imaginative. By far the most pervasive function in language use is the ideational function, by which we express meaning in terms of our experience of the real world (Halliday 1973:20). This includes the use of language Communicative language ability 9Í to express propositions or to exchange information about knowledge or feelings. For example, language is used ideationally to present knowledge in lectures or scholarly articles. It is also ideationally used to express feelings, as when one pours out one's emotions to a good friend or in a diary, with or without any intention of eliciting advice or help. The manipulative functions are those in which the primary purpose is to affect the world around us. One such function is the instrumental function with which we use language to get things done. For example, we may get someone, including ourselves, to do something by forming or uttering suggestions, requests, orders, commands, or warnings. We may accomplish other things by saying what we intend to do, as, for example, with offers, promises, or threats. The regulatory function is used 'to control the behavior of others — to manipulate the persons and, with or without their help, the objects in the environment' (Halliday 1973:18). In addition, this function is performed in formulating and stating rules, laws, and norms of behavior. The interactional function of language is its use to form, maintain, or change interpersonal relationships. Any act of interpersonal language use involves two levels of message: context and relationship. Haley (1963) makes this point: When any two people meet for the first time and begin to establish a relationship, a wide range of behavior is possible between them. They might exchange compliments or insults or sexual advances or statements that one is superior to the other, and so on. As the two people define their relationship with each other, they work out what type of communicative behavior is to take place in this relationship. Every message they interchange by its very existence either reinforces this line or suggests a shift in it to include a new kind of message. In this way, the relationship is mutually defined by the presence or absence of messages interchanged by the two people. (Haley 1963:6-7) Phatic language use, such as in greetings, ritual inquiries about health, or comments on the weather, is primarily interactional in function. Its propositional content is subordinate to the relationship maintaining function. The heuristic function pertains to the use of language to extend our knowledge of the world around us, and occurs commonly in such acts as teaching, learning, problem solving, and conscious memorizing. Teaching and learning may be either formal, as in an academic setting, or informal, as in self-study. The use of language in problem 94 Fundamental Considerations in Language Testing solving is exemplified in the writing of papers in which one goes through the processes of invention, otganization, and revision. The use of language to aid the conscious retention of information is exemplified in the memorization of facts, words, formulae, or rules. It is important to note that this function also pertains to the use of language for the purpose of extending one's knowledge of language itself, that is, for acquiring or learning a language. For example, when a language teacher points to a book on a table and says, 'The book is on the table', he is not conveying information. That is, he is not performing an ideational function, but rather a heuristic function of illustrating the meaning of the preposition 'on' in English. The imaginative function of language enables us to create or extend our own environment for humorous or esthetic purposes, where the value derives from the way in which the language itself is used. Examples are telling jokes, constructing and communicating fantasies, creating metaphors or other figurative uses of language, as well as attending plays or films and reading literary works such as novels, short stories, or poetry for enjoyment. While these have been discussed as distinct functions, clearly most instances of language use fulfill several functions simultaneously. Such is the case when a teacher makes an assignment (ideational, manipulative, and heuristic functions) in an amusing way (imaginative function), or when one reads a magazine article for enjoyment (imaginative function) and in so doing acquires useful information (heuristic function). Furthermore, although language functions have been discussed as if they occurred in individual, unconnected utterances, it should be emphasized that the majority of language use involves the performance of multiple functions in connected utterances, and it is the connections among these functions that provide coherence to discourse. Sociolinguistic competence While illocutionary competence enables us to use language to express a wide range of functions, and to interpret the illocutionary force of utterances or discourse, the appropriateness of these functions and how they are performed varies from one language use context to the next, according to a myriad of sociocultural and discoursal features. Sociolinguistic competence is the sensitivity to, or control of the conventions of language use that are determined by the features of the specific language use context; it enables us to perform language functions in ways that are appropriate to that context. Without attempting to identify and discuss the features of the language use situation that determine the conventions of language use, I will Communicative language ability 95 discuss the following abilities under sociolinguistic competence: sensitivity to differences in dialect or variety, to differences in register and to naturalness, and the ability to interprer cultural references and figures of speech. Sensitivity to differences in dialect or variety In virtually every language there are variations in use that may be associated with language users in different geographic regions, or who belong to different social groups. These regional and social varieties, or dialects, can be characterized by different conventions, and the appropriateness of their use will vary, depending on the features of the language use context. An example of the way different contexts require the use of different varieties of English is that of a Black student who indicated that she would not consider using Black English in class, where 'Standard American English' would be appropriate. On the other hand, she would probably be understood as either affected and pretentious or joking, were she to use Standard American English in informal conversations with Black friends. Sensitivity to differences in dialect or variety of language is thus an important part of sociolinguistic competence. Sensitivity to differences in register Halliday, Mcintosh, and Strevens (1964) used the term 'register' to refer to variation in language use within a single dialect or variety.4 They distinguished differences in register in terms of three aspects of the language use context: 'field of discourse', 'mode of discourse', and 'style of discourse' (pp. 90^t). The field of discourse may consist simply of the subject matter of the language use, as in lectures, discussions, or written expositions; it may also refer to the entire language use context, as in the registers of playing football, planting trees, or computer 'hacking'. Variations in register also occur as a function of differences berween modes of discourse - spoken and written. Anyone who has attempted to capture 'genuine' dialogues in writing, or ro present a written paper 'conversationally' can attest to the differences between written and spoken regisrers. Another term that has been used to describe the features or conventions that characterize the language used within a particular area or for specific functions is 'discourse domain'. Swales (1987), for instance, discusses entries in philarelic catalogues and the language used in written mail requests for reprints of papers or articles as examples of domains that characterize discourse communities. The discourse domain in which language use occurs, whether it be spoken, as in lectures or job interviews, or written, as in business 96 Fundamental Considerations in Language Testing letters, job announcements, or scholarly papers, will determine the register of language use, including the specific functions and organizational features that are appropriate to that register. And just as the use of a particular dialect or variety is associated with membership in a speech community, using the register of a particular discourse domain can establish one's membership in a 'discourse community'. How often, for example, do we seek out people at large parties with whom we can 'talk shop'? Likewise, we very quickly feel like an outsider when we cannot participate in a given domain of discourse. The following test item from Bachman and Palmer (1982d) provides an example of sensitivity to discourse domain (that of love letters): In the blanks below, wrire the appropriate greeting and ending: ...iMÁsf........................... I've been thinking about you all day, and can hardly wait to hold you in my arms again. I'll be waiting for you under the apple rree. George ô (Some of the responses of advanced non-native speakers of American English to this question, such as, 'To the Loved One', and 'Cheers', suggest that this is a relatively esoteric area of language competence, to say the least!) The third of Halliday, Mcintosh and Strevens' dimensions for characterizing variations in register is 'style of discourse, which refers to the relations among the participants' (p. 92). The classic discussion of style is still Joos (1967), who distinguished five different levels of style, or register, in language use: frozen, formal, consultative, casual, and intimate. These five styles are characterized primarily in terms of the relationships that obtain between the participants in the language use context, so that the use of the inappropriate style can be interpreted as presumptuous or even rude. Consider, for example, the inappropriate familiarity of the salesperson who telephones people at random from numbers in the telephone directory, and says something like, 'Hi, Lyle, this is Herb from Ail-American Storm Windows. How are you tonight? That's just great. Say, you know winter is just around the corner, and I was just calling to let you know that. . ,' Sociolinguistic competence thus involves sensitivity to variations in register, since the illocutionary force of utterances virtually always Communicative language ability 91 depends on the social contexts in which they are used. These variations occur in both highly formalized language use, as in greetings, introductions, or leave takings, and in extended language use, as when we use more elaborate syntactic structures and cohesive devices in formal writing, or when we sustain a conversation in a regional dialect with childhood friends and family members. Sensitivity to naturalness A third aspect of sociolinguistic competence is that which allows the user to either formulate or interpret an utterance which is not only linguistically accurate, but which is also phrased in what Pawley and Syder (1983) call a nativelike way, that is, as it would be by speakers of a particular dialect or variety of a language who are native to the culture of that dialect or variety. For example, consider the interpretation of the second line of the following exchange: A: Why are you yelling? B: Because I have much anger with him. While this example merely sounds strange, or archaic, non-naturalness of language use can also affect interpretability. Compare, for example, i wish you wouldn't do that' with 'I would feel better by your not doing that', or T have my doubts' with 'I have several doubts'. Ability to interpret cultural references and figures of speech The final aspect of sociolinguistic competence to be dealt with here is that which allows us to use and interpret cultural references and figures of speech. Many of these will be incorporated, with set meanings, into the lexicon of any language, and can thus be considered part of lexical, or vocabulary, competence. Nevertheless, knowledge of the extended meanings given by a specific culture to particular events, places, institutions, or people is required whenever these meanings are referred to in language use. For example, to interpret the following exchange, the language user would have to know that 'Waterloo' is used linguistically to symbolize a major and final defeat with awful consequences for the defeated: A: I hear John didn't do too well on his final exam. B: Yeah, it turned out to be his Waterloo. Knowledge of only the referential meaning of the place name without knowing what the name connotes in American and British English would not allow the correct interpretation of the second utterance. Similarly, interpreting figurative language involves more than 98 Fundamental Considerations in Language Testing simply knowledge of referential meaning. For example, the correct interpretation of hyperboles such as, '1 can think of a million good reasons for not smoking' and cliches like 'It's a jungle out there', require more than a knowledge of the signification of the words and grammatical structures involved, while similes such as Faulkner's 'the sound of tires on the hot tar was like the tearing of new silk', and metaphors like Eliot's The river sweats Oil and tar', invoke images far beyond those of the concrete objects to which they refer. Although individuals from different cultural backgrounds will, no doubt, be able to attach meaning to figures of speech, the conventions governing the use of figurative language, as well as the specific meanings and images that are evoked are deeply rooted in the culture of a given society or speech community, which is why I have included them as part of sociolinguistic competence. To summarize, language competence comprises two types of competence, organizational and pragmatic. Organizational competence includes the knowledge employed in creating or recognizing grammatically correct utterances, in comprehending their proposi-tional content, and in organizing them to form oral or written texts. Pragmatic competence includes the types of knowledge which, in addition to organizational competence, are employed in the con-textualized performance and interpretation of socially appropriate illocutionary acts in discourse. These competencies include the knowledge of language functions, of sociolinguistic rules of appropriateness, and of cultural references and figurative language. Strategic competence As mentioned above, one characteristic of recent frameworks of communicative competence is the recognition of language use as a dynamic process, involving the assessment of relevant information in the context, and a negotiation of meaning on the part of the language user. This dynamic view of communication is also reflected in the literature on interlanguage communication strategies. There have been essentially two approaches to defining communication strategies: the 'interactional' definition and the 'psycholinguists' definition (Fairch and Kasper 1984). The interactional definition, as stated by Tarone (1981), characterizes a communication strategy as 'the mutual attempt by two interlocutors to agree on a meaning in situations where the requisite meaning structures do not seem to be shared' (p. 288). Tarone includes both linguistic and sociolinguistic rule structures in her Communicative language ability 99 notion of meaning structure, and considers communication strategies distinct from this meaning structure. In their review of the literature, TxTch and Kasper (1984) observe that an interactional view of communication strategies is too narrow in scope, since it only applies to 'the negotiation of meaning as a joint effort between two interlocutors' (p. 51), while much communicative language use, such as reading novels or writing textbooks, involves only one individual, with no feedback from a second interlocutor. Tarone (1981) does, however, describe another type of strategy, the production strategy, as 'an attempt to use one's linguistic system efficiently and clearly, with a minimum of effort' (p. 289). Like communication strategies, production strategies are distinct from the language user's language competence. Unlike communication strategies, however, they 'lack the interactional focus on the negotiation of meaning' (ibid.). Recent frameworks of communicative competence that have incorporated the notion of strategies have generally accepted the interactional definition. Thus, Canale and Swain (1980), citing the research on communication strategies, include strategic competence as a separate component in their framework of communicative competence. They describe strategic competence as providing a compensatory function when the linguistic competence of the language users is inadequate: Strategic competence . . . will be made up of verbal and nonverbal communication strategies that may be called into action to compensate for breakdowns in communication due to performance variables or to insufficient competence. (Canale and Swain 1980:30) Canale (1983) has extended this definition of strategic competence to include both the compensatory characteristic of communication strategies and the enhancement characteristic of production strategies: Strategic competence: mastery of verbal and nonverbal strategies both (a) to compensate for breakdowns in communication due to insufficient competence or to performance limitations and (b) to enhance the rhetorical effect of utterances. (Canale 1983:339) While these definitions provide some indication of the function of strategic competence in facilitating communication, they are limited in that they do not describe the mechanisms by which strategic competence operates. I would also note that these definitions include 100 Fundamental Considerations m Language Testing non-verbal manifestations of strategic competence, which are clearly an important pan of strategic competence in communication, but which will not be dealt with in this book. In an attempt to provide a more general description of strategies of communication, Fa^rch and Kasper (1983) have described a 'psycho-linguistic' model of speech production. Drawing on the work of cognitive psychologists such as Miller et al. (1960) and Clark and Clark (1977), they describe a model of speech production that includes a planning phase and an execution phase. The planning phase consists of communicative goals and a planning process, the product of which is a plan. Communicative goals consist of (1) an actional element, associated with speech acts; (2) a modal element associated with the role relationship holding between the inter-actants; and (3) a propositional element, associated with the content of the communicative event (p. 24). Fierch and Kasper further describe the planning process as an interaction of three components: the communicative goal, the communicative resources available to the individual, and the assessment of the communicative situation (p. 27). The execution phase of Fserch and Kasper's model consists of 'neurological and physiological processes' that implement the plan, resulting in language use. Fa;rch and Kasper's model is intended only to explain the use of communication strategies in interlanguage communication. However, I view strategic competence as an important part of all communicative language use, not just that in which language abilities are deficient and must be compensated for by other means, and would therefore extend Fajrch and Kasper's formulation to provide a more general description of strategic competence in communicative language use. I include three components in strategic competence: assessment, planning, and execution. Assessment component The assessment component enables us to (1) identify the information - including the language variety, or dialect - that is needed for realizing a particular communicative goal in a given context; (2) determine what language competencies (native language, second or foreign language) are at our disposal for most effectively bringing that information to bear in achieving the communicative goal; (3) ascertain the abilities and knowledge that are shared by our interlocutor; and (4) following the communication attempt, evaluate the extent to which the communicative goal has been achieved. The Communicative language ability 101 importance of assessing our interlocutor's capabilities has been underscored by Corder (1983): The strategies adopted by speakers, of course, depend upon their interlocutors. What we attempt to communicate and how we set about it are determined not only by our knowledge of the language but also by our current assessment of our interlocutor's linguistic competence and his knowledge of the topic of discourse. (Corder 1983: 15) The process of assessment can be illustrated by the following example. When I lived in Bangkok, it was frequently a major undertaking to explain to dinner guests how to reach my house. After having struggled to give directions on several occasions, I eventually discovered, through the process of assessment, that it was necessary to first determine what part of the city the person would be coming from. This was learned at great embarrassment after I had sent many a hungry dinner guest off in the wrong direction, assuming that they would be coming to my house the same way I usually did. Being able to extract this information necessitated determining the most effective and appropriate forms and structures (in both English and Thai) to do so over the phone without sounding impolite or eccentric. Even the most polite attempt to convey such information was of little avail, however, if the person I had invited had little sense of direction or was unfamiliar with the major districts of the city. In such cases, the conversation rather quickly turned from giving directions to attempting to ascertain some landmark we both knew. Once this was determined, a new set of directions could be provided. In summary, the information I needed to effectively attain my communicative goal was the part of the city the person was coming from, and the bulk of the conversation frequently involved ascertaining what geographical knowledge of the city my intended dinner guest and I shared. Planning component The planning component retrieves relevant items (grammatical, textual, illocutionary, sociolinguistic) from language competence and formulates a plan whose realization is expected to achieve the communicative goal. In the case of a monolingual speech context, relevant items will be drawn from the native language (Li) competence, while in a bilingual, second, or foreign language use context, the items may be retrieved from the native language, from 102 Fundamental Considerations in Language Testing the language user's interlanguage rule system (Lj), or from the second or foreign language (L2). In the example above, I retrieved the appropriate forms of address and the questioning routines I had learned specifically for the occasion, and formulated a plan for utilizing them to acquire the information needed. Depending on how the conversation evolved, other items would be retrieved and other plans formulated. This description of the assessment and planning components in communicative language use is similar to Johnson's 1982 characterization of the processes involved in communication: There are at least three processes which [a listener] must undertake if he is to fulfill his role as interactant. Firstly, he must 'scan' [the speaker's] utterance to extract... its pragmatic information ... [which is] that part of the total information conveyed which contributes to the information required by the speaker. It is, in short, information which the listener wants to receive.. .. [The listener] approaches the task of listening comprehension prepared to search for certain pieces of information in his interactanťs words. Once this information comes, it has to be assessed according to the speaker's aim, and this is the second process which [the listener] must undertake. . . . [The listener] compares, then, what he is told with what he wants to know, identifies any mismatch and then - as a third process - formulates his next utterance. (Johnson 1982:149) It is exactly these characteristics of communicative language use that I associate with strategic competence. As indicated above, communication involves a dynamic interchange between context and discourse, so that communicative language use is not characterized simply by the production or interpretation of texts, but by the relationship that obtains between a text and the context in which it occurs. The interpretation of discourse, in other words, requires the ability to utilize available language competencies to assess the context for relevant information and to then match this information to information in the discourse. It is the function of strategic competence to match the new information to be processed with relevant information that is available (including presuppositional and real world knowledge) and map this onto the maximally efficient use of existing language abilities. Communicative language ability 103 Execution component Finally, the execution component draws on the relevant psychophysiological mechanisms to implement the plan in the modality and channel appropriate to the communicative goal and the context. The interactions among the components of strategic competence, language competencies, and the language use context are illustrated in Figure 4.3 below, which represents an extension of Fa;rch and Kasper's model (1983: 25). GOAL Inlerpret or express speech with specific function, modality, and content 1 SITUATIONAL ASSESSMENT PLANNING PROCESS Retrieve items from language competence LANGUAGE COMPETENCE Organizational competence Pragmatic Competence L, L, I PLAN Composed of items, Ihe realization of which is expected lo lead to a communicative goal I EXECUTION A neurological and physiological process PSYCHOPHYSIOLOGICAL MECHANISMS t UTTERANCE Express or interpret language Figure 4.3 A model of language use 104 Fundamental Considerations in Language Testing In comprehending, or interpreting, the same processes - assessment, planning, and execution - are involved. In attending a lecture, for example, our communicative goal may be to better understand a given area of knowledge. In assessing the situation, we consider the topic, what we know about the speaker, the likely audience, and our own knowledge and abilities, and form expectations regarding the utterances we will have to interpret and our likelihood of being able to comprehend them. This assessment may be quite deliberate, as in the case of a classroom lecture, and we may actually make conscious preparations, such as reading an assigned article, to better achieve our communicative goal. In other cases, our assessment may be less premeditated, and we may need to make on-the-spot adjustments. For example, if we find that the lecturer's speech is heavily accented, we will need to employ competencies associated with interpreting non-native speech. If we are lacking these, we may find the lecture incomprehensible, and fail in our communicative goal. Recent research into the role of schemata in reading (for example, Carrell 1982, 1986) provides, I believe, further exemplification of the role of plans and resultant expectations in receptive language use. The influence of strategic competence on language test performance At this point we may well wonder about the extent to which strategic competence affects scores on language tests. Suppose that two non-native speakers of a language were to take three tests: a test of usage, a test of contextualized receptive performance in which the scores are influenced in part by practical outcomes, and a test of productive oral performance. Suppose we find that the two subjects' scores are the same on the first two tests but different on the third. When we analyze tapes of the third test, we find that the more effective test taker made use of more of the various different ways of performing illocutionary acts than did the second, and that her propositions made more references to relevant objects in the environment. We ask the less effective test taker why she did not try all of the different ways of performing the required illocutionary acts that the more effective test taker used, and why she did not make reference to relevant objects in the environment to get her message across. She replies, 'I just didn't think of them at the time', or 'I didn't notice the objects in the environment', or it didn't seem worth the effort.' In such a case we might characterize the more effective language Communicative language ability 105 user as more willing and adept at making use of what she knew was available in order to perform a function using language, but we would be reluctant to say that the two speakers' language competence differed. In other words, we would consider both persons to have the same control of the rules of usage and use, but to differ in their willingness to exploit what they knew and their flexibility in doing so. And while this example deals with the generality of strategic competence for non-native speakers, I believe it applies to native speakers as well. Some types of test tasks may measure strategic competence, almost by design. In tests of reading comprehension, for example, it is common practice to include questions requiring inference. I believe that correctly answering such questions requires strategic competence, in that the test taker must recognize what information outside the discourse itself is relevant to answering the question, and then must search for that information in his memory. There would also appear to be some types of language test tasks that are particularly susceptible to the effects of test takers' strategic competence, in that they can successfully complete such tasks by utilizing their strategic competence to compensate for deficiencies in other competencies. An example of such a test is the picture description test described by Palmer (1972, 1981), in which the test taker is required to describe a picture as quickly as possible in order to enable the examiner to distinguish the picture described from a number of similar pictures. As a test of organizational competence, it should require the examinee to employ a variety of vocabulary items. In fact, the subject matter of the pictures was selected with this in mind. However, Palmer (1981) noticed that some subjects with obviously small vocabularies described the pictures in terms of their placement on the page, in terms of how dark or light they are, or how big or small. These subjects appear to have adopted the strategy of ignoring the propositional content of the pictures and communicating instead about the nonverbal visual code {the lines and shapes) used to represent them. In tests such as these, it may well be that performance is affected more by strategic competence than by the specific language ability the test was originally intended to measure. The type of scoring used can also be expected to influence the effect of strategic competence on test performance. If a test is scored solely on the basis of the practical effect of the language performance, the potential contribution of strategic competence may be high. Consider, for example, a test in which an examinee is required to write a set of instructions describing how to perform some task, such 106 Fundamental Considerations in Language Testing as preparing a scattergram: drawing two intersecting axes and labeling them, and entering a small set of data correctly. Suppose the test is scored only on the extent to which the scattergram drawn by the examiner matches that shown to the examinee, and that the examinee is informed that this is the criterion for correctness. An examinee willing to use what she knows to be simplified and perhaps ungrammatical structures might nevertheless be able to write a set of understandable instructions adequate for the task. On the other hand, an examinee not willing to do so might spend an inordinate amount of time in attempting to produce linguistically accurate sentences, and not complete the task. The same test scored partly on the basis of different criteria, say the range of structures used and the accuracy with which they are used, might reward the strategically adept examinee — the one willing to use simple and ungrammatical language - to a lesser degree. Can we measure strategic competence? I have argued that strategic competence influences language performance in that there do appear to be language users who make better use of their language competence in performing various functions. Similarly, some types of language test tasks seem to involve strategic competence to a greater degree than do others. However, rather than considering strategic competence solely an aspect of language competence, I consider it more as a general ability, which enables an individual to make the most effective use of available abilities in carrying out a given task, whether that task be related to communicative language use or to non-verbal tasks such as creating a musical composition, painting, or solving mathematical equations. And it is here that we begin to enter the realm of general cognitive abilities, or intelligence, which is beyond the scope of this book. Oiler (1983a) has hypothesized that what he calls 'a general factor of language proficiency', that involves 'the process of "pragmatic mapping" of utterance forms . . . into the contexts of experience' (p. 356) 'is the principal function of intelligence' (p. 355). John B. Carroll, on the other hand, (personal communication) holds that intelligence, while not totally independent, is distinct from language abilities. I would agree that it may be inaccurate to identify strategic competence with intelligence. At the same time, to simply dismiss strategic competence as a general ability whose effects on language test performance we cannot measure is to beg the question. Determining the effects of various abilities on test performance is Communicative language ability 107 ultimately an empirical question - that of construct validation. It is therefore my hope that the formulation of strategic competence presented here will prove useful for generating hypotheses about test performance, and for designing tests that will enable us to examine these hypotheses through empirical research. Psychophysiological mechanisms Thus far 1 have discussed those competencies that I believe are part of communicative language ability. In order to fully characterize language use, however, it is also necessary to consider the psychophysiological mechanisms that are involved in language use. These are essentially the neurological and physiological processes that Fserch and Kasper (1983) include in their discussion of the execution phase of language use. Thus, we can distinguish the visual from the auditory channel and the productive from the receptive mode. In receptive language use, auditory and visual skills are employed, while in productive use the neuromuscular skills (for example, articulatory and digital) are employed. For instance, in the example on pp. 87-8 above, the test taker correctly used her linguistic competence to torm the sentence, The girl is taller than the boy.' She also used her visual skill to gain access to the non-linguistic information in the picture, her auditory skill to gain access to the information in the administrator's instructions, and her articulatory skill to pronounce the words correctly and to provide appropriate stress and intonation. Summary In this chapter, a framework for describing communicative language ability as both knowledge of language and the capacity for implementing that knowledge in communicative language use has been proposed. Communicative language ability consists of language competence, strategic competence, and psychophysiological mechanisms. Language competence includes organizational competence, which consists of grammatical and textual competence, and pragmatic competence, which consists of illocutionary and sociolinguistic competence. Strategic competence is seen as the capacity that relates language competence, or knowledge of language, to the language user's knowledge structures and the features of the context in which communication takes place. Strategic competence performs assessment, planning, and execution functions in determining the most 108 Fundamental Considerations in Language Testing effective means of achieving a communicative goal. Psychophysiological mechanisms involved in language use characterize the channel (auditory, visual) and mode {receptive, productive) in which competence is implemented. This framework is presented as one part of a theory of factors that affect performance on language tests. The second part of this theory, a description of the characteristics of test methods that affect performance on language tests, will be described in the next chapter. Notes 1 Although words that refer to various aspects of using language have been quite clearly defined over the years, there appears to be some disagreement in their interpretation. Thus, at the risk of adding to this confusion, I will try to indicate exactly how I understand and use these words. I use knowledge and competence more or less synonymously, to refer to entities which we may hypothesize to be in the minds of language users. Furthermore, I use competence in Hymes's (1972b) sense, and do not limit this to 'linguistic competence', as originally defined by Chomsky (1965). The terms trait and construct are more precise synonyms for knowledge and competence. The term ability includes both knowledge or competence and the capability for implementing that competence in language use - the 'ability to do X'. I consider communicative language ability to provide a more inclusive definition of proficiency than has been defined in the context of oral language testing (for example, American Council on the Teaching of Foreign Languages 1986; Liskin-Gasparro 1984; Lowe 1985,1986). The terms use and perform (and their noun counterparts) are more or less synonymous, referring to the execution, or implementation of abilities. Use and perform thus subsume a wide range of terms such as, listen, speaks read, write, produce, interpret, express, receive, understand, and comprehend, which themselves have more specific meanings. 2 Carroll (1968) discusses linguistic performance abilities', such as speed and diversity of response, complexity of information processing, and awareness of linguistic competence, but considers these as essentially outside the construct of language proficiency. These will be touched on in the discussion of test method factors in Chapter 5. 3 Searle (1969) points out that not all illocutionary acts have Communicative language ability 109 perlocutionary effects associated with them, and argues that perlocutionary effects cannot be considered part of illocutionary acts. 4 A comprehensive discussion of register is given in Besnier (1986). Further reading The framework described by Canale and Swain (1980) is seminal to research on communicative competence. This paper includes an excellent review of the research related to the four main components of the framework: grammatical, discourse, sociolinguistic, and strategic competence. Widdowson's (1978) distinctions between usage and use and between text and discourse provide a basis for understanding the relationship between the organizational and illocutionary aspects of language competence. Hymes (1972b) develops the notion of sociolinguistic appropriateness, distinguishing between what is possible, what is feasible, what is appropriate, and what is actually done, in communicative language use. Halliday (1976) discusses the outlines of his theory of functional grammar. Hymes (1972b, 1982) and van Dijk (1977) provide extensive analyses of the features that constitute the context of language use. Richards and Schmidt (1983b) provide extensive examples that illustrate the role of strategic competence in conversation. Richards and Sukwiwat (1983) discuss the effects of transferring native language (Thai) conversational conventions to conversational discourse in a second language (English). Finegan and Besnier (1988) provide excellent discussions of dialects (Chapter 12), and registers (Chapter 13). Discussion questions 1. Does every native speaker have the same degree of communicative language ability? On what competencies are native speakers most likely to be the same? 2. In what aspects of communicative language ability are native speakers most likely to differ? What competencies arise as a result of being in the speech community? Which are a result of education? 3. Discuss the propositional, illocutionary, and perlocutionary acts in the following utterances: a. 'There isn't enough sugar in my coffee.' 110 Fundamental Considerations in Language Testing b. 'Do you have the time?' c. 'I want to speak to the manager!' d. 'Trick or treat?' e. 'Smoking causes lung cancer, heart disease, emphysema, and may complicate pregnancy.' 4. Is an order of acquisition or learning of language competencies implied by the proposed framework? Must some be mastered before others can be learned? Can some be mastered independently without mastering others? 5. Describe how you might measure an individual's strategic competence in language use. 6. Discuss the roles of language competence and strategic competence in the following exchange between a native speaker (NS) of English and a learner of English (L) (Fasrch and Kasper 1983:20}: NS: ... what do you read at home . . . what do you er L: mmmm ... er historie NS: do you have history lessons in school L: ... kings NS: when you learn about er I don't know ... old kings L: oh yes I have that. . . NS: but you like reading books about history L: er this history . . . you know ... er young histories ... er NS: aha L: not not with this old things you know kings or . . . all that... NS: do you mean . . . recent... in more recent years like that L: er ... a history is . . . maybe on a boy . . . girl and ... er young people life and . . . yer NS: oh you mean a story . .. just a story about people . . . not not 5 Test methods Introduction In Chapter 4 a framework for characterizing those abilities that are of interest in language tests was described. That framework provides a means for characterizing the traits, or constructs, that constitute the 'what' of language testing. However, we know that test performance is also affected by the characteristics of the methods used to elicit test performance. Some test takers, for example, may perform better in the context of an oral interview than they would sitting in a language laboratory speaking into a microphone in response to statements and questions presented through a pair of earphones. And individuals who generally perform well in oral interviews may find it difficult to speak if the interviewer is someone they do not know. Some test takers find a cloze test intimidating, and do poorly, while at the same time performing well on a test consisting of the completion of individual sentences, or a multiple-choice test based on a reading passage. 'Live' versus recorded presentation of aural material, personality of examiner, filling in the blanks in isolated sentences as opposed to completing blanks in a text, and the amount and type of text to be processed, are but a few examples of the ways in which the methods we employ in language tests can vary. These characteristics, or 'facets', of test methods constitute the 'how' of language testing, and are of particular importance for designing, developing, and using language tests, since it is these over which we potentially have some control. These test method facets are the primary focus of this chapter. One way to conceive of a language test is as a means for controlling the context in which language performance takes place. From this perspective, the characteristics of the test method can be seen as analogous to the features that characterize the context of situation, or the speech event, as this has been described by linguists (for example, Firth 1957; Hymes 1964, 1972a; Labov 1972; van Dijk 1977; Brown and Yule 1983). Firth (1957), for example, described