Prevention Science, Vol. 2, No. 2, 2001
Effects of the Positive Action Program on Achievement and Discipline: Two Matched-Control Comparisons1
Brian R. Flay,2,4 Carol G. Allred,3 and Nicole Ordway2
This paper reports on the effectiveness of an integrated comprehensive school model for character development, problem behavior prevention, and academic achievement enhancement. The Positive Action program consists of a school curriculum, together with schoolwide climate, family, and community components. As evaluated here, the yearly K-6 curriculum consists of over 140 fifteen-to-twenty-minute lessons per year delivered in school classrooms on an almost daily basis. The program is based on theories of self-concept, learning, behavior, and school ecology. We use a matched control design and school-level achievement and disciplinary data to evaluate program effects on student performance and behavior in two separate school districts. The program improved achievement by 16% in one district and 52% in another, and reduced disciplinary referrals by 78% in one district and 85% in the other. We discuss implications of these replicated findings for the prevention of substance abuse and violence, the improvement of school performance, and the reform of American schools.
KEY WORDS: prevention;achievement;matched-control;self-concept;violence;discipline.
Public and official demands for improvement in student achievement have been never ending (Coleman etal., 1966), and have increased of late (U.S. Department of Education (DoE), 1997; President Clinton's State of the Union Address, 1999, 2000). Schools are also expected to prevent such problem behaviors as violence (Eron et al., 1994), substance use (Johnston et al., 1998), and other behaviors requiring disciplinary action (Chandler et al., 1998). A number of different kinds of programs have been developed to address problems of academic achievement (Slavin & Fashola, 1998), smoking (Flay, 1985; Sussman et al., 1995), substance use (Peters & McMahon, 1996), violence (Tolan & Guerra, 1994), and many
1The program evaluated in the reported study was developed by the second author. The reported evaluation was conducted independently by the first and third authors.
2Health Research and Policy Centers, School of Public Health, University of Illinois at Chicago, Illinois. 3Positive Action, Inc., Twin Falls, Idaho.
4 Correspondence should be directed to Health Research and Policy Centers, School of Public Health, University of Illinois at Chicago, 850 W. Jackson Blvd., Chicago, Illinois 60607;e-mail: bflay@uic.edu
others. Although many of these programs are initially promising, most are problem specific and unable to provide comprehensive sustainable effects. One possible reason for this is because most of these programs address the microlevel predictors of the problem, and do not attempt to affect the multifaceted, distal factors. A comprehensive approach that includes self-concept development, schoolwide environmental change, and parental and community involvement may successfully affect not just one outcome, such as academic performance, violence, and so forth, but may affect all outcomes together. Recent changes in Title 1 legislation have acknowledged and facilitated the development/funding of comprehensive school reform programs; however, there are few that have been fully evaluated.
REVIEW OF LITERATURE
School Performance
In 1990, President Bush, and 50 state governors, created Goals 2000 to guide our children's educational
71
1389-4986/01/0600-0071$19.50/1 © 2001 Society for Prevention Research
72
Flay, Allred, and Ordway
future. The United States is making some progress toward those goals, but they clearly were not met by the year 2000. The March 1999 National Center for Educational Statistics report stated that there has been no change in fourth-grade reading achievement since 1992, with only 31% of our fourth-grade students demonstrating proficiency in reading (Donahue et al, 1999). Only 21% of U.S. students met mathematics competency for the 1996 National Assessment of Education Progress Test (NAEP; National Education Goals Panel, 1998).
Problem Behavior
The public is upset about lack of discipline and increased violence in schools. Some schools across the United States are responding by installing metal detectors, requiring school uniforms, hiring full-time security guards, and enforcing zero-tolerance policies, thus creating a prison-like atmosphere—not very conducive to learning. However, even with these measures, 1 in 10 eighth graders feels unsafe at school (Barton et al., 1998). During the 1996-97 school year, 1 in 10 schools reported at least one serious violent crime, including murder, rape, suicide, physical attack/fight with a weapon or robbery (U.S. DoE, 1998). Nationwide, 4.0% of students missed a day of school within the previous 30 days because they felt unsafe at school or traveling to and from school (Centers for Disease Control and Prevention, 1998). Forty-five percent of elementary schools have reported one or more violent incidents (U.S. DoE, 1998). Petty theft crimes are more common than violent crimes, and they occur as often in elementary schools as in middle schools. The prevalence of discipline problems has been correlated with the number of incidents of serious violent crimes; schools reporting more serious discipline problems were more likely to have one or more incidents of serious violent crimes (U.S. DoE, 1998). Eighty-four percent of public-school principals report that discipline problems were a minor to moderate problem in their school. Tardiness, absenteeism/class cutting, and physical conflicts are the most frequently reported problems (U.S. DoE, 1998). The occurrence of discipline and violence problems within our schools is affecting academic achievement. In a recent Policy Information Report, Barton et al. (1998) found frequency of offenses negatively related to achievement in mathematics, reading, science, and social science.
Risk and Protective Factors
Substance use, violence, and problem behaviors are all multifactorial problems that need to be addressed in a comprehensive manner. Research has shown that all three behaviors have correlates at the individual, family, school, and community level. Many of these correlates were summarized as follows with respect to substance use and violence:
... laws and norms favorable toward drug use;avail-ability of drugs;extreme economic deprivation; neighborhood disorganization; certain psychological characteristics; early and persistent behavior problems, including aggressive behaviors in males, other conduct problems, and hyperactivity in childhood and adolescence; a family history of alcoholism and parental use of illegal drugs; poor family management practices; family conduct; low bonding to fam-ily;academic failure;lack of commitment to school; early peer rejection;social influences to use drugs; alienation and rebelliousness; attitudes favorable to drug use;and early initiation of drug use (Hawkins etal., 1992).
Violence has also been examined at the individual, family, and societal level, and researchers have come to the conclusion that violence is a problem with multiple correlates, including individual, family, peer, community and societal influences (Andrews & Trawick-Smith, 1996).
Substance use, violence, and other problem behaviors are correlated (e.g., Furlong et al., 1997), and share several of the same predictors. Common predictors include poor school performance, risk-taking behaviors, peer association, normative beliefs, socioeconomic status, and living in a neighborhood with high rates of crime and poverty. Targeting youth substance use may affect youth violence (Slaby et al., 1994). Substance use and other problem behaviors are probably not directly causal, but instead co-occur (Derzon et al., 1999; Derzon & Lipsey, 1998). Problem behavior has also been associated with increased risk of school failure, involvement in the criminal justice system, and health problems (Dryfoos, 1990). Poor social skills, perceptions of low social competence, academic underachievement, negative attitudes toward school, and lack of parental guidance have also been identified as modifiable risk factors for problem behavior (Simons-Morton et al., 1999).
Multiple studies have documented the relationships between problem behaviors of many kinds and academic achievement (e.g., Barton et al., 1998; Bryant et al., 2000; Paulson et al., 1990). However, the
Effects of the Positive Action Program
73
direction of the relationship is unclear: Does poor academic achievement lead to increased disruptive behavior, violence, and/or substance use or vice versa? In one recent study using Monitoring the Future data, Bryant et al. (2000) found that, between grades 8 and 10, school misbehavior and poor performance predicted cigarette smoking, rather than the reverse. Whatever the answer to this question, there can be no doubt that a program that both improves academic achievement and reduces violence, substance use, and other problem behaviors can only be valued.
Studies show that self-concept is correlated negatively with several problem behaviors and academic performance (Coleman et al., 1966; Filozof et al., 1998; Paulson et al., 1990; Purkey & Novak, 1984; Symons et al., 1997). However, the causal ordering among them remains in question. For instance, early studies of self-concept and academic success suggested that self-concept directly affected academic success
(Purkey, 1970; Hansford & Hattie, 1982), whereas
later studies suggested a bidirectional or reverse relationship between the two (Bandura et al., 1996; Filozof et al., 1998; Hamachek, 1995; Hay et al., 1998; Hoge et al., 1995). Other studies have found that academic achievement affects self-concept (Skaalvik & Valas, 1999). The direction of the relationship between self-concept and behavioral problems is also inconclusive (Jang & Thornberry, 1998; McCarthy & Hoge, 1984; Rigby & Cox, 1996). Another debate within the literature concerns self-esteem and its correlation to substance use. Some recent studies have found that low self-esteem is significantly correlated with tobacco use (Crump et al., 1997; Jones & Heaven, 1998) and alcohol use (Loveland-Cherry et al., 1996), whereas others have found no correlation with substance use (Moore et al., 1996). Despite lack of clarity regarding causal directions in relationships between self-concept, behavior, and achievement, improving self-concept is an important goal in and of itself.
School Climate and Parental Involvement
School and home environment are associated with various factors affecting children's mental and physical well-being. A positive school environment both improves academic achievement (Bulach et al., 1995) and reduces the risk of substance use and delinquency (Battistich & Hom, 1997). Parental involvement is very important to a child's overall academic success.
Parental school involvement is also associated with academic achievement, student motivation to learn, improved student behavior in school, improved grades, test scores, and long-term achievement (Griffith, 1996; Shaver & Walls, 1998). Family SES does not appear to impact parent involvement with their child's schooling (Sui-Chu & Willms, 1996), but a school's physical features, organizational structure, and staff attitudes may do so (Dauber & Epstein, 1989; Griffith, 1998).
Prevention
Multiple reviews and commentaries during recent years indicate that prevention science is advancing our knowledge of what is efficacious in prevention of problem behaviors. Social influences programs had promise for the prevention of smoking in reviews as early as 1985 (Flay, 1985). Subsequent research and reviews have established this (Tobler, 1986, 1992).5 For example, Tobler and Stratton (1997) found in their meta-analysis of drug-prevention programs that interactive programs with 18+ program hours that included skills development and changes of normative beliefs (see also Hansen, 1992) are effective in decreasing student's substance use. Reviewers of the violence prevention literature (Derzon et al., 1999; Tolan & Guerra, 1994) have come to similar conclusions. Unfortunately, some programs having positive effects on violence and other antisocial behavior have reported negative effects on achievement (Derzon et al., 1999). Such a pattern of results suggests that we need to be very careful to establish the effects of interventions on both behavior and achievement.
There has also been a trend toward more comprehensive and multimodal programs that address multiple behaviors and that involve families and community, and these are generally more effective (Derzon et al., 1999). Several research groups have already reported comprehensive schoolwide programs that both reduce problem behaviors and enhance achievement (see Flay, 2000, for a partial review). In the earliest of such reports, Elias et al. (1991) found that a social-emotional learning program both reduced problem behaviors and enhanced achievement. In the smallest of these kinds of studies, Kellam and colleagues (Kellam et al., 1994; Kellam & Anthony,
5Unfortunately, subsequent studies found that these effects did not last through high school (Flay et al., 1989;Murray et al., 1989).
74
Flay, Allred, and Ordway
1998) found that two interventions designed to diminish aggressive and disruptive behavior and poor academic achievement among students in the first and second grades reduced the incidence of smoking initiation by boys through age 14.
In three small studies with mixed results, Gottfredson et al. (1998) evaluated three programs designed to improve teaching methods and to change the whole school experience for students. Project PATHE produced improvements in drug use but declines in achievement (Gottfredson, 1986). Project STATUS used innovative teaching methods, encouraged active student participation, and included field experiences, guest speakers, role play exercises and simulations, and independent and small group research projects to produce positive changes in both behavior and achievement in both middle school and high school students in a small quasi-experimental study (Gottfredson, 1990). The 5-year Multimodal School-Based Prevention Demonstration was designed to change the learning environment of the schoolbyinitiatingschoolwide changesinpoliciesand practices and by delivering a social competency curriculum. The main finding was that the program was not implemented as well as anticipated (Gottfredson et al., 1998) and therefore did not work.
The Child Development Project was designed to change the learning environment of the school by modifying teacher/classroom practices, changing classroom and whole school policies, and fostering connections between the school and home. In a quasi-experimental design (12 schools on both treatment and control conditions), Battistich et al. (1996, in press) found that alcohol and marijuana use was less among intervention than comparison students at a 2-year followup. Intervention students in the five program schools where the program implementation was high were less likely than comparison students to use alcohol or marijuana, carry a weapon, steal a car, skip school, or threaten another with harm. Poor implementation and negative effects in seven of the program schools (McGuire, 1998), raises questions about adoption of the program and, therefore, its replicabil-ity and long-term viability. This is of major concern given the reliance on the training-the-trainer model.
In a preliminary report, Cairns and Cairns (1999) reported improvements in violence, attendance, and achievement as a result of an intervention that enhanced existing competencies, emotional regulation, and school linkages. The Seattle Social Development Group have reported positive effects on both behavior and achievement as a result of their multifaceted
intervention that included training of teachers in classroom management, a classroom intervention in Grades 1-6, and parent training. The team has reported program effects on behavior, school bonding, and achievement for the complete sample (Hawkins et al., 1992) and for a high-risk subsample (O'Donnell et al., 1995). Six years after the end of the intervention, when the students were nearing the end of high school, Hawkins et al. (1999) found strong positive effects on substance use and other behaviors, including academic achievement. Unfortunately, a complex, and everchanging design makes it difficult to interpret the reported results with confidence.
Summary
Although the direction of causal relationships may not always be clear, there is no doubt that parent involvement with their child and school, and the self-concept, behavior, and academic achievement of the student are all strongly related. However, the evidence for the effects of changing one of these variables on changes in another is slim and contradictory.
A comprehensive, long-term, school-wide intervention that involves families, but is not too difficult to implement, is logical given the risk and protective factors for behavior and achievement and recent findings in prevention research. Schools that actively respond to problem behaviors, and cultivate a positive, healthy environment, have lasting effects on students' long-term behaviors in adolescence and beyond (St Leger, 1999). Providing an environment that is prochild, and that responds to a child's needs, will increase a child's behavior and academic performance (Simons-Morton et al., 1999). The set of studies reviewed here suggests that comprehensive programs that involve curriculum, teacher training, schoolwide climate change, and involvement of parents and community hold promise. However, no single study was of high enough quality to establish this definitively. One issue that arises with many prior programs is the ability of schools to implement them fully and continuously. A noteworthy quality of the Positive Action program evaluated here is that it was designed over many years to be easily trained and maintained.
With the exception of those reviewed earlier, most current programs address one particular subject such as reading (Slavin & Fashola, 1998) or a particular behavior such as drug abuse (Botvin, 1996) and do not focus on the child and the environment
Effects of the Positive Action Program
75
as a whole. Few programs focus on self-concept development. Few programs address the role of parents in school governance and student learning (Comer, 1988). Clearly, we need one program that addresses all of student self-concept, achievement, discipline, and related issues in a comprehensive and integrated way that also involves and meets the needs of teachers and parents. The Positive Action program was designed to do so.
THE POSITIVE ACTION PROGRAM
Theoretical Basis
The Positive Action (PA) program, first developed in 1977 by Carol Gerber Allred,6 and revised since then as a result of process and monitoring evaluations, is grounded in a broad theory of self-concept (Combs, 1962; Purkey, 1970; Purkey & Novak, 1984). This theory posits that people determine their self-concepts by what they do; that actions, more than thoughts or feelings, determine self-concept; and that making positive and healthy behavioral choices results in feelings of self-worth. The program teaches children what actions are positive, that they feel good when they do positive actions, and that they then have more positive thoughts and future actions. The recent development of "Positive Psychology" (Seligman, 1998), particularly recent results and theoretical developments reported by Fredrickson (2000), fully support this notion. Fredrickson (2000) reports that when children feel positive, they subsequently have more positive thoughts and engage in more positive behavior. By explicitly linking thoughts, feelings, and actions, the program is believed to enhance the development and integration of affective and cognitive brain functions (Damasio, 1994). The program is also consistent with educational theories of brain development (Caine & Caine, 1997), higher level thinking skills (Bloom, 1981, 1984), and multiple intelligences (Gardener, 1991; Goleman, 1995). It also trains teachers, other school staff, and parents to identify and reinforce positive feelings, thoughts, and actions by students, leading to continual reinforcement of positive actions and enhanced student bonding with parents and school, consistent
6Contact information: Dr. Carol Gerber Allred, President/ Developer, Positive Action, Inc., 264 4th Avenue South, Twin Falls, Idaho 83301;e-mail: info@positiveaction.net;Website: www.positiveaction.net.
with multiple social learning theories (Akers, 1977, 1998; Bandura, 1977a,b, 1986). PA is also consistent with other current approaches to social development, health promotion, and prevention of unhealthy behaviors (Battistich & Hom, 1997; Hawkins & Weis, 1985; Peters & McMahon, 1996), and a wide array of theories of behavior change integrated into the Theory of Triadic Influence (Flay & Petraitis, 1994; Petraitis etal., 1995).
The PA model is very different from that of most other programs, because it is comprehensive, integrated, and holistic. Current approaches to school ecology focus on parent involvement in school governance and reorganization, while not addressing the students' needs very effectively (Comer, 1988; Haynes et al., 1989,1997; Haynes & Comer, 1996). Recent approaches to improving academic achievement, even many of those classified as whole school reform, focus on enhancing particular curricular content and instruction methods (Bloom, 1981) or particular skills such as reading (Slavin & Fashola, 1998), but not many other needs of students. Recent approaches to prevention of mental-health problems, drug abuse, or violence rely on teaching of knowledge, correction of normative beliefs, and teaching of self-management and social skills (Peters & McMahon, 1996). Each of these approaches attempts to identify and correct particular risk or protective factors. Even so-called comprehensive or integrated approaches tend to address each of many proximal predictors of student behavior and performance separately.
Broad- and long-term effectiveness in reducing problem behaviors and increasing school performance will require addressing more distal (more causally remote) factors in a more comprehensive and integrated way. The PA program attempts this with a holistic approach to school reorganization, teacher-student relations, parent involvement, instructional practices, and development of the self-concept of all parties (students, teachers, parents, community members). With the PA model, students and adults are expected to gain not only the knowledge, attitudes, norms, and skills that they might gain from other programs, but also improved family bonding, peer selection and communication, and appreciation of school. That is, PA is designed to affect more distal (and ultimately more important) influences on behavior and performance than most other programs do. The theoretically expected result is improvement in a broad array of measurable student behaviors in the physical, intellectual, social and emotional arenas (e.g., decreased disruptive behaviors, other disciplinary
76 Flay, Allred, and Ordway
Table 1. Positive Action Goals for the Individual (Student, Teacher, Parent, Others), Family, School, and Community Individual goals
To give everyone the opportunity to learn and practice physical, intellectual, and emotional and social positive actions To understand that success and happiness means feeling good about who you are and what you are doing (being the best you can be) To develop good character, morals, and ethics Family goals
To create a positive learning environment in the home
To contribute to adult literacy and to develop life skills in adult family members To prepare children to be effective learners prior to entering school School goals To bring about comprehensive school reform
To develop lifelong skills that lead to success and happiness in school and society To create a positive environment conducive to teaching and learning To create a safe, drug-free school environment
To promote the personal and professional development of teachers, staff members, and administrators
To completely unite the efforts of the school, home, and community organizations in promoting the social, academic, and emotional growth of children
To teach the leadership skills that will promote high achievement and expert performance in the global marketplace Community goals
To involve the whole community in learning and practicing the positive actions necessary for a good self-concept and a successful life To contribute to a community environment
problems, substance use, violence and suspensions), and school performance (decreased absenteeism, improved academic achievement).
The Program Structure
PA was developed over 6 years (1977-83) of planned pilot work, formative evaluation, and revision and further evaluation, and was funded by the Office of Juvenile Justice and Delinquency Prevention (OJJDP) and the Centers for Disease Control and Prevention (CDC). This development and evaluation work took place in one school in Twin Falls, Idaho, and involved a comparable control school. The evaluation studies were conducted by external/independent evaluators (see later, e.g., Cottrell, 1980; Shaver, 1982; Stephenson, 1978, 1979). A subsequent study in additional schools was conducted by the program developer as part of her dissertation (Allred, 1984a). In each study, students answered questionnaires at the beginning and end of each academic year. Parents and teachers also answered questionnaires at various times. Measures included self-concept, life-adjustment skills, student achievement, law-enforcement bookings, parent involvement, and parent and teacher opinions of the program. As a result of the formative evaluation results, the PA program was altered, extended, and improved. Based on results and feedback from participating schools, the
developer has continually improved and added to the program.7
The PA program includes a detailed curriculum with almost daily lessons, a schoolwide climate program, and family- and community-involvement components, each of which uses research-proven educational strategies and methods such as active learning and positive classroom management. The program has goals at each of the individual, family, school, and community levels as shown in Table 1. The stated goals help align student, teacher, family, and community.
The K-6 classroom curriculum consists of over 140 lessons per grade. Using Teacher's Kits (that include the teacher's manuals and all materials needed for all activities for a whole class), classroom teachers present 15-20-min lessons almost every day (i.e., 35-45 hr each year). For each school, a Principal's Kit provides directions for a school-climate program to promote the practice and reinforcement of positive actions in the entire school. It also includes parent-and community-involvement activities.
In the classroom curriculum and all other materials, the content is taught through six units:
Unit 1. Self-Concept: What It Is, How It's Formed, and Why It's Important. The relationship of
7Positive Action Family Kit (Allred, 1995;see Gorsky, 1996), the Positive Action Counsellor's Kit (Allred, 1997, 1998b), and the Positive Action Community Kit (Allred, 1998a) were developed and added since the schools in this study adopted the program.
Effects of the Positive Action Program
77
thoughts, feelings, and actions (behavior). Units 26 teach children what actions are positive in various domains of life, that they feel good when they do positive actions, and that they then have more positive thoughts and future actions. Unit 2. Positive Actions for Body (Physical) and Mind (Intellectual). Physical: exercise, hygiene, nutrition, avoiding harmful substances, sleeping and resting enough, safety. Intellectual: creative thinking, learning/studying, decision making, problem solving.
Unit 3. Social/Emotional Positive Actions for Managing Yourself Responsibly. Manage human resources of time, energy, thoughts, actions, feelings (anger, fear, loneliness, others), talents, money, possessions. Includes self-control.
Unit 4. Social/Emotional Positive Actions for Getting Along with Others. Treat others the way you like to be treated, code of conduct (respect, fairness, kindness, honesty, courtesy, empathy, caring, responsible, reliable), conflict resolution, communicating positively (communication skills), forming relationships, working cooperatively, community service. [These are the essence of character education.]
Unit 5. Social/Emotional Positive Actions for Being Honest with Yourself & Others. Self-honesty, doing what you will say you will do (integrity), not blaming others, not making excuses, not rationalizing; self-appraisal (look at strengths and weaknesses); and being in touch with reality (mental health).
Unit 6. Social/Emotional Positive Actions for Improving Yourself Continually. Goal setting (physical, intellectual, and social/emotional), problem solving, decision making, believe in potential, have courage to try, turn problems into opportunities, persistence.
Unit 7. Review of all of above.
Scripted lessons are completely prepared and teacher-friendly, employing a variety of methodologies and addressing different learning styles. Activities include stories, role playing, modeling, games, music, questions/answers, activity booklets and sheets, posters, and manipulatives. The program content teaches students how to use positive actions, to recognize feeling good about themselves, to manage themselves (including thoughts, actions, and feelings), and to treat others the way they want to be treated.
The multiple components of the PA program are administered by the school principal (or designate)
who, using the Principal's Kit (Allred, 1987), is responsible for (1) initiating the adoption process, (2) appointing a PA Coordinator and a PA Committee, (3) coordinating training and professional development workshops and work groups, and (4) coordinating multiple resources. PA, Inc (PAI) staff trainers provide training and professional development opportunities for faculty, staff, parents, and community members. PAI staff train teachers in the PA method of instruction by actively role modeling the use of positive actions and use of the strategies listed above. Students and teachers are encouraged to set goals and to follow through with them. Teachers are trained to focus classroom management on encouragement and reinforcement of positive behavior, including the positive actions/behaviors that are being taught. PAI provides all materials for both preservice and inservice sessions. A PAI trainer leads further training and facilitates workshops and adoption plans.
The school-climate program reinforces the curriculum learning by coordinating the efforts of the entire school in practicing positive actions to promote improved behavior and performance. The school-climate activities are inclusive, varied, and comprehensive, serving all students including those learning English as a second language, low-income students, and students with disabilities. Positive Action schools are strongly encouraged to adapt and adopt the activities that best serve both individual populations of students and the student body as a whole. The PA program for school-climate change also supports the counselor's role in creating a positive school climate and ways for involving parents and other community members in school and program activities.
A logic/theoretical model of the expected effects of PA is shown in Fig. 1.8 Figure 1A shows the expected effects of the program components on school, family, and community climates, and interrelationships between people in the school, families, and the community. For example, the school-climate component should lead to measurable changes in the school climate, such as administration-staff relations (including teacher to teacher, teacher to staff, teacher to student), reinforcement of positive behavior, and parent-school relations; the family component (kit) should lead to measurable changes in family involvement with the school, and school-parent relations
8Prior evaluations, and the data reported herein, do not address the hypothesized causal paths, but focus only on program effects on self-concept, behavior, and achievement.
78
Flay, Allred, and Ordway
Fig. 1. Expected pathways of change due to the Positive Action (PA) program.
and parent-child relations; the teacher/staff training should contribute to improved teacher-student relations, and lead to changes in classroom management and instructional strategies, and teacher-student relations; the classroom curriculum should contribute to improved teacher-student, student-student, and student-parent relations, and lead to student engagement with community; and the community component should contribute to student involvement in the community and lead to greater community involvement in the school. All of these changes should contribute to an improved learning environment, and they should also lead to improved implementation of the program.
Figure 1B shows the student-centered immediate (mediational) and long-term effects of the program units—which are common to the Principal's kit (school-wide climate), the classroom curriculum, the family kit, and the community kit. The major immediate effects on students are improved self-concept, thinking skills, motivation to learn, study habits, self-management, involvement in all learning, time on task, prosocial behaviors, and fewer antisocial behaviors, including fewer conflicts. These changes, in turn, should lead to fewer problem behaviors (disciplinary behaviors, substance use, and violence) and improved school performance (attendance, grades, test scores).
The changes in school performance and behavior are thought to influence each other, and we hypothesize a larger effect from behavior to performance than the reverse, because better behavior might free up more class and personal time for learning.
Prior Evaluations of PA
Formative evaluations of the PA curriculum were conducted in a quasi-experimental evaluation at two elementary, suburban sites, an experimental school and a control school (Stephenson, 1978,1979). After 2 years of PA, students in the PA school scored 33% higher on self-concept than at pretest compared to a 23% improvement in the control school. On the Iowa Test of Basic Skills, scores for second- and fourth-grade students in the PA school improved in reading performance at more than 31/2 times the rate of their counterparts in the control school.9
9The treatment school had consistently ranked lower in academic achievement, had the most social and behavioral problems, and had more students qualifying for Chapter 1. Students in the control school were more middle class, and usually ranked in the top half of the district on achievement scores and in the bottom half of students qualifying for Chapter 1. These background differences make the reported results all the more credible.
Effects of the Positive Action Program
79
After 31/2 years of PA, the number of students booked by law-enforcement officials had decreased 94% (from 31 to 2 per year) in the PA school compared with a 14% increase (from 7 to 8 per year) in the control school (Cottrell, 1980). Parental awareness of the program was high, and parental willingness to volunteer assistance to the school or attend parental classes were exceptionally high (45% and 71% respectively).
The program developer then studied program acceptance and implementation in four elementary schools in a different school district (Allred, 1984a). Using a Solomon four-group design with one school per cell, she found that students in PA schools scored significantly higher on self-concept than did control school students (in both pretested and posttest-only schools) after 1 year of PA. School personnel perceived that student self-concept and behavior had improved after participation in PA. Students, teachers, parents, and principals all regarded the PA program as effective and valuable. Teachers implemented the program with a high degree of fidelity. Ease of use is believed to have contributed to subsequent adoption, replicability, and maintenance.
The changes in self-concept have been replicated in different schools, using different measures in three other school districts researched by the developer and doctoral students in subsequent years, using single-group, pretest-posttest designs. Allred (1984b) found significant increases in student self-concept in Royal School in Hawaii. Burcham (1992) found significant increases in self-concept on the Behavioral Academic Self-Esteem instrument (Coopersmith & Gilberts, 1981) for students in all grades (K-5) in a Maryland school. In a Georgia elementary school, Woodward (1996) reported the percentage of students scoring below average on the Academic Self-Concept Scale of the Multidimensional Self-Concept Scale (Bracken, 1992) dropping from 64 to 46% after 2 years of PA.
Recently, we have collected data available in school records or School Report Cards (SRC) (Flay & Ordway, 1999). Using before and after PA data, we documented strong improvements in achievement and decreases in problem behavior in a wide array of elementary schools. For example, percentile rankings on standardized tests improved from as low as the 30th percentile to as high as the 90th percentile over the course of only 1-3 years. Some schools improved from being the worst in their district to being the best. Admittedly, these are not the average results that might be expected in a more controlled
study. The study reported here was designed to provide such estimates.
METHODS
The PA program has been implemented in thousands of schools within hundreds of school districts around the United States and in several other countries. Some districts are now able to provide school-level archival data (SRC) on student performance and disciplinary referrals/actions and have a significant number of elementary schools that have implemented PA for a number of years. For this study, we chose two districts that had these data easily available (on the web) for 2 or 3 years10 and that had a significant number of schools that had implemented PA for 3 or more years. In a large Nevada school district 13 schools had implemented PA for 3 or more (up to 10) years before the 1995-96 school year. In Hawaii, eight schools had implemented PA for 4 or more (up to 10) years before the 1994-95 school year.
In each case, we used SRC data to find two matching control schools for the schools implementing PA. In order to find matched controls, we first rank-ordered all schools on percent free/reduced lunch, then mobility, and then selected schools with similar ethnic distributions. We have found poverty (percent free/reduced lunch) and mobility rates to be the strongest predictors of both average student performance and disruptive behavior. They account for 35-50% of the school-level variance in achievement, a little less of the variance in disruptive behaviors. Ethnic distribution is also often a significant predictor of lower achievement scores and higher violence.11 Tables 2 and 3 show the comparability of the program schools and their matched control schools compared with all non-PA schools in the district. In Nevada, program schools were similar to non-PA schools in terms of means, but not on distribution around the means (variance). Matched controls were not only similar on means but also on variance. In Hawaii, PA schools were substantially different from non-PA schools, having higher proportions of Japanese/Chinese students and lower proportions of White and Hawaiian students, higher rates of mobility, and lower
10 We desired 2 or more years of data to reduce error variance.
11 These matching variables are not expected to change as a result of PA. Therefore, they may be presumed to infer pretest matching on the outcome variables of interest (behavior, attendance, and achievement).
80
Flay, Allred, and Ordway
Table 2. Comparability of Nevada" PA' Schools, All Non-PA Schools' and Matched Control Schools—Means and
Standard Deviations (SD)
	PA schools (N = 12)		All non-PA schools (N = 87)		Matched controls (N = 24)	
	Mean	SD	Mean	SD	Mean	SD
School enrollment	690	181	704	225	728	160
Average class size	23.7	2.00	23.0	1.5	23.5	1.7
Mobility	38.1	9.3	38.9	13.1	39.9	10.0
% African American	18.0	22.6	15.4	17.0	18.0	24.5
% Hispanic	22.9	12.0	24.7	18.4	23.4	12.4
% White	53.6	22.9	53.9	24.3	53.2	23.6
% Free/reduced lunch	43.0	18.9	42.6	27.5	42.5	21.7
"From one large school district. 'PA = Positive Action.
' Of 132 schools, nine were deleted because of incomplete reporting.
proportions of students receiving free/reduced lunch. As expected, matched control schools were similar to PA schools.
SRC outcome data consist of standardized test scores and disciplinary reports. In Nevada, achievement scores are the average of the 1995-96 and 1996-97 district level Grade 4 percentile ranks on the Terranova Comprehensive Test of Basic Skills (5th Edition, Form A). Disciplinary data consisted of reports of incidents of student-to-student violence, student-to-staff violence, and possession of weapons, for the same 2 years, each of which we analyzed raw as well as by total incidents per 1,000 students. We also analyzed rates of absenteeism. In Hawaii, achievement data consist of the percent of students scoring above average on the Stanford Achievement Test for
three school years (1994-95, 1995-96, and 1996-97).
Hawaii reports disciplinary data in four categories: felonies, misdemeanors, department rules, and school rules. We also used a total count of incidents and incidents per 100 students across the 3 years as indicators. For Hawaii, we also analyzed number and rates ofsus-pensions, and absenteeism rates.
Preliminary analyses found no significant differences between the 2 years of data for Nevada or the 3 years of data for Hawaii, so we combined data across the years for all reported analyses.12 We conducted analyses of covariance, using the stratifying variables as covariates. For achievement data, we first used multivariate analyses to determine if there were effects overall, then univariate analyses. For disciplinary data, we conducted independent tests. In all cases, we tested for interactions of condition (program or not) with the covariates.
RESULTS
Nevada
Table 4 shows results from Nevada. For achievement, the three variables used for matching schools— percent free/reduced lunch, student mobility, and percent African American students—were all significant
12When data from each academic year were analyzed separately, results followed the same pattern as reported here.
Table 3. Comparability of Hawaii PA Schools, All Non-PA Schools and Matched Control Schools—Means and Standard
Deviations (SD)
PA schools (N = 8)    All non-PA schools (N = 117)    Matched controls (N = 16)
	Mean	SD	Mean	SD	Mean	SD
School enrollment	557	136	607	259	560	238
Average # students/teacher	18.9	1.8	17.5	2.0	17.9	2.0
Instability (%)	5.5	2.62	8.6	5.16	6.3	2.00
% Hawaiian	17.3	11.3	27.7	21.6	9.0	8.4
% Japanese/Chinese	37.6	23.1	14.4	14.4	27.9	16.2
% White	12.8	4.1	19.3	16.4	12.5	7.6
% Free/reduced lunch	28.2	22.5	46.7	22.7	31.6	19.8
Effects of the Positive Action Program
81
Table 4. Effects of PA on Achievement and Violence Scores (1996-97) in Nevada Schools (Means, Standard Deviations, Statistical Tests,
and Percent Improvement From Controls)
PA schools          All non-PA schools            Matched controls _ _ _ % Improvement
	Mean	SD	Mean	SD	p	Mean	SD	P	from controls
Achievement: Grade 4 Percentile ranks on Terranova CTBS"'									
Math	55.7	11.50	51.6	14.12	.039	46.2	11.27	.000	21
Reading	49.5	9.60	46.4	12.71	.025	43.8	11.04	.001	13
Language	56.5	9.04	53.6	14.82	.075	49.0	12.59	.002	15
Science	44.1	11.85	43.2	12.97	.280	39.1	13.80	.067	13
Combined math/reading/language	53.9	9.54	50.6	13.53	.028	46.4	11.30	.000	16
Violence: (Number of incidents)'									
Student to student	0.25	0.62	0.76	2.54	.048	1.96	4.75	.048	87
Student to staff	0.00	0.00	0.12	0.32	.000	0.17	0.38	.022	100
Possession of weapons	0.08	0.29	0.15	0.45	.233	0.29	0.46	.055	72
Total per school	0.33	0.65	1.03	2.70	.017	2.40	4.98	.028	86
Per 1000 students	0.44	0.89	1.42	2.98	.007	2.98	5.14	.013	85
Absenteeism	6.82	1.06	6.72	1.65	>.05	7.13	1.33	> .05	4.5
aFor all non-PA schools 1-tailed significance in MANOVA is used in which significant covariates are % free/reduced lunch (F4i 115 = 16.43, p < .0001), student mobility (F4115 = 4.51, p = .002),and% African American students (F4115 = 3.09, p = .019), and condition is not significant (p >.05). Between subject F tests (shown in table) are significant for condition for math and reading, and marginally significant for language (and significant for a combined measure).
bFor all matched controls 1-tailed significance in MANOVA is used in which % free/reduced lunch is a significant covariate (F4 30 = 37.78, p < .0001), and condition is significant at p <. 01 (F430 = 5.21);p level for between subjects F s for condition shown in table. cFor all non-PA schools and matched controls 1-tailed significance in independent tests is used.
in MANOVAs for comparisons with both all non-PA schools and with matched controls. For comparisons with all non-PA schools, condition was not significant ( p > .05). However, univariate tests were significant for math and reading, and marginally significant for language (and significant for a combined measure of math, reading, and language). For the matched control comparisons, only one covariate (percent free/reduced lunch) was significant, and condition was also highly significant; in univariate tests program effects were significant for math, reading, language, and the combined measure (math, reading, language). The program improved test scores in PA schools by an average of 16% compared to matched controls.
For violence data in Nevada, significant program effects were observed in both the comparisons with all schools and the comparisons with matched controls for student-to-student and student-to-staff violence and for both summary measures (total number of incidents and incidents per 1,000 students). Marginally significant effects were observed for possession of weapons in only the matched control comparison. The PA program reduced incidents of violence by 85% on average (72-100%). There were no significant results regarding rates of absenteeism.
Table 5 shows results from Hawaii. For achievement, three covariates were significant predictors in the multivariate ANOVA comparing PA schools
with all other schools—parent education, percent free/reduced lunch, and percent Japanese/Chinese. Condition was also significant, and univariate tests were significant for all three indicators (math, reading, and a combined score). When comparing PA schools with matched controls, the results were parallel (with the exception of parent education). Viewing the data in Table 5 shows that mean differences were smaller with the matched controls, but at the same time more significant. On average, the PA program improved achievement scores by 52%.
For disciplinary data in Hawaii, all indicators were significantly different when PA schools were compared with all non-PA schools, and all but misdemeanors were significant when compared with matching controls. On average, the PA program reduced disciplinary referrals by 77% (from 51% for misdemeanors to 100% for school rules). For total incidents/rates of disciplinary actions and suspensions, the effects of PA interacted with percent free/reduced lunch, indicating stronger program effects in schools with higher proportions of students receiving free/reduced lunches. This effect is illustrated in Fig. 2. Within schools without the program, incidents/rates are much higher in schools with higher proportions of students receiving free/reduced lunch; the PA program reduces these otherwise much higher rates down to rates equal to rates in schools with low proportions of students receiving free/reduced lunch.
82
Flay, Allred, and Ordway
Table 5. Effects of PA on Achievement and Violence Scores (1995-97) in Hawaii Schools (Means, Standard Deviations, Statistical Tests and
Percent Improvement From Controls)
PA schools        All non-PA schools         Matched controls        „, T _ _ _ % Improvement
	Mean	SD	Mean	SD	p	Mean	SD	P	from cont
Achievement: (% A, SAT 3-year average)a,b									
Math	41.3	18.89	23.7	13.50	.028	27.4	14.53	.000	51
Reading	33.6	17.45	19.5	11.28	.021	22.1	11.80	.002	52
Combined math/reading	37.5	18.00	21.6	12.12	.016	24.7	13.01	.000	52
Behavior: (Number of incidents)c									
Felonies	0.13	0.35	2.06	6.49	.001	1.13	2.28	.052	88
Misdemeanors	1.13	2.03	6.44	12.12	.000	2.31	5.46	.225	51
Department rules	0.13	0.35	1.30	2.25	.000	0.94	1.73	.045	86
School rules	0.00	0.00	0.77	2.49	.001	1.38	3.10	.048	100
Total incidents	1.38	2.39	10.57	17.65	.000	5.75	8.51	.035d	76
Total N per 100 students	0.22	0.32	1.96	5.02	.000	0.98	1.27	.018d	78
N suspensions	1.00	1.41	7.51	10.83	.000	4.81	6.40	.018d	79
N Suspensions/100 students	0.17	0.21	1.29	2.19	.000	0.84	1.01	.010d	80
Absenteeism (av # days)	7.73	2.17	9.66	2.62	.021	8.36	1.69	.05	7.5
Absenteeism rate (%)	4.40	1.22	5.81	3.24	.008	4.77	0.96	.05	7.8
aFor all non-PA schools 1-tailed significance in MANOVA is used in which significant covariates are parent educations (F2117 = 16.08, p < .0001), % free/reduced lunch (F2117 = 13.26, p <.0001) and % Japanese/Chinese (F2117 = 12.95, p <.0001), and condition is marginally significant (F2 17 = 2.37, p = .098). Between subjects F tests (shown in table) are significant for condition for all three indicators of achievement.
bFor all matched controls 1-tailed significance in MANOVA is used in which significant covariates are % free/reduced lunch (F2> 19 = 11.61, p < .001) and % Japanese/Chinese (F2 19 = 6.13, p <.01), and condition is significant at p = .001 (F219 = 11.06);p level for between subjects Fs for condition shown in table.
cFor all non-PA schools and matched controls 1-tailed significance independent tests is used.
dIn ANOVAs, % free/reduced lunch also interacts with PA (p <.05), indicating stronger program effects in schools with greater poverty (see Fig. 1).
Fig. 2. Interaction effects of the Positive Action (PA) program and poverty on disciplinary actions (felonies, misdemeanors, and rule-breaking).
Effects of the Positive Action Program
83
The program also reduced suspensions (significant in both comparisons) by 80%. Absenteeism appeared to be reduced compared with non-PA schools, but was not statistically significant in comparison with matched controls.
DISCUSSION
Findings
Data from evaluative research during program development suggest that the PA program is very effective at increasing self-concept, reducing problem behaviors, and improving school performance. School-level data from this matched-control study replicated the earlier results regarding achievement and discipline. The PA program appears to be very effective at both improving school performance and reducing behavioral incidents requiring disciplinary referral or suspensions. Also, program effects seem to be obtained in a wide variety of schools. Positive results were obtained by regular classroom teachers with no prior association with the program developers, in schools of various socioeconomic levels (including Title 1 schools) and ethnic groupings. It is also noteworthy that the results were replicated in two diverse school districts in widely different parts of the country.
Limitations
One limitation of this study concerns the lack of data about program implementation in study schools. Early studies found high degrees of acceptance and high integrity of implementation of the curriculum. Despite the lack of implementation data, the program effects reported in this paper provide estimates of effectiveness under real-world conditions. Many evaluations reported in the scholarly literature report efficacy results under conditions controlled by researchers or when the program is implemented by researcher-controlled staff (see Flay, 1986, for a discussion of efficacy and effectiveness). Effects as large as those reported here are not common for programs implemented and tested under such real-world conditions.
Another limitation is that the reported data concern only the elementary-level curriculum and school-wide climate components of the program. Thus, no data have been reported to date on the effectiveness of the parent or community components, nor
on the middle-school or high-school programs. Many schools are now using these components, so evaluation is overdue.
A third limitation concerns the use of archival data on risk factors for matching schools. Risk factors such as poverty, mobility, and ethnic distribution are imperfect predictors of student behavior and achievement; and, thus, imperfect for matching schools. However, any changes over time in these variables should not be in response to the PA program; so they should be reasonable proxies for pretest comparability on outcome variables. Nevertheless, prior (pretest) behavior and achievement data would have been preferable, both for matching at pretest and for assessing change over time. Unfortunately, prior data were not available.
Reliance on school-level indicators of outcomes is another potential limitation of this study. For example, no school in this study had any data on substance use. Individual-level data on achievement would allow assessments of individual improvements as a result of participation in the program. Individual-level data on behavior would allow assessment of program effects on prevention of problem behaviors, such as substance use, comparable with results from other prevention studies. A further limitation with the discipline data is likely inconsistencies in reporting across time as leadership, environments, policies, and demographics change within schools.
It must be noted, however, that the school-level data reported here are very strong. The achievement data indicate that the PA program can change the performance level of the majority of students within a school to raise its relative ranking by 16-52%. The disciplinary referral data indicate that the PA program can reduce problem behavior in a school by an average of over 80%. These estimates are also consistent with previously reported results. These are very large effects that ecological validity concerns are unlikely to explain away.
This study was also limited by the small number of school districts with readily available school-level data on student achievement and disciplinary actions. This means that the set of schools studied could be biased, and less (or more) impressive program effects may be obtained in a larger sample of schools. However, anecdotal evidence from many more schools that have adopted and continue to use the PA program suggests otherwise—many school districts report adopting the program in low scoring schools and having them improve to be among the best performing schools. Nevertheless, we shall not know for sure the
84
Flay, Allred, and Ordway
average impact of the program until further studies with larger samples of schools are conducted. The increasing availability of SRC data and other historical archival data will be useful in assessing the robustness of the effects of the program over longer periods of time and across many more schools. From a methodological perspective, the matched case-control design with school-level archival data provides an approach that could be used more often to evaluate popular programs for school reform, Safe and Drug Free Schools, Character Education, and other programs addressing social development.
A final limitation of this study (and all prior studies of this program) concerns the age of the students (elementary school). Most studies of prevention programs have involved middle-school or high-school students. This makes sense, in that the onset of most problem behaviors occurs during middle or high school. Thus, future studies will need to follow elementary students exposed to the PA program into middle and high school and track long-term results.
Discussion
The PA program was designed to address central determinants of both problem behavior and school performance. Self-concept was hypothesized to be central to both as well as to student development, family interaction, school climate, and community involvement. Educational and prevention researchers have had difficulty with positing a central role for self-concept in recent years. It has proven difficult to change and, when changed, its relationship to subsequent behavior and school performance has been elusive. The results of this and prior work suggest that the PA program can change all three, though it still does not address the causal ordering among them.
There has been much debate about the causal relationships among self-concept, problem behavior, and school performance (Coleman et al., 1966; Myers et al., 1987; Rutter et al., 1979). PA was developed on the premise that encouraging students to engage in positive behavior (actions) with positive consequences for themselves and others would lead them to feel better about themselves, develop improved self-concepts and, in turn, engage in further positive actions. Thus, PA students who engage in positive rather than problem behaviors are self-reinforced for that behavior, they feel good about it, and they are likely to engage in it again in the future. Positive behaviors may be health related, leading to feeling good physically; educational or school-related, leading to feeling good
mentally; or social, leading to feeling good emotionally. Any one of these kinds of positive behaviors can also lead to others feeling good about the person, thus providing social reinforcement.
Recent years have seen multiple attempts to identify and produce lists of programs of proven effectiveness (e.g., Center for the Study and Prevention of Violence, 1998; Center for Substance Abuse Prevention [CSAP], 1997,2000; Character Education Partnership, 2000; CSAP, 2001; Drug Strategies, 1996, 1998; Education Commission of the States, 1999; Eron et al., 1994; Mrazek & Haggerty, 1994; Price et al., 1988; Reiss & Roth, 1993; Safe Disciplined and Drug Free Schools, 2001; Slavin & Fashola, 1998; Sloboda & David, 1997; Title 1,1999; U.S. Department of Health and Human Services. [USDHHS], 1994). Given other less encouraging research findings, it is rather noteworthy that this program has effects across multiple domains. Schools, school districts, and state and national educational bodies have all been frustrated by the lack of programs of proven effectiveness when it comes to the prevention of substance use, violence, or school drop-out, or the improvement of school performance. The PA program is one program that may have the potential to address all of these problems at once.13
The PA model was designed to insure the following: (1) the inclusion and strengthening of all aspects of the school; (2) provision for program component interaction with opportunities to learn, practice, and reinforce PA concepts throughout the program; (3) a logical and sequential rationale for component development and flow; (4) reduced fragmentation and distraction; (5) cost-effectiveness and resource sensitivity; (6) efficient service for the multiple needs and special-need students (low income, limited-English proficiency, disabilities); (7) increased academic achievement by supporting teachers in meeting state content and performance standards; (8) higher behavioral standards for students as determined by
13The Positive Action program is included on several of these lists. For example: the Northwest Regional Educational Laboratory's Catalog of School Reform Models (NWREL, 1999) sponsored by the U.S. Department of Education;the Education Commission of the States list of programs for Comprehensive School Reform
(1999) ;the Centerfor Substance Abuse Prevention National Registry of Effective Prevention Programs Substance Abuse Matrix
(2000) ;the Department of Education's list of promising programs for Safe, Disciplined, and Drug Free Schools (2001);the Character Education Partnership resource guide (2000); and multiple State Education Department and School District lists of approved programs.
Effects of the Positive Action Program
85
school and community expectations derived from needs assessments; and (9) a common purpose to improve society by teaching principles that lead individuals to being the best they can be.
The PA program also has the characteristics of a quality program as identified by the National Education Association (1998) survey of "high-achieving" school (schools in which students perform well on standardized tests and which teachers perceive as having all the essential features of a learning organization). Of special relevance are the following: (a) provides review, reinforcement, and extension (generalization) necessary to produce meaningful change in achievement-related behavior; (b) works with key cultural environment or social contexts (family and school) that strengthen individual behavior and enhance classroom effects; (c) is developmentally appropriate with programs for grades K-6 that target the biological, cognitive, and social relational issues of this developmental period; and (d) provides teacher inservices that give a basic understanding of learners' need for a comprehensive approach—procedures are given for coordinated school services for internal and/or external referrals; (e) increases protective factors for students by increasing their involvement with communal institutions of family, school, and home.
The PA program is one of very few programs to date to report strong effects on both achievement and multiple problem behaviors, from many diverse types of schools; effects that are both statistically and practically significant. How can the PA program be so effective? We believe that the developer's broad experiences—teaching in both high and elementary schools, in school administration, and in completing a PhD while evaluating the program—all provided rich insights for the development of PA. Most developers, including researchers, do not have all of these experiences. PA seems to incorporate the best that is known from both practice and research. The program has strong effects in multiple behavioral domains and on achievement for the following reasons derived from current research and theory. It is delivered to every class in every grade at the same time by every teacher. PA trains teachers, other staff, students, and parents how to reinforce positive behaviors all day, every day. It engages and involves parents and community partners, who then contribute to and expand the reinforcing environment. The program focuses on social and other asset development and general positive behaviors regarding self and others, rather than negative behaviors as so many other programs do. It also includes the social and personal
cognitive and behavioral skills development and the correction of norms found effective in research-based programs in substance use and violence prevention. Of critical importance, program activities link all behaviors/actions to feelings and thoughts, and emphasizes universal values and principles, thus supporting diversity. It targets distal influences, proximal influences, and actual behaviors all at once, and in a way that is self-reinforcing at a system level as well as the individual level. It is also consistent with theories from education and sociology as well as psychology. Finally, the design and format of the program materials and delivery schedule make it easy for schools and teachers to adopt and continue to implement.
If we are serious about ensuring the success of every child, both academically and socially, school-wide reform will be necessary. This will require something more central than a focus on curriculum, instructional methods, family support, assessment, or remediation would imply, and even more basic than a focus on systemic change would imply. It also requires a substantial change in the attitudes, values, and self-concept of teachers, students, parents, and community members, and increased positive interactions among them. The PA program appears to provide an approach to these ends, focusing on the development of character, positive values, and behaviors among students, including those related to studying. Parallel changes by teachers, principals, parents, and community members are required to provide the environment to support such changes. Changes at many levels (both societal levels and theoretical levels) reinforce and promote changes at other levels, and the educational and social system becomes one of reciprocal feedback and self-actualization that leads to success and happiness (being the best you can be) for all individuals, families, schools, and communities. We note, however, that no study of PA to date has measured all of the above expected effects (or all those shown in Fig. 1) to determine that (a) they occur or (b) they mediate program effects on behavior, achievement, or both. This remains for future research.
Perhaps the ideal school-wide reform might include the PA program, specific systemic change (a la Comer, 1988) and specific curriculum, instructional methods, assessment, and remediation (cf. Bloom, 1981). However, empirical results and anecdotal evidence suggest that the PA program achieves most of the desired effects by itself—by leading to changes in school organization/structure, staff behavior and instructional skills, and curriculum content (including incorporation of PA principles in competency areas).
86
Flay, Allred, and Ordway
As students, faculty, parents, administrators, support staff members, and community members begin to learn, practice, and reinforce positive actions, sustained social changes occur—sometimes rapidly, sometimes over the course of several years. These changes are positive and include decreased disciplinary problems and violence, and increased academic achievement. These positive social changes define enhanced school functioning and equate with positive school reform.
REFERENCES
Akers, R. L. (1977). Deviant behavior: A social learning approach
(2nd ed.). Belmont: Wadsworth. Akers, R. L. (1998). Social learning and social structure: A general
theory of crime and deviance. Boston: Northeastern University
Press.
Allred, C. G. (1984a). The development and evaluation of Positive Action: A systematic elementary school self-concept enhancement curriculum, 1977-1983. Unpublished doctoral dissertation, Brigham Young University, Provo, UT.
Allred, C. G. (1984b). The Positive Action program: An evaluation. Honolulu, HI: Royal School, Honolulu School District.
Allred, C. G. (1987). Positive Action Principal's Kit. Twin Falls, ID: The Positive Action Company.
Allred, C. G. (1995). Positive Actions for Living. Twin Falls, ID: The Positive Action Company.
Allred, C. G. (1997). Positive Action Counselor's Kit. Twin Falls, ID: The Positive Action Company.
Allred, C. G. (1998a). Positive Action Community Kit. Twin Falls, ID: The Positive Action Company.
Allred, C. G. (1998b). Using Positive Action for the prevention, intervention and treatment of youth at risk. In R. Warkentin & D. Rea (Eds.), Youth-adult partnerships: Unity in diversity (pp. 35-40). New York: McGraw-Hill.
Andrews, L., & Trawick-Smith, J. (1996). An ecological model for early childhood violence prevention. In R. L. Hampton, P. Jenkins, & T. P. Gullotta (Eds), Preventing violence in America: Issues in children's and families' lives (Vol. 4, pp. 233261). Thousand Oaks, CA: Sage.
Bandura, A. (1977a). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A. (1977b). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.
Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (1996). Multifaceted impact of self-efficacy beliefs on academic functioning. Child Development, 67, 1206-1222.
Barton, P. E., Coley, R. J., & Wenglinsky, H. (1998). Order in the classroom: Violence, discipline and student achievement. Princeton, NJ: Policy Information Center, Educational Testing Service.
Battistich, V., & Hom, A. (1997). The relationship between student's sense of their school as a community and their involvement in problem behavior. American Journal of Public Health,
87(12), 1997-2001. Battistich, V., Schaps, E., Watson, M., & Solomon, D. (1996).
Prevention effects of the Child Development Project: Early findings from an ongoing multisite demonstration trial. Journal of Adolescent Research, 11(1), 12-35. Battistich, V., Schaps, E., Watson, M., Solomon, D., & Lewis, C. (2000). Effects of the Child Development Project on students'
drug use and other problem behaviors. Journal of Primary Prevention, 21(1), 75-99. Bloom, B. S. (1981). All our children learning. New York: McGraw
Hill.
Bloom, B. S. (1984). Taxonomy of educational objectives, book 1. New York: Longman Publishing.
Botvin, G. (1996). Substance abuse prevention through Life Skills Training. In R. D. Peters & R. J. McMahon (Eds.), Preventing childhood disorders, substance abuse, and delinquency. (pp. 215-240). Thousand Oaks, CA: Sage. Banff International Science Series, Vol. 3.
Bracken, B. A. (1992). Multidimensional self-concept scale examiner's manual. Austin, TX: Pro-Ed.
Bryant, A., Schulenberg, J., Bachman, J. G., O'Malley, P. M., & Johnston, L. D. (2000). Understanding the links among school misbehavior, academic achievement, and cigarette use: A national panel study of adolescents. Prevention Science, 1(2), 7187.
Bulach, C., Malone, B., & Castleman, C. (1995). Investigation of
variables related to student achievement. Mid-Western Educational Researcher, 8(2), 23-29.
Burcham, S. (1992). Improving the academic self-esteem ofelemen-tary students (p. 119). Unpublished doctoral dissertation, Nova University, Fort Lauderdale, FL.
Caine, R. N., & Caine, G. (1997). Education on the edge ofpossibil-
ity. Alexandria, VA: ASCD.
Cairns, B., & Cairns, R. (1999). Paper presented at the Life History Society meetings, Kauai, HI, September 1999.
Center for Substance Abuse Prevention. (1997). Reducing tobacco use among youth: Community-based approaches Prevention Enhancement Protocols Systems. DHHS Publication
No. SMA 97-3146). Washington, DC: USDHHS, SAMHA, CSAP.
Center for Substance Abuse Prevention. (2000). National registry on effective prevention programs: The substance
abuse data matrix. USDHHS, SAMHA, CSAP, Washington,
DC, September 2000. Available at: http://www.samhsa.gov/ CSAP/modelprograms. Center for the Study and Prevention of Violence. (1998). Blueprints. Boulder, CO: University of Colorado, Institute for Behavioral Sciences.
Centers for Disease Control and Prevention. (1998). CDC surveillance summaries, August 14, 1998. MMW 47 (no. SS-3). Chandler, K. A., Chapman, C. D., Rand, M. R., & Taylor, B. M.
(1998). Student's report on school crime: 1989 and 1995 (NCES
98-241/NCJ-169607). Washington, DC: U.S. Departments of
Education and Justice. Character Education Partnership. (2000). Character education resource guide. Washington, DC: Character Education Partnership.
Coleman, J., Campbell, E., Hobson, C., McPartland, J., Mood, A., Weinfeld, F. D., & York, R. (1966). Equality of educational opportunity. Washington, DC: U.S. Government Printing Office.
Combs, A. (1962). Perceiving, behaving, becoming: A new focus for educators. ASCD 1962 Yearbook. Washington, DC: Association for Supervision and Curriculum Development.
Comer, J. P. (1988). Educating poor minority children. Scientific
American, 259(5), 42-48.
Coopersmith, S., & Gilberts, R. (1981). Behavioral academic self-esteem: A rating scale. Palo Alto, CA: Consulting Psychologists Press.
Cottrell, M. C. (1980). The Positive Action program: An evaluation. Twin Falls, ID: Twin Falls School District.
Crump, R., Lillie-Blanton, M., and Anthony, J. (1997). The influence of self-esteem on smoking among African-American school children. Journal ofDrug Education, 27(3), 277-291.
CSAP (2001).  Center for Substance Abuse Prevention,
Washington. DC, CSAP's Prevention Portal: Model Programs, http://www.samhsa.gov/centers/csap/modelprograms/.
Effects of the Positive Action Program
87
Damasio, A. R. (1994). Descartes' error: Emotion, reason, and the human brain. New York, NY: Avon Books.
Dauber, S. L., & Epstein, J. L. (1989). Parent's attitudes and practices of involvement in inner-city elementary and middle schools. Report No. 33, The Johns Hopkins University, Center for Research on Elementary and Middle Schools, Baltimore, MD.
Derzon, J. H., & Lipsey, M. W. (1998). A synthesis of the relationship of marijuana use with delinquent and problem behaviors. School Psychology International, 20(1), 57-68.
Derzon, J. H., Wilson, S. J., & Cunningham, C. A. (1999). The
effectiveness of school-based interventions for preventing
and reducing violence. Center for Evaluation Research and
Methodology, Vanderbilt University. Donahue, P. L., Voelkl, K. E., Campbell, J. R., & Mazzeo, J. (1999).
NAEP 1998 reading report card for the nation and the states
[on-line]. Available at: http://nces.ed.gov/nationsreportcard/
pubs/main1998/1999500.html. Drug Strategies. (1996). Making the grade: A guide to school
drug prevention programs (pp. 21-24). Washington, DC: Drug
Strategies.
Drug Strategies. (1998). Safe schools, safe students: A guide to violence prevention strategies (pp. 34, 37, 39). Washington, DC: Drug Strategies.
Dryfoos, J. D. (1990). Adolescents at risk: Prevalence and prevention. New York: Oxford University Press.
Education Commission of the States. (1999). Promising practices for comprehensive school reform. Education Commission of the States, Denver, CO, October 1999. Available at: http://www.ecs.org/ecs/ecsweb.nsf
Elias, M. J., Gara, M., Schuyler, T., Branden-Muller, L., & Sayette, M. (1991). The promotion of social competence: Longitudinal study of a preventive school-based program. American Journal of Orthopsychiatry, 61, 409-417.
Eron, L. D., Gentry, J. H., & Schlegel, P. (1994). Reason to hope: A psychosocial perspective on violence and youth. Washington, DC: American Psychological Association.
Filozof, E., Albertin, H., Jones, C., Sterne, S., Myers, L., & McDermott, R. (1998). Relationship of adolescent self-esteem to selected academic variables. Journal of School Health, 68(2),
68-72.
Flay, B. R. (1985). Psychosocial approaches to smoking prevention:
A review of findings. Health Psychology, 4(5), 449—488. Flay, B. R. (1986). Efficacy and effectiveness trials (and other phases
of research) in the development of health promotion programs.
Preventive Medicine, 15, 451-474. Flay, B. R. (2000). Approaches to substance use prevention utilizing
school curriculum plus social environment change. Addictive
Behaviors, 25(6): 861-885.
Flay, B. R., Koepke, D., Thomson, S. J., Santi, S., Best, J. A., &
Brown, K. S. (1989). Six-year follow-up of the first Waterloo School Smoking Prevention Trial. American Journal ofPublic Health, 79(10), 1371-1376.
Flay, B. R., & Ordway, N. (1999). Changes in school-level percentile rankings as a measure of effectiveness. Unpublished report, University of Illinois at Chicago.
Flay, B. R., & Petraitis, J. (1994). The theory of triadic influence: A new theory of health behavior with implications for preventive interventions. In G. S. Albrecht (Ed.), Advances in medical sociology, Vol. IV: A Reconsideration of models ofhealth behavior change (pp. 19-44). Greenwich, CN: JAI Press.
Fredrickson, B. L. (2000). Cultivating positive emotions to optimize health and well-being. Prevention and Treatment [on-line] 3, article 0001a, March 7, 2000. Available at: http://journals. apa.org/prevention/volume3/pre0030001a.html.
Furlong, M., Casas, J., Corral, C., Chung, A. & Bates, M. (1997).
Drugs and school violence. Education and Treatment of Children, 20(3), 263-280.
Gardener, H. (1991). The unschooled mind: How children think and how schools should teach. New York: Basic Books.
Goleman, D. (1995). Emotional intelligence. New York: Bantam Books.
Gorsky, R. A. (1996). Resource reviews: Positive Actions For Living. American Journal of Health Promotion, 11(1), 15.
Gottfredson, D. C. (1986). An empirical test of school-based environmental & individual interventions to reduce the risk of delinquent behavior. Criminology, 24(4), 705-731.
Gottfredson, D. C. (1990). Changing school structures to benefit high-risk youth. In P. E. Leone (Ed.), Understanding troubled and troubling youth: Multidisciplinary perspectives. Newbury
Park, CA: Sage.
Gottfredson, D. C., Gottfredson, G. D., & Skroban, S. (1998). Can
prevention work where it is needed most? Evaluation Review,
22(3), 315-340.
Griffith, J. (1996). Relation of parental environment, empowerment, and school traits to student academic performance. Journal of Educational Research, 90(1), 33—41.
Griffith, J. (1998). The relation of school structure and social environment to parent involvement in elementary schools. The Elementary School Journal, 90(1), 53-80.
Hamachek, D. (1995). Self-concept and school achievement: Interaction dynamics and a tool for assessing the self-concept component. Journal of Counseling and Development, 73(4),
419-425.
Hansen, W. B. (1992). School-based substance abuse prevention: A review of the state of the art in curriculum, 1980-1990. Health Education Research, 7(3), 403-430.
Hansford, B. C., & Hattie, J. A. (1982). The relationship between
self and achievement/performance measures. Review of Educational Research, 52(1), 123-142. Hawkins, J. D., Catalano, R. F., Kosterman, R., Abbott, R., & Hill, K. G. (1999). Preventing adolescent health-risk behaviors by strengthening protection during childhood. Archives ofPedi-atrics and Adolescent Medicine, 153, 226-234.
Hawkins, J. D., Catalano, R. F., & Miller, J. Y. (1992). Risk and
protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin, 112, 64-105. Hawkins, J. D., Catalano, R. F., Morrison, D. M., O'Donnell, J.,
Abbott, R. D., & Day, L. E. (1992). The Seattle Social Development Project: Effects of the first four years on protective factors and problem behaviors. In: J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: Interventions from birth through adolescence (pp. 139-161). New York: Guilford Press.
Hawkins, J. D., & Weis, J. G. (1985). The social development model: An integrated approach to delinquency prevention. Journal of Primary Prevention, 6, 73-97.
Hay, I., Ashman, A. F., & Van Kraayenoord, C. E. (1998). Educational characteristics of students with high or low self-concept.
Psychology in the Schools, 35(4), 391-400.
Haynes, N. M., & Comer, J. P. (1996). Integrating schools, families, and communities through successful school reform: The School Development Program. School Psychology Review,
25(4), 501-506.
Haynes, N. M., Comer, J. P., & Hamilton-Lee, M. (1989). School
climate enhancement through parental involvement. Journal
ofSchool Psychology, 27, 87-90.
Haynes, N. M., Emmons, C., & Ben-Avie, M. (1997). School climate as a factor in student adjustment and achievement. Journal of Educational and Psychological Consultation, 3, 321-329.
Hoge, D., Smit, E., & Crist, J. (1995). Reciprocal effects of self-
concept and academic achievement in sixth and seventh grade. Journal of Youth and Adolescence, 24(3), 295-314. Jang, S. J., & Thornberry, T. P. (1998). Self-esteem, delinquent peers, and delinquency: A test of the self-enhancement thesis. American Sociological Review, 63, 586-598.
88
Flay, Allred, and Ordway
Johnston, L. D., O'Malley, P. M., & Bachman, J. G. (1998). National survey results on drug use from the Monitoring the Future Study, 1975-1997. Washington DC: National Institute on Drug Abuse, U.S.D.H.H.S., Public Health Service, National Institutes of Health.
Jones, S., & Heaven, P. (1998). Psychosocial correlates of adolescent drug-taking behaviour. Journal of Adolescence, 21(2), 127-134.
Kellam, S. G., & Anthony, J. C. (1998). Targeting early antecedents to prevent tobacco smoking: Findings from an epidemiologi-cally based randomized field trial. American Journal of Public Health, 88(10), 1490-1495.
Kellam, S. G., Rebok, G. W., Ialongo, N., & Mayer, L. S. (1994).
The course and malleability of aggressive behavior from early first grade into middle school: Results of a developmental epidemiologically-based preventive trial. Journal of Child Psychology and Psychiatry and Allied Disciplines, 35, 259-281.
Loveland-Cherry, C., Leech, S., Laetz, V., & Dielman, T. (1996). Correlates of alcohol use and misuse in fourth grade children: Psychosocial, peer, parental, and family factors. Health Education Quarterly, 23(4), 497-511.
McCarthy, J. D., & Hoge, D. R. (1984). The dynamic of self-esteem and delinquency. American Journal of Sociology, 90, 396-410.
McGuire, P. A. (1998). School-based prevention: One size does not fit all. APA Monitor, 29(10).
Moore, S., Laflin, M., & Weis, D. (1996). The role of cultural norms in the self-esteem and drug use relationship. Adolescence, 31,
523-542.
Mrazek, P. J., & Haggerty, R. J. (Eds.). (1994). Reducing risks for mental disorders: Frontiers for preventive intervention research. Washington, DC: National Academy Press.
Murray, D. M., Pirie, P., Luepker, R. V., &Pallonen, U. (1989). Five-and-six-year follow-up results from four seventh-grade smoking prevention strategies. Journal of Behavioral Medicine,
12(2), 207-218.
Myers, D., Milne, A., Baker, K., & Ginsburg, A. (1987). Student discipline and high school performance. Sociology ofEducation, 60, 18-33.
National Education Association. (1998). Keys to excellence for your school. From NEA Web site http://www.newhorizons.org/ restr_neakeys.html.
National Education Goals Panel. (1998). Mathematics and science achievement state by state, 1998. Washington, DC: U.S. Government Printing Office.
Northwest Regional Educational Laboratory. (1999). Catalog of school reform models. Northwest Regional Educational Laboratory, January 1999. Available at: http://www. nwrel.org/scpd/natspec/catalog/ (See write-up for Positive Action at http://www.nwrel.org/scpd/natspec/catalog/positive.
htm)
O'Donnell, J., Hawkins, J. D., Catalano, R. F., Abbott, R. D., & Day, L. E. (1995). Preventing school failure, drug use, and delinquency among low-income children: Long-term intervention in elementary schools. American Journal of Orthopsychiatry,
65(1), 87-161.
Paulson, M., Coombs, R., & Richardson, M. (1990). School performance, academic aspirations, and drug use among children and adolescents. Journal ofDrug Education, 20(4), 289-303.
Peters, R. D., & McMahon, R. J. (Eds.). (1996). Preventing childhood disorders, substance abuse, and delinquency. (pp. 215240). Thousand Oaks, CA: Sage. Banff International Science Series, Vol. 3.
Petraitis, J., Flay, B. R., & Miller, T. Q. (1995). Reviewing the theories of adolescent substance abuse: Organizing pieces of the puzzle. Psychological Bulletin, 117(1), 67-86.
President Clinton's State of the Union addresses (1999, 2000). http://clinton3.nara.gov/WH/.
Price, R. H., Cowen, E. L., Lorion, R. P., & Ramos-McKay, J. (1988). Fourteen ounces ofprevention: A casebook for practitioners. Washington, DC: American Psychological Association.
Purkey, W. W. (1970). Self-concept and school achievement. Englewood Cliffs, NJ: Prentice-Hall.
Purkey, W., & Novak, J. (1984). Inviting school success: A self-concept approach to teaching and learning. Belmont, CA: Wadsworth.
Reiss, A. J., &Roth, J. A. (Eds.). (1993). Understanding and preventing violence. Washington, DC: National Academy of Sciences, National Academy Press.
Rigby, K., & Cox, I. (1996). The contribution of bullying at school and low self-esteem to acts of delinquency among Australian teenagers. Personality and Individual Differences, 21(4), 609612.
Rutter, M.,Maughan,B., Mortimore, P., & Ouston, J. (1979). Fifteen
thousand hours. London: Open Books. Safe, Disciplined and Drug Free Schools (2001) US Department
of Education., Washington, DC, Exemplary and Promising
Safe, Disciplined, and Drug-Free Schools Programs, www.ed.
gov/offices/OESE/SDFS/programs.html. Seligman, M. E. P. (1998). What is the 'good life'? APA Monitor,
October 1998, p. 2.
Shaver, J. P. (1982). Independent evaluation offour replicated sites for the Positive Action Program—Final report. Logan, UT: Wasatch Institute for Research and Evaluation.
Shaver, A., & Walls, R. T. (1998). Effect of Title 1 parent involvement on student reading and mathematics achievement. Journal of Research and Development in Education, 31(2), 9097.
Simons-Morton, B. G., Crump, A. D., Haynie, D. L., & Saylor, K. E. (1999). Student-school bonding and adolescent problem behavior. Health Education Research, 14(1), 99-107.
Skaalvik, E. M., & Valas, H. (1999). Relations among achievement, self-concept, and motivation in mathematics and language arts: A longitudinal study. The Journal of Experimental Education, 67(2), 135-149.
Slaby, R., Barham, J., Eron, L., & Wilcox, B. (1994). Recommendations: Prevention and treatment of youth violence. In L. D. Eron, J. H. Gentry, & P. Schlegel (Eds.), Reason to hope: A psychological perspective on violence & youth (pp. 447-461). Washington, DC: American Psychological Association.
Slavin, R. E., & Fashola, O. S. (1998). Show me the evidence! Proven and promising programs for America's schools. Thousand
Oaks, CA: Sage.
Sloboda, Z., & David, S. L. (1997). Preventing drug use among children and adolescents: A research-based guide (NIH publication No. 97-4212). National Institute on Drug Abuse,
Rockville, MD.
St Leger, L. H. (1999). The opportunities and effectiveness of the health promoting primary school in improving child health—a review of the claims and evidence. Health Education Research, 14(1), 51-69.
Stephenson, D. (1978). Evaluation of the Twin Falls Positive Action program 1977-78. Twin Falls, ID: College of Southern Idaho.
Stephenson, D. (1979). Evaluation of the Twin Falls primary Positive Action program 1978-79. Twin Falls, ID: College of Southern Idaho.
Sui-Chu, E., & Willms, J. D. (1996). Effects of parental involvement on eighth-grade achievement. Sociology Quarterly, 69, 126141.
Sussman, S., Dent, C. W., Burton, D., Stacy, A. W., & Flay, B. R.
(1995). Developing school-based tobacco use prevention and cessation programs. Thousand Oaks, CA: Sage. Symons, C. W., Cinelli, B., James, T. C., & Groff, P. (1997). Bridging student health risks and academic achievement through comprehensive school health programs. Journal ofSchool Health,
67(6), 220-227.
Title I (1999). Northwest Regional Educational Laboratory, Catalog of Comprehensive School Reform Demonstration Programs for improving academic achievement (US Department of Education, Title I). www.nwrel.org/scpd.natspec/catalog.
Effects of the Positive Action Program
89
Tobler, N. S. (1986). Meta-analysis of 143 adolescent drug prevention programs: Quantitative outcome results of program participants compared to a control or comparison group. Journal of Drug Issues, 16(4), 537-567.
Tobler, N. S. (1992). Drug prevention programs can work: Research findings. Journal of Addictive Diseases, 11(3), 1-28.
Tobler, N. S., & Stratton, H. H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. The Journal of Primary Prevention, 18(1), 71-128.
Tolan, P., & Guerra, N. (1994). What works in reducing adolescent violence: An empirical review of the field. Boulder, CO: The Center for the Study and Prevention of Violence, Institute of Behavioral Sciences, University of Colorado.
U.S. Department of Education. (1997). What really matters in American Education: White paper prepared for U.S. Secre-
tary of Education Richard W. Riley for speech at the National Press Club, Washington, DC.
U.S. Department of Education, National Center for Education Statistics. (1998). Violence and discipline problems in U.S. public schools: 1996-1997 (NCES 98-030). By Sheila Heaviside, Cassandra Rowand, Catrina Williams, and Elizabeth Farris. Projects Officers, Shelley Burns and Edith McArthur. Washington, DC.
U.S. Department of Health and Human Services. (1994). Preventing tobacco use among young people: A report of the Surgeon General. Atlanta, GA: Office on Smoking or Health.
Woodward, J. R. (1996). Improving academic achievement offourth-grade students through a program of self-concept enhancement activities. Unpublished doctoral practicum report, Nova Southeastern University, Jacksonville, FL.