Inclusive and Supportive Education Congress International Special Education Conference Inclusion: Celebrating Diversity? 1st - 4th August 2005. Glasgow, Scotland

home about the conference programme registration accommodation contact

Behavior Management in the Elementary Inclusive Classroom:
Effects of a Social Problem-Solving Intervention

Ann P. Daunic, Ph.D.
University of Florida, Gainesville, Florida
adaunic@coe.ufl.edu

Stephen W. Smith, Ph.D.
Eve M. Brank, Ph.D.
University of Florida

Randall D. Penfield
University of Miami

Abstract

Teachers, especially in inclusive settings, need effective, efficient strategies to prevent and ameliorate destructive student behaviors. During the past two decades, researchers have found that cognitive strategies can decrease student disruption/aggression and strengthen pro-social behavior. Following preliminary studies, we conducted a randomized efficacy trial to determine whether a classwide, social problem-solving curriculum affected measures of knowledge and behavior for 165 4 th and 5 th grade target students in high-risk schools. We found significant positive treatment effects on knowledge of problem solving concepts, teacher ratings of aggression, and student self-reports of anger suppression. Outcomes differed across teachers/classrooms, and there was no evidence that booster lessons affected treatment efficacy. Teacher ratings of social validity were generally positive. We discuss classroom-based prevention research issues and future research needs.

The proactive assurance of safe and productive school environments is a critical objective for education policy makers. Implementing preventive interventions may be particularly challenging in high-risk schools with a high proportion of children at risk for academic failure and conduct problems (Bierman, Greenberg, & CPPRG, 1996). The pressure to improve student academic performance and standardized test scores creates significant demands that compete with social programs for instructional time. Moreover, school administrators and classroom teachers need access to evidenced-based practices to avoid adopting prevention programs that are intuitively appealing but unsubstantiated by empirical research (see e.g., Vaughn & Dammann, 2001).

Cullinan (2002) describes universal prevention as the application of interventions to a broadly defined group (e.g., classroom) to reduce risk and maintain student health and safety. Selective prevention involves activities designed for at-risk groups who may share characteristics that put them at risk for developing mental disorders or a school diagnosis of emotional or behavioral disorders (EBD). Thus, classroom-based interventions can act as both universal and selective prevention. Walker, Colvin, and Ramsey (1995) maintain that universal interventions (e.g., classwide) are especially effective for students who are “on the margins,” or beginning to behave in ways that compromise their future school success, noting that such interventions allow children with or at risk for EBD to learn effective coping strategies with support from socially appropriate peers.

One approach to universal and selective prevention is the classwide application of a cognitive-behavioral intervention (CBI). A research-based approach to teaching students positive coping strategies (e.g., Kendall & Braswell, 1985; Robinson, Smith, Miller, & Brownell, 1999), CBI techniques incorporate behavior therapy (e.g., modeling, feedback, reinforcement) and cognitive mediation (e.g., think-alouds) to build what Kendall (1993) called a new “coping template.” The underlying assumptions are that overt behavior is mediated by cognitive events and that a person can learn to influence cognitive events to change behavior. Literature reviews and meta-analyses (cf. Abikoff, 1991; Dush, Hirt, & Schroeder, 1989; Robinson et al., 1999; Whalen, Henker, & Hinshaw, 1985) have substantiated CBI’s usefulness for the prevention and remediation of specific behavioral deficits and the maintenance of appropriate behavior for mainstream students. Teaching students cognitive strategies has been found to decrease hyperactivity/impulsivity and disruption/aggression, strengthen pro-social behavior, increase social cognition, and improve peer relations (cf. Ager & Cole, 1991; CPPRG, 2002a, 2002b; Dodge, 1986; Lochman, Coie, Underwood, & Terry, 1993; Robinson, Smith, & Miller, 2002; Smith, Siegel, O’Connor, & Thomas, 1994).

In addition to efficacy, researchers are necessarily concerned with intervention efficiency and sustainability. The degree of exposure necessary to achieve and maintain a desired behavioral effect has been difficult to specify across treatments because of the variability of intervention packages. In general, longer treatments tend to result in better outcomes (e.g., Heinicke, 1988; Waltman & Zimpfer, 1988; Whalen et al., 1985). ) For example, Lochman (1985) manipulated duration of exposure to an anger coping program and found that students who received 18 weeks vs. 12 weeks (one lesson per week) achieved greater gains. More recently, Larson and Lochman (2002) noted that booster sessions designed to supplement and reinforce initial instructional content helped sustain student learning and improvements in pro-social behavior.

Research Purpose

In light of the CBI literature and the call for evidenced-based practice, we developed, piloted, and investigated the efficacy of Tools for Getting Along: Teaching Students to Problem Solve (TFGA), a universal, cognitive-behavioral social problem-solving curriculum. Our purpose was to determine whether a CBI implemented by classroom teachers (universal prevention) could engender and sustain positive outcomes, particularly for students at risk for disruptive and/or aggressive behavior (selective prevention). Our preliminary work suggested that exposure to TFGA significantly decreased teacher ratings of reactive and proactive aggression for “target students” (those nominated by teachers as having or being at risk for developing disruptive or aggressive behavior patterns). The target students and their non-target peers increased their knowledge of problem-solving concepts, and target students’ ratings on the teacher reports of aggression became more similar to non-target students’ ratings following the intervention.

Findings were inconsistent, however, across multiple measures in our pilot work. There were also methodological weaknesses (e.g., self-selecting experimental groups, small sample size, demographic differences between groups) that necessitated further investigation of treatment efficacy with a more adequate design and analysis. Using a more robust treatment, a larger sample, a randomized design, and more sophisticated statistical techniques, therefore, we designed the current study to answer the following research questions:

Does TFGA increase target students’ knowledge of social problem-solving strategies?

Does TFGA improve ratings of student aggression and anger expression/control?

Does degree of treatment exposure (i.e., number of lessons/opportunities for skill practice) affect treatment efficacy?

In addition, we examined TFGA’s social validity (appeal to consumers) for regular elementary school teachers in high-risk schools, a factor critical to the feasibility and sustainability of universal and selective prevention.

Method

Intervention

We designed TFGA to help 4 th- and 5 th-grade students develop positive solutions to social problems, particularly in anger-provoking situations. We selected grades four and five because students at this level are cognitively ready to accommodate fairly sophisticated problem-solving concepts and because they are approaching the transition to middle school, which requires increasing independence and the ability to resist negative peer influence. Moreover, Lochman, Dunn, and Klimes-Dougan (1993) suggested that teachers introduce CBI skills within targeted domains. The focus of the problem-solving framework in TFGA is, therefore, understanding and dealing with frustration and anger, frequent correlates of disruptive and aggressive behavior (Averill, 1982).

Content . TFGA is designed to help students learn six social problem-solving steps, use them as self-statements to guide decision-making, and ultimately, use them automatically in challenging social situations at school and elsewhere.

Following an introductory lesson, three lessons are devoted to problem recognition (Step 1). This step includes recognizing anger in oneself and others and understanding how frustration and anger can create and/or exacerbate problems. The next two lessons detail Step 2 strategies: To prevent the escalation of frustration and anger and to engage students’ cognition (i.e., “calm down and think”). Remaining lessons cover problem definition, solution generation, strategy selection, and outcome evaluation. Each of the 15 lessons devoted to teaching these 6 problem-solving steps begins with a cumulative review and ends with an opportunity to practice learned skills through an associated activity. In addition, there are five role-play lessons placed strategically throughout the curriculum to allow students to practice each problem-solving step after learning the relevant skills.

Assuming that repeated behavioral practice enhances learning (Bandura, 1986), we developed six booster lessons to be taught at 1-2 week intervals during the second half of the school year. The first consists of a general review of the problem-solving steps and rationale. In the second, students act out scripted role-plays demonstrating the six problem-solving steps, and in booster lessons 3 and 4, teachers divide students into small groups in which they design their own role-plays and act them out for the class. Student volunteers are asked to share appropriate real life problems with the class in lesson 5; the class helps each choose a solution, and the volunteers report to the class during the final lesson about how the chosen strategy worked. (For a complete description of TFGA development, see Daunic and Smith [2003]).

Curriculum format and features . TFGA lessons last approximately 30 minutes and, ideally, are taught at the rate of one or two per week. Each includes a specific step with objectives, a cumulative review, teacher presentation of new material, and opportunities for guided and independent practice. Teachers were directed to use elements of self-instructional training to help students develop self-management of behavior through the purposeful manipulation of overt speech and eventually, covert verbalizations. We recommended frequent pairing or grouping of students and included an optional point system at the end of each “Tool Kit” (a student guided practice activity) to allow students to reward themselves for self-reflection and appropriate participation.

Setting

Participating schools had large at-risk populations (40 to 86 percent of students eligible for free or reduced-price lunch) and were situated in varied environments from rural to small city. We solicited participation through area school district personnel and school principals, explaining prior to their commitment that a particular school might fall into one of three conditions: 20 lessons, 20 lessons plus boosters, or control.

After obtaining a commitment from seven schools in three districts, we matched schools (two sets of three and one set of two)on (a) state assigned school grade (based on student achievement and highly correlated with SES), (b) SES as determined by percent of students receiving free or reduced price lunch, and (c) school size. We randomly assigned members of each matched set to the three conditions and met with 4 th and 5 th grade teachers at each school to solicit their involvement. The resulting sample consisted of 2 schools (10 teachers) in the 20-lesson condition, 3 schools (17 teachers) in the 20-lesson plus booster condition, and 2 schools (8 teachers) in the control condition. From one to eight teachers participated at any given school.

Participants

We asked each teacher to complete a target student nomination form to identify students whom they believed to be the most disruptive or aggressive relative to others in their class.We solicited parent permission to use data from approximately 800 students across the 7 schools; a total of 525 students (66 percent) returned signed parental consent forms. Of the 210 nominated target students, 165 (79 percent) returned parental consent forms.(The higher return rate for targets probably resulted from our request to make every effort to obtain consent for these students.)The number of targets in the 20-lesson, 20-lesson plus booster, and control groups consisted of 42, 86, and 37, respectively. (The considerable difference in size of groups resulted from the number of teacher volunteers within each school and the number of target students nominated by each.) Targetstudents’ gender, race, SES, and educational program for each condition are shown in Table 1.

Table 1

Target Student Gender, Race, SES, and Education Program for Three Conditions

a The majority of special education students received services for learning disabilities and were mainstreamed for most or all of the school day.

b The sample size for each condition is the number of target students with complete data across assessments, i.e., the sample used in statistical analyses.

Measures

Problem-Solving Questionnaire . The Problem-Solving Questionnaire consists of 14 questions (24 possible points) developed from TFGA content by project staff. For some questions, only one answer among several alternatives is appropriate; other items require students to “check all that apply” (e.g., Check all the ways your body may feel when you are angry). Two additional items require students to supply information (e.g., What are three levels of anger, from lowest to highest?) Scale development included (a) a pilot test with pre- and post-treatment administrations to 35 students taught TFGA and (b) subsequent item revisions. Reliability estimates using post-test data yielded a Cronbach’s alpha for the total scale of .72.

Pediatric Personality (PPS-1 & 2) and Anger Expression PAES-3) Scales. The PPS and PAES, derived by Jacobs (Jacobs, Phelps, & Rohrs, 1989) from the State-Trait Anger Expression Inventory (see Spielberger, Sydeman, Owen, & Marsh, 1999), include five subscales within three sections: State and trait anger (PPS-1 & 2), and anger control, suppression, and expression (PAES-3). (Note: The PPS-1 & 2 also include items related to anxiety, which we administered but did not include in our analyses.) Items associated with state anger ask respondents to describe how they currently feel (e.g., I feel like yelling at somebody; I feel grouchy) using three response options: very much so, somewhat, or not at all. Trait anger items ask respondents to describe how they usually feel (e.g., I have a bad temper; I get angry quickly) with response options of hardly-ever, sometimes, or often. The last three subscale items ask respondents to select how frequently they feel or act a particular way when they are angry or very angry and include anger control (e.g., I do something totally different until I calm down; I control my temper), anger suppression (e.g., I hold my anger in; I get mad inside but I don’t show it), and anger expression (e.g., I say mean things; I do things like slam doors). Cronbach’s alphas yielded post treatment internal subscale reliabilities of .91 for state anger, .86 for trait anger, .74 for anger suppression, .77 for anger control, and .81 for anger out.

Reactive-Proactive Aggression Scale (R/P). Developed by Dodge and Coie (1987), the reactive and proactive aggression subscales include 3 items each, imbedded in 19 total items about behavioral qualities. These six items ask respondents to indicate how true a statement is (from never to always) for a particular child (e.g., When this child has been teased or threatened he/she gets angry easily and strikes back [reactive] and This child threatens or bullies others in order to get his/her own way [proactive]). Cronbach’s alphas computed with teacher-reported outcome scores post treatment yielded internal reliabilities of .92 for reactive aggression and .90 for proactive aggression.

Social Skills Rating System (SSRS). The SSRS includes social competence and problem behavior subscales, both relevant to the skills taught in TFGA. The scale is widely used and described thoroughly elsewhere (Gresham & Elliott, 1990).

Teacher Social Validity Questionnaire . To solicit feedback about the appeal and utility of TFGA, eight scale items focus on ease of use (e.g., The curriculum was easy to use; I completed each lesson in the time allotted), six on appeal/utility to students (e.g., The curriculum concepts were age-appropriate for my students), and eight on effectiveness for reducing disruption and aggression (e.g., The curriculum improved my students’ behavior). The scale was revised after teachers and university faculty reviewed all 22 items and suggested improvements. Each of the three subscales has a likert-type response format, with responses ranging from 1 (strongly disagree) to 5 (strongly agree). Further adjustments followed administration to TFGA pilot teachers.

Design and Analyses

Using a randomized design, we collected outcome measures from students and teachers in all conditions three times during the school year: Prior to fall treatment (Assessment 1), when both treatment groups completed the core 20 lessons (Assessment 2), and at the end of the academic year (Assessment 3). We determined intervention effects for target students at Assessment 2 using hierarchical linear modeling techniques (HLM) on residual change scores, with experimental condition as a between-subjects factor and occasion (Assessment) as a within-subjects factor. HLM is the recommended procedure when the experimental design violates the independence of observations assumption required by traditional analysis of variance (ANOVA) designs (Raudenbush & Bryk, 2002). In our study, F tests based on ANOVA would not provide accurate estimates of the Type I error rate (Weinfurt, 2000) because students are nested within classrooms, where the intervention occured. In addition to the residual change score model, we used a raw change (simple difference score) model, a recommended procedure when it is likely that a high rank-order stability in outcome measures exists over time and when a powerful relation between pre and post treatment scores could obscure the detection of a third (i.e., treatment) effect (Stoolmiller & Bank, 1995).

Procedures

Although only data for students with parental consent were analyzed, all students in participating classrooms completed self-report assessments as part of classroom activities, and all students in treatment classrooms received instruction in TFGA. We asked teachers to complete the R/P and SSRS on all target students and an approximately equal number of randomly selected non-target students. Teachers in both treatment conditions taught core lessons during the first half of the school year (between Assessments 1 and 2) approximately 2-3 times per week for a total of 7-10 weeks. Following Assessment 2, students in the 20-lesson plus booster group received 6 booster lessons, approximately one every other week. All students and teachers completed Assessment 3 at year-end.

One project staff member was assigned to each treatment school, addressing ongoing questions or concerns. To monitor treatment fidelity, we asked teachers to complete “feedback forms” containing checklists about instructional content completed and questions about lesson duration, strengths, weaknesses, and student responses. Each form covered four consecutive lessons and was to be completed as soon after teaching the lessons as possible. Project staff also observed 15 teachers (1 lesson each) using a checklist to note whether the teacher followed the lesson plan, included all lesson concepts and activities, and provided student feedback. Following the intervention, treatment group teachers completed the Teacher Social Validity Questionnaire and were debriefed about implementation and social validity.

Results

Target vs. non-target comparisons . We found target students across all three groups to be higher on self-reports of Anger Out (t (184) = -2.593, p < .05) and lower on self-reports of Anger Control (t (428) = 3.036, p < .01) at Assessment 1 than their non-target (typical) peers. (Note: Due to a printing error, portions of the PPS and PAES subscales were missing from a large number of surveys at pre test. This resulted in the above discrepancy in degrees of freedom because we analyzed data only from students with complete scales.) Teachers also rated target students higher on proactive (t (226) = -19.627, p < .001) and reactive (t (226) = -15.263, p < .001) aggression, lower on self-control (t (214) =14.6, p < .001), and higher on externalizing behavior (t (166) = -8.3, p < .001), as shown in Table 2.

Table 2

Pretest Means, Standard Deviations, Sample Sizes, and Significance Tests for Target vs. Non-Target Students.

* statistically significant

Treatment efficacy. Subscale means, standard deviations, and sample sizes across three administrations are shown in Table 3. To determine treatment effects, we asked two primary questions: (a) After controlling for pretest scores, were mean outcome scores related to experimental condition? (b) After controlling for pretest scores and experimental condition, was there significant between-classroom variability in mean outcome scores? The first question addresses whether treatment induces a change in the outcome of interest, and the second addresses whether changes in outcome variables can be attributed to classroom-level factors other than intervention condition.

Table 3

Means, Standard Deviations, and Sample Sizes Across Measures and Conditions for Assessments on Three Occasions.

To answer these questions, we employed HLM 5 to fit a series of nested linear models, as follows.

Level-1 (individual):

Level-2 (classroom):

In the level-1 model, Y ij corresponds to the outcome variable score of the ith individual in the jth classroom, b 0j corresponds to the outcome mean of the jth classroom, b 1j corresponds to the coefficient associated with the covariate (pretest score), and r ij represents the error in the level-1 model for the individual in question. In the level-2 model, the outcome variable mean for the jth classroom is expressed as a function of the total outcome mean (g 00), the effect of the intervention (g 01), and error (u 0j). The second level-2 model (for b 1j, the within-classroom slope associated with the regression of posttest on pretest) is set as a fixed effect model because prior analyses showed that the error term variance did not differ significantly from zero.

In total, six versions of the level-1 and level-2 models were fit, differing according to outcome variable and comparison of interest. (For initial HLM analyses, we selected only those measures on which descriptive data indicated a possible treatment effect.) Thus, three versions compared reactive aggression (RA), proactive aggression (PA), and knowledge (KNOW) outcomes at time-2, using scores at time-1 as a covariate, to compare the control group to the two treatment groups combined. The other three versions compared RA, PA, and KNOW outcomes at time 3, using scores at time 2 as a covariate, to examine differences between the 20-lesson group and the 20-lesson plus booster group.

Target students in treatment classrooms scored higher on KNOW at time-2 than did their control counterparts. Table 4 shows that the coefficient (6.842) relating condition to mean KNOW at time-2 after controlling for KNOW at time-1 differs significantly from zero (t = 5.766, df = 29, p < .001). Thus, intervention classrooms had an adjusted mean KNOW score at time-2 that was 6.842 units higher than that of control classrooms. In addition, the variance of level-2 errors differed significantly from zero ( c 2 = 120.470, df = 29, p < .001), indicating that after adjusting for time-1 scores and treatment effect, there was significant variation in classroom mean KNOW scores at time-2. Analyses using simple difference scores replicated these findings.

To investigate whether the addition of booster lessons affected KNOW at time-3, HLM models used time-3 KNOW as the outcome variable and time-2 KNOW as the covariate. The results indicated that participation in the booster lessons was not significantly related to adjusted time-3 KNOW scores (t = -0.036, df = 23, p = .972), suggesting that booster lessons did not add to treatment effects. Nevertheless, the adjusted mean time-3 KNOW scores displayed a significant nonzero variance ( c 2 = 6.728, df = 23, p < .001), suggesting that after adjusting for time-2 KNOW scores and condition, there was still significant variation in mean time-3 KNOW scores among classrooms (probably a result of the significant time-2 variance in adjusted classroom means).

The relation of treatment to adjusted time-2 scores for RA and PA was similar to that observed for KNOW (see Table 4). Experimental condition was significantly related to the adjusted classroom mean RA (g 01 = -1.583, t = -3.441, df = 26, p = .002) and PA (g 01 = -1.647, t = -3.490, df = 26, p = .002) score at time-2. These results indicate that after controlling for time-1 scores, time-2 mean RA and PA scores were 1.583 and 1.647 units lower for treatment than for control classrooms. After adjusting for time-1 scores and condition, there also was significant variation among classrooms in time-2 mean scores for RA ( c 2 = 106.913, df = 26, p < .001) and PA ( c 2 = 66.163, df = 26, p < .001). As with KNOW, we did not find a significant relation between participation in booster lessons and adjusted time-3 RA and PA scores. Again, analyses using simple difference scores replicated these findings.

On SSRS self-control and externalizing behavior subscales, we noted little or no variation among target student means at Assessment 2 and thus no indication of a treatment effect. Similarly, mean scores on anger disposition and expression subscales showed little evidence of a treatment effect, but HLM analyses using simple difference scores indicated that treatment did affect student self-reports of anger suppression significantly (g 01 = -1.493, t (12) = -2.669, p = .021). These findings should be interpreted with caution because of the small number of target students who completed assessments, particularly in the control condition.

Estimating effect sizes can be problematic in a multi-level design (Raudenbush & Bryk, 2002). We used a "proportion of variance explained" approach, computing the proportion of between-group variance explained by the intervention effectat time-2, co-varying out time-1, for measures on which we found significant effects. With this procedure, we found that 0.418 of the between-group variance in mean KNOW scores, 0.169 of the between-group variance in mean RA scores, and 0.353 of the between-group variance in mean PA scores could be explained by the intervention. According to general criteria used with this method (see Cohen, 1977), these effect sizes range from moderately large to large.

Treatment fidelity and social validity . Of the 27 teachers in treatment conditions, the number returning each of the four treatment fidelity forms ranged from 18-22. For the 15 core content lessons (5 of the 20 core lessons were role-plays), 98 percent of teachers who responded reported they had covered all lesson content; these lessons averaged 30 minutes in length. The vast majority of respondents (88 percent) reported that students completed the Tool Kits, worksheets, and activities during class time; a smaller percentage (61) reported that students completed Tool Kit activities at home with a family member, as suggested to teachers in 8 TFGA lessons.In the 5 role-play lessons included in the 20-lesson core, 91 percent of respondents indicated that students participated in practice activities associated with role-plays, and 98 percent indicated that students engaged in the role-plays. Observational data substantiated that most of the 15 teachers observed included all lesson concepts, activities, and components (e.g., role-plays, discussion questions, worksheets) during the teaching period. One of the 15 did not include some parts of the lesson (previous lesson review, overhead transparencies), expressing doubt that the curriculum could affect her students’ behavior. Some teachers paired students or used small group instruction as suggested in lesson directions, and others used a whole-class delivery format.

We used descriptive data from the Teacher Social Validity Questionnaire to examine teacher satisfaction with TFGA. The appeal/utility to students on a 1 to 5 point subscale had a mean of 4.18 (SD = .77); appropriateness and ease of use 3.99 (SD = 1.03); and efficacy in addressing problem behavior 3.55 (SD = .95). The overall scale mean was 3.86 (SD = .97).

Discussion and Limitations

Efficacy

Study findings indicated that following a cognitive-behavioral curricular intervention (TFGA), target students’ knowledge about problem solving increased, teacher ratings of target student aggression improved, and improvements in knowledge and teacher-reported behavior appeared to be maintained over several months. We did not find evidence that treatment affected teacher-reports of self-control and externalizing behavior (SSRS subscales). Using an HLM simple difference score model, we replicated these findings and also found evidence that treatment affected student self-reports of anger suppression. Thus, the curriculum may have lowered students’ tendency to deny they were angry or hold their anger in. We interpret this result with caution, however, because of the small number of students with complete data across classrooms. We did not find evidence that the intervention affected student self-reports on other subscales involving anger disposition/expression (state/trait anger, anger out, anger control), but again, the small number of students completing these subscales on Assessments 1 and 2 limited statistical power.

We believe that consistent positive outcomes across our preliminary work and this study provide some evidence of intervention efficacy and build a case for further investigation. First, the skill of problem solving requires procedural knowledge that is foundational to sufficient performance and subsequent reinforcement (Bandura, 1986).The consistent combined improvements in knowledge and teacher ratings of aggression (both proactive and reactive) lend tentative evidence that TFGA influenced student behavior, or at the very least, altered teacher perceptions of that behavior. Teacher ratings of behavior, however, are admittedly not without limitations, particularly within this design. As others have (e.g., CPPRG, 2002b),we relied in part on reports from teachers who implemented the intervention, recognizing that these ratings could be biased by teacher investment in student outcomes.We feel strongly, however, that teacher perception is still a valid indicator of intervention impact because teachers are the ones most likely to refer students for special services. Although perhaps less compelling than direct observations by those blind to experimental condition, teacher reports are an essential measure of student behavior in the classroom, especially if used in conjunction with other data sources (Ollendick & King, 1999).

The lack of congruence between teacher-reported change in aggressive behavior and student self-reports of how they manage anger is more problematic. Although anger control and anger out subscales, particularly, assess self-reports of behaviors targeted by TFGA content, our findings did not indicate that studentschanged their perceptions of how they act on feelings of anger, with the exception of anger suppression (i.e., whether they try typically to ignore or hide their anger).It is possible that following treatment, students were more aware of their own angry feelings and realized they needed to do something about them, thus suppressing them less often, but they did not yet feel they had lowered their angry outbursts (anger out) or increased their anger control (e.g., calming down, talking to someone). All findings about anger expression must be interpreted with caution because of the small number of students with complete assessment data, particularly in the control condition.

Based on what is known about cognitive skill development, we hypothesized that adding varied practice opportunities in the form of booster lessons would strengthen student learning and generalization (Bandura, 1986). We did not find evidence, however, that students in the booster condition outperformed students in the 20-lesson only condition on outcome measures at Assessment 3. Though all teachers in the booster condition reported that they taught the additional six lessons, booster instruction took place late in the school year following statewide high-stakes testing. These teachers may, therefore, have delivered booster lessons less diligently than they taught the 20-lesson core, and we may not have focused adequately on these lessons during training, unintentionally conveying the message that they were less important.

On a final note, the students in our sample were generally not likely to exhibit behavior classified as chronic or severe, or scores differing as markedly from the norm on measures like the PPS/PAES, as students already placed in special programs. Inherent in working with this population (and in prevention research in general), therefore, is the challenge of ensuring enough statistical power (e.g., through effect size or sample size) to detect relatively small improvements that result from intervention, particularly over the short term (see Muehrer & Koretz, 1992).

Fidelity and Social Validity

While requiring that instruction be implemented with consistency across classrooms, we necessarily had to accommodate competing demands on teacher time and a variety of classroom/school schedules, and we relied heavily on the accuracy of teacher reports about instruction time and content completion in our assessments of treatment fidelity. Limited resources and the distant location of several intervention sites constrained our ability to observe teachers with enough frequency to gain insight into instructional nuances such as points of emphasis, modeling of self-talk, or level of enthusiasm. Although a majority of teachers indicated they had covered all TFGA lesson content and most activities, more observational data would have (a) strengthened confidence about treatment fidelity across sites and (b) added qualitative information about how teacher characteristics relate to student outcomes.

Moreover, as state and national accountability efforts continue to rise, teachers are increasingly preoccupied with meeting performance standards and often view curricula not directly related to academics as distractions from more pressing concerns. Such attitudes could, in turn, affect student responses to intervention (Polsgrove & Smith, 2004). Although all participating teachers in this study volunteered, it is difficult to discern the priority they placed on teaching TFGA relative to other demands. The fact that classroom means on outcome measures differed (after controlling for experimental condition and pre-treatment scores) indicated that some teachers were more successful than others in effecting positive changes in student knowledge and behavior.Despite this finding, it is noteworthy that teacher responses on the social validity survey across studies were generally positive about the curriculum’s ease of use and value to students. During informal follow-up discussions, most teachers expressed a desire to teach TFGA in the future and said they would recommend its use to other teachers and school personnel.

Implications and Future Research

Notably, the vast majority of teachers with whom we have worked expressed the opinion that students know what to do to control their behavior but often do not act on what they know because of competing behavioral models at school, at home, or in the community. Although peer, family, and community influences, including reinforcement contingencies, may compete strongly with a skill-based, cognitive-behavioral intervention, we believe teachers can increase student resilience in the face of behavioral risk factors through what they teach and model in schools. Our findings, though limited, support the continuing study of such efforts.

Researchers have noted that prevention programs are most powerful when they include family, peer, and community components (see CPPRG 1999a, 1999b). Although multi-component programs requiring support staff and specialized materials can be difficult for schools to sustain after external leadership and funding have ended (Sindelar & Brownell, 2001), these approaches are clearly warranted when adequate resources exist. We also recognize that the skills involved in social problem solving are complex and require repeated cognitive and behavioral practice. We recommend the continued study of factors that contribute to the efficacy of classroom-based CBIs, therefore, including optimal treatment exposure, treatment fidelity, sustainable parent involvement, efforts to ensure generalization, and in light of evidence for a teacher-level effect, teacher characteristics that enhance or diminish treatment outcomes and could be addressed in training, such as classroom discipline style and attitudes toward students with challenging behavior. In addition, longitudinal measures to examine prevention effects over time (see, e.g., CPPRG, 1999b), such as referrals for special education, rates of behavioral incidents, and measures of academic performance, would add significant information to findings based on shorter-term outcomes.

By continuing these efforts, researchers can define the parameters of feasible, sustainable, classwide preventive strategies that can be used in tandem with other approaches to reduce risk and enhance resilience. We concur with Muehrer and Koretz (1992) andCPPRG (2002a)that prevention research is challenging, but it offers the potential of avoiding negative outcomes and the costs associated with specialized treatment. Quality of life for many students and educators may rest in early and effective efforts to prevent anger and aggression. These efforts are worth making, and the most promising strategies can be found only through continued collaboration between researchers and school practitioners.

References

Abikoff, H. (1991). Cognitive training in ADHD children: Less to it than meets the eye. Journal ofLearning Disabilities, 24, 205-209.

Achenbach, T. M., & Edelbrock, C. S. (1983). Manual for the Child Behavior Checklist and revised child behavior profile. Burlington, VT: Department of Psychiatry, University of Vermont.

Ager, C. L., & Cole, C. L. (1991). A review of cognitive-behavioral interventions for children and adolescents with behavior disorders. Behavioral Disorders, 16, 276-277.

Averill, J. R. (1982). Anger and aggression: An essay on emotion. New York: Springer-Verlag.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall, Inc.

Bandura, A. (1969). Principles of behavior modification. New York: Holt, Rinehart, & Winston.

Bierman, K. L., Greenberg, M. T., & the Conduct Problems Prevention Research Group (CPPRG). (1996). Social skills training in the Fast Track Program. In R. D. Peters & R. J. McMahon (Eds.), Preventing Childhood Disorders, Substance Abuse, and Delinquency (pp. 65-89). Thousand Oaks, CA: Sage Publications, Inc.

Bower, E.M. (1981). Early identification of emotionally handicapped children in school (3rd ed.). Springfield, IL: Charles C. Thomas.

Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Revised Ed.). Academic Press. New York.

Conduct Problems Prevention Research Group (2002a). The implementation of the Fast Track program: An example of a large-scale prevention science efficacy trial. Journal of Abnormal Child Psychology, 30(1), 1-17.

Conduct Problems Prevention Research Group (2002b). Evaluation of the first 3 years of the Fast Track prevention trial with children at high risk for adolescent conduct problems. Journal of Abnormal Child Psychology, 30(1), 19-35.

Conduct Problems Prevention Research Group (CPPRG). (1999a). Initial impact of the Fast Track prevention trial for conduct problems: I. The high-risk sample. Journal of Consulting and Clinical Psychology, 67(5), 631-647.

Conduct Problems Prevention Research Group (CPPRG). (1999b). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting and Clinical Psychology, 67(5), 648-657.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.

Cullinan, D. (2002). Students with emotional and behavior disorders: An introduction for teachers and other helping professionals. Upper Saddle River, NJ: Pearson.

Daunic, A. P. & Smith, S. W. (2003). Technical Report #9: Tools for Getting Along: Teaching Students to Problem Solve. Department of Special Education: University of Florida.

Dodge, K. A. (1986). A social information-processing model of social competence in children. In M. Perlmutter (Ed.), The Minnesota Symposium on Child Psychology (pp. 77-126). Hillsdale, NJ: Lawrence Erlbaum.

Dodge, K. A., & Coie, J. D. (1987). Social-information-processing factors in reactive and proactive aggression in children’s peer groups. Journal of Personality and Social Psychology, 53(6), 1146-1158.

Dush, D. M., Hirt, M. L., & Schroeder, H. E. (1989). Self-statement modification in the treatment of child behavior disorders: A meta-analysis. Psychological Bulletin, 106, 97-106.

Gresham, F. M., & Elliott, S. N. (1990). The Social Skills Rating System. Circle Pines, MN: American Guidance Service.

Heinicke, C. M. (1988). Psychodynamic psychotherapy with children: Current status and guidelines for future research. In B. B Lahey & A. E. Kazdin (Eds.), Advances in Clinical Child Psychology (Vol. 12). New York: Plenum Press.

Jacobs, G. A., Phelps, M., & Rohrs, B. (1989). Assessment of anger expression in children: The Pediatric Anger Expression Scale. Personality Individual Differences, 10(1), 59-65.

Kendall, P. C. (1993). Cognitive-behavioral therapies with youth: Guiding theory, current status, and emerging developments. Journal of Consulting and Clinical Psychology, 61, 235-247.

Kendall, P. C., & Braswell, L. (1985). Cognitive-behavioral therapy for impulsive children. New York: Guilford.

Larson, J., & Lochman, J.E. (2002). Helping schoolchildren cope with anger: A cognitive-behavioral intervention. New York: Guilford Press.

Lochman, J. E. (1985). Cognitive-behavioral intervention with aggressive boys: Three-year follow-up and preventive effects. Journal of Consulting and Clinical Psychology, 60, 426-432.

Lochman, J. E., Coie, J. D., Underwood, M. K., & Terry, R. (1993). Effectiveness of a social relations intervention program for aggressive and nonaggressive, rejected children. Journal of Consulting and Clinical Psychology, 61(6), 1053-1058.

Lochman, J. E., Dunn, S. E., & Klimes-Dougan, B. (1993). An intervention and consultation model from a social cognitive perspective: A description of the Anger Coping Program. School Psychology Review, 22(3), 458-471.

Muehrer, P., & Koretz, D. (1992). Issues in preventive intervention research. Current Directions in Psychological Science, 1(3), 109-112.

Ollendick, T. H., & King, N. (1999). Child behavioral assessment and cognitive-behavioral interventions in schools. Psychology in the Schools, 36, 427-436.

Polsgrove, L., & Smith, S.W. (2004). Informed practice in teaching self-control to children with emotional and behavioral disorders. In Rutherford, R. B., Quinn, M., & Mathur, S.R. (Eds.), Handbook of research in emotional and behavioral disorders. New York: Guilford.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2 nd ed.). Thousand Oaks, CA: Sage.

Robinson, T. R., Smith, S. W., & Miller, M. D. (2002). Effect of a cognitive behavioral intervention on responses to anger by middle school students with chronic behavior problems. Behavioral Disorders, 27, 256-271.

Robinson, T. R., Smith, S. W., Miller, M. D., & Brownell, M. T. (1999). Cognitive behavior modification of hyperactivity/impulsivity and aggression: A meta-analysis of school-based studies. Journal of Educational Psychology, 91, 195-203.

Sindelar, P. T., & Brownell, M. T. (2001). Research to practice dissemination, scale, and context: We can do it, but can we afford it? Teacher Education and Special Education, 24, 348-355.

Smith, S. W., Siegel, E.M., O’Connor, A. M., & Thomas, S. B. (1994). Effects of cognitive-behavioral training on aggressive acts and anger behavior of three elementary-aged students. Behavioral Disorders, 19, 126-135.

Spielberger, C. D., Sydeman, S. J., Owen, A. E., & Marsh, B. J. (1999). Measuring anxiety and anger with the State-Trait Anxiety Inventory (STAI) and the State-Trait Anger Expression Inventory (STAXI). In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (2 nd ed.) Mahwah: Lawrence Erlbaum.

Stoolmiller M., & Bank, L. (1995). Autoregressive effects in structural equation models: We see some problems. In J.M. Gottman. The analysis of change. Mahwah, NJ: Lawrence Erlbaum Associates.

Vaughn, S., & Dammann, J. E. (2001). Science and sanity in special education. Behavioral Disorders, 27, 21-29.

Walker, H. M. Colvin, G., & Ramsey, E. (1995). Antisocial behavior in school: Strategies and best practices. Pacific Grove, CA: Brookes/Cole Publishing.

Waltman, D. E., & Zimpfer, D. G. (1988). Composition, structure, and duration of treatment: Interacting variables in counseling groups. Small Group Behavior, 19, 171-184.

Whalen, C. K., Henker, B., & Hinshaw, S. P. (1985). Cognitive-behavioral therapies for hyperactive children: Premises, problems, and prospects. Journal of Abnormal Child Psychology, 13, 391-410.

Weinfurt, K. P. (2000) Repeated measures analyses: ANOVA, MANOVA, and HLM. In L. G. Grimm & P. R. Yarnold (Eds.), Reading and Understanding More Multivariate Statistics (pp. 317-362). Washington, D.C.: American Psychological Association.