Research in Developmental Disabilities 32 (2011) 2894–2900

Contents lists available at ScienceDirect

Research in Developmental Disabilities

Evaluating the validity of the Nisonger Child Behavior RatingForm – Parent Version

Megan Norris a,b, Luc Lecavalier a,*a Nisonger Center, The Ohio State University, 305 McCampbell Hall, 1581 Dodd Drive, Columbus, OH 43210, USAb University of Rochester Medical Center, Department of Pediatrics, Division of Neurodevelopmental and Behavioral Pediatrics, Box 601, 627 Elmwood Ave,

Rochester, NY 14620, USA

A R T I C L E I N F O

Article history:

Received 3 May 2011

Accepted 5 May 2011

Keywords:

Intellectual disability

Assessment

Psychopathology

Emotional and behavior problems

NCBRF

Validity

A B S T R A C T

Youth with intellectual and developmental disabilities (IDD) experience high rates of

emotional and behavior problems. The Nisonger Child Behavior Rating Form (NCBRF) is

one of the few tools developed to assess these problems in this population. It consists of a

10-item Social Competence section and a 66-item Problem Behavior section. The goal of

this study was to examine the factorial, criterion, and convergent validity of the NCBRF.

Data from individuals aged five to 18 years (n = 399) were submitted to confirmatory

factor analysis (CFA). In addition, scores of different clinical groups (disruptive behavior

disorders, mood/anxiety disorders, taking psychotropic medicines) were compared to

youth with no significant clinical concerns and associations with Developmental Behavior

Checklist (DBC) scores were examined in a subsample of participants. Results of the CFA

indicated that the Social Competence subscales better fit the data (RMSEA = .056), than the

Problem Behavior subscales (RMSEA = .086). Comparisons across clinical groups and

correlations with DBC scores were significant and in the expected directions. This study

supports the use of the NCBRF in youth with IDD, though the factor structure of the

Problem Behavior section may need to be revisited.

� 2011 Elsevier Ltd. All rights reserved.

1. Introduction

Multiple studies have indicated that individuals with intellectual and developmental disabilities (IDD) have higher ratesof psychopathology than the typically developing population (e.g., Linna et al., 1999; Tonge & Einfeld, 2003). Thoughprevalence of emotional, behavioral, and psychiatric problems varies by study methodology, rates in children andadolescents with ID generally range from 30% (Molteno, Molteno, Finchilescu, & Dawes, 2001) to 50% (Dekker, Koot, van derEnde, & Verhulst, 2002; Rutter, Tizard, Yule, Graham, & Whitmore, 1976). Numerous additional problems are associated withpsychopathology in those with ID, including difficulties at school, limited employment opportunities, adaptive behaviordeficits, increased caregiver stress, and difficulties integrating into community residential settings (Anderson, Lakin, Hill, &Chen, 1992; Baker et al., 2003; Bruininks, Hill, & Morreau, 1988; Emerson et al., 2001; Lecavalier, Leone, & Wiltz, 2006).

Although the frequency and severity of behavior and emotional problems in youth with ID has been recognized, there is adearth of research on their assessment (Lecavalier & Aman, 2005). One available instrument is the Aberrant Behavior Checklist

(ABC; Aman, Singh, Stewart, & Field, 1985a, 1985b). Although originally developed for adolescents and adults, its factorstructure held in children (Brown, Aman, & Havercamp, 2002). Adaptive behavior scales are another means of assessing

* Corresponding author. Tel.: +1 614 292 2378; fax: +1 614 688 5522.

E-mail addresses: megan_norris@urmc.rochester.edu (M. Norris), luc.lecavalier@osumc.edu (L. Lecavalier).

0891-4222/$ – see front matter � 2011 Elsevier Ltd. All rights reserved.

doi:10.1016/j.ridd.2011.05.015

M. Norris, L. Lecavalier / Research in Developmental Disabilities 32 (2011) 2894–2900 2895

behavior and emotional problems, but the primary aim of these scales is not to assess such problems. Often multiple problembehaviors are lumped together within one question, decreasing reliability. For example, on the Scales of Independent

Behavior-Revised (SIB-R; Bruininks, Woodcock, Weatherman, & Hill, 1996), the item ‘‘Disruptive Behavior,’’ which isendorsed yes or no, includes clinging, picking fights, and laughing or crying for no reason, as well as other behaviors.

Among instruments developed for youth with a dual diagnosis, Hurley, Levitas, Lecavalier, and Pary (2007) highlightedtwo well-respected, broad-band instruments with sound psychometric properties: the Developmental Behavior Checklist

(DBC; Einfeld & Tonge, 2002) and the Nisonger Child Behavior Rating Form (NCBRF; Aman, Tasse, Rojahn, & Hammer, 1996;Tasse, Aman, Hammer, & Rojahn, 1996).

The NCBRF was developed with a sample of 369 subjects. Separate analyses were conducted for the Problem Behavior andSocial Competence items and for parent and teacher ratings. Exploratory factor analysis (EFA) of the parent ratings suggesteda six-factor solution. Subscales were named as follows: Conduct Problem (16 items), Insecure/Anxious (15 items),Hyperactive (9 items), Self-Injury/Stereotypic (7 items), Self-Isolated/Ritualistic (8 items), and Overly Sensitive (5 items). Atwo-factor solution emerged for the Social Competence section with subscales named as follows: Compliant/Calm (6 items)and Adaptive Social (4 items). Internal consistency was sound for both the Social Competence and Problem Behaviorsubscales (alpha ranged from .73 to .93). Convergence with the ABC indicated strong associations between those subscalesexpected to correlate (median Pearson r = .72, range from .49 to .80) (Aman et al., 1996).

In 2004, Lecavalier, Aman, Hammer, Stoica, and Mathews performed factor analysis of the NCBRF in a sample of 330children with autism spectrum disorders (ASDs). They again found a largely similar factor structure, although a five-factorsolution was chosen for the Problem Behaviors section. More recently, the reliability and factorial and convergent validitywere examined in a school-based sample of children and adolescents with DDs (n = 237; Rojahn et al., 2010). Evidence offactorial validity and internal consistency was mixed, but convergent validity with the Behavior Problems Inventory (BPI-01;Rojahn, Matson, Lott, Esbensen, & Smalls, 2001) was good.

The NCBRF has been used as the primary outcome measure in large clinical trials of risperidone in children withdisruptive behavior disorders (DBD), and the Conduct Problems subscale in particular was remarkably sensitive to the effectsof treatment (Aman et al., 2002; Snyder et al., 2002). Also, the NCBRF has been recommended by experts as having potentialfor assessing individuals with ID (Rush & Frances, 2000). It has been translated into several languages (e.g. Mircea, Rojahn, &Esbensen, 2010; Tasse, Morin, & Girouard, 2000) and has been used with typically developing youth (Aman et al., 2008).

Despite its increasing popularity, there have been few psychometric studies of the instrument. In addition, examinationof its factor structure has yielded mixed results. And while the DBC and NCBRF are two of the few tools appropriate for theassessment of psychopathology in youth with ID, they have never been examined within the same sample. Such analysis canshed some light on the relative strengths and weaknesses of these instruments. This study attempted to fill these gaps byusing a large, heterogeneous sample of individuals. It expanded on previous research by assessing the construct validity ofthe NCBRF through (a) confirmatory factor analysis, (b) comparisons between different clinical subgroups, and (c)correspondence with the DBC.

2. Method

2.1. Participants

Participants were 399 individuals with ID aged five to 18 years. The sample consisted of two subgroups: one recruitedfrom special education classrooms in Ohio (n = 199) and one consisting of clients in an outpatient behavioral support clinic(n = 200). Sample characteristics are presented in Table 1. The average age of the total sample was 11.6 years (SD = 3.8). Onehundred forty-one participants were female (35%) and 254 were male (64%); four (1%) did not report gender. The ethniccomposition was the following: 75% White (n = 301), 16% Black (n = 65), 2% Hispanic (n = 9), 5% ‘‘Other’’ (n = 21) (e.g.,Cherokee-American), and 3% (n = 1) not known. The most frequently endorsed psychiatric disorders were PDDs (n = 149,37%). Two hundred twenty-five participants (56%) reported some other medical condition (e.g., Epilepsy). One hundredninety-six youth (49%) were taking at least one psychotropic medication.

2.2. Instruments

The Nisonger Child Behavior Rating Form – Parent Version (NCBRF-P; Aman et al., 1996) is a 76-item scale designed to becompleted by the child’s primary caregiver. The Social Competence items are rated on a four-point Likert scale ranging from0 (not true) to 3 (completely or always true). The Problem Behavior items are rated from 0 (behavior did not occur or was not a

problem) to 3 (occurred a lot or was a severe problem). The two social competence subscales are: Compliant/Calm and AdaptiveSocial. Problem behavior items are distributed on the following six subscales: Conduct Problems, Insecure/Anxious,Hyperactive, Self-Injury/Stereotypic, Self-Isolated/Ritualistic, and Overly Sensitive. Informants are instructed to ratebehaviors within the last month.

The Developmental Behavior Checklist – Primary Caregiver Version (DBC-P; Einfeld & Tonge, 2002) is a 96-item scaledeveloped to be completed by the child’s primary caregiver. It is scored on a three-point scale, ranging from 0 (not true as far

as you know) to 2 (often true or very true). Informants are instructed to rate behaviors within the past 6 months.A demographic form was completed by parents/caregivers included the previously reported participant information.

Table 1

Sample characteristics.

Community, n = 199 Clinical, n = 200 Total, N = 399

Age: Mean (SD) 12.7 (3.7) 10.5 (3.6) 11.6 (3.8)

Sex: n (% male) 110 (55%) 144 (72%) 254 (64%)

Level of ID: n (%)

Borderline 18 (9%) 31 (16%) 49 (13%)

Mild 25 (13%) 79 (40%) 104 (26%)

Moderate 13 (7%) 62 (31%) 75 (19%)

Severe 7 (4%) 18 (9%) 25 (6%)

Profound 2 (1%) 2 (1%) 4 (1%)

Not specified 134 (67%) 8 (4%) 142 (36%)

Subscale scores: mean (SD)

Compliant/Calm 9.20 (3.63) 6.12 (2.11) 7.65 (3.34)

Adaptive Social 5.78 (2.36) 3.83 (1.70) 4.80 (2.27)

Conduct Problems 11.00 (9.24) 25.76 (10.53) 18.40 (12.35)

Insecure/Anxious 8.04 (7.34) 12.56 (9.67) 10.28 (8.88)

Hyperactive 10.09 (6.36) 16.69 (6.49) 13.40 (7.22)

SIB/Stereotypic 1.65 (2.89) 5.04 (4.39) 3.35 (4.08)

Self-isolated/Ritualistic 4.38 (3.79) 7.63 (4.98) 6.00 (4.70)

Overly Sensitive 5.49 (3.26) 8.01 (3.45) 6.75 (3.58)

M. Norris, L. Lecavalier / Research in Developmental Disabilities 32 (2011) 2894–29002896

2.3. Procedure

Inclusion criteria for the study were that children be between the ages of five and 18 years. Youth were not excluded dueto any medical or physical impairment. Individuals were excluded if they did not speak English, or if the NCBRF had morethan 10% of the data missing. For scales missing less than 10% of item ratings, the mean subscale score was assigned.

Community children were recruited through Ohio special education classes with the cooperation of special educationdirectors. Packets were mailed to directors (or appointed liaisons), which included NCBRF scales, demographicquestionnaires, and DBCs in every fifth packet, along with letters to parents explaining the study and seeking theirparticipation. These items were enclosed in business-reply envelopes. Liaisons sent the envelopes home in book bags withchildren. Follow-up letters were distributed in the same manner. Primary caregivers completed the demographicquestionnaires and the NCBRF (and the DBC) and mailed them back to the researchers in self-addressed, stamped envelopes.

Clinically referred children were consecutive cases meeting inclusion criteria from a university-based outpatientbehavior clinic. Data were gathered retrospectively via chart review.

2.4. Statistical analyses

2.4.1. Factorial validity

Confirmatory factor analysis (CFA) was performed using LISREL (Joreskog & Sorbom, 2004). Analyses were run on theSocial Competence and Problem Behavior items separately. Specifications were based on the original factor structurereported by Aman et al. (1996). Polychoric correlation matrices were used due to the ordinal nature of the data. Diagonallyweighted least squares (DWLS) was used for estimation due to the categorical nature of the data, and because it requiressmaller sample sizes than other methods (Wirth & Edwards, 2007).

There is little consensus regarding which measure of fit researchers should use to evaluate models (Bollen, 1989). Indicesin this study were chosen to reflect recommendations which suggest selecting indices from different categories whichconsider different criteria (Brown, 2006; Garson, 2009). RMSEA was selected as the primary measure because it is one of themeasures least impacted by sample size, has a confidence interval, and does not involve a null model. Additional measuresconsidered were the NNFI, CFI, and SRMR. Generally accepted cut-offs for indices were used to determine acceptable fit (i.e.,for the RMSEA, values less than .08, for the NNFI and CFI values greater than .95, and for the SRMR, values less than .10).

2.4.2. Criterion validity

Criterion validity was assessed with a series of ANOVAs. Children with DBD diagnoses (i.e., Attention Deficit/Hyperactivity Disorder, Oppositional Defiant Disorder, and/or Conduct Disorder; n = 95) were compared to those without aDBD diagnosis (n = 304), and children with a mood and/or anxiety disorder (n = 31) were compared to those without a moodor anxiety disorder (n = 368). Additionally, individuals taking at least one psychotropic medication (n = 196) were comparedto those not taking any psychotropic medications (n = 203). To correct for multiple comparisons, the significance level wasset at .01.

2.4.3. Convergent validity

To assess convergent validity, Spearman ranked correlations were computed between NCBRF and DBC subscales. Rawscores were used to compute these correlations. Alpha was set at .05 since the number of comparisons was few and thesample size was small (n = 29).

M. Norris, L. Lecavalier / Research in Developmental Disabilities 32 (2011) 2894–2900 2897

3. Results

3.1. Factorial validity

3.1.1. Social Competence

Analysis of the Social Competence items based on the original item assignment indicated good fit (RMSEA = .056, 90%confidence interval = .039–.073), supporting the two-factor structure originally proposed by Aman et al. (1996). Additionalindices of fit (e.g., NNFI, CFI, and SRMR) also indicated acceptable fit of the data to the model. The mean factor loading for theCompliant/Calm subscale was .73, while the mean loading for the Adaptive Social subscale was .70. Correlation between thetwo factors was strong (r = .91). Coefficient alpha was computed for the subscales and was as follows: Compliant/Calm = .84,Adaptive Social = .73. Due to the high inter-factor correlation, a second CFA was conducted testing a one-factor model. Thisalso indicated acceptable fit, but not better than the two-factor model.

3.1.2. Problem Behavior

Analysis of the Problem Behavior items based on the original item assignment indicated a mediocre fit (RMSEA = .086, 90%confidence interval = .084–.088). The NNFI, CFI, and SRMR all also indicated less than desired fit. Mean factor loadings for thesubscales were as follows: Conduct Problems = .78, Insecure/Anxious = .69, Hyperactive = .78, Self-Injury/Stereotypic = .73,Self-Isolated/Ritualistic = .64, and Overly Sensitive = .72. Correlations between factors ranged from .43 (Insecure/Anxiousand Self-Injury/Stereotypic) to .75 (Insecure/Anxious and Overly Sensitive). Coefficient alphas were calculated for thesubscales and were as follows: Conduct Problems = .94, Insecure/Anxious = .89, Hyperactive = .90, Self-Injury/Stereo-typic = .78, Self-Isolated/Ritualistic = .78, and Overly Sensitive = .77.

Due to previous support of a five-factor solution and the mediocre fit of the original six-factor model in this sample, thefive-factor solution proposed by Lecavalier, Aman, Hammer, Stoica, and Mathews (2004) was specified and results againindicated suboptimal fit (RMSEA = .087, 90% confidence interval = .085–.089). Coefficient alphas for subscales of the five-factor solution were identical to those for the six-factor solution, with the exception of the Self-Isolated Ritualistic factor,which had the slightly lower alpha of .75. CFAs for the five- and six-factor solutions also were repeated on the clinical andcommunity subsamples separately but model fit did not improve.

3.2. Criterion validity

Results of ANOVA are presented in Table 2. Comparisons between those taking psychotropic medications and those nottaking medications indicated significant differences on all subscales between groups, with those taking psychotropicmedications scoring lower on Social Competence subscales and higher on Problem Behavior subscales. Comparisonsbetween those with and without DBD diagnoses revealed significant differences between groups on all subscales, exceptSelf-Isolated Ritualistic. Those with a DBD diagnosis scored lower on Social Competence subscales and higher on the ProblemBehavior subscales, as expected. Comparisons between those with Mood and/or Anxiety Disorders to those without a Moodor Anxiety Disorder indicated significant differences between groups on all subscales, except Adaptive Social. Youth notdiagnosed with a Mood or Anxiety disorder scoring higher on the Compliant/Calm subscale and lower on the ProblemBehavior subscales than those with a Mood or Anxiety Disorder diagnosis.

3.3. Convergent validity

Correlations between the DBC and NCBRF subscales are shown in Table 3. Expected correlations between similarsubscales were as follows: Conduct Problems and Disruptive/Antisocial (r = .85, p < .01); Insecure/Anxious and Anxiety

Table 2

Criterion validity.

Subscalesa Medb No Medb F DBDc No DBDc F M/Ad No M/Ad F

n = 196 n = 203 n = 95 n = 304 n = 31 n = 368

M (SD) M (SD) M (SD) M (SD) M (SD) M (SD)

C/Calm 6.7 (2.5) 8.5 (3.8) 30.4* 6.4 (2.6) 8.1 (3.5) 19.5* 5.6 (1.8) 7.8 (3.4) 12.9*

A. Social 4.3 (1.9) 5.3 (2.5) 22.4* 4.1 (1.8) 5.0 (2.4) 11.4* 4.1 (1.3) 4.9 (2.3) 3.6

Conduct 22.1 (11.8) 14.9 (11.9) 37.0* 25.0 (11.7) 16.3 (11.8) 39.3* 28.5 (11.8) 17.5 (12.0) 23.7*

Insecure 11.5 (9.4) 9.1 (8.2) 6.9* 14.8 (8.9) 8.9 (8.4) 34.9* 21.7 (8.9) 9.3 (8.2) 65.0*

Hyperactive 15.0 (6.6) 11.8 (7.4) 20.3* 16.0 (6.8) 12.6 (7.2) 17.3* 18.1(6.6) 13.0 (7.1) 14.9*

Self-Injury 4.2 (4.3) 2.5 (3.7) 16.7* 4.4 (4.4) 3.0 (3.9) 7.9* 5.6 (4.0) 3.2 (4.0) 10.3*

Self-Isolated 6.7 (4.8) 5.3 (4.5) 9.3* 7.1 (4.7) 5.7 (4.7) 6.7 9.0 (4.9) 5.8 (4.6) 14.1*

Sensitive 7.3 (3.6) 6.2 (3.4) 10.9* 8.1 (3.4) 6.3 (3.5) 18.0* 9.8 (2.9) 6.5 (3.5) 26.0*

a Summary of subscale name.b Medication.c Disruptive Behavior Disorder.d Mood/Anxiety Disorder.* p <.01.

Table 3

Convergent validity between NCBRF and DBC subscales (n = 29).

NCBRF subscalesa DBC subscalesa

Disrup/Anti Self-Absorb Com Disturb Anxious Social Rel

Compliant Calm �.69** �.46* �.29 �.49 �.54**

Adaptive Social �.56** �.46* �.28 �.21 �.54**

Conduct Problems .85** .38* .37* .48** .45*

Insecure/Anxious .59** .02 .15 .39* .24

Hyperactive .61** .51** .57** .33 .48**

SIB/Stereotypy .28 .28 .41* .40* .25

Self-Isolated/Rit .64** .63** .60** .43* .80**

Overly Sensitive .53** .24 .44* .50** .25

Expected correlations in bold.a Abbreviated subscale name.* p < .05.** p < .01.

M. Norris, L. Lecavalier / Research in Developmental Disabilities 32 (2011) 2894–29002898

(r = .39, p < .05); and Self-Isolated/Ritualistic and Self-Absorbed (r = .63, p < .01). Additionally, the DBC Disruptive/Antisocialsubscale was significantly correlated with five of the six NCBRF subscales: the previously mentioned Conduct Problems,Insecure/Anxious (r = .59, p < .01), Hyperactive (r = .61, p < .01), Self-Isolated/Ritualistic (r = .64, p < .01), and OverlySensitive (r = .53, p < .01). The NCBRF Conduct Problems subscale was correlated with all five DBC subscales: the previouslymentioned Disruptive/Antisocial, Self-Absorbed (r = .38, p < 05), Communication Disturbance (r = .37, p < .05), Anxiety(r = .48, p < .01), and Social Relatedness (r = .45, p < .05).

4. Discussion

4.1. Factorial validity

Evidence of factorial validity was mixed. Data from the Social Competence subscales were found to have good fit with theoriginal factor model, mirroring findings of Lecavalier et al. (2004). This is in contrast to results reported by Rojahn et al.(2010), who found poor fit with the original factor structure. Average factor loadings were high, as was the inter-factorcorrelation. Despite the strong relationship between the subscales, examination of a one-factor model did not improve fit.

Examination of the Problem Behavior data indicated sub-optimal fit with both the original six-factor and subsequentlyproposed five-factor models. This is similar to Rojahn et al. (2010) who found unacceptable fit for the six-factor solution, andpointed to differences in level of functioning across samples a possible reason for the discrepant results. While it is difficult tocompare samples in terms of level of functioning due to missing IQ data in studies, available information indicates that thepresent study and the original study samples were composed of children at similar levels of functioning. An alternativeexplanation for differing results could relate to the level of psychopathology experienced by participants. Contrary toLecavalier et al. (2004), Rojahn et al. (2004), and the current study, the original sample (Aman et al., 1996) consistedprimarily of clinically referred individuals. It is possible that the factor structure of the instrument differs for children andadolescents who do not have clinically elevated levels of emotional and behavioral disorders. However, we did not findsupport for this idea in our subsample analyses. Of course, sample size was notably reduced in the subsample analyses andconsequently results of these analyses should be interpreted with caution.

It also is noteworthy that while indices for the Problem Behavior section were not particularly good, average factor loadingswere quite high. The factor loading measures the strength of relationship between an item and its latent construct, indicatingthat in this sample the items were strongly related to their factors. By contrast, indices of fit provide a measure of how closely thedata (i.e., the observed correlation matrix) compares to the suggested model (i.e., the implied correlation matrix). Therefore,other variables such as correlated error (i.e., redundancy in items) could explain the less-than-ideal fit indices.

4.2. Criterion validity

Results largely supported the criterion validity of the NCBRF. Groups of children taking psychotropic medicines or withprevious psychiatric diagnoses obtained significantly different scores than children who were not taking medicines orwithout previous psychiatric diagnoses. In each set of comparisons, the subgroup expected to score higher on ProblemBehavior subscales and lower on Social Competence subscales did. Only two comparisons did not yield significant results,and each occurred on a subscale with relatively few items (the Adaptive Social subscale contains 4 items and the Self-Isolated/Ritualistic subscale contains 8 items).

4.3. Convergent validity

Although the NCBRF and DBC are two of the few tools designed specifically for the assessment of behavior problems inchildren with ID, no previous study has examined both instruments within the same sample. Expected correlations between

M. Norris, L. Lecavalier / Research in Developmental Disabilities 32 (2011) 2894–2900 2899

similar subscales were found, indicating that the NCBRF and DBC subscales assess similar constructs. Nonetheless, they donot measure the same constructs, and each instrument can account for unique variance.

Interestingly, the patterns of correlation found were not as clear-cut as expected. The DBC Disruptive/Antisocial subscalewas significantly correlated with all of the NCBRF subscales, with the exception of Self-Injury/Stereotypic. Also, the NCBRFConduct Problems subscale was significantly correlated with all five DBC subscales, and the NCBRF Hyperactive subscale wassignificantly correlated with all DBC subscales except Anxiety. These patterns of correlation between externalizing subscalescould be due to undifferentiated behavior disturbance (i.e., children exhibiting one behavior problem are likely to exhibitmany problems). It is also possible that the patterns are a result of externalizing behaviors being relatively easy to observeand report reliably. Indeed, the strongest correlation was found between the Conduct Problems and Disruptive/Antisocialsubscales.

4.4. Limitations

Approximately half of the current sample had clinically significant behavioral disturbance while half was a community-based sample. Though this may be viewed as a limitation, this can also be considered a strength as a heterogeneous group islikely to have greater variability in scale scores, increasing the likelihood of stable factor solutions. Another limitation of thecommunity subsample is that parents/caregivers reported diagnoses, level of functioning, and other demographicinformation. There is reason to be confident that the individuals sampled have some level of ID, as this requirement wasstressed to directors/liaisons and the children were recruited from special education programs. However, it would have beenpreferable for participants to have undergone a clinical evaluation in order to confirm diagnoses. Additionally, in order tohave sufficient power, diagnostic categories (e.g., Oppositional Defiant Disorder, ADHD) were collapsed into single groups(e.g., DBD).

5. Conclusions

The assessment of behavior and emotional problems in children and adolescents requires psychometrically soundinstruments. The NCBRF is one such instrument which includes items measuring both positive and problem behaviors.Results of the current study generally supported the validity of the NCBRF – Parent Version, although the factor structure ofthe Problem Behavior section may need to be refined. It also provides evidence of criterion and convergent validity of theinstrument. Future studies need to establish cutoff scores for clinically significant behavior problems. The NCBRF’ssensitivity to change should also be examined, allowing the instrument to be used to evaluate treatment of behavior andemotional problems.

Acknowledgements

The authors wish to thank all of the Ohio special education directors, teachers, children, and parents who participated inthis study. They also thank the Ohio Department of Mental Health, who provided funding for this project.

References

Aman, M. G., DeSmedt, G., Derivan, A., Lyons, B., Findling, R. L., & The Risperidone Disruptive Behavior Study Group, (2002). Double-blind, placebo-controlled studyof risperidone for the treatment of disruptive behaviors in children with sub average intelligence. American Journal of Psychiatry, 159, 1337–1346.

Aman, M., Leone, . G. , Lecavalier, S., Park, L., Buican, L., & Coury, B. D. (2008). The Nisonger Child Behavior Rating Form-TIQ Version, for children with typical IQ.International Clinical Psychopharmacology, 23, 232–242.

Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985a). The Aberrant Behavior Checklist: A behavior rating scale for the assessment of treatment effects.American Journal of Mental Deficiency, 89, 485–491.

Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985b). Psychometric characteristics of the Aberrant Behavior Checklist. American Journal of MentalDeficiency, 89, 492–502.

Aman, M. G., Tasse, M. J., Rojahn, J., & Hammer, D. (1996). The Nisonger CBRF: A child behavior rating form for children with developmental disabilities. Research inDevelopmental Disabilities, 17, 41–57.

Anderson, D., Lakin, K., Hill, B., & Chen, T. (1992). Social integration of older persons with mental retardation in residential facilities. American Journal of MentalRetardation, 96, 488–501.

Baker, B. L., McIntyre, L. L., Blacher, J., Crnic, K., Edelbrock, C., & Low, C. (2003). Pre-school children with and without developmental delay: Behavior problems andparents stress over time. Journal of Intellectual Disability Research, 47, 217–230.

Bollen, K. A. (1989). Structural equations with latent variables. USA: John Wiley & Sons, Inc.Brown, T. A. (2006). Confirmatory factor analysis for applied research. NY: The Guilford Press.Brown, E. C., Aman, M. G., & Havercamp, S. M. (2002). Factor Analysis and norms for parent ratings on the Aberrant Behavior Checklist-Community for young

people in special education. Research in Developmental Disabilities, 23, 45–60.Bruininks, R., Hill, B. K., & Morreau, L. E. (1988). Prevalence and implications of maladaptive behaviours and dual diagnosis in residential and other service

programs. In J. A. Stark, F. J. Menolascino, M. H. Albarelli, & V. C. Gray (Eds.), Metal retardation and mental health: Classification, diagnosis, treatment, services (pp.1–29). New York: Springer-Verlag.

Bruininks, R. H., Woodcock, R. W., Weatherman, R. F., & Hill, B. K. (1996). Scales of Independent Behavior-Revised: Comprehensive Manual. Boston: RiversidePublishing Company.

Dekker, M. C., Koot, H. M., van der Ende, J., & Verhulst, F. C. (2002). Emotional and behavioral problems in children and adolescents with and without intellectualdisability. Journal of Child Psychology and Psychiatry, 43, 1087–1098.

Einfeld, S. L., & Tonge, B. J. (2002). Manual for the Developmental Behavior Checklist (Second Edition). Clayton, Melbourne, and Sydney: Monash University Center forDevelopmental Psychiatry, University of South Wales.

M. Norris, L. Lecavalier / Research in Developmental Disabilities 32 (2011) 2894–29002900

Emerson, E., Kiernan, C., Alborz, A., Reeves, D., Mason, H., Swarbrick, R., et al. (2001). The prevalence of challenging behaviors: A total population study. Research inDevelopmental Disabilities, 22, 77–93.

Garson, G. D. (2009). Structural Equation Modeling. http://faculty.chass.ncsu.edu/garson/PA765/structur.htm retrieved February 2, 2010.Hurley, A. D., Levitas, A., Lecavalier, L., & Pary, R. J. (2007). Assessment and diagnostic procedures. In J. Fletcher, E. Loschen, & P. Sturmey (Eds.), Diagnostic Manual

for the Dually Diagnosed. Kingston, NY: National Association for the Dually Diagnosed Press.Joreskog, K. G., & Sorbom, D. (2004). LISREL 8.7 for Windows [Computer software]. Lincolnwood, IL: Scientific Software International, Inc.Lecavalier, L., & Aman, M. G. (2005). Rating instruments. In Matson, J. L., Laud, R. B., & Matson, M. L. (Eds.), Behavior modification for persons with developmental

disabilities: Treatments and supports (Vol. I, pp. 160–189). Kingston, NY: National Association for the Dually Diagnosed Press.Lecavalier, L., Aman, M. G., Hammer, D., Stoica, W., & Mathews, G. L. (2004). Factor analysis of the Nisonger Child Behavior Rating Form in children with autism

spectrum disorders. Journal of Autism and Developmental Disabilities, 34, 709–721.Lecavalier, L., Leone, S., & Wiltz, J. (2006). The impact of behavior problems on caregiver stress in young people with autism spectrum disorders. Journal of

Intellectual Disability Research, 50, 172–183.Linna, S., Moilanen, I., Ebeling, H., Piha, J., Kumpulainen, K., Tamminen, T., et al. (1999). Psychiatric symptoms in children with intellectual disability. European

Child & Adolescent Psychiatry, 8, 77–82.Mircea, C. E., Rojahn, J., & Esbensen, A. J. (2010). Psychometric evaluation of Romanian translations of the Behavior Problems Inventory-01 and Nisonger Child

Behavior Rating Form. Journal of Mental Health Research in Intellectual Disabilities, 3, 51–65.Molteno, G., Molteno, C. D., Finchilescu, G., & Dawes, A. R. L. (2001). Behavioural and emotional problems in children with intellectual disability attending special

schools in Cape Town, South Africa. Journal of Intellectual Disability Research, 45, 515–520.Rojahn, J., Matson, J. L., Lott, D., Esbensen, A. J., & Smalls, Y. (2001). The Behavior Problems Inventory: An instrument for the assessment of self-injury, stereotyped

behavior, and aggression/destruction in individuals with developmental disabilities. Journal of Autism and Developmental Disorders, 31, 577–588.Rojahn, J, Rowe, E. W., Macken, J., Gray, A., Delitta, D., Booth, A., et al. (2010). Psychometric evaluation of the Behavior Problems Inventory-01 and Nisonger Child

Behavior Rating Form with Children and Adolescents. Journal of Mental Health Research in Intellectual Disabilities, 3, 28–50.Rush, A. J., & Frances, A. (2000). Expert consensus guideline series: Treatment of psychiatric and behavioral problems in mental retardation. American Journal on

Mental Retardation, 105, 159–228.Rutter, M., Tizard, J., Yule, W., Graham, P., & Whitmore, K. (1976). Isle of Wight studies. Psychological Medicine, 6, 313–332.Snyder, R., Turgay, A., Aman, M. G., Binder, C., & Fisman, S. The Risperidone Conduct Study Group. (2002). Effects of risperidone on conduct and disruptive

disorders in children with subaverage IQ’s. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 1026–1036.Tasse, M. J., Aman, M. G., Hammer, D., & Rojahn, J. (1996). The Nisonger Child Behavior Rating Form: Age and gender effects and norms. Research in Developmental

Disabilities, 17, 59–75.Tasse, M. J., Morin, N., & Girouard, I. N. (2000). French Canadian translation and validation of the Nisonger Child Behavior Rating Form. Canadian Psychology, 41,

116–123.Tonge, B. J., & Einfeld, S. L. (2003). Psychopathology and Intellectual Disability: The Australian child to adult longitudinal study. International Review of Research in

Mental Retardation, 26, 61–91.Wirth, R. J., & Edwards, M. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.