232 RICHARD M. PIERCE AND THOMAS H. ZARLE

self-exploration. It was found that those who were more effective interpersonally attended to the feelings of others more often during self-exploration. It further was found that increased interpersonal effectiveness led to more reference to the feelings of others during self-exploration.

REFERENCES 1. CAHKHUFF, It. I t . Helping and Human Relations: A Primer for Lay and Professional Helpers.

2. CARKHUFF. R. R. and BEIWNSON, B. G. Begond Counseling and Therapw. New York: Holt. (Vol. I) New York: Holt, Rinehart & Winston, 1969.

_ . Rinehart & Winston, 1967.

3. LEITNER, L. A. Client self-exploration as a function of reference to significant others. J . ctin.

4. TnUAx, c. B. and CAnKHuFF, It. It. Toward Eflectiue Counseling and Psychotherapy. Chicago:

5. TRUAX, C. B. and WITTMIZR, J. Nonpersonal reference during psychotherapy and therapeutic

Ps?/ChOl., 1969, 26, 339-340.

Aldine, 1967.

outcome. J. clin. Psychol., 1971, 27, 300-302.

RELIABILITY O F PROCESS RATINGS OF PSYCHOTHERAPY WITH CHILDREN*

LOGAN WRIGHT CHARLES B. TRUAX AND KEVIN M. MITCHELL

Children's Hospital of thc University of Oklahoma

Univcrsity of A Lberla Tiniversity of Gcorgia

PROBLEM

A major challenge that confronts child therapists is the refinement of effective means of intervention when the course of personality development is deviant. Most efforts have involved psychotherapy, milieu treatment, or parent-provided assistance. Considerable controversy (4 ) has centered around whether or not psychotherapy as a form of developmental intervention is effective and, if so, why.(I2) Truax and Carkhuff(lO) have reviewed a large number of studies that suggest that the presence or absence of three conditions, accurate empathy (AE), nonpossessive warmth (NPW), and genuineness (GEN) in human contact designed to benefit personality development, do much to determine the effectiveness of such inter- vention. This finding makes measurement of these conditions important for the study of effective means of intervention in abnormal emotional or personality de- velopment. Truax", 9, has constructed separate scales to measure AE, NPW and GEN.1 Truax and Carkhuff ( lo , 1'. 4 5 ) cite approximately 25 studies that demonstrate the test-retest and interrater reliability of these scales that determine whether the three conditions were present in a variety of situations that involve many different types of Ss. However, no data have been reported that establish the reliability of the three Truax scales in the rating of these conditions in the behavior of thera- peutic interveners with children.

The issue of whether reliable ratings of AE, NPW and GEN can be obtained from child therapists is a real one. The fact that personality development of children is less complete makes the consequence of intervention a t this stage more far-reach-

*This investigatioir was supported in part through NIMH Grant 12306-02 and by ltesearch and Training Center Division Grant (RT-13) from the Social and Rehabilitation Service, Department of Health, Education, and Welfare, Washington, 11. C. Thanks are due to Ed Williams for assistance i r t data analysis.

*A brief description of these scales can be found in Truax, C. B. and Carkhuff, H. I{. Experi- mental manipulations of therapetitic conditions. J . consult. Psychol., 1965, 29, 119-124.

RELIABILITY OF PROCESS RATINGS OF PSYCHOTHERAPY WITH CHILDREN 233

ing. In addition, the above research appears to indicate that the presence of such conditions could have a great deal to do with the success of any attempt to intervene in the personality development of children. However, the nature of intervention with children is often quite different from that with adults, especially because it is considerably less verbal. It would seem inappropriate to assume from the reliability ratings of AE, NPW and GEN with therapists of adult patients that reliable ratings of these conditions can be obtained with children. Shapiro(", for instance, has con- cluded that most raters trained to utilize the Truax scales are verbally oriented and that audio recordings of therapy sessions with adult patients provide equal re- liability and validity to those obtained from audio-video recordings. Since such an extremely high proportion of the communication and activity of child therapy is nonverbal in nature, it seemed important to assess the extent to which the presence of AE, NPW and GEN could be judged reliably during psychotherapy with children.

METHOD Subjects. Ss were 16 practicing psychotherapists affiliated with one of two

psychiatric training facilities located in the Southwest. Four were Ph.D. level clinical child psychologists who served as staff members a t one of the facilities, 6 were child psychiatry residents in their third year of training, and 6 were clinical psychology interns. Videotape recordings of Ss' activity during regularly scheduled therapy sessions were obtained on an Ampex #VR 1500 video recording machine. Three, 5-minute segments were taken from each of two sessions with a child patient. These sessions occurred one week apart. The &minute film segments were obtained 20 to 25 minutes, 30 to 35 minutes, and 40 to 45 minutes into each session. The extent to which AE, NPW and GEN were present in the 32 therapy sessions was rated by three trained observers. The types of raters and the procedures employed are identical to those outlined in an earlier study. (11) Ebel's ( l ) intraclass correlations were estimated between the three raters' ratings for the three scales. Pearson rs that measured the extent to which AE, NPW and GEN tended to be present in the same session also were obtained.

RESULTS AND DISCUSSION The average interrater reliability coefficients (Ebel's Tkk) were .72 for empathy

measures ( p < .OOl), .53 for warmth ( p < .005) and .34 for genuineness ( p < .025). Correlations between the three separate conditions provided by the child therapists were AE us. NPW, .50 ( p < .005); AE us. GEN, .19 ( p = NS); NPW us. GEN, .10 ( p = NS). These data indicate that while a significant relationship exists between AE and NPW, it was not possible to obtain significant correlations between GEN and either of the other two variables.

These results demonstrate the reliability of ratings of empathy and warmth during psychotherapeutic intervention with children. This finding should be helpful to pave the way for further investigations to determine whether these conditions are related to the success of such intervention. However, it must be noted that the reliability ratings obtained in this study, while significant, are generally lower than those obtained in studies with adult patients (see Truax and Carkhuff (10. p. 45)).

This would seem to indicate that ratings of these conditions are somewhat less reliable in the more nonverbal activity of child therapy. The reliability coefficient of .34 for GEN is particularly low and may indicate that meaningful correlations between therapists' genuineness and various outcome measures cannot be obtained because of the low reliability of the genuineness scale when applied to the activity of child therapists. The fact that therapists' genuineness could not be correlated with the other therapist behaviors of AE and NPW, while AE and NPW were significantly correlated with each other, further supports this point.

New scales to measure AE, NPW, and especially GEN could be constructed specifically for use with child therapists. Such instruments would differ from those

234 LOGAN WRIGHT, CHARLES B. TRUAX AND KEVIN M. MITCHELL

devised by Truax in that they would focus more on therapists’ nonverbal be- haviors. The existing scales do not mention nonverbal behaviors, but rather refer to either covert therapist activity (e.g., defensiveness, boredom) or verbal behavior (e.g., agreeing, giving advice). A considerable body of literature now exists con- cerning the relationship of nonverbal behaviors (e.g., position of arms and trunk, eye contact, posture) and the attitudes and relationships that exist between a com- municator and addressee. An excellent review of these studies is provided by Mehrabiar~‘~). One investigation(2) on the effects of a speaker’s visual behavior upon the perception of his “authenticity” could be particularly useful for the develop- ment of a scale to measure GEN during intervention with children. Also, a 7-point scale developed by Guerney, Stover and DeMeritt(3) to rate empathy and accept- ance in parentlchild interaction might be considered by any researchers who at- tempt to produce new measures of AE and NPW for child therapists.

SUMMARY Videotape segments of the psychotherapeutic activity of 16 therapists during

32 separate sessions with children were rated for accurate empathy (AE), non- possessive warmth (NPW), and genuineness (GEN). The average interrater re- liabilities of these ratings were estimated by Ebel’s Tkk and found to be .72 for AE, .52 for NPW and .34 for GEN. Though moderate, these values were interpreted as a support for the hypothesis that reliable ratings of AE and NPW can be obtained for psychotherapeutic activity with children. However, the reliability scores were sufficiently weak to suggest that the largely nonverbal nature of interaction with children produces ratings of lower reliability than those that generally obtained with adults. The reliability for ratings of therapists’ genuineness was sufficiently low to raise serious questions as to whether this variable could be used meaningfully in future outcome studies with children.

REFERENCES 1. &BILL, It. L. Estimation of the reliability of ratings. Psychometrika, 1951, 16, 407-424. 2. EXLINIS, It. V. and E L D I t i D w , C. Effects of two patterns of a speaker’s visual behavior upon the percept,ion of the authenticity of his verbal message. Paper presented a t the annual meeting of the Eastern Psychological Association, Boston, April, 1967.

GuimNcY, Jit., B. G., STOVllR, L. and ~ ~ : M I ~ x I T T , S. A measure of empathy in parent/child interaction. J . gnet. Psychol., 1968, 112,49-55.

LINITT, E. E. The results of psychotherapy with children: An evaluation. J . consult. Psychol., 1957, 21, 189-196.

MICHKABIAN, A. Significance of posture and position in the communication of attitude and status relationships. Psychol. Bull., 1969, 71, 359-372.

SHAPIIIO, J. G. Itelationships between visual and auditory cues of therapeutic effectiveness. J. elin. Psychol., 1968, 24, 1-7.

TIWAX, C. B. A scale for the measurement of accurate empathy. Psychiatric Institute Butletin, Wisconsin Psychiatric Institute, University of Wisconsin, 1961, 1 , 10.

Tau Ax, C. B. A tentiitive scale for the rneasurcnmit of unconditional positive regard. Psychi- atric Institute Sulletin, Wisconsin Psychiatric Institute, University of Wisconsin, 1963, 2, 1 (a).

TituAx, C. B. A tentative sc:LIe f o r the measilrernent of therapist genuineness or self-congruence. Discussion Papers, Wisconsin Psychiatric Iristitute, University of Wisconsin, 1962, 35(b).

TituAx, c. B. and CmKHuFF, It. 11. Toward Effcctive Counseling and Psychotherapy: Training and Practice. Chicago: Aldine, 1967.

T ~ ~ U A X C. B., CAIIKHUFF, It. 11. and KODMAN, F., JII. Itelationships between therapist-offered conditions and patierit change in group psychotherapy. J . clin. Psychol., 1‘36.5, % I , 327-329.

Titunx, C. L1. and WAltQO, 1). G. Psychotherttpeutic ericounters that change behavior: for better or for worse. Amer. J . Psycholhcr., 1966, 20, 499-520.

3.

4.

5.

6.

7.

8.

9.

J 0.

11.

12.