Section 1. Research

Measuring the Effectiveness ofTransactional Analysis: An International Study

Theodore B. Novey

AbstractThe effectiveness of psychotherapy car­

ried out by a group of 27 international cer­tified transactional analysts as a function oftherapy length is compared to the effective­ness of groups of psychiatrists, psycholo­gists, social workers, marriage and familycounselors, and physicians as measured in aresearch study carried out by the staff ofConsumer Reports, with Martin Seligman astheir consultant ("Mental Health," 1995;Seligman, 1995).Comparison isalso made tothe results from a group of psychoanalyticpsychotherapists (Freedman, Hoffenberg,Vorus, & Frosch, 1999). The results com­piled from the responses of 932 clients fromfour language groups confirm that therapylasting more than six months is considerably(40%) more effective than that lasting forless than six months. The data also deter­mines that the effectiveness of certifiedtransactional analysts issignificantly higher(p« 0.001) than the effectiveness of any ofthe groups from the Consumer Reportsstudy.

How do we know that our clients are pro­gressing, getting better, reaching their goals,being cured, or whatever the treatment goalsare for individuals in therapy? We can listen towhat they describe is happening in their livesand relationships. We can watch how they in­teract with us or with group members. We canadminister standardized tests. In one way or an­other, however, we rely on what they commu­nicate in words or behavior. If they tell us thatthey are satisfied, do we remain unsatisfied? Ifthey say that they feel better or are happy orexperience other positive feelings, do we dis­count that?

The progress of therapy is most generallymeasured by client satisfaction-an internal

8

experience of the client-whether by directverbal report, observation of their behavior, orthe administration of a validated test instru­ment. There is usually a high correlation be­tween the various ways of measuring positivechange (Strupp, 1996). Exceptions to this cor­relation wiII certainly occur if clients wish tocover up problems and/or avoid therapy, socare must be taken that such reports are madevoluntarily and with no possible negative con­sequences for the client.

Measuring change in human behavior andexperience occurs in many arenas. For ex­ample, public opinion polls are conductedregularly on every subject imaginable, frompolitics, to food, to work, to sexual preferences.Testing organizations exist to carry out thesemeasurements in statistically reliable fashion sothat there can be confidence in using the resultsfor whatever purpose they are designed.

This article reports the results of one suchmeasurement project: to measure client satis­faction or the effectiveness of therapy carriedout by certified transactional analysts who weretrained and certified using similar training andexamination criteria. The results of these mea­surements are compared to a basic set of datagenerated by a seminal study ("Mental Health,"1995) designed and carried out by a well­known and highly reliable testing group on thestaff of Consumer Reports magazine. Theirstudy compared the satisfaction of clients whohad worked with psychiatrists, psychologists,social workers, marriage and family counselors,and family physicians. Further details concern­ing this study were provided by Martin Selig­man, who was a consultant to the ConsumerReports study (Seligman, 1995, 1996a, 1996b),and by Consumer Reports staffmembers (Kot­kin, Daviet, & Gurin, 1996).

The results of the study reported here arealso compared to those from another study us­ing the Consumer Reports questionnaire and

Transactional AnalysisJournal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OFTRANSACTIONAL ANALYSIS: AN INTERNATIONAL STUDY

scoring system carried out by a group of psy­choanalytic psychotherapists at IPTAR, theInstitute for Psychoanalytic Training and Re­search (Freedman, Hoffenberg, Vorus, &Frosch, 1999).

An earlier (Novey, 1999) pilot set of mea­surements already provided, with highly signi­ficant statistical reliability, results from usingthe Consumer Reports questionnaire with cli­ents of a group of transactional analysts. Thequestionnaire and its scoring system provide arelatively simple and useful instrument forcomparing any well-defined group oftherapistswith the basic set measured by the ConsumerReports study. The questionnaire, used with thepermission of Consumer Reports, comprises aset of 35 questions. Five of the questions areused to develop the scores used in the mea­surements. The pilot study compared the satis­faction of the clients of certified transactionalanalysts to the clients ofvarious groups ofpro­fessionals measured by the Consumer Reportsstaff. It demonstrated that the clients of certi­fied transactional analysts were more satisfiedwith their therapy than those of any of thegroups measured by the Consumer Reportsstudy.

The present study expands this research toclients of a much larger group of 27 transac­tional analysis therapists from around theworld. The questionnaire was sent to clients invarious countries in four different languages:English, Spanish, French, and Italian. A totalset of 932 returned questionnaires were ana­lyzed. The results confirm and extend the re­sults of the pilot study. The clients ofcertifiedtransactional analysts, as measured over a five­year period, were once again, and with muchimproved statistical reliability, significantlymore satisfied with their therapy than the cli­ents of any of the groups of psychiatrists, psy­chologists, social workers, marriage and familycounselors, and physicians evaluated in theoriginal Consumer Reports study. The resultsreproduce for all of the language groups andcountries studied.

Similar results are obtained from a compari­son with the results of the Institute for Psycho­analytic Research and Training (IPTAR) study(Freedman et aI., 1999).

Vol. 32, No.1, January 2002

Measurements of EffectivenessOne ongoing question is how best to measure

what kinds of therapy and what kinds of thera­pist training lead to the best results for psycho­therapy. Two general methods have been de­veloped to answer this question: the efficacystudy as well as the effectiveness study. Num­erous publications have discussed the differ­ences between these two types of study andquestions about limitations and validity raisedby each. For details see Nathan, Stuart, andDolan (2000); Howard, Moras, Brill, Mar­tinovitch, and Lutz (1996); Clarke (1995); andSeligman (1995). Relevant comments fromthese papers are presented later in this article.

Many efficacy studies have been done anddiscussed extensively in the literature. Gen­erally these involved contrasting the results ofa particular approach to therapy with a com­parison group under well-defined conditions(see, for example, Howard, Kopta, Krause, &Orlinsky, 1986; Kazdin, 1986; Nathan, 1999).This method has considerable sophistication inorder to increase the objectivity of the mea­surements. Patients are assigned randomly tocontrol or treatment situations, with rigorouscontrols. Treatment methods are organized inreproducible ways; treatment time is fixed andlimited; outcomes are carefully defined; andratings are done blindly in that raters do notknow to which group patients were assigned.Well-defined single diagnosis disorders areselected. Therapy assessments are done at ter­mination and at follow-up intervals.

While many credible results have been ob­tained by this methodology, there are limi­tations to efficacy measurements. Vandenbos(1996) points out the need for more input fromboth efficacy and effectiveness measurements.Strupp (1996) states that rather than continuingthe controversy between efficacy and effec­tiveness measurements, it will be more usefulto continue to improve the quality of both typesof research and that the Consumer Reportsresults "provide another powerful argument infavor ofpsychotherapy" (p. 1017). Seligman's(1995) most cogent criticism is that there is awide disparity between how efficacy measure­ments are carried out and what actually hap­pens in the field of psychotherapy. He states

9

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

that the efficacy study is the wrong method forempirically validating psychotherapy as it ac­tually occurs because such a study omits toomany crucial elements of what is done in thefield. As he points out, actual psychotherapy isnot offixed duration, and it is self-correcting inthat if one approach does not seem to work, acompetent therapist will use another. Clientsoften take an active role in shopping for and se­lecting their therapist, and often there are mul­tiple problems to be treated. Finally, therapyoutcome is concerned with improvement in clie­nts' general functioning, not just with ameliora­tion of a specific symptom or disorder, which iswhat efficacy studies are designed to measure.

In line with its traditional methods of evalu­ating automobiles, refrigerators, and variousother consumer items-including medical andother health-related services-the ConsumerReports staffdeveloped a questionnaire to mea­sure the overall satisfaction of therapy clientswith psychotherapy treatmentthey had receivedin recent years (Annual Questionnaire, 1994).Using their carefully researched methods ofproduct evaluation, they set up an "effective­ness" study for their subscribers to measurehow satisfied clients were with therapy thatthey had received from mental health profes­sionals and physicians. Martin Seligman, pro­fessor of psychology at the University of Pen­nsylvania, was a consultant on this project. Thequestionnaire was sent out with the annualproduct survey to some 180,000 subscribersselected at random from the magazine's list ofmore than four million subscribers. Some 7,000responses to the mental health questionnairewere received, ofwhich 4100 were from clientsof one or more mental health professionals,family doctors, and support groups, and 2900were from specifically clients of a mentalhealth professional. The analysis and methodsto validate the data were discussed in theoriginal article in Consumer Reports ("MentalHealth," 1995) as well as by Seligman (1995,1996a, 1996b) and Kotkinet al. (1996). Detailsofthis analysis are discussed later in this articlein the section on "Sampling."

The Instrument and the MethodThe questionnaire used in the original

10

Consumer Reports study presented 35 ques­tions about the following factors: the kind oftherapist and therapy that were involved; thepresenting problems and the client's emotionalstate at the outset of therapy and at present;whether group or individual therapy was cho­sen; the duration and frequency of therapy,cost, health plan coverage (questions on insur­ance coverage were deleted for countries inwhich these questions were not relevant), andlimitations; how much the therapy helped invarious areas (e.g., with the original problem,in work and social domains, in personal do­mains); how the client evaluated therapistcompetence and therapy satisfaction; and thereasons for termination.

From the questions about the mental healthprofessionals and the results of therapy, thedata analysis involved a triple measure of ef­fectiveness based on answers to four questions.

One question measured specific improve­ment: "How much do you feel your therapyhelped you with the specific problem that ledyou to therapy?" (made things a lot better,made things somewhat better, made no dif­ference, made things somewhat worse, madethings a lot worse, not sure). The answers werescaled to yield scores of 100 to O.

A second question measured satisfaction:"Overall, how satisfied were you with thistherapist's treatment ofyour problems?" (com­pletely satisfied, very satisfied, fairly well satis­fied, somewhat dissatisfied, very dissatisfied,completely dissatisfied). Again the answerswere scaled to yield scores of 100 to O.

Two other questions measured global im­provement: how the clients described theiroverall psychological state at the time of thesurvey as compared to when they started treat­ment (very poor-I barely manage to deal withthings; fairly poor-life is usually pretty toughfor me; so-so-l have my ups and downs; quitegood-I have no serious complaints; very good-life is much the way I like it to be). Thedifferences in the answers to these before andafter questions were again scaled to give ascore of 0 to 100. For example: No changegave a score of 50, change from very poor tovery good gave a score of 100, change fromvery good to very poor gave a score ofO.

Transactional AnalysisJournal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OF TRANSACTIONAL ANALYSIS: AN INTERNATIONAL STUDY

The scores from questions one, two, andthree/four were summed to give an overallscore ranging from 0 to 300.

The question on specific improvement alsoincluded responses to measure improvement inwork and social domains and improvement inpersonal domains. These questions were scoredseparately as percentages of clients reportingthat "treatment made things a lot better."

Responses to questions about the duration oftherapy and the classification of the therapists,psychiatrists, psychologists, social workers,marriage counselors, and physicians allowedthe scoring for the Consumer Reports study tobe sorted for these therapy durations and thera­pist classifications. The Consumer Reports re­sults were presented and discussed by the Con­sumer Reports staff ("Mental Health," 1995)and Seligman (1995). The data were also re­produced by Novey (1999) and are presentedagain for comparison purposes in Figures 1through 5 of this article.

ProceduresThe study reported on here used the identical

section of the original Consumers Reportsquestionnaire referring to mental health exceptfor the questions on insurance coverage, which,as mentioned earlier, were deleted in countrieswhere they were not relevant. The question­naire was translated into three additionallanguages: Spanish, French, and Italian. A let­ter inviting participation in the study was sentto certified transactional analysts in Latin andCentral America, Spain, Australia, New Zea­land, Switzerland, France, Italy, the UnitedKingdom, the United States, and Canada. Ifthenumber of relevant therapists in a given areawas less than 40, each certified transactionalanalyst was sent an invitation. In countrieswhere there are much larger numbers of cer­tified therapists, groups of 40 were chosen byrandom computer selection for the mailings. Atotal of22 (in addition to the five original vol­unteers from the pilot study) certified trans­actional analysts volunteered to send the ques­tionnaire to clients they had seen during theprior five years. The original Consumer Re­ports study asked for responses to therapy overthe preceding three years. Given that therapy

Vol. 32. No. J. January 2002

effects tend to diminish with time, any effect ofthe difference in time can be expected to re­duce rather than increase the scores of the pre­sent study. Table 1 shows the distribution ofvolunteers and responses.

Table 1Distribution of Therapists and Returns

Country # Therapists # Returns

USA 10 416

Canada 2 90

Mexico 3 60

Spain 1 38

Australia 4 162

Switzerland 4 92

France 1 26

Italy 2 48

Totals 27 932

The volunteers were asked to mail the ques­tionnaires to clients they had seen over theprior five years with a letter saying that theyhad been invited to join in a research project tomeasure the effectiveness of transactionalanalysis. They asked the clients to fill out thequestionnaire and return it unsigned to a neutraladdress in that particular country. In a fewcases the return address was the therapist's of­fice. Clients were assured that their therapistwould not see their responses and were askedto answer as objectively as possible. All re­sponses were collected at each central returnaddress and shipped unopened to me for dataanalysis. Each therapist sent mailings to from30 to 150 clients seen during the prior fiveyears. Ifthe therapist saw more than 150 clientsduring this period, they were asked to mail to150 ofthem at random. The average return ratewas about 40%. Thirty-five questionnaireswere eliminated from analysis because theylacked a response to at least one crucial ques­tion. Analysis was done on 932 completedquestionnaires. The data were assembled into acomputer spreadsheet that could be used to

lJ

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

check and recheck for accuracy ofevaluationsand to carry out statistical evaluations

The scores were obtained using the exactscoring system used by the Consumer Reportsstaff. Student t tests and chi-squared tests wereused to evaluate the significance of the com­parisons with the ConsumerReportsdata as re­ported by Seligman (1995).

ResultsThe three subscales for specific improve­

ment, satisfaction, and global improvementwere evaluated by the identical scoring systemused by Consumer Reports. Each subscale wasscored on a 0-100 range, adding up to a totaloverall scale of0-300.

Figure 1shows the variation ofeffectivenesswith length of therapy for the Consumer Re­ports data (Seligman, 1995) and for the datafrom the present study of clients of certifiedtransactional analysts. The total number ofcli­ents for all groups reported by Consumer Re­ports was N = 2846. The total number ofclients

reported in the present study for transactionalanalysts is N = 932 (including the 248 clientsreported in an earlier pilot study [Novey,1999]). The overall ratings are plotted on the 0­300 scale defined earlier as a function of thelength of treatment. The results demonstrateclearly that longer-term therapy producesmore improvement than short-term therapy.The results are robust and hold up over allstatistical models. In addition, in each timecategory the results for transactional analystsare higher than the Consumer Reports resultsfor other professionals. A chi-squaredcomparison ofthe complete data set shows thatthe transactional analysis results are overallvery significantly higher (p much smaller than0.001 [p « .001], where p is the probabilitythat a repeat of the measurement would notproduce a significant difference between thetwo measurements). These results confirm theresults reported from an earlier pilot study(Novey, 1999) with data from 248 clientresponses.

270 -r---------------------~-

II Cons. Rep. • Trans.Anal.260 -j-----------------------.{

250-j-------------------e$240

1:!E230 -j----------r.;;;;J---.---.---

~[220 -t-------r:;:;:-;~---

.5210 -1---1

200

190

<=1Mo. 1-2 Mo. 3-6 Mo. 7-11 Mo. 1-2Yrs. >2Yrs.

Figure 1The Effect of Duration of Therapy

12 Transactional Analysis Journal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OFTRANSACTIONAL ANALYSIS: ANINTERNATIONAL STUDY

According to Seligman (1995), the Consu­merReports data showed that all mental healthprofessionals helped their patients more thandid family physicians who discussed mentalhealth problems with their patients. Within thegroups of mental health professionals studied,psychiatrists, psychologists, and social workerswere rated similarly and better than marriagecounselors. Their patients' overall improve­ment scores (0-300 scale) were 220, 226, 225(not significantly different from each other), re­spectively, with marriage and family counselorsrating 208 (significantly worse that the firstthree). From the data collected for this presentstudy, the average improvement score was 240for transactional analysts. This is significantlyhigher than the results for psychologists, psy­chiatrists, and social workers (p « 0.001).Interestingly, members of Alcoholics Anony­mous gave an even higher score (251) to theirself-help groups.

The Consumer Reports study also measuredself-reported improvement for presenting

symptoms, improvement in work and social do­mains, and improvement in personal domains.The results are shown in Figures 2, 3, and 4. Inthese figures, the number of clients reportingwere: psychiatrists, N = 639; psychologists, N= 1062; social workers, N = 381; marriagecounselors, N = 250; and physicians, N = 616.The number ofclients of transactional analystsreporting is N = 932.

Figure 2 shows the Consumer Reports datafor improvement on the presenting problem aspresented by Seligman (1995, Figure 2) withthe addition ofresults from the present study onimprovement on the presenting problem. Inagreement with the Consumer Reports data,treatment that lasted longer than six monthsresulted in considerable (40%) additional im­provement. The percentages of improvementon the presenting problem reported by clientsof transactional analysts is significantly higherthan the scores for the clients of any profes­sional reported on in the Consumer Reportsdata (p« 0.001).

70 -,-----------------------.,

• >6Mo.• <=6 Mo.

50-+--"

60-+-------------------

oPsychiatrists PsychologistsSoc.Workers Marr.Couns Physicians Trans.Anal.

Figure 2Improvement for Presenting Symptoms

Vol. 32. No. I. January 2002 13

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

Figure 3 shows the Consumer Reports dataas presented by Seligman (1995, Figure 3) forimprovement in three areas of work and socialdomains-ability to relate to others, produc­tivity at work, and coping with everyday stress-along with the results ofthe present study forimprovement in these same domains. Again,treatment lasting longer than six months re­sulted in considerable additional improvement,and the percentages of improvement in theseareas reported by clients of transactionalanalysts is significantly higher than the scoresreported for any other professional group fromthe Consumer Reports data (p < 0.001).

Figure 4 shows the Consumer Reports dataas presented by Seligman (1995, Figure 4) forimprovement in four areas ofpersonal domains-enjoying life more, personal growth and in­sight, self-esteem and confidence, and alleviat­ing low moods-along with the addition oftheresults ofthe present study for improvement inthese personal domains. Again, treatment last­ing longer than six months gave considerable

additional improvement, and the percentages ofimprovement in these areas reported by clientsof transactional analysts is significantly higherthan the scores reported for any other pro­fessional group from the Consumer Reportsdata (p« 0.001).

In the section in Seligman (1995) entitled"Consumer Reports Study: MethodologicalFlaws and Rebuttals," under the issue of sam­pling, he questions whether the benefits oflong-term treatment could be an artifact ofsam­pling bias. Is it possible that people who aredoing well in treatment selectively remain intreatment, and people who are doing poorlydrop out earlier? In other words, the early drop­outs are mostly people who fail to improve, butlater dropouts are mostly people whose prob­lems resolve. The Consumer Reports data dis­agree with this possibility empirically: Respon­dents reported not only when they left treat­ment, but why, including leaving because theirproblems were resolved. The dropout rates dueto the resolution of problems were uniform

50-.--------------------------.II <= 6 Mo. • >6Mo.

40 -+---------------------.--..-l

10

oPsychlatrlslB PsychologiBtB Soc.Workers Marr. Couns Physicians Trans.Anal.

Figure 3Improvement Over Work and Social Domains

/4 Transactional Analysis Journal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OFTRANSACTIONAL ANALYSIS; ANINTERNATIONAL STUDY

60 -.----------------------,

<= 6 Mo. >6Mo.

50 -1---------------------

C 40 -1------------------------Q)

E~2~30

...o.CU'::E 20'#.

10

oPsychlatrl81B PsychologlstBSoc.Workers Marr.Couns Physicians Trans.Anal.

Figure4Improvement Over Personal Domains

across duration of treatment (less than onemonth = 60%; 1-2 months = 66%; 3-6 months= 67%; 7-11 months = 67%; 1-2 years = 67%;over two years = 68%). The data in the presentstudy support this relative uniformity, with cor­responding dropout rates of 52%, 54%, 50%,38%,46%, and 46%, respectively.

As a check of the internal consistency of theresults of the present study, Figure 5 shows acomparison ofthe various improvement scoresbetween the combined data from the UnitedStates and Canada and the combined data fromthe other international regions. The scores forthe US/Canada clients (N = 506) and the non­US/Canada clients (N = 428) are not signifi­cantly different.

These results can also be compared to thosefrom a similar study carried out by Freedman etat. (1999) under the sponsorship ofthe Institutefor Psychoanalytic Research and Training(IPTAR) in New York. Their paper, entitled"The Effectiveness of Psychoanalytic Psycho­therapy: The Role of Treatment Duration,

Vol. 32, No. I. January 2002

Frequency of Sessions, and the TherapeuticRelationship," describes the use ofa question­naire and scoring system that are essentiallyidentical to those used in both the ConsumerReports study and in the study described in thisarticle. The Institute for Psychoanalytic Re­search and Training study is more analogous tothe one presented here in that the therapists allhad a similar training background and treatmentapproach, and the questionnaire was given toall of their clients, past and present. Two hun­dred and forty questionnaires were sent out.Ninety-nine (N = 99) were returned, a rate of41%. The results showed the same type of in­crease in effectiveness with length of therapy.The scores were not significantly different fromthose obtained in the Consumer Reports studyfor the groups of psychiatrists, psychologists,and social workers. The scores increase up toabout 220 for the long-term therapy greaterthan two years. They are significantly lowerthan the scores reported in the present study (p« 0.001).

15

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

80 -r-------------------.--,II <= 6 Mo. • >6 Mo.

70 -1----------------------1

SO-+--

...o'Cir3Q~

(/.20

10

oPresenting symptoms Work and Social Dom. Personal Domains

US/Canada Non- US/Canada Non- US/Canada Non-US/Canada US/Canada US/Canada

Figure 5Comparison of US/Canadawith Non-US/Canada Results

Methodology CheeksThere have been numerous discussions in re­

cent years of methodological advantages andpossible methodological flaws in the type ofre­search reported on here. Much of this discus­sion is summarized in the following sections.

Efficacy and EffectivenessMany questions arise incomparingeffective­

ness studies to efficacy studies, the latter ofwhich aim to provide more objective resultsthan those that primarily depend on subjectiveevaluations of individuals that cannot be mea­sured by external observers. In the end, how­ever, with regard to the positive results ofpsy­chotherapy, it is how clients feel about them­selves and their lives that is the measure ofsuccess. This is also true with efficacy mea­surements, even with all the controls and limits

16

applied by using specific techniques for a spe­cified length of time with specifically trainedtherapists, followed by comparisons with spe­cifically organized control groups. These testsin large part still rely on the responses of cli­ents about how they feel and what changes theyhave experienced.

Effectiveness measurements inactual therapyworking conditions can provide an average,over a large number of variables, by focusingon the self-evaluation of clients. It allows forthe existence ofmultiple problems and for self­correction in the therapy process (i.e., if oneapproach does not seem to be working, anothercan be chosen). The final question is, correctly,did clients get what they came to therapy for?

Strupp (1996) discussed the connections be­tween his tripartite model for determining ther­apeutic outcomes and the Consumer Reports

Transactional Analysis Journal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OF TRANSACTIONAL ANALYSIS: AN INTERNATIONAL STUDY

approach to outcome research. He acknow­ledged the contribution that effectiveness re­search can make to determining the contribu­tions ofpsychotherapy to positive outcomes inclients' lives. He emphasized the need for amodel for evaluating change based on positiveadaptive behaviors, positive sense of well be­ing, and positive personality structure changesas judged by a professional observer.

Hollon (1996) considered efficacy and effec­tiveness comparisons taking into account theeffects ofpharmacological treatment. He statedthat the Consumer Reports study emphasizedthe importance of defining outcomes by bothapproaches and that these comparisons alsohave important implications in the comparisonsof drugs and psychotherapy. Kriegman (1996)also considered the Consumer Reports study tobe enlightening and pointed out that the dataindicated a lack ofeffectiveness of medicationas compared to the effectiveness of psycho­therapy.

Hunt (1996) offered a critique ofSeligman'spresentation of data from the Consumer Re­ports study. He questioned whether the com­parison of improvement in less than six monthsas compared to longer-term therapy could beexplained by differing percentages of clientsseeing psychiatrists, psychologists, or socialworkers. He also suggested that the uniformityof effectiveness for these three groups oftherapists implied that the cost effectiveness oftherapy would be best if all clients were re­ferred to social workers. Seligman (1996a) re­sponded that the detailed data analysis did notshow that shorter- to longer-term improvementwas dependent on any difference in percentagesof this type. He also stated that the results didnot demonstrate better cost effectiveness forsocial workers because the study did not in­volve random assignment to various therapistgroups but reflected a nonrandom choice or re­ferral to social workers. In the study presentedin this article, all of the clients had chosen towork with a transactional analyst, and it there­fore measures the effectiveness of therapiststrained and certified in this modality. This be­gins to measure differences in effectivenessamong various types oftherapist training ratherthan differences in academic degrees.

Vol. 32, No. I, January 2002

Nathan (1998) and Nathan et at. (2000), whohave published extended reviews of efficacyand effectiveness measurements of psycho­therapy effects, also reviewed the ConsumerReports study. They raised a number of ques­tions concerning the reliability of the meth­odology, in particular, the sampling procedure,which will be discussed later in this articleunder "Sampling." Kotkin and Daviet (1999)gave additional supportive evidence for the re­liability of the Consumer Reports measure­ments based on their decades ofstatistical stud­ies in the consumer field. Clearly, furtherstudies and methodological clarification will berequired to provide general acceptance of ef­fectiveness studies.

Self-ReportsThe research staff of the Consumer Reports

organization has carried out both efficacy andeffectiveness types of research for many dec­ades and has found both types of studies to beuseful in evaluating a wide variety ofconsumerproducts. Also, the many polls taken almostdaily across the world on various political sub­jects to measure people's subjective opinionsare widely used to give direction to democraticsocieties and their governing bodies. Theremust be a great deal of merit to effectivenesstypes of measurements or they would havebeen discredited long ago.

Strupp (1996) supported the significance ofself-reports. He stated that if his tripartitemodel is to be taken seriously, then self-reportsreflect one of the perspectives and must betaken seriously, especially if robust improve­ment is reported, as was the case in the Con­sumer Reports study and in the study reportedon here.

Brock, Green, Reich, and Evans (1996) andBrock, Green, and Reich (1998) criticized theConsumer Reports study on the basis of its useofself-report data. They saw less validity in allself-reports because there was no random as­signment of clients in the Consumer Reportsstudy and also because they believe self-reportsare distorted by the desire of clients to validatetheir therapy. Mintz, Drake, and Crits­Christoph (1996) also questioned the validityof such effectiveness studies, suggesting that

/7

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

random assignment and the use of controlgroups are the essence of reliability in measur­ing the benefits of psychotherapy.

Seligman has responded to these criticisms.In his paper "Science as an Ally of Practice"(Seligman, 1996b), he stressed the importanceof both efficacy and effectiveness studies andof improving the methodology ofeffectivenessstudies. Efficacy studies also have their limi­tations in developing and applying long-termmanualized therapy (i.e., therapy inwhichprac­titioners use a manual of procedures) in prac­tice. Validation of longer-term therapy willneed to depend more on similar types of ef­fectiveness studies. In a comment entitled "ACreditable Beginning," Seligman (1996a) gaveadditional responses to the criticisms of Brocket al. (1996) and Mintz et al. (1996). He statedthat methodological problems exist in both ef­ficacy and effectiveness studies and call forcontinual study and improvement. He wrotethat results of effectiveness studies such asthose based on the ConsumerReports question­naire are more than consumer satisfaction stud­ies and that they do provide considerable statis­tical validity ofthe results oftherapy in generalclinical practice.

Personal opinions, which are labeled subjec­tive data, have always been considered lessreliable than objective data that can be obtainedfrom independent external observations of asystem. This latter approach is common in thephysical sciences in studies about inanimatenatural objects. In the social and biologicalsciences, in which human beings and other liv­ing things are being studied, there is a strongchance of reciprocal interaction, which caninfluence the measurements. As therapists, wemust rely mainly on clients' opinions abouttheir progress and success in therapy. If theysay that they feel better and that their lives aregoing better or that they are cured of a symp­tom that cannot be observed externally, then wehave little choice but to accept these opinionsas the best data available. Ofcourse, as with allmeasurements, it is important to look for sys­tematic influences that can distort the results(e.g., whether the clients may be attempting toplease their therapists with their responses tothe questionnaire).

18

The procedures for taking the measurementsin the present study were designed to avoid thiskind of systematic error. For example, clientswere asked to return the questionnaires un­signed and, except for a few cases, to a neutraladdress other than that of their therapist. In thefew cases in which the return address was thatof the client's therapist, the average scoreswere observed to be no different from those ofquestionnaires delivered to a neutral address.The letters asking the clients to fill out thequestionnaires informed them that the therapisthad been invited to participate in a researchproject and that they should fill out the ques­tionnaire as objectively as possible and withoutregard to the therapist's feelings. They wereassured that the therapist would not see theirresponses.

SamplingAnother possible source of systematic error

can arise from nonrandom sampling. Cansampling problems have distorted the data inthe Consumer Reports study or in the presentstudy? For the original Consumer Reportsstudy, the overall return percentage of ques­tionnaires was 12% (22,000 returns from180,000 mailed). Ofthese 22,000 respondents,7000 filled out the mental health part of thequestionnaire (3.9%), and ofthese 7000, about2900 (1.6%) talked to friends and family mem­bers or clergy. The balance of the 7000, 4100(2.3%), utilized mental health professionals,physicians, or support groups, and of these4100,2900 (1.6%) utilized a mental health pro­fessional.

Thus, of the total mailing of 180,000, 1.6%or 2900 filled out the mental health question­naire and had seen a mental health profes­sional. This low percentage has been criticizedas a source of systematic error (Brock et aI.,1996; Brock et aI., 1998; Nathan, 1998, 1999).The Consumer Reports staff was cognizant ofpossible problems in this area. To check thispoint, a separate data collection was carried outin which return postage was paid and the re­spondents were compensated. This resulted ina much larger (38%) response as compared tothe original 13% return. However, the scoresobtained from this quite different motivation

Transactional AnalysisJournal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OF TRANSACTIONAL ANALYSIS: AN INTERNATIONAL STUDY

procedure were not significantly different fromthose obtained from the original data collec­tion. Responses to the criticisms about samp­ling were made in detail by Seligman (1996a,1996b), Kotkin et al. (1996), and Kotkin andDaviet (1999). The Consumer Reports staff,being from long experience well aware of pos­sible sampling problems, did rechecks on theirsampling procedures as just mentioned andfound no differences to support the criticisms.They were aware of and acknowledged in theoriginal publication that because the readershipof Consumer Reports probably did not includepeople with severe mental problems, this factorcould be expected to lead to somewhat higherscores.

No such limitation existed in the presentstudy because either all of a therapist's clientsor a random sample of 150 clients of eachcertified transactional analyst studied wereincluded in the study. As noted earlier,effectiveness scores from the present studywere robustly higher than those obtained fromthe Consumer Reports subscribers. In thepresent study, return postage was paid andthere was no other compensation given. Thereturn percentages varied from 30% to 80%.Neither the questionnaires nor the enclosedstamped return envelope were identified by anylabel in order to allow for anonymity of theresponses. A small minority ofthe respondentssigned the questionnaires. An examination ofthe questionnaires and the responses ofpeoplefrom different countries and different languagegroups suggested that the clients took time andmade efforts to be complete, as objective asthey could be, and serious about supplying thisdata. They recognized that both the therapy thatthey had undergone and the research werebeing done to improve their lives and the livesofothers who might be involved in the therapyprocess.

The Consumer Reports study provided arelative measure of satisfaction for clients whosought treatment from psychiatrists,psychologists, social workers, marriage andfamily counselors, and physicians. The presentstudy gives a relative comparison to the resultsof the Consumer Reports study for thesevarious groups. An advantage of this type of

Vol. 32, No.1, January 2002

comparison is that errors due to any self­selection effect will tend to cancel out.

In the present research, a test for internalconsistency can be made by comparing thescores from a number ofnations or regions anda number of different language groups. Figure5 shows one such comparison. No significantvariation in scoring is observed for this case orfor intercomparisons between countries.

Control GroupsNo nontherapy control groups are used in

any of these studies, so the results cannot becompared to what would happen without treat­ment; however, the results for physicians, un­trained as therapists, can serve as an upper limitfor a no-therapy control group. The resultsshow the relative effectiveness of the variousgroups covered by the Consumer Reports dataand in the present study comparison to a set oftherapists trained in a specific therapist trainingprogram. As can be seen from the figures, thereis considerable variation in effectiveness asmeasured as a function of both therapy durationand therapist grouping.

Hollon (1996) discussed the importance ofcontrol groups and possible problems with ef­fectiveness measurements that do not use con­trol groups. This criticism was echoed byMintz et al. (1996) and Jacobson and Chris­tensen (1996), all of whom emphasized theneed for controls in order to assign improve­ment in a valid manner. Seligman (1996a,1996b) has discussed this criticism in detail. Hedoes not discount the value of control groupsfor efficacy studies in which the therapy pro­cess can be organized, manualized, timed, andevaluated by multiple means; however, he alsopointed out the value of effectiveness measure­ments in more open clinical practice. This in­cludes the original Consumers Report study aswell as the study presented here and the one byFreedman et al. (1999).

Seligman's main points are that while controlgroups are important, they also exact a price interms of research complications that may re­duce gains from studies. He states that controlreferences can also be obtained from internaldata analysis (e.g., the increase of scores withlength of therapy and the differences in scores

19

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

among various groups of therapists, both edu­cationallevel differences and therapy modalityand training differences). The significance ofvarious measurements can be seen from the ra­tios of scores and a scale of improvement de­rived from these ratios. In relation to the pre­sent study, it is clear, statistically, that theeffectiveness offamily physicians provides anupper limit to the effectiveness of nontreat­ment. Ifjust talking to someone or the passageof time was as good as highly skilled therapy,then we could expect that the results reportedfor clients of family physicians would be ashigh as those reported by clients of other pro­fessionals.

These results, with robust statistical validity,strongly indicate that all approaches to therapydo not lead to the same results. This contradictsthe dodo bird hypothesis, which suggests thateven conversation with a bird will lead to thesame improvement as therapy with a profes­sional therapist (Rosenzweig, 1936; Wampoldet aI., 1997). In the Consumer Reports study,marriage and family counselors and physiciansare judged to be less effective than psychia­trists, psychologists, and social workers. In ad­dition, as reported in the section on resultsearlier in this article, the average scores forthose involved in Alcoholics Anonymousgroups were higher than all of the other groups(251). In the present study, the average scoresfor certified transactional analysts were higher(241) than those for psychiatrists, psycholo­gists, and social workers as measured by theConsumer Reports study and higher than theaverage scores for psychoanalytic therapy asreported in the IPTAR study (209). This typeof study, then, supports the position that dif­ferent therapeutic modalities and/or levels oftraining will differ in terms of their effective­ness in assisting clients to resolve mental prob­lems.

The Therapists in This StudyThe importance of therapist competence and

the quality of the therapeutic relationship hasbeen discussed widely in the psychology litera­ture. Research on the efficacy or effectivenessof psychotherapy has continually faced theproblem of separating the effects of specific

20

therapy interventions and the effect of thetherapeutic relationship. Kazdin (\986) sur­veyed the methodological issues and strategiesin comparative outcome studies and in part fo­cused on therapist factors. He stated that theimpact of therapists as a group cannot be sepa­rated from treatment effects, that therapists andtreatment are confounded, and that treatmentdifferences can also be interpreted as therapistdifferences.

Strupp (1996)discussed the controversy con­cerning whether the trained therapist possessesunique skills. He concluded that while carefullyselected untrained persons can function effec­tively in the therapeutic role, the quality of thetraining of professional therapists is an im­portant factor to study.

Jacobson and Christensen (\996) claimedthat the Consumer Reports study confirmedwhat was already known: that increased ex­perience, skill, and education do not make forimproved therapy. They based their argumenton the results reported in their earlier paper(Christensen & Jacobson, 1993), in which theyclaimed that the data showed that nonpro­fessional therapists did as well as profession­ally trained therapists. Seligman (\996b) ex­pressed strong disagreement with this position.He claimed that Jacobson and Christensen's1993 results were based on case studies inwhich problems were mild and did not requiremuch therapeutic skill. In any event, the Con­sumer Reports data demonstrated a clear dif­ference in global improvement between clientsofpsychiatrists, psychologists, and social work­ers and those of marriage and family counse­lors and physicians. As mentioned earlier, theConsumer Reports studyacknowledged that thereadership of the magazine probably did notinclude people with severe mental problems. Itseems likely that more severe client problemswill require therapists to have higher qualityprofessional training and experience to be ef­fective. In the present study there was no se­lection made on the basis of client problemseverity. The results thus represent a normalrange of client problems from mild to severe.

The 27 therapists in the present study have avariety ofeducational backgrounds as psychia­trists, psychologists,social workers, counselors,

Transactional Analysis Journal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OF TRANSACTIONAL ANALYSIS: AN INTERNATIONAL STUDY

and related health professions. They have alsoundertaken long-term advanced training intransactional analysis and extensive supervisionof their work in the application of transactionalanalysis and other therapy modalities. Finally,they have been certified as transactional ana­lysts by successfully passing written and oralexaminations. These therapists are members ofthe same educational groups studied in theoriginal Consumer Reports study. They differfrom these groups in that they have all com­pleted an extended training process to becomecertified transactional analysts. It is possiblethat this training and experience factor alonecan account for higher client satisfaction.

Previous studies have not provided generallyaccepted evidence that any specific type oftherapy or training or any amount of therapistexperience has a significant effect on client sa­tisfaction. The Consumer Reports data does notdifferentiate for therapist training or experi­ence. The therapists were divided into educa­tionalldegree groups-psychiatrists, psycholo­gists, social workers, marriage and familycounselors, and physicians-withoutany speci­fication of the kind of therapist training, super­vision, and personal therapy they had received.Nevertheless, the results of the measurementsfor both the Consumer Reports study and thepresent one challenge the idea that different ap­proaches to therapy or therapist training andexperience have no effect on the effectivenessof treatment. The measurements for the undif­ferentiated groups of marriage counselors andphysicians showed significantly lower resultsthan those for undifferentiated groups of psy­chiatrists, psychologists, and social workers,and the results for those involved in AlcoholicsAnonymous groups demonstrated even higheraverage satisfaction, although these groups arenot specifically therapy groups.

The present research focuses on a moreclearly defined group of therapists as far astraining is concerned because all underwentsimilar training programs and, except for oneperson, similar certifying examinations. It alsoprovides a model for comparing groups oftherapists who have followed training programsin other modalities. The indications of thepresent measurements are that the clients of

Vol. 32. No.1. January 2002

transactional analysts are likely to be moresatisfied with their therapy in comparison togroups oftherapists identified only by commonacademic degrees who have probably experi­enced a wide range oftherapist training, super­vision, and personal therapy.

The results of the present study, which indi­cate a higher level of effectiveness for thetransactional analysts, are based on client self­evaluations; however, as discussed in the be­ginning of this article, client satisfaction is theultimate goal of therapy and thus is a realisticmeasure from the client's point of view. Sincethere is no indication of distortions due tosampling nonuniformities, this provides va­lidity for intercomparisons ofany given group­ing of therapists.

DiscussionThere are two main possibilities for explain­

ing the results of this research. First, there issomething unique about the transactional analy­sis approach that provides better results. Sec­ond, there is something unique about training intransactional analysis that results in more ef­fective therapists.

With regard to the first hypothesis, there aresome special foci to the transactional analysisapproach to therapy: Clients are given infor­mation in a relatively simple nonmedical lan­guage to help them understand their problemsbetter. Therapists use a contractual approach totherapy. Safety and protection for the client anda strong focus on the establishment of a warmand accepting client-therapist relationship areemphasized. Transactional analysts, however,are generally quite eclectic in their approaches.Transactional analysis can be seen primarily asa cognitive-behavioral therapy combined, whenuseful, with script analysis, which is similar tothe psychodynamic approach to insight-basedtherapy. Workers in this discipline usuallyintegrate many therapeutic approaches in theirwork and in general pursue goals similar tomany other therapeutic modalities. Indeed,training and certification programs in trans­actional analysis require knowledge ofand theability to use a variety of methods. Transac­tional analysts generally use a combination ofaffective, behavioral, cognitive, and

21

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

physiological approaches to psychotherapy de­pending on the needs ofand most effective ap­proach with any particular client (Erskine &Moursund, 1988; Goulding & Goulding, 1979;James, 1981; Stewart, 1996; Stewart & Joines,1987).

A more likely factor in explaining the resultsof the present study is the training and certi­fication of transactional analysts. The trainingand certification program is quite extensive andrequires a number of years to complete. Moreimportantly, it requires a combination of per­sonal therapy and supervision unlikely to befound in academic degree programs, which arelimited in the levels of successful therapy thatcan be required of students. It is not likely thata student who has passed the academic re­quirements ofa course ofstudy will be refusedgraduation because of unresolved personal is­sues. In the transactional analysis training pro­gram, however, the student's supervisor and theexamining board pay close attention not only tothe achievement of academic knowledge butalso to the resolution ofpersona I issues that canlead to countertransference problems in therapyand interfere with the therapist's potency Thetransactional analysis training program is de­scribed in detail in the Transactional AnalysisCertification Council Manual, which is uni­formly applied in the United States and inter­nationally (Training and Certification Councilof Transactional Analysts, 1997).

The training program involves a tutorial typeofeducation in which one supervisor contractswith the trainee to guide and supervise his orher theoretical and practical experience. Thisincludes making sure the trainee obtains anynecessary therapy to ensure that he or shelearns how to protect and take care of both theclient and himself or herself as well as to beaware of and avoid interfering countertrans­ference that can bring the therapist's personalissues into the therapy process. Some super­visors combine therapy with supervision if thatseems appropriate and boundary problems donot interfere, while others separate therapyfrom supervision. Most, however, use a peer­group form of training in which members of agroup oftrainees take turns at being clients andtherapists for each other under supervision ofa

22

Teaching and Supervising Transactional Ana­lyst over an extended period oftime (generally2-5 years, depending on the depth and persis­tence of the trainee's personal issues).

The peer training group is an extraordinarytraining experience. The trainees in peer groupsreceive immediate feedback from the super­visor and other group members. This allowshere-and-now emphasis on therapy issues thatarise for the therapist and immediate thera­peutic focus on the issue or a referral to ongo­ing personal therapy to resolve the problem.There is also a significant emphasis on tape su­pervision in the peer group and other supervi­sion work. Trainees present sections of tapedclient sessions in order to obtain supervisionand information on therapeutic options andproblems.

Thus, in the training oftransactional analyststhere is a strong emphasis on the personal de­velopment of the therapist. This often requiresconsiderably more time than the accumulationof theoretical knowledge. Ultimately, bothknowledge and therapeutic ability are tested inthe examination processes. Theoretical knowl­edge is tested through one of several writtenexaminations used internationally. Therapeuticeffectiveness is tested by an oral examinationcarried out at national and international con­ferences by examination committees consistingofcertified transactional analysts trained in theexamination process. A number ofqualities areexamined and scored in the oral examinations(which typically require about an hour). Theseinclude: knowledge of theory, various transac­tional analysis approaches, treatment direction,awareness ofdiscounts and incongruities, clar­ity ofclient assessment, potency, effectiveness,professionalism, intuition and creativity, andoverall rating as a therapist. Samples of tapedtherapy sessions are presented during this ex­amination so that the trainee's actual perfor­mance can be evaluated by the members of theexamination board.

A second level of training, which involvesfurther training, supervision, and examination,is available for those who wish to be certifiedas Teaching and Supervising TransactionalAnalysts (Training and Certification Council ofTransactional Analysts, 1997). This requires a

Transactional Analysis Journal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

MEASURING THE EFFECTIVENESS OF TRANSACTIONAL ANALYSIS: AN INTERNATIONAL STUDY

further commitment offrom three to five yearsof additional training in teaching and supervi­sion processes and an additional set of threeone-hour examinations in theoretical knowl­edge, teaching ability, and supervision ability.Ability in supervision is tested in the oral ex­amination by means oftwo live demonstrationsof supervisory capability.

The transactional analysis system oftrainingfocuses on producing competent therapists andcompetent teachers and supervisors in a uni­form and reproducible training process. This isnot necessarily true for training obtained foracademic degrees. For example, psychiatristshave varied clinical training, some much morehighly focused on psychopharmacological ap­proaches than psychotherapy. This can also betrue for psychologists and social workers, forwhom training may vary widely from testingprocesses and social work processes to therapyprocesses. If it were possible in the ConsumerReports study to select from the groups ofpsy­chiatrists, psychologists, and social workersthose who were specially trained and examinedfor competency as therapists, the results couldbe expected to be quite different.

With regard to the effectiveness of transac­tional analysts, one could say that the good re­sults do not depend on their specific use oftransactional analysis methods but rather ontheir human assets of warmth, interest, and un­derstanding. However, I contend that if theseare the qualities that lead to highly competenttherapy, that they are specifically developed inthe training of transactional analysts. Duringthis training there is a strong emphasis on thepersonal development of the therapist as wellas on accumulating a wide variety oftherapeu­tic approaches, the better to fit the wide varietyof client mental systems encountered in prac­tice.

Summary and ConclusionsThis article presents the results ofan interna­

tional study of clients of 27 certified trans­actional analysts in eight countries (the Unitedstates, Canada, Mexico, Spain, Australia,Switzerland, France, and Italy) using identicalquestionnaires in four languages (English,Spanish, French, and Italian). A total of 932

Vol. 32. No.1. January 2002

questionnaires were scored to provide the basicdata.

Based on the data collected in this study, it ispossible to conclude that the effectiveness oftherapy as measured by client satisfaction issignificantly greater for certified transactionalanalysts than it is for any of the professionalgroups identified by Seligman from the Con­sumerReports database (i.e., psychiatrists, psy­chologists, social workers, marriage and familycounselors, and physicians). The results alsoconfirm the results presented by Consumer Re­ports and Seligman that longer-term therapy(>6 months) is more effective than shorter-term«6 months) therapy. In addition, the resultsshow a higher level ofeffectiveness than the re­sults reported for a group of psychoanalyticalpsychotherapists in the IPTAR study.

Another important conclusion can be drawnfrom using the questionnaire developed by theConsumer Reports staff with Martin Seligmanas consultant. From as little as 250 client re­sponses, any specific group of psychotherapistscan now determine their relative effectivenessas compared to the large database of practi­tioners collected by the Consumer Reports pro­ject and this and other similar research studies.

This thus offers a low-cost and relativelysimple means to compare various approachesto psychotherapy, various types of therapisttraining programs, and various types of aca­demic education programs. Statistically signi­ficant results can be relatively easily obtainedand can provide an actual measure of the ef­fectiveness ofpsychotherapy as determined byclient satisfaction.

Theodore B. Novey, Ph.D. (Physical Chem­istry), MS. (Counseling Psychology), is aTeaching and Supervising Transactional Ana­lyst in clinical, educational, and organiza­tional specialties. He is in private practice inGlenview, Illinois, and is past editor of theTransactional Analysis Journal. Please sendre­print requests to Theodore B. Novey, 815Indian Road, Glenview, IL 60025, U.S.A., oremail Tnovey@Compuserve.com .

AcknowledgmentsThe author wishes to acknowledge the

23

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from

THEODORE B. NOVEY

cooperation and assistance of Mark Kotkin,Ph.D., who directed the ConsumerReportspro­ject, and Martin Seligman, Ph.D., consultantfor the project. He also gratefully acknowl­edges the assistance of certified transactionalanalysts around the world for their help in ob­taining the client data. In particular, he thanksthe following therapists who shared in the workof organizing the research in their countries:Linda Gregory in Australia, Gloria Noriega inMexico, Paola Nadas in Switzerland, IsabelleCrespelle in France, and Marco Sambin inItaly. The author also wishes to thank MervynHine, Ph.D. and Charles Strom, M.D., Ph.D.,for their assistance with the data analysis andthe statistical evaluations. This research wassupported by a grant from the Eric Berne Fundfor the Future.

REFERENCESAnnual questionnaire. (1994). Consumer Reports.Brock,1. c.,Green, M. c., & Reich, D. A. (1998). New

evidenceof flaws in the consumer reports study of psy­chotherapy. American Psychologist, 53(1),62-63.

Brock, T. C., Green, M. C., Reich, D. A., & Evans, L. M.(1996). The consumer reports study of psychotherapy:Invalid is invalid. American Psychologist, 51(10),1083.

Christensen, A., & Jacobson, N. (1993). Who (or what)can do psychotherapy:The status and challengeof non­professionaltherapies. Psychological Science, 5, 8-14.

Clarke, G. N. (1995). Improvingthe transition from basicefficacy research to effectiveness studies: Methodo­logical issues and procedures. Journal of Consultingand Clinical Psychology, 63(5), 718-725.

Erskine, R. G., & Moursund,J. P. (1988). Integrative psy­chotherapy in action. Newbury Park, CA: Sage.

Freedman,N., Hoffenberg,1. D., Vorus, N., & Frosch, A.(1999). The role of treatment duration, frequency ofsessions, and the therapeutic relationship. Journal ofthe American Psychoanalytic Association, 47(3),741­772.

Goulding, M. M., & Goulding, R. L. (1979). Changinglives through redecision therapy. New York: Brunner!Mazel.

Hollon, S. D. (1996). The efficacy and effectiveness ofpsychotherapy relative to medications. American Psy­chologist, 51(10), 1023-1030.

Howard, K. L., Kopta, S. M., Krause, M. S., & Orlinsky,D. E. (1986). The dose-effect relationship in psycho­therapy. American Psychologist. 41(2), 159-164.

Howard,K. L.,Moras, K., Brill, P. L.,Martinovitch, Z., &Lutz, W. (1996). Evaluation of psychotherapy: Effi­cacy, effectivenessand patient progress.American Psy­chologist, 51(10), 1059-1064.

Hunt, E. (1996). Errors in Seligman's "The effectivenessof psychotherapy: The consumer reports study."American Psychologist, 51(10), 1082.

24

Jacobson, N., & Christensen, A. (1996). Studying the ef­fectiveness of psychotherapy: How well can clinicaltrials do thejob? American Psychologist, 51(10),1031­1037.

James, M. (1981). Breakingfree: Self-reparentingfor anew self. Reading, MA: Addison-Wesley.

Kazdin, A. E. (1986). Comparative outcome studies ofpsychotherapy: Methodological issues and strategies.Journal ofConsulting and Clinical Psychology. 54(1),95-105.

Kotkin, M., & Daviet, C. (1999). Mistaken inferences.American Psychologist, 54 (12), 1130.

Kotkin, M., Daviet, C., & Gurin.J, (1996). The consumerreports mental health survey. American Psychologist.51(10),1080-1082.

Kriegman, D. (1996). The effectiveness of medications:The consumer reports study. American Psychologist,51(10),1086.

Mental health: Does therapy help? (1995). Consumer Re­ports, 60(11),734-739.

Mintz, J., Drake, R. E., & Crits-Christoph, P. (1996).Efficacyand effectivenessof psychotherapy:Two para­digms, one science. American Psychologist. 51(10),1084-1085.

Nathan, P. E. (1998). Not yet ideal. American Psycholo­gist, 53(3),290-299.

Nathan, P. E. (1999). Too many assumptions. AmericanPsychologist. 54(12), 1130-1131.

Nathan, P. E., Stuart, S. P., & Dolan, S. I. (2000). Re­search on psychotherapy efficacy and effectiveness:BetweenScyllaand Charybdis?Psychological Bulletin.126(6),964-981.

Novey,T. (1999).The effectivenessoftransactional analy­sis. Transactional Analysis Journal, 29, 18-30.

Rosenzweig,S. (1936). Some implicit common factors indiversemethodsinpsychotherapy.American Journal ofOrthopsychiatry. 6,412-415.

Seligman,M. (1995). The effectivenessof psychotherapy.American Psychologist, 50(12),965-974.

Seligman, M. (1996a). A creditable beginning. AmericanPsychologist. 51(10),1086-1088.

Seligman, M. (I 996b). Science as an ally of practice.American Psychologist, 51(10), 1072·1079.

Stewart, I. (1996). Developing transactional analysiscounseling. London: Sage Publications.

Stewart, I., & Joines, V. (1987). TA today: A new intro­duction to transactional analysis. Nottingham, Eng­land, and Chapel Hill, NC: Lifespace Publishing.

Strupp, H. H. (1996). The tripartite model and the con­sumer reports study. American Psychologist, 51(10),1017-1024.

Training and Certification Council of Transactional Ana­lysts. (1997). The transactional analysis certificationcouncil manual (rev.). San Francisco: Author.

Vandenbos, G. R. (1996). Outcome assessment of psy­chotherapy. American Psychologist. 51(10), 1005­1006.

Wampold, B. E., Mondin, G. W., Moody, M., Stich, F.,Benson, K., & Ahn, H. N. (1997). A meta-analysis ofoutcome studies comparing bona fide psychotherapies.Empirically, "All must have prizes." PsychotherapyBulletin, 122,203-215.

Transactional Analysis Journal

at Bobst Library, New York University on June 2, 2015tax.sagepub.comDownloaded from