ARTICLE · 2012-07-24 · PSYCHIATRIC REHABILITATION JOURNALAn Update on Randomized Controlled...
Transcript of ARTICLE · 2012-07-24 · PSYCHIATRIC REHABILITATION JOURNALAn Update on Randomized Controlled...
Psychiatrie Rehabilitation Journal2008, Volume 31, No. 4, 280-290
Copyright 2008 Trustees of Boston University
DOl: 10.2975/31.4.2008.280.290
ARTICLE
An Update onRandomized Controlled Trials
of Evidence-BasedSupported Employment
V
Gary R. Bond
Indiana University-Purdue University
Indianapolis
Robert E. Drake and
Deborah R. Becker
Dartmouth Medical School
Background: The Individual Placement and Support (IPS) model of supported em-
ployment for clients with severe mental illness has been described as a standardi-
zation of evidence-based supported employment. Although several reviews on the
literature on its effectiveness have been conducted, the completion of several new
studies suggests an updated review is warranted.
Methods: We conducted a comprehensive literature search for randomized con-
trolled trials of IPS, limiting our review to programs with high-fidelity IPS pro-
grams, locating li studies. We examined the following competitive employment
outcomes: employment rates, days to first job, annuatized weeks worked, and job
tenure in longest job held during the follow-up period.
Findings: Across the 11 studies, the competitive employment rate was 61 % for IPS
compared to 23% for controls. About two-thirds of those who obtained competitive
employment worked 20 hours or more per week. Among those who obtained a
competitive job, IPS participants obtained their first job nearly lo weeks earlier
than did controls. Among IPS participants who obtained competitive work, dura-
tion of employment after the start of the first job averaged 24.2 weeks per year, or
47% of the 52-week year.
Conclusions: The current review is consistent with earlier reviews, although the evi-
dence for high-fidelity IPS appears to be somewhat stronger here than in reviews
evaluating studies with more heterogeneity in the supported employment models
examined. The number, consistency, and effect sizes of studies of evidence-based
supported employment establish it as one of the most robust interventions avail-
able for persons with severe mental illness.
Keywords: supported employment, employment, evidence-based practices, ran-domized controlled trials
280
SPRING 2008—VOLUME 31 NUMBER 4
Introduction
1 he Individual Placement and Support(IPS) model of supported employmentfor clients with severe mental illnesshas been described as a standardiza-tion of evidence-based supported em-ployment (Bond et al., 2001). It hasbeen so identified because it is thebest-described (Becker & Drake, 2003)and most extensively researched modelof supported employment for this popu-lation. The core principles of this modelare (1) a focus on competitive employ-ment, (2) eligibility based on consumerchoice, (3) rapid job search, (4) integra-tion of mental health and employmentservices, (5) attention to consumer pref-erence in the job search, (6) individual-ized job supports and (7) personalizedbenefits counseling (Bond, 2004).
Starting in the late 1990s, the literatureon supported employment for individu-als with severe mental illness has beenreviewed on numerous occasions(Bond, 2004; Bond et al., 2001; Bond,Drake, Mueser, & Becker, 1997; Burnset al., 2007; Crowther, Marshall, Bond,& Huxley, 2001; Twamley, Jeste, &Lehman, 2003). The evidence from ran-domized controlled trials (RCTs) contin-ues to accumulate quickly, and reviewsfrom only a couple years ago are al-ready obsolete. The purpose of the cur-rent review is to provide acomprehensive summary of competi-tive employment outcomes for RCTsevaluating evidence-based supportedemployment for this population. A sec-ond difference from earlier reviews isthat we restricted our review to evalua-tions of programs with documented ad-herence to the aforementioned IPSprinciples. Our rationale was based onthe literature suggesting that fidelity toIPS is associated with better competi-tive employment outcomes (Becker,Smith, Tanzman, Drake, & Tremblay,2001; Becker, Xie, McHugo, Halliday, &Martinez, 2006; Gowdy, Carlson, &
Rapp, 2003; McGrew & Griss, 2005;
McGrew, 2007).
Methods
Study Inclusion Criteria
To be included in this review, a studywas required to be a randomized con-trolled trial design examining longitu-dinal competitive employmentoutcomes for individuals with severemental illness in which participantswere randomly assigned to two or moreconditions, one of which used a high-fidelity IPS supported employmentmodel. IPS has been well described inthe literature (Becker & Drake, 2003),and a psychometricatly validated fideli-ty scale has been developed to deter-mine which programs achieve highfidelity (Bond, Becker, Drake, & Vogler,1997). A further requirement for inclu-sion in the review was that the controlgroup or groups must have received ei-ther services as usual or some otherform of vocational rehabilitation be-sides IPS.
Literature Search Strategies
Three main sources were used to iden-tify studies. The first was to draw onpublished literature reviews (Bond,2004; Bond et al., 2001; Crowther etal., 2001; Twamley et al., 2003). Thesecond was to examine each of thestudies in the Employment InterventionDemonstration Project (EIDP) (Cook etal., 2005). The third was through con-tacting principal investigators and con-tinuous review of the publishedliterature.
Sample
The 11 studies included in the currentreview are shown in Table 1, sequencedaccording to date of publication. Tenstudies have been published in peer-reviewed journals, one is in press(Killackey, Jackson, & McGorry, inpress). One study aimed at youngadults with schizophrenia comparing
IPS to services as usual was not includ-
ed because the results have not yet
been reported (Nuechterlein et al.,
2005; Nuechterlein, Subotnik, Turner,
Ventura, Becker, & Drake, this issue).
Ten of the 11 studies used a 2-groupdesign (IPS versus control); theConnecticut study compared IPS to 2control groups (Mueser et aL, 2004).The number of IPS sites ranged fromone in 8 studies, two in 2 studies(Bond et al., 2007; Drake, McHugo,Becker, Anthony, & Clark, 1996), andsix in 1 study (Burns et aL, 2007). Interms of control groups, two studiesused nonintegrated supported employ-ment control groups (Drake et aL,1996; Mueser et al., 2004). Otherwise,all of the control groups consisted ofeither treatment as usual (typically re-ferral to the state vocational system orequivalent) or alternative vocationalmodels. In every study, high fidelity toIPS was ensured through systematicmonitoring using the IPS Fidelity Scale(Bond, Becker et al., 1997). Many otherdetails of the study protocols, includ-ing sample inclusion criteria and meas-urement batteries, were the same orsimilar across studies. In all the stud-ies, participants were adults who meteach state's or province's criteria forsevere mental illness, typically aDiagnostic arid Statistical Manual ofMental Disorders—4th Edition(American Psychiatric Association,1994) Axis I or II diagnosis plus severeand persistent impairment in psy-chosocial functioning. In most studies,participants were recruited from men-tal health centers. In all the studies,participants were unemployed at thetime of study admission. In all but onestudy, the study inclusion criteria in-cluded an expressed desire to work;the single exception was the Marylandstudy (Lehman et al., 2002). Anothereligibility criterion common acrossmost studies was the absence of signif-icant medical conditions that would
PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials of Evidence-Based Supported Employment
TABLE I -RANDOMIZED
WITH SEVERE
study
Drake et al. (1996)
Drake et al. (1999)
Lehman et al. (2002)
Mueser et al. (2004)
Gold et al. (2006)
Latimer et al. (2006)
Burns et al. (2007)
Bond et al. (2007)
Wong et al. (2008)
Twamleyetal.(2008)
Killackey et al. (in press)
CONTROLLED TRIALS OF INDIVIDUAL PLACEMENT AND
MENTAL ILLNESS
Control Condition Study Population
& Salient
Eligibility Criteria
Skills training,
nonintegrated
Sheltered workshop
PSR
(1) Brokered SE; (2) PSR
Sheltered workshop
Traditional vocationalservices
Traditional vocationalservices
Diversified placementapproach
Stepwise conventionalvocational services
VR referral
Traditional vocationalservices
CMHC clients
Case management
program clients
CMHC clients, includingthose without voc goals
CMHC clients
CMHC clients
Clients receiving mental
health services
Clients receiving mentalhealth services
New admissions toPSR agency
Outpatients at hospitaloccupational therapyprogram
Middle aged and
older adult
Early psychosis program
SUPPORT FOÜ
study Site
Location
NH
DC
MD
t INDIVIDUALS
Months of
Follow-up
18
18
24
CT 24
SC 24
Québec, Canada 12
6 European
cities
IL
HK
CA
Melbourne,Australia
18
24
18
12
6
Label
96 NH
99 DC
02 MD
04 CT
06 SC
06 QUE
07 EUR
07 IL
08 HK
08 CA
08 AUST
Acronyms: SE = supported employment; IPS = Individual Placement and Support; ACT = assertive community treatment;PSR = psychosocial rehabilitation; VR = State-federal vocational rehabilitation system; CMHC = community mental health center;ICCD = International Center for Clubhouse Development
Note: Two control groups in the Mueser et al. (2004) study combined into a single control group in the tables and figure to follow.
preclude working or participating in as-sessment interviews. In many of theIPS studies, participants were requiredto attend multiple research informationmeetings in which the project was ex-plained before giving consent to partic-ipate in the study.
Excluded Randomized Controlled Trials
Excluded RCTs fall into 3 categories:RCTs evaluating supported employ-ment prior to the development of theIPS model; recent supported employ-ment RCTs evaluating a different model
than IPS or a program for which fidelityof IPS is uncertain; and RCTs compar-ing an enhanced form of IPS with IPSwithout an enhancement.
Five "pre-IPS" RCTs were found. Inevery case they evaluated a supportedemployment model falling short on oneor several IPS criteria. Four of these(Bond, Dietzen, McGrew, & Miller, 1995;Chandler, Meisel, Hu, McGowen, &Madison, 1997; Gervey & Bedell, 1994;McFarlane et al., 2000) were examinedin earlier reviews (Bond, 2004; Bond et
al., 2001). One comparing the Programof Assertive Community Treatment(PACO to usual services has never beenpublished (Test, 1992; Test, Allness, &Knoedler, 1995).
The second category consists of 4 recentRCTs that either have evaluated a differ-ent model of supported employment(Macias et al., 2006; Rogers, Anthony,Lyass, & Penk, 2006) or an approach inwhich the fidelity to the IPS model wasunknown or uncertain (Shafer, 2005;Tsang, 2006). The specifics of these 4studies are as follows:
SPRING 2008—VOLUME 31 NUMBER 4
The Massachusetts EIDP study findingsare reported in two papers (Macias etal., 2oo6; Schonebaum, Boyd, & Dudek,2oo6). The former paper uses the stan-dardized definition of competitive em-ployment and limits the analysis toparticipants enrolling in the study withan avowed interest in working. The lat-ter paper uses the full randomized sam-ple with an expanded definition ofcompetitive employment. The reportedexperimental differences given in twopapers are fairly similar. This study eval-uated a newly established PACT team inwhich a supported employment positionwas created. Although in many respectsadhering to IPS principles, the role de-scription for the supported employmentposition followed the PACT model inwhich the employment specialist hassignificant clinical duties, which com-promises model effectiveness.
Although the "choose-get-keep" modeldeveloped by Anthony and his col-leagues (Danley & Anthony, 1987) wasoriginally viewed as a kind of support-ed employment model, in a recently re-ported randomized controlled trial ofthis model (Rogers et al., 2006), theauthors label their approach "psychi-atric vocational rehabilitation." Thechoose-get-keep model differs in im-portant respects from the IPS model,centering on its emphasis on prevoca-tional career planning.
The Shafer EIDP study, which has notbeen published, compared a support-ed employment approach to servicesas usual (Shafer, 2005). The investiga-tors did not specifically monitor theirservices to achieve high fidelity to IPS.We have been unable to obtain a finalreport of the study.
The study conducted by Tsang et al.(2006) had two experimental conditions:one was IPS only and the second was IPSplus skills training. These two experi-mental conditions were compared to atreatment-as-usual control group.
Although the authors state that they im-plemented IPS with high fidelity, the crit-ical ingredients of the model were notfollowed. In addition, the findings fromthis study are not usable in the currentreview, because their employment out-come measures combined competitiveemployment and what the report labels"partially competitive employment."
The third category of excluded studiesconsists of RCTs comparing IPS to anenhancement of IPS. Two studies fromthe EIDP (Maine and Texas studies)compared supported employment to anenhanced form of supported employ-ment and would be excluded for thisreason (although in neither study didthe investigators closely adhere to theIPS model, so they would be excludedfor that reason as well). Since the com-pletion of the EIDP, there have been agrowing number of studies comparingsupported employment to supportedemployment plus an enhancement.Enhancements have included bothskills training (Mueser et al., 2005;Wallace & Tauber, 2004) and cognitivetraining strategies (Bell, Greig, Zito, &Wexler, 2007; McGurk, Mueser, &Pascaris, 2005). While these studies areanswering important questions aboutsupported employment, they lie out-side the scope of the current review,which compares supported employ-ment to other vocational approaches.
Outcome Measures
This review focuses exclusively on com-petitive employment outcomes, de-fined as jobs paying at least minimumwage in integrated community settings(i.e., employing nondisabled workers)and are jobs that anyone could hold,not just individuals with disabilities. Bythis definition, protected jobs, such astransitional employment (Propst,1992), sheltered employment, andother set-aside jobs (Black, 1988) wereexcluded.
All studies reported competitive em-ployment rates, defined as working acompetitive job at any time during fol-low-up. Several other competitive em-ployment outcome measures were alsoexamined. When reported, we com-piled the following outcomes: days tofirst job (i.e., time from study entry tofirst job start), annualized weeksworked (that is, number of weeksworked per year; dividing total weeksworked by 1.5 for studies with 18-month follow-up and dividing totalweeks work by 2.0 for studies with 2-year follow-up), and job tenure (i.e.,weeks worked) in longest job held dur-ing the follow-up period. Oob tenure ismeasured from job start date. The enddate is determined from either job end-ing date or end of follow-up, whichevercomes first.) As seen below, informa-tion was not available on these vari-ables for all 11 IPS studies.
For the first measure of duration of em-ployment—weeks worked per year—wecalculated the findings for both the fullintent-to-treat sample (everyone en-rolled in the study) and the workersample only (those who obtained atleast one competitive job during fol-low-up). We calculated weeks workedin a third way as well: weeks workedafter obtaining first competitive job.Each of these methods for estimatingduration of employment assesses it ina different way.
Data Analyses
Data were either recorded directly frompublished reports or hand calculatedfrom information presented in these re-ports. As noted above, the Connecticutstudy was the only study included withtwo control groups (Mueser et al.,2004). Campbell (2007) examined ef-fect sizes for the differences on 3 com-petitive employment outcomesbetween the two control groups in thisstudy and found the effect sizes for thedifferences to be small. Based on these
PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials of Evidence-Based Supported Employment
FIGURE I-COMPETITIVE EMPLOYMENT RATES IN l i RANDOMIZED
CONTROLLED TRIALS OF INDIVIDUAL PLACEMENT AND SUPPORT
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
78%7
40%
5% 74%70%
34%28%
6
29%
1
51° 64%
10%
26%
19%
-
5
i1
b%47%
28% 1 27%
1 | i 8 % 1
1 1»1 196 NH 07 IL 04 CT 08 HK 08 AUST 06 SC 99 DC 08 CA 07 EUR o< QUE 03 MD
• IPS nControl •Control2
findings, we collapsed the outcomes
for the two Connecticut control groups
into a single control group for all em-
ployment outcomes.
Effect size for each study for the differ-ence in employment rate between sup-ported employment and controls wascalculated using the arc sine approxi-mation (Lipsey, 1990). An unweightedoverall effect size was calculated as thesimple mean of the individual effectsizes. For all other outcome measures,means are reported without standarddeviations, which are not availablefrom the published studies. Overallmeans were calculated weighting indi-vidual means by sample sizes.
Effect sizes were also estimated forpublished review articles using the arcsine method on overall percentageworking. It should be noted that thismethod of estimation gives slightlydifferent estimates than the unweight-ed overall effect size method de-scribed above.
Results
Competitive employment rates. In all l istudies, the competitive employmentrate was significantly higher for the IPScondition than for controls, as shownin Figure l . Averaging the rates acrossstudies, the competitive employmentrate was 61% (Median = 64%) for IPScompared to 23% (Median = 27%) forcontrols. The average difference in per-centage employed between supportedemployment and controls was 38%,ranging from 20% to 55%. The individ-ual study effect sizes ranged from .56to 1.23. The overall unweighted effectsize was .83.
Using a stem-and-leaf diagram (Tukey,
1977), the Maryland IPS sample
(Lehman et al., 2002) was a statistical
outlier, falling more than the interquar-
tile range (19%) below the lower quar-
tile (55%) in the distribution of
competitive employment rates. With
this outlier removed, the mean compet-
itive employment rate for IPS was 65%.
The next two lowest employment rates
were for two non-U.S. studies in Canada
(Latimer et al., 2006) and Europe (Burns
et al., 2007). However, two other non-
U.S. studies were above the median
(Killackey et al., in press; Wong, Chiu,
Tang, Mak, Liu, & Chiu, 2008).
Other competitive employment out-comes. Four IPS studies reported thefrequencies of participants whoworked 20 hours or more per week, asshown in Table 2. Aggregating acrossthese studies, 134 (43.6%) of 307 IPSparticipants and 53 (14.2%) of 374 con-trols held such jobs, yielding an effectsize of .67.
Days to first competitive job was re-ported in 7 IPS studies, as shown inTable 3. Overall, the average time was50% faster in first competitive job forIPS compared to controls (138 days ver-sus 206 days). Excluding the extremevalues (less than 3 months in theQuébec (Latimer et al., 2006) and HongKong studies (Wong et al., 2008) andover 6 months in the Hartford study(Mueser et al., 2005), the range was
TABLE 2-PERCENTAGE OF PARTICIPANTS EMPLOYED IN
HOURS OR MORE A WEEK IN 4 IPS STUDIES
Study IPS
96 NH
99 DC
07 IL
04 CT
Total
34 (46.6%)
34 (45.9%)
43 (46.7%)
23 (33-8%)
134 (43.6%)
A COMPETITIVE JOB 2 0
(BASED ON TOTAL SAMPLES)
Controls
15 (22.4%)
4 (5-3%)
22 (23.1%)
12 (8.8%)
53 (14.2%)
SPRING 2008—VOLUME 31 NUMBER. 4
between 4 months and a little over 5
months as the average length of time
to first job for IPS participants.
Proportion of time in competitive em-ployment, as measured by annualizedweeks worked, is reported for the same7 IPS studies, as shown in Table 4. Theresults are reported for both all studyparticipants, including those whonever worked, and for the worker sam-ple. The Maryland study (Lehman et al.,2002) reported substantially lowerrates of weeks work in the full sample.Excluding the Maryland study, annual-ized average weeks worked for IPSranged from 10.0 weeks for the SouthCarolina study (Gold et al., 2006) to17.0 weeks for the Québec study(Latimer et al., 2006). Overall, the aver-age weeks worked for IPS was overtwice that for controls. When the sam-ples are limited to participants who ob-tained competitive employment duringfollow-up, the weeks worked are virtu-ally the same for IPS and controls.
Duration in longest held competitive jobfor those who worked at least one suchjob, is shown in Table 5. The average jobtenure for IPS participants in these stud-ies ranged from 37 weeks for the Illinoisstudy (Bond et al., 2007) to 10 weeks in
TABLE 3-MEA14
study
08 HK
06 Qué
99 DC
06 SC
07 IL
02 MD
04 CT
Total
DAYS TO FIRST COMPETITIVE JOB
IPS
72(A/=32)
84 (A/=51)
126 (A/= 45)
133 (A/= 42)
156 (A/= 69)
164 (A/=47)
197 (A/ = 51)
137.6 (A/=337)
IN 7 IPS STUDIES
Control
118 (A/ = 13)
89 (A/= 39)
293 (A/= 7)
322 (A/= 20)
193 (A/ = 32)
287 (A/ = 12)
277(A/=3i)
205.9 (N = 154)
the New Hampshire study (Drake et al.,1996). It should be noted that this meas-ure was influenced by length of follow-up. Except for the Connecticut study(Mueser et al., 2004), which found fargreater job tenure for IPS participants,IPS and control samples looked fairlysimilar on this measure.
Excluded Studies
Although not a focus of the current re-view, some comments on the excludedstudies are warranted. Nine studieswere located comparing a non-lPS sup-ported employment program to some
other vocational model. Four had signifi-cant findings suggesting superior com-petitive employment findings forsupported employment (Bond et al.,1995; Gervey & Bedell, 1994; McFarlaneet al., 2000; Test et al., 1995), while tworeported significantly better employ-ment outcomes for their supported em-ployment intervention but did notclearly differentiate between competi-tive employment and set-aside jobs, sothat the interpretation of their results isclouded (Chandler et al., 1997; Tsang,2006). Two studies found no differencesin employment outcomes for supported
TABLE 4-
06 Qué
07 IL
04 CT
08 HK
99 DC
06 SC
02 MD
Total
-ANNUALIZED WEEKS WORKED IN COMPETITIVE JOBS IN 7
All study Participants
IPS
17.0 (A/ = 75)
16.2 (A/ = 92)
14.9 (A/ = 68)
13.0 (N = 46)
10.1 (A/ = 74)
10.0 (A/= 66)
6.0 (A/ = 113)
12.1 (W=534)
Control
14.1 (A/= 74)
8.2 (A/= 95)
2.3 (A/=136)
7-0 (A/= 46)
0.8 (N = 76)
2.9 (A/= 77)
1.6 (A/ = 106)
4.8 (W=6io)
IPS STUDIES
Participants Working a Competitive Job
Total Follow-up Period
IPS Control
25.0 (A/= 51)
21.6 (A/ = 69)
19.8 (A/=51)
18.6 (A/ = 32)
16.6 (A/= 45)
15.8 (A/= 42)
14.4 (A/ = 47)
19.2 («=337)
26.8 (A/= 39)
24.3 (A/= 32)
9.8 (A/= 31)
24.9 (N = 13)
8.7 (A/= 7)
11.3 (N = 20)
14.1 (A/ = 12)
18.9 (/V=154)
Time Period from
IPS
32.5 (A/ = 51)
25.6(A; = 69)
27.1 (A/=51)
21.4 (A/ = 32)
21.6 (A/= 45)
19.3 (A/ = 42)
18.6 (A/= 47)
24.2 (Ai=337)
ist Job Start
Control
35.4 (A; = 39)
26.1 (A; = 32)
15.8 (A/= 31)
31.7 (A/=13)
iS.7(A/ = 7)
20.2 (A/ = 20)
23.2 {N = 12)
25.5 (Ai=i54)
PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials af Evidence-Based Supported Employment
employment and the comparison group
(Rogers et al., 2006; Shafer, 2005).
Finally, one study had mixed results, on
balance favoring the comparison group
over supported employment (Macias et
al., 2006). Combining the employment
rates for 4 early studies (Bond et al.,
1995; Chandler et al., 1997; Gervey &
Bedell, 1994; McFarlane et al., 2000),
we found that the combined employ-
ment rate was 53% for supported em-
ployment and 16% for controls, for an
effect size of .82.
Comparison with Other Reviews
We compared our competitive employ-ment rates with 5 earlier reviews, asshown in Table 6. The aggregated com-petitive employment rate for supportedemployment was highest in the currentreview, while, with the exception of theCooi< (2005) review, the mean competi-tive employment rate for controls wasalso higher than the other 4 reviews.All five reviews concluded that thecompetitive employment rate for sup-ported employment is at least twicethat for clients receiving some otherform of vocational assistance. Usingthe arc sine method, the effect sizes forthe overall IPS/control differenceranged from .43 to .79 in previous re-
TABLE
study
07 IL
04 CT
02 MD
06 SC
06 Qué
96 NH
Total
5-WEEKS WORKED AT LONGEST COMPETITIVE JOB DURINGFOLLOW-UP IN 6 IPS STUDIES (WORKER SAMPLE ONLY)
IPS
36.8 (/V = 69)
25.5 (W= 51)
21.6 (W = 47)
19.0 (W = 42)
14.6 (/V = 51)
10.0 (W = 57)
22.0 («=317)
Control
32.7 (W = 32)
4.4(W = 3i)
23.1 (W = 12)
20.0 (W = 20)
12.7 (A/= 39)
10.0 (W = 27)
16.3 (/V = 161)
views, compared to .79 in the currentreview.
Discussion
This review examined evidence-basedsupported employment in order to esti-mate expected outcomes. We concludethat the majority of iPS participants ob-tain competitive employment, at a farhigher rate than clients enrolled inother vocational services. Most IPSclients work part time; about two-thirdsof those who obtain competitive em-
ployment work 20 hours or more perweek. This could be due to preferences,limited stamina, or fear of losing healthinsurance. Undoubtedly the number ofhours worked per week is influenced bythe rules governing receipt of disabilitypayments and Wledicaid eligibility. Lessthan 1% of IPS participants left disabili-ty rolls during the follow-up period.Consistent with the principle of rapidjob search, the time to first competitivejob for IPS participants is nearly 10weeks less than for controls, althoughthe average for IPS of 20 weeks to first
TABLE 6-AGGREGATED COMPETITIVE EMPLOYMENT RATES IN 6 REVIEWS OF SUPPORTED EMPLOYMENT
Effect Size Effect SizeStudy Supported Reported Using Arc Number of
Employment Control by Authors Sine Studies
Bond (2004) 56%
Burns et al.(2OO7) ^ 58%
Cook et al. (2005) 55%
Crowther et al. (2001) 34%
Twamley et al. (2003) 51%
Current Review 61%
19%
21%
34%
12%
18%
23%
0.85
0.79
0.83
0.79
0.78
0.43
°-54
0.72
0.79
9
6
7
5
5^
11
Number ofStudies of
High Fidelity IPS
5 (56%)
6 (100%)
3 (43%)
2 (40%)
3 (60%)
11
3 Findings aggregated by the current authors for this table
'' Twamley review included 11 RCTs of vocational programs, 5 of which were supported employment studies used in this table.
SPRING 2008—VOLUME 31 NUMBER 4
job is somewhat of a surprise. Time tofirst job strongly affects longitudinalcompetitive employment outcomes; inmost studies, the large majority whowork at all do so in the first six months.Thus, this might be an area for modelimprovement, which we speculatemight require better job developmentstrategies (Carlson & Rapp, 2007). Inaddition, the use of the vocational pro-file to help identify job types and worksettings that match the individual'spreferences, skills, and experiences isanother area that may help improveand speed up the job seeking processand increase job tenure as well (Becker,Drake, Farabaugh, & Bond, 1996).
This review represents an advance overearlier reviews in several respects.First, it has the largest and (for the mo-ment) the most up-to-date collection ofpertinent randomized controlled trials.Second, it is one of only two reviewslimited to rigorous evaluations of IPSprograms. Thus, our review is based ona homogeneous set of studies, givingthe clearest picture of the potential forevidence-based supported employ-ment. None ofthe previous reviews in-cluded more than 6 high-fidelity IPSstudies. Notably, the Cook et al. (2005)analysis included two studies compar-ing enhanced supported employmentto supported employment only, whichconfounds the impact of supported em-ployment. We also note that the currentis a substantial update ofthe Cochranereview on vocational rehabititation(Crowther, Marshall, Bond, & Huxley,2000; Crowther et al., 2001). We attrib-ute our somewhat stronger results togrowing maturation ofthe field withboth higher fidelity supported employ-ment programs and stronger controlgroups. The differential advantage forevidence-based supported employmentremains about the same, but our con-clusions are stronger because the com-parisons appear to be more rigorous.
Could the current review be subject tothe "file drawer" problem - failing to in-clude studies that have not been pub-lished (Rosenthal, 1984)? We can give apartial answer to this question: Of fourunpublished studies, three are knownto have strongly positive results(Nuechterlein et al., 2005; Test et al.,1995; Tsang, 2006), while a fourthstudy, the Arizona study from the EIDP,had null results (Shafer, 2005).Moreover, of these unpublished studies,only the Nuechterlein et al. study (2005)conforms to the sampling criterion ofhigh-fidelity IPS. We deem it implausiblethat any unpublished studies could tipthe balance of evidence present here.
Some comment is warranted about theinclusion of Maryland study (Lehman etal., 2002) as one ofthe 11 IPS high-fi-delity studies. This study was a statisti-cal outlier. In terms of sample inclusioncriteria, it clearly deviated from theother IPS studies in that it was the onlystudy among those reviewed that didnot require participants to have a goalof competitive employment. The poorercompetitive employment outcomes areconsistent with this difference in sam-ple inclusion criteria.
Three widely repeated criticisms ofsupported employment concern exclu-sion of clients who do not have voca-tional goals, high dropout rates (i.e.,the contention that supported employ-ment has a high attrition rate) and briefjob tenure (supported employmenthelps clients get jobs but not to keepthem, i.e., most supported employ-ment jobs are short term). The currentcollection of studies sheds light onthese three issues.
Exclusion of individuals who do nothave competitive employment goals.Some observers have criticized evi-dence-based supported employmentfor its lack of outreach to individualswho do not have competitive employ-ment goals (Macias, DeCarlo, Wang,
Frey, & Barreira, 2001; Roberts & Pratt,2007). This criticism violates the ethi-cal principle of informed choice. Ofcourse we agree that all consumerswith severe mental illness should beencouraged to pursue work, and casemanagers and others within the mentalhealth center should create a culture inwhich work is valued (Gowdy, Carlson,& Rapp, 2004). At the same time, fromthe shared decision-making perspec-tive (O'Connor et al., 2007), the deci-sion to pursue work should be anactive choice based on a clear under-standing of what the decision means.
Early termination rates in iPS studies.Although only a handful of studies re-port the findings regarding program re-tention, the studies that do show astrong advantage for supported employ-ment. Five IPS studies compared ratesfor dropping out of vocational services.Although the criterion for early termina-tion differed across studies, all 5 studiesfound substantially greater program re-tention for IPS. In the NH study, early at-trition (dropping out of services withinfirst two months) was higher in the con-trol condition (38%) than in IPS (0%)(Drake et al., 1996). In the DC study, de-spite the addition of an extra staff per-son to help link clients with providers inthe comparison condition, early attrition(dropping out of services within first twomonths) was greater in the control con-dition (16%) than in IPS (5%) (Drake etal., 1999). In the Hartford study, approxi-mately 50% ofthe participants assignedto both control conditions dropped outwithin a few weeks, whereas less than10% of IPS participants dropped outduring this early period (Mueser et al.,2004). In the Québec study, 9% of IPSparticipants failed to receive at least oneservice contact during each ofthe firstand second 3-month periods in thestudy, compared to 70% of those inusual services (Latimer et al., 2006). Inthe Illinois study, 18% of IPS partici-pants dropped out within 6 months.
PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials of Evidence-Based Supported Employment
compared to 35% of controls (Bond et
al., 2007).
Thus it is time to suspend the concernthat supported employment programshave high dropout rates. This conclusionwas warranted in 1997 based on theavailable data at that time (Bond, Drakeet al., 1997), but the research now showsthat high-fidelity supported employmentprograms have minimal attrition.
Job tenure. Another widely repeated criti-cism of supported employment is that ithelps people get jobs, but not retainthem (Mueser et al., 2005). The currentreview suggests that the averagelongest-held job among IPS participantswho obtain work is 22 weeks. Two fac-tors affect the interpretation of this find-ing. First, there is wide variation acrossstudies. Second, the longest-held job isa biased statistic in a short-term study,because it gives no credit for jobs heldpast the foUow-up. Annualized weeks ofwork is probably a better statistic; on av-erage, IPS participants who obtain atleast one competitive job work 19 weeksout of 52, or about 36% of the availabletime. Moreover, this figure is not adjust-ed for the initial period of unemploymentlooking for a job, which averaged 21weeks for the first job. If we remove theinitial job search period and calculatethe duration of employment over the pe-riod of time after the start of the first job,IPS participants average 24.2 weeks peryear, or 47% of the 52-weekyear.
We found that among those who ob-tained at least one competitive job, IPSand control participants did not differin job tenure and other employmentoutcomes. This finding sometimes hasbeen misinterpreted to suggest that al-though IPS helps individuals get a firstjob sooner, it provides no advantageover usual services in helping themkeep the job. However, these sub-groups are no longer equivalent (e.g.,the IPS subgroup of those with a job in-cludes more than twice the proportion
of the enrolled sample as the compari-
son group). The resulting sampling
bias undoubtedly favors the non-IPS
comparison program because IPS
helps a wider range of participants ob-
tain employment.
To study duration of employment prop-erly, longer-term studies are needed.Two such studies present a clearer pic-ture of the stability of long-term em-ployment than do the studies in thecurrent review. Salyers and colleagues(2004) conducted follow-up interviews10 years after clients had enrolled in anIPS program, finding that 47% werecurrently working and 33% had workedat least 5 years during this period. Evenmore impressive results were obtainedby Becker and colleagues (2007) whointerviewed clients 8 to 12 years afterenrollment in IPS, finding that 71%were currently working and the identi-cal percentage had worked for morethan half of the follow-up period.
One contribution of this review is tohighlight common outcome measuresthat every supported employmentstudy should measure and report.Although the field is making goodprogress in converging on standard-ized measures, the inability to includeall studies in all comparisons is a limi-tation of this review.
Study Limitations
One confound in our comparativeanalysis is the lack of standardizationof follow-up period. Follow-up was 6months in one study, 1 year in 2 stud-ies, 18 months in 4 studies, and 2years in 4 studies. For the competitiveemployment rate variable, participantsin studies with longer follow-up havemore opportunity to obtain work.However, the reality is that the proba-bility of obtaining a first job diminishesover time. As already noted, a secondlimitation of this set of studies is theirrelatively short follow-up period, espe-
cially for measuring job tenure, as
measured by time on longest job held.
This is an inadequate indicator of em-
ployment outcome in short-term fol-
low-up studies, because the measure
does not capture the successful job
tenure for a participant who is em-
ployed at the end of the follow-up peri-
od and continues to work for years
thereafter.
Future Directions
The number, consistency, and effectsizes of studies of evidence-based sup-ported employment establish that it isone of the most robust interventionsavailable for persons with severe mentalillness. Recognizing this, researchershave moved ahead to examine a varietyof enhancements to amplify outcomesthrough early interventions, motivation-al interventions, cognitive interventions,alteration of benefits, and other ap-proaches. We review these efforts at theend of this special section (Drake &Bond, this issue).
In the meantime, research should alsoaddress the myriad issues related todissemination of evidence-based prac-tices (Drake & Skinner, in press). TheScylla and Charybdis of U.S. mentalhealth services continue to be the fail-ure to provide access to evidence-based health care and excessivespending on ineffective health care(Wang, Demler, & Kessler, 2002).
References
American Psychiatric Association. (1994).Diagnostic and statistical manual of men-tal disorders: DSM-IV(iith ed.).Washington, DC: American PsychiatricAssociation.
Becker, D. R., & Drake, R. E. (2003). A workinglife for people with severe mental illness.New York: Oxford Press.
Becker, D. R., Drake, R. E., Farabaugh, A., &Bond, G. R. (1996). Job preferences ofclients with severe psychiatric disordersparticipating in supported employmentprograms. Psychiatric Services, 47,1223-1226.
SPRING 2008—VOLUME 31 NUMBER4
Becker, D. R., Smith, J., Tanzman, B., Drake, R.E., & Tremblay, T. (2001). Fidelity of sup-ported employment programs and em-ployment outcomes. Psychiatric Services,52, 834-836.
Becker, D. R., Whitley, R., Bailey, E. L., &Drake, R. E. (2007). Long-term employ-ment outcomes of supported employmentfor people witii severe mental illness.Psychiatric Services, 58,922-928.
Becker, D. R., Xie, H., McHugo, G. )., Halliday,|., & Martinez, R. A. (2006). What predictssupported employment program out-comes? Community Mental Health Journal,42, 303-313-
Bell, M. D., Greig, T. C, Zito, W., & Wexler, W.(2007). An RCT of neurocognitive enhance-ment therapy with supported employ-ment: Employment outcomes at 24months. Schizophrenia Bulletin, 33,420-421.
Black, B. J. (1988). Work and mental illness:Transitions to employment. Baltimore:Johns Hopkins Press.
Bond, G. R. (2004). Supported employment:Evidence for an evidence-based practice.Psychiatric Rehabilitation Journal, 27,345-359-
Bond, G. R., Becker, D. R., Drake, R. E., Rapp,C. A., Meisler, N., Lehman, A. F., et al.(2001). Implementing supported employ-ment as an evidence-based practice.Psychiatric Services, 52, 313-322.
Bond, G. R., Becker, D. R., Drake, R. E., &Vogler, K. M. (1997). A fidelity scale for theIndividual Placement and Support modelof supported employment. RehabilitationCounseling Bulletin, 40, 265-284.
Bond, G. R., Dietzen, L. L., McGrew, J. H., &Miller, L. D. (1995). Accelerating entry intosupported employment for persons withsevere psychiatric disabilities.Rehabilitation Psychology, 40, 91-111.
Bond, G. R., Drake, R. E., Mueser, K. T., &Becker, D. R. (1997). An update on sup-ported employment for people with severemental illness. Psychiatric Services, 48,335-346.
Bond, G. R., Salyers, M. P., Dincin, J., Drake, R.E., Becker, D. R., Fraser, V. V., et al. (2007).A randomized controlled trial comparingtwo vocational models for persons withsevere mental illness. Journal ofConsulting and Clinical Psychology, 75,968-982.
Burns, T., Catty, |., Becker, T., Drake, R. E.,Fioritti, A., Knapp, M., et al. (2007). The ef-fectiveness of supported employment forpeople with severe mental illness: A ran-domised controlled trial. Lancet, 370,1146-1152.
Campbell, K. (2007). Consumer predictors ofcompetitive employment outcomes in sup-ported employment. Unpublished doctoraldissertation, Indiana University-PurdueUniversity Indianapolis, Indianapolis.
Carlson, L., & Rapp, C. A. (2007). Consumerpreference and individualized job search.American Journal of PsychiatricRehabilitation, 10,123-130.
Chandler, D., Meisel, |., Hu, T., McGowen, M.,& Madison, K. (1997). A capitated modelfor a cross-section of severely mentally illclients: Employment outcomes.Community Mental Health Journal, 33,501-516.
Cook, I. A., Leff, H. S., Blyler, C. R., Gold, P. B.,Goldberg, R. W., Mueser, K. T., et al.(2005). Results of a multisite randomizedtrial of supported employment interven-tions for individuals with severe mental ill-ness. Archives of General Psychiatry, 62,505-512.
Crowther, R., Marshall, M., Bond, G. R., &Huxley, P. (2000). Vocational rehabilitationfor people with severe mental disordersICochrane Review Art. No.: CD003080],Cochrane Library (Issue 1, ISSN 1464-780X); www.cochrane.org/. Oxford,England: Update Software.
Crowther, R. E., Marshall, M., Bond, G. R., &Huxley, P. (2001). Helping people with se-vere mental illness to obtain work:Systematic review. British Medical Journal,322, 204-208.
Danley, K. S., & Anthony, W. A. (1987). TheChoose-Get-Keep Model: Serving severelypsychiatrically disabled people, ̂ menconRehabilitation, 13(4), 6-9, 27-29.
Drake, R. E., & Bond, G. R. (2008). The futureof supported employmentfor people withsevere mental illness. PsychiatricRehabilitation Journal, 31, 367-376.
Drake, R. E., McHugo, G. J., Bebout, R. R.,Becker, D. R., Harris, M., Bond, G. R., et al.(1999). A randomized clinical trial of sup-ported employment for inner-city patientswith severe mental illness. Archives ofGeneral Psychiatry, 56, 627-633.
Drake, R. E., McHugo, G. J., Becker, D. R.,Anthony, W. A., & Clark, R. E. (1996). TheNew Hampshire study of supported em-ployment for people with severe mentalillness: Vocational outcomes. Voumo/ ofConsulting and Clinical Psychology, 64,391-399.
Drake, R. E., & Skinner, ). S. (in press).Technology diffusion in mental health. InH. H. Goldman (Ed.), Mental health in themainstream of public policy: Research is-sues and policy. Chicago: John D. andCatherine T. MacArthur Foundation.
Gervey, R., & Bedell, J. R. (1994). Supportedemployment in vocational rehabilitation.In J. R. Bedell (Ed.), Psychological assess-ment and treatment of persons with severemental disorders (pp. 151-175).Washington, DC: Taylor & Francis.
Gold, P. B., Meisler, N., Santos, A. B.,Carnemolta, M. A., Williams, 0. H., &Kelleher, J. (2006). Randomized trial ofsupported employment integrated withassertive community treatment for ruraladults with severe mental illness.Schizophrenia Bulletin, 32, 378-395.
Gowdy, E. A., Carlson, L. S., & Rapp, C. A.(2003). Practices differentiating high-per-forming from low-performing supportedemployment programs. PsychiatricRehabilitation Journal, 26, 232-239.
Gowdy, E. A., Carlson, L S., & Rapp, C. A.(2004). Organizational factors differentiat-ing high-performing from low-performingsupported employment programs.Psychiatric Rehabilitation Journal, 28,150-156.
Killackey, E., Jackson, H. J., & McGorry, P. D. (inpress).Vocational intervention in first-episode psychosis: A randomised con-trolled trial of Individual Placement andSupport vs. treatment as usual. BritishJournal of Psychiatry.
Latimer, E., Lecomte, T., Becker, D., Drake, R.,Duelos, 1., Piat, M., et al. (2006).Generalisablity of the individual place-ment and support model of supported em-ployment: Results of a Canadianrandomised controlled trial. BritishJournal of Psychiatry, 189, 65-73.
Lehman, A. F., Goldberg, R. W., Dixon, L B.,McNary, S., Postrado, L., Hackman, A., etal. (2002). Improving employment out-comes for persons with severe mental ill-ness. Archives of General Psychiatry, 59,165-172.
Lipsey, M. W. (1990). Design sensitivity.Newbury Park, CA: Sage.
Macias, C, DeCarlo, L. T., Wang, Q., Frey, ]., &Barreira, P. (2001). Work interest as a pre-dictor of competitive employment: Policyimplications for psychiatric rehabilitation.Administration and Policy in MentalHealth, 28, 279-297.
Macias, C, Rodican, C. F., Hargreaves, W. A.,Jones, D. R., Barreira, P. J., & Wang, Q.(2006). Supported employment outcomesof a randomized controlled trial of ACT andclubhouse models. Psychiatric Services,57,1406-1415.
PSYCHIATRIC REHABILITATION JOURNAL An Update on Randomized Controlled Trials of Evidence-Based Supported Employment
McFarlane, W. R., Dushay, R. A., Deakins, S.M., Stastny, P., Lukens, E. P., Toran, )., etal. (2000). Employment outcomes inFamily-aided Assertive CommunityTreatment. American Journal ofOrthopsychiatry, 70, 203-214.
McGrew, J. H. (2007). IPS fidelity survey of 17supported employment programs inIndiana: Final report to SECT Center.Unpublished report. Indianapolis: IndianaUniversity-Purdue University Indianapolis.
McGrew, J., & Griss, M. (2005). Concurrent andpredictive validity of two scales to assessthe fidelity of implementation of support-ed employment. Psychiatric RehabilitationJournal, 29, i^i-tij.
McGurk, S. R., Mueser, K. T., & Pascaris, A.(2005). Cognitive training and supportedemployment for persons with severe men-tal illness: One year results from a ran-domized controlled trial. SchizophreniaBulletin, 31, 898-909.
Mueser, K. T., Aalto, S., Becker, D. R., Ogden, J.S., Wolfe, R. S., Schiavo, D., et al. (2005).The effectiveness of skills training for im-proving outcomes in supported employ-ment. Psychiatric Services, 56,1254-1260.
Mueser, K. T., Clark, R. E., Haines, M., Drake, R.E., McHugo, G. J., Bond, G. R., et al.(2004). The Hartford study of supportedemployment for persons with severe men-tal illness. Journal of Consulting andClinical Psychology, 72,479-490.
Nuechterlein, K. H., Subotnik, K. L, Turner, L.R., Ventura, J., Becker, D. R., & Drake, R. E.(2008). Individual Placement and Supportfor individuals with recent-onset schizo-phrenia: Integrating supported educationand supported employment. PsychiatricRehabilitation Journal, 31, 340-349.
Nuechterlein, K. H., Subotnik, K. L, Ventura, |.,Gitlin, M. )., Green, M. F., Wallace, C. )., etal. (2005). Advances in improving and pre-dicting work outcome in recent-onsetschizophrenia. Schizophrenia Bulletin, 31,530.
O'Connor, A. M., Wennberg, J. E., Legare, F.,Llewellyn-Thomas, H. A., Moulton, B. W.,Sepucha, K. R., et al. (2007). Toward the"tipping point": Decision aids and in-formed patient choice. Health Affairs, 26,716-725.
Propst, R. N. (1992). Standards for clubhouseprograms: Why and how they were devel-oped. Psychosocial Rehabilitation Journal,16, 25-30.
Roberts, M. M., & Pratt, C. W. (2007). Putativeevidence of employment readiness.Psychiatric Rehabilitation Journal, 30,175-181.
Rogers, E. S., Anthony, W. A., Lyass, A., &Penk, W. E. (2006). A randomized clinicaltrial of vocational rehabilitation for peoplewith psychiatric disabilities. RehabilitationCounseling Bulletin, 49,143-156.
Rosenthal, R. (1984). Meta-analyticproceduresfor the social sciences. Beverly Hills: Sage.
Salyers, M. P., Becker, D. R., Drake, R. E.,Torrey, W. C, & Wyzik, P. F. (2004). Ten-year follow-up of clients in a supportedemployment program. PsychiatricServices, 55, 302-308.
Schonebaum, A. D., Boyd, J. K., & Dudek, K. ).(2006). A comparison of competitive em-ployment outcomes for the clubhouse andPACT models. Psychiatric Services, 57,1416-1420.
Shafer, M. (2005). Employment InterventionDemonstration Project: Arizona site (un-published study).
Test, M. A. (1992). Training in CommunityLiving. In R. P. Liberman (Ed.), Handbookof psychiatric rehabilitation (pp. 153-170).New York: Macmillan.
Test, M. A., Allness, D. )., & Knoedler, W. H.(1995, October). Impact of seven years ofassertive community treatment. Paper pre-sented at the American PsychiatricAssociation Institute on PsychiatricServices, Boston, MA.
Tsang, H. W. (2006). Enhancing employmentopportunities of people with mental illnessthrough an Integrated SupportedEmployment approach (ISE) of IndividualPlacement and Support (IPS) and SocialSkills Training (SST) (HSRC Report #S121014). Hong Kong: Health ServicesResearch Committee DisseminationReport.
Tukey, |. W. (1977). Exploratory data analysis.Reading, MA: Addison-Wesley.
Twamley, E. W., Jeste, D. V., a Lehman, A. F.(2003). Vocational rehabilitation in schizo-phrenia and other psychotic disorders: Aliterature review and meta-analysis of ran-domized controlled trials./ourno/ ofNervous and Mental Disease, 191,515-523-
Twamley, E. W., Narvaez, J. M., Becker, D. R.,Bartels, S. )., & leste, D. V. (2008).Supported employment for middle-agedand older people with schizophrenia.American Journal of PsychiatricRehabilitation, 11, 76-89.
Wallace, C. |., & Tauber, R. (2004).Supplementing supported employmentwith workplace skills training. PsychiatricServices, 55, 513-515.
Wang, P. S., Demler, 0., & Kessler, R. C.(2002). Adequacy of treatment for seriousmental illness in the United States.American Journal of Public Health, 92,92-98.
Wong, K. K., Chiu, R., Tang, B., Mak, D., Liu, J.,& Chiu, S. N. (2008). A randomized con-trolled trial of a supported employmentprogram for persons with long-term men-tal illness in Hong Kong. PsychiatricServices, 59, 84-90.
GARYR. BOND, PHD, IS CHANCELLOR'SPROFESSOR, DEPARTMENT OF PSYCHOLOGY,INDIANA UNIVERSITY-PURDUE UNIVERSITYINDIANAPOLIS, INDIANAPOLIS, IN.
ROBERT E. DRAKE, MD, PHD, IS PROFESSOR OFPSYCHIATRY AND COMMUNITY AND FAMILYMEDICINE, DARTMOUTH MEDICAL SCHOOL,LEBANON, NH.
DEBORAH R. BECKER, M ED, IS ASSISTANTPROFESSOR, DEPARTMENT OF COMMUNITY ANDFAMILY MEDICINE, DARTMOUTH MEDICAL SCHOOLAND DARTMOUTH PSYCHIATRIC RESEARCH CENTER.
CORRESPONDING AUTHOR:
GARYR. BOND
DEPARTMENT OF PSYCHOLOGY
INDIANA UNIVERSITY-PURDUE UNIVERSITYINDIANAPOLIS402 N. BLACKFORD ST., LD124INDIANAPOLIS, IN 46202-3275PHONE: 317-274-6752FAX: 317-274-6756EMAIL: [email protected]