Implementation of Goal Attainment Scaling in Community Intellectual Disability Services

Journal of Policy and Practice in Intellectual Disabilities Volume 3 Number 2 pp 119–128 June 2006

© 2006 International Association for the Scientific Study of Intellectual Disabilities and Blackwell Publishing, Inc.

Blackwell Publishing IncMalden, USAPPIPolicy and Practice in Intellectual Disabilities1741-11222006 International Association for the Scientific Study of Intellectual Disabilities and Blackwell Publishing Ltd? 200632119128Original ArticleGoal Attainment Scaling and Community Intellectual Disability ServicesM. Chapman et al.

Received March 14, 2005; accepted April 25, 2005Correspondence: Mark Burton, Head of Service, Manchester Learning Disability Partnership, Mauldeth House, Mauldeth Road West, Chorlton, Manchester M21 7RL, UK. Tel: +44 161958 4050; Fax: +44 161958 4149; E-mail: [email protected]

Implementation of Goal Attainment Scaling in Community Intellectual Disability ServicesMelanie Chapman*, Mark Burton*, Victoria Hunt*, and David Reeves†

*Manchester Learning Disability Partnership; and †National Primary Care Research & Development Center, University of Manchester, Manchester, UK

Abstract The authors describe the evaluation of the implementation of an outcome measurement system (Goal Attainment Scaling– GAS) within the context of an interdisciplinary and interagency intellectual disability services setting. The GAS database allowedanalysis of follow-up goals and indicated the extent of implementation, while a rater study evaluated the quality of goals. While staffwere able to produce adequate goals and scales, fewer goals were set than anticipated, and the overall quality was not high. Althoughimplementation resulted in a number of perceived benefits, various barriers to implementation were experienced. These hinged onstaff perceptions of the value, ease of use, appropriateness, and soundness of the method. Widespread adoption of GAS in communityintellectual disability teams is not supported by the findings of this study. The authors suggest that staff perceptions, ease of use, andthe implementation process play a key role in the successful adoption of an outcome measurement system. They conclude thatalternative ways of measuring individually oriented outcomes may be more useful.

Keywords: community setting, Goal Attainment Scaling, outcome measurement

INTRODUCTION

Health and social care organizations, policy, and professionalguidance have emphasized the importance of outcome measure-ment and the monitoring of the quality of health and social careservices (e.g., within the British context see Department ofHealth, 1997; 2001; NHS Executive, 1998; United KingdomCentral Council for Nursing, Midwifery & Health Visiting, 2001).An outcome focus can demonstrate the effectiveness and impactof existing and new services; improve accountability; enable usersand practitioners to choose between alternative approaches; andhelp service providers improve services, make more effective useof available resources, and demonstrate effectiveness to commis-sioners (i.e., funders of services). Routine outcome measurementis particularly important when there is a lack of published infor-mation about the effectiveness of different approaches, as is oftenthe case in intellectual disability practice.

This paper describes an evaluation of the implementation ofone outcome measurement system, Goal Attainment Scaling(GAS), as used in a British intellectual disability service. GAS

involves the service recipient (or his or her representative) agree-ing to the most important and feasible goals with the serviceprovider (Kiresuk & Sherman, 1968). The expected level ofattainment, using a 5-point scale anchored at zero, is then con-structed for each goal. This scale is used to score the outcome ata date specified in advance.

Initially developed for use in North American communitymental health centers, GAS has been used in a variety of servicecontexts, and a substantial literature has appeared (see Kiresuk,Smith, & Cardillo, 1994, for a review). It has been reported thatGAS has particular virtues as a change measure, has adequatevalidity, and its reliability is at least as good as those of conven-tional rating scales (Cardillo, 1994; Cardillo & Smith, 1994a;1994b).

The pragmatic features of GAS also make it attractive forapplications to community services. GAS uses a common metricfor all types of goals, workers, and clients, anchoring the scale atthe “expected level of attainment.” Cardillo (1994) reports thatprofessional staff can identify likely levels of outcome, and so setthe “expected level” anchor realistically across goals. Potential biasfrom the professional’s participation in scoring the outcome isreduced, as the criteria for outcome measurement are establishedin advance of the outcome being measured. The participation ofthe person with intellectual disabilities (or a key ally) in the goal-setting process should enhance goal relevance and commitmentto the intervention. The measurement of outcomes through thegoals deliberately set means that measurement has appropriate

Journal of Policy and Practice in Intellectual Disabilities Volume 3 Number 2 June 2006

M. Chapman et al. • Goal Attainment Scaling and Community Intellectual Disability Services

120

sensitivity to the intervention in contrast to broad-brush strate-gies that will inevitably measure aspects of the person’s circum-stances or functioning that will be irrelevant to, or unlikely torespond to, the intervention.

However, there have been limited applications of GAS in intel-lectual disability services, although there has been interest in andadvocacy for its use (Burns & Tobell, 1997; Turnbull, 1998; Young& Chesson, 1997). Published studies have concluded that GAS isapplicable to intellectual disability services, can be implementedat minimal expense in terms of staff time (Bailey & Simeonsson,1988; Glover, Burns, & Stanley, 1994), and appears helpful inclarifying the purpose of therapy (Young, Harvais, Joy, &Chesson, 1997). However, these studies have taken place inresidential or day service settings, or with goals restricted tobehavioral change (Bailey & Simeonsson, 1988).

Unpublished studies of GAS in the context of intellectualdisability field services report difficulties in implementation,largely because of professional resistance or at least conflict withexisting procedures and practices (Ball, 1996; Lemon & Burns,1996). However, these studies indicated that staff could set real-istic, observable, and attainable goals, and that service recipientsvalued involvement in goal setting. Ball (1996) concluded thatthe difficulties encountered could be minimized by adequatetraining, planning, and management. Although Ball found thatmeasurable change was produced, and GAS was sensitive to smallchanges, the study was of limited scope, covering 30 clients witha mean of two goals each, set and monitored by eight staff. WhileLemon and Burns (1996) reported that staff were able to setrealistic, observable, and attainable goals, the diversity of goaltypes with such a small sample meant that quantitative analysisof the GAS scores was not attempted.

Therefore, there has been surprisingly limited evaluation ofthe application of GAS in intellectual disability services generallyand in intellectual disability field services specifically. Moreover,where GAS has been applied, it has been used with restricted goalsthat are broadly therapeutic in nature, whereas community intel-lectual disability teams typically work with a broader range ofinterventions involving a variety of types of outcome (e.g.,behavior change, affective change, learning, status maintenanceor change, social and physical environmental change, arrange-ment of other interventions, and services). In addition, there is alack of research on the practitioner experience of using GAS. Thisstudy aimed to evaluate the implementation and usefulness of anoutcome measurement system based on GAS within communityintellectual disability teams, and how staff experienced the intro-duction and use of the system.

The Service Setting

This project involved the implementation of GAS in the com-munity teams of a British intellectual disability service. The ser-vice is a partnership between the local Social Services Department(local government) and community health services (National

Health Service), and supports approximately 1,400 people withintellectual disabilities living in a large city (Manchester). At thetime of this study, four community teams consolidated care man-agement/social work, intellectual disability nursing, psychology,speech and language therapy, occupational therapy, physiother-apy, additional support workers, and domiciliary care under asingle management structure. Each team served a catchment pop-ulation of approximately 105,000 inhabitants, and had around 12full-time equivalent staff, not including management, adminis-tration, and domiciliary care staff. Community intellectual dis-ability team staff work with people in the family home and inservice settings managed by the service or other provider organi-zations (e.g., day service provision, supported housing, andshort-term support respite services).

Implementation of GAS

The GAS system consists of the GAS form (reproduced asFigure 1), a manual and examples booklet for staff, and a databasethat allows entry of data for progress monitoring and analysiscentrally. Figure 2 outlines the process of using GAS. In our study,the system was implemented by a project worker in cooperationwith local managers. It was maintained by Author 3, who also hada research role in the project.

Management ownership was considered “built-in” as asenior manager in the service (Author 2, who was then the headof Development and Clinical Services) initiated the project.Organizational ownership was encouraged through briefingsand regular meetings with managers and senior members ofeach professional group. Adoption by staff was encouraged byregular updates included in the monthly service update forstaff, clear messages about the nonnegotiability of the system,training sessions and materials, “GAS Clinics” and individualappointments, feedback on goals set, and use of “early adopt-ers” to help other staff. The original GAS recording form wasmodified to include a rating of external factors affecting goalattainment and mode of delivery of intervention (a help/hindrance scale).

EVALUATION METHODS

The evaluation incorporated a number of methods. The qual-ity of information provided by the GAS system was examinedthrough statistical analysis and rating of the quality of goals, whileimplementation issues were examined through staff question-naires and interviews, and by a journal kept by the researcher.

Analysis of Data from the Outcome Measurement System

A research database was established (using MicrosoftACCESS) to record the goals and scales, and to aggregate and



121

FIGURE 1

Goal Attainment Scaling (GAS) form.

G.A.S. FormNa me of Se rv ic e U ser: Goal:

Na me of Pe rs on Setti ng th e G oa l:

Occu pa ti on of Pe rs on Setti ng Go al:

L EVEL S O F A TT AI NM EN T Da te Go al Se t

Pre-G.A.S. Sc or e ( Ma rk wi th a ‘ X’ )

Pr ed ic te d Da te fo r 6

Mo nt h R evi ew

Ac tu al Da te of 6 M ont h

R evi ew

G. A. S. Sc or e at 6 M on th

R evi ew (M ar k w it h a

‘X ’)

Da te of Discharge

(if notat 6

Mo nt hs )

G. A. S. Sc or e at Di sch ar ge

(i f n ot di sch ar ge d a t

6 M on th s) Mu ch Le ss

So me w hat Les s

Ex pec te d Lev el of

At ta in me nt So me w hat

Mo re

Mu ch Mo re

Mo de of De li ver y Ty pe of Go al He lp /H in dr an ce Sc or e Ou tc om e

The form should be completed as follows:

1. Name of the Service User The member who the goal is being set for.

2. Name of Person Setting Goal Member of the Joint Service.

3. Occ. of Person Setting Goal I.e. profession.

4. Goal The goal set.

5. Date Goal Set The date the goal is decided and written onto the form.

6. PreG.A.S. Score The level the service user is at before the work with them began.

7. Pred. Date for 6 Month Review The date six months afterthe goal is set.

8. Actual Date of 6 Month Review The date when the 6 MonthReview actually takes place.

9. G.A.S. Score at 6 Month Review The level the service useris at on the 6 Month Review.

10. Date of Discharge(if not at 6 Month Review)

If the client is discharged either before or after the 6 Month Review,please write the date down in this column.

11. G.A.S. Score at Discharge(if not at 6 Month Review)

The level the service user is atwhen discharged, if dischargedbefore or after 6 Month Review.

NB. Numbers 10 & 11 (above) do not need to be completed if the service user is discharged at 6 Month Review.

12. Mode of Delivery Please choose the option which bestdescribes the way the goal will be achieved:

–

–

–

–

–

–

–

–

–

–

–

–

1) (You, the goal setter, will do the work yourself)2) (Staff in a house/day centre, etc. have responsibility for

work being completed)3) (The family of the service user are responsible for the

the work being completed)4) (The outcome of the goal will now depend on the

Another Service response of another service)

13. Type of Goal Please choose the option which best describes the type of goal which has been set:

1) Accommodation2) Arrange service3) Assessment4) Counseling5) Environmental Change6) Give or Obtain Information7) Health Related8) Maintenance of Function9) Mobility10) Skill Development11) Support Family/Carers12) Therapy and/or Behaviour Change

14. Help/Hindrance Score Please rate whether external factors helped orhindered goal attainment, using the following guide:

+2+1

0–1

====

–2 = Considerable hindrance from external factors

15. Outcome Please choose the option which best describes the outcome of the goal at this time:

1) Goal Completed2) Goal Continued3) Inappropriate Goal4) Reduction in Resources5) New Goal Written6) Service User Illness7) Service User Discharged Before Goal Completed8) Service User is Withdrawn9) Service User Withdraws10) Staff Illness11) Death of Service User

DirectVia Staff

Via Family

Via Referral to

Considerable help from external factorsSome help from external factorsNo significant effect from external factorsSome hindrance from external factors



122

analyze this information. SPSS (SPSS, 2001) was used to conductmore detailed statistical analysis.

A Rating Study Exploring the Quality of Goals

Some examples of goals are provided in Table 1. Three ratersindependently examined a selection of 60 goals, selected ran-domly and anonymously from those available. The ratersinvolved were all connected with the project: the research assis-tant (Author 3), the project leader (Author 2), and the statisticaladvisor (Author 4). Goals were rated on clarity, specificity,relevance, practicality, and measurability. Scales were rated onwhether they involved multiple goals, multidimensional scales,overlapping levels or gaps between levels, blank levels, achievabil-ity, and appropriateness of factor used to scale. For the ratingstudy, the quality of goals was scored on a 5-point or 3-pointrating scale; the quality of scales was scored on a “Yes/No” basis.

Questionnaires to Staff

Three batches of questionnaires were used:

Batch 1 (Month 12): Sent to all staff who had completed morethan three goals.Batch 2 (Month 18): Sent to all staff who had completed morethan three goals.Batch 3 (Month 20): Sent to all staff who had not submitted atleast three goals.

The questionnaires were anonymous and sought informationabout how staff viewed GAS, where staff first heard of the system,difficulties they experienced, and their suggestions for furthertraining. Fixed response questions, open questions, and opinionrating scales were used, along with space for any furthercomments.

Interviews with Staff

Two sets of interviews were carried out. Eight interviews werecarried out relatively early in the project, and five further inter-views at a later stage. The interview schedule explored opinionson GAS, training sessions, experiences of using GAS, benefits anddrawbacks of the system, and suggestions for improvements tothe implementation process.

Interviews were taped and transcribed. Analysis consisted of“open coding” of interview transcripts by Authors 2 and 3 usinga set of starter codes that were added to as required. Themes andpatterns were then discussed and refined, paying particular atten-tion to the different views expressed by participants, including“negative cases.” Matrices (Miles & Huberman, 1994) were usedto summarize the information and to guide analysis of emergent

FIGURE 2

The process of using Goal Attainment Scaling (GAS).

Referral to community intellectual

disability team

As se ss me nt

Co mp lete GAS fo rm (s ): Serv ic e r ec ipie nt (o r t he ir

re pr esentativ e) an d c om mu ni ty team member agree on goals and

exp ected le ve l o f a ch ievement

Afte r 6 mo nt hs : Re vi ew situatio n

Go al ach ie ve d Goal not achieved/ ongoing/inappropriate

Ne w g oa l i de nt if ie d

En d o f i np ut re ga rd in g g oa l(s)

Co mp lete GAS fo rm , r ec or di ng curr en t p os it io n o n leve l o f achie ve me nt scal e a nd an y h el p/ hi nd ra nc e factor s

Co mp lete ne w GAS fo rm (s )

TABLE 1Examples of goals set

To enable Lara to live in a residential environment which incorporates all her needs

That Susan may attend chosen “drop in” with support every 4–6 weeks

Gayle will be able to have a bath with support until her arm is out of the sling

Derek will be provided with equipment that enables him to sit on the toilet safely and comfortably

Assessment of independent living skillsYazid will attend the art group 2/3 times a monthFor Ellie to complete an occupational therapy skills assessmentThat Melanie has a new set of splints to meet her needsJack will have increased access to leisure activitiesMobeen will be able to step out of bath independently using a

suitable handrail

Note: All names have been changed.



123

themes (Coffey & Atkinson, 1996). A thematic framework wasthen developed through familiarization, constant internal andexternal comparison (e.g., with literature and other project find-ings), and discussion with the research team. There was noattempt to make themes “representative” of the sample in a quan-titative sense, but rather to reflect as full and complete as possiblea picture of the issues perceived by participants (Strauss &Corbin, 1990).

The two coders (Authors 2 and 3) made explicit their assump-tions about the project in written statements prior to beginningthe analysis. This was to increase reflexivity in terms of sensitivityto the “sense making” (Bannister, Burman, Parker, Taylor, & Tin-dall, 1994) by researchers who had specific interests in the project.

Research Journal

The researcher (Author 3) kept a journal throughout theproject describing factual details concerning the everyday run-ning of the project and reflexive analysis of the progress of theproject. This was analyzed by identifying key themes, using asimilar method to that outlined above.

FINDINGS

This section describes the number of goals set during theresearch project to demonstrate the extent of implementation ofGAS, and analysis of follow-up scores. Key issues related to imple-mentation—identified from analysis of the questionnaires, inter-views, and research journal—are then presented.

Analysis of Data from the Outcome Measurement System

The number of goals set provides an index of the extent ofimplementation and routinization of GAS in the work of thecommunity teams. A total of 789 goals were set across the wholeservice over a 2-year period. As Figure 3 shows, there was a slowstart, with all teams showing a similar initial pattern of slowsetting of goals. The rate rose 1 year into the project, but tailedoff again once the implementation phase drew to a close and theproject devoted more attention to the evaluation.

During the year 1998–99, 1,395 referrals to the relevant dis-ciplines in the community teams were recorded. Of these, some10% were classified as inappropriate or requiring no further

FIGURE 3

Cumulative goals set by the four teams.

Cumulative goals set

0

100

200

300

400

500

600

700

800

Janu

ary 19

98

Febru

ary 19

98

Mar

ch19

98

April 1

998

May19

98

June

1998

July

1998

Augus

t 199

8

Septe

mber 19

98

Octobe

r 1998

Novem

ber 19

98

Decem

ber 19

98

Janu

ary 19

99

Febru

ary 19

99

Mar

ch19

99

April 1

999

May19

99

June

1999

July

1999

Augus

t 199

9

Septe

mber 19

99

Octobe

r 1999

Novem

ber 19

99

Decem

ber 19

99

Time

Nu

mb

er o

f g

oal

s 1

2

3

4

All

Team



124

action. This would suggest a minimum expected number of 1,255goals per year, if only one goal were set per referral; however, theproject suggested that 2–3 goals would usually capture the mainthrust of the work being done. Therefore, the number of goals setwas far fewer than anticipated.

Mean baseline and follow-up scores for 374 goals with bothan initial and follow-up score are presented in Table 2, along witha breakdown of these scores by discipline.

There were no statistically significant associations betweenthe extent of change in GAS score reported and discipline (χ2

(5, n = 374) = 9.8, p > 0.05) or mode of delivery (χ2 (3,n = 372) = 1.8, p > 0.05). However, there were significant associ-ations between change in GAS score and both help/hindrance(Spearman rank correlation coefficient: rs = 0.453, p < 0.01) andtype of goal (χ2 (6, n = 372) = 20.1, p < 0.01). Smaller changes inGAS scores were associated with a greater hindrance factor, whilehigher changes in GAS scores were associated with a greater helpfactor. The data seem to indicate that smaller changes in GASscores were associated with therapy/behavior change and skilldevelopment goals, whereas greater changes in GAS score wereassociated with goals around giving and obtaining informationand environmental change.

Rating Study Findings

Table 3 summarizes the results of the rating study relating tothe quality of goals. Score means have been translated into acommon scale ranging from 0 (lowest quality) to 10 (highestquality). These results are based on all three raters combined.

Table 4 contains the average results (summed across all threeraters) on the existence of multiple goals, and the quality of scales.

Staff Satisfaction with GAS

Forty-four questionnaires were returned (a response rate of38.3%). Table 5 gives details of satisfaction with and usefulness

of GAS as perceived by staff who returned questionnaires. Use-fulness was scored on a 5-point scale (1 being “not at all,” and 5being “very much so”), while satisfaction was scored on a10-point scale (1 being “very happy,” and 10 being “veryunhappy”).

TABLE 2Scores by discipline

Mean baseline scorea Mean follow-up score Mean change in score

Nurses −1.6 (0.6) 0.1 (1.4) 1.8 (1.4)Care managers −2.0 (0.2) 0.5 (1.0) 2.5 (1.0)Psychology −1.7 (0.6) 0.3 (1.1) 2.1 (1.0)Physiotherapy −1.6 (0.6) 0.6 (1.4) 2.2 (1.6)Speech and language therapy −1.9 (0.3) 0.1 (1.1) 2.1 (1.1)Occupational therapy −1.7 (0.6) 0.3 (1.3) 2.0 (1.3)All −1.8 (0.5) 0.3 (1.3) 2.0 (1.3)

Note: Numbers in brackets are the standard deviation.aMeans are on the scale of −2 (much less than expected) to +2 (much more than expected).

TABLE 3Quality of goals

DimensionAverage score

out of 10Acceptable to two

or more raters (%)

Clarity 6.9 92Specificity WHO 7.5 100Specificity WHAT 6.0 93Specificity WHEN 4.0 40Relevance 7.0 100Practicality 9.0 100Measurability 8.0 98Technical language 8.5 75

TABLE 4Multiple goals and quality of scales

Dimension % YesAgreement between

all ratersa(%)

Multidimensional scale 47 61Other factors better used? 42 47Gaps between levels 23 75Multiple goals 16 73Overlapping levels 15 85Achievable? 11 79Blank levels 7 97

aThat is, on 73% of goals, all three raters agreed that either there were or werenot multiple goals implied in the goal statement.



125

Similar issues arose in interviews, the day-to-day talk recordedin the research journal, and the open sections of the question-naires, providing a triangulated check on the plausibility of thesethemes. These shed light on potential reasons for the disappoint-ing returns of GAS forms and the low levels of satisfaction amongstaff.

Perceived benefits and drawbacks with GAS are summarizedin Table 6. These broadly surround the implementation process,use of the information, value to clients and community intellec-tual disability team staff, appropriateness and methodologicalstrength of GAS, and ease of use of the system.

Of particular note with the current emphasis on person-centered approaches is that GAS was not being used in a dialoguewith service recipients or families in the majority of cases. The

actual circumstances of using the system did not seem to enablethe use of GAS in an empowering way with people with intellec-tual disabilities or carers. Employees were completing the formsat their desks, and most interviewees felt GAS was too compli-cated to be shared with service recipients or families, sometimescommenting that it would not be appropriate to introduce GASinto already complex situations. Those who tried to use GAS asa basis for negotiating work with service users and families foundit difficult to explain. Many interviewees who said that they werenot able to share GAS with service recipients mentioned that theydid discuss the general aims and focus of their work, and there-fore felt that it was not necessary to share GAS in its specific form.Follow-up interviews suggested that it was commonly felt thatservice user consultation was already a part of staff routine, andthat GAS was not required to formalize this.

DISCUSSION

This study adds to existing evidence of the difficulties inimplementing GAS, demonstrating serious problems with GASboth with its implementation in a community intellectual disabil-ity service setting, and as a measurement tool. It is possible thatthe experiences here and the views of staff may apply to anycomplex system of outcome measurement and are not necessarilypeculiar to GAS.

TABLE 6Perceived benefits and drawbacks of Goal Attainment Scaling (GAS) as viewed by staff

Benefits Drawbacks

Implementation process Training positively received Lack of consultation, poorly “sold,” introduction through CIDTs (community intellectual disability teams) rather than professional group, management pressure

Reasons for GAS Provision of information on outcomes for service and staff

Concern about potential use by management to assess performance

Service monitoring and planning

Value to clients Increase accountability to clients Seen as a management tool with no benefits for clientsEmpower clients Forms rarely, if ever, completed with clientsTool to negotiate input with clients Too complicated to share with clients

Seen as tokenistic by staff who felt that clients were unlikely to demand goals are met

Appropriateness Clarify and focus input Does not fit complexity of inputReflects work Provides only selected information and a distorted view of

work

GAS system Ease of use Difficulties completing forms, particularly writing and scaling goals

Adds to, and duplicates, paperworkTime-consuming process

Methodological soundness

Methodological soundness of the GAS approach, e.g., writing easily achievable goals, writing goals in hindsight

TABLE 5Staff satisfaction with Goal Attainment Scaling and perceived usefulness

Satisfaction Usefulness

Batch 1 (>3 goals) 7 Not requestedBatch 2 (>3 goals) 7.7 2.9Batch 3 (<3 goals) 7.2 3.2



126

At a service level, although the GAS system was established, itwas clearly not to the satisfaction of the majority of the staff whohad to use it. Other published and unpublished studies on GASshow that attempts to use this system have often foundered onimplementation. Our findings suggest that the success ofimplementation appears to hinge on two interdependent issues:staff perceptions of the system and the implementation processitself.

Staff varied in their experiences with the introduction and useof the system. While those staff used to working in a goal-orientedfashion (e.g., occupational therapists and speech and languagetherapists) made relatively little criticism of the approach, thosefrom other traditions were generally critical. It is vital that staffare committed to the need for outcome measurement and con-sulted during the development and implementation of any sys-tem. In this situation, the benefits to clients and professionalgroups do not appear to have been communicated well enough.It appears that reassurance and openness about how outcomeinformation would be used was not communicated fully; doingso would be necessary to allay staff concerns.

The ease of use of any outcome measurement system alsoplays a vital part in the success of implementation. Specific diffi-culties relating to GAS included problems with setting and scalinggoals. We observed that the complexity of the method discour-aged staff from sharing GAS with users and carers. It may havebeen easier to use standardized goals (as these are more readilyused by some disciplines), but this may have been more difficultfor some professionals who subscribe to a broad remit, and henceuse very heterogeneous goals.

Problems of GAS as a Measurement Tool

Our goal in this study was to assess the impact of the imple-mentation of the GAS on an intellectual disability service setting.The statistical analysis of GAS follow-up scores seems to indicatethe feasibility of analyzing aggregate GAS scores to highlightdifferential performance of the service. Other analyses could beconducted if the relevant data were recorded (e.g., degree of dis-ability, living situation, and ethnicity). If the data were of suffi-cient quality to allow such analyses, the results are both of localinterest, and could be of use in making comparisons betweenservices.

However, the number of goals set indicates that the systemwas not being used routinely; therefore, how well the informationreflects the work of the community teams is open to question.The rating study indicated that while the majority of the sampleof goals were at least usable, in general the goals did not providesufficient information about when the goal was to be carried out,scales were often multidimensional, and other factors may havebeen more appropriate for scaling goals (e.g., frequency ratherthan duration).

Further difficulties follow from our finding that some workersreported setting artificially low expected levels of attainment, and

that some goals were set in hindsight. It could be argued that thismeans that our system was inadequately “policed,” but a keyfinding from our study was that staff resistance was a feature ofthe system, and attempts to impose conformity perhaps notsurprisingly exacerbated the problem.

A further problem area with GAS concerns its usefulness incombining individual goal attainment scores to use as an overallmeasure of service effectiveness, and for making comparisonswithin and between services. GAS is unusual in that its 5-pointscales are “self-anchored” at the center point. The person con-structing the GAS scale sets the mid-point at the expected levelof attainment. This potentially provides for difficulties in makingcomparisons between GAS scores derived from scales set bydifferent workers, or by the same worker under differentcircumstances.

For example, worker A is highly effective, uses the best avail-able methods appropriately, and with appropriate intensity.Worker B is less effective, using a less well-selected set of methodsfor particular goals, and with less than optimal intensity. Bothworkers set goals and construct scales appropriately, with theexpected level of attainment as the mid-point. In both cases, thisexpected level is based on the worker’s experience of what tendsto happen. Worker B, however, consistently sets a less ambitiousmid-point than does worker A, but because both work in theirhabitual fashion, each on average reaches their expected level ofattainment. A quantitative comparison (such as that in the pre-vious section) would indicate similar attainment, despite workerA delivering more benefit than worker B. Only a comparison ofthe actual goals set and their scales would reveal this difference.

Similarly, the self-anchoring at the center of GAS scales wouldbe likely to obscure differences in effectiveness between differentgroups of clients. If, for example, it was more difficult to producepositive behavior change in members of families with an intellec-tual disability coming from larger than smaller families, then theworkers’ expectations could be correspondingly reduced in theformer, and yet GAS scores in each case would have an averageof zero. The argument could be extended to the other points onthe scale. Cardillo and Smith (1994a) argue that the validity ofGAS scores can be assessed on the basis of a mean score of zeroand a particular standard deviation (of the T-scores computedfrom GAS scores). The problem described here identifies a weak-ness with this procedure, because finding such a data structurewould indicate that GAS was being used consistently, but wouldoffer no guarantee that it was measuring anything that was exter-nally anchored. Internal consistency does not imply externalvalidity.

It may be that the problem of self-anchoring at zero makesthe GAS scores at best of limited usefulness in making aggregatedcomparisons. Such a difficulty would only occur if the workerssetting the goals and creating scales did (as the Kiresuk groupasserts) make reasonably accurate prognostications regardingtheir degree of success, and did so differentially along the dimen-sions of interest in the evaluation of differential effectiveness.Such a possibility would require further investigation, which



127

would first require the establishment of a consistently imple-mented GAS system.

A further problem concerns the structure of the data pro-duced by a GAS system. Because GAS follows the natural historyof work by community intellectual disability team members,there is considerable heterogeneity in goal content, professionaldiscipline, client characteristics, number of goals, time intervalbetween goal setting or scaling and outcome measurement, and,finally, mode of delivery of intervention. This variation may haveimplications both for analyses of predictors of goal attainment,and – because a complex multiple regression model would berequired in order to thoroughly analyze GAS data sets – for thesize of the service that can use GAS to gain meaningful and robustinformation on a regular basis. Perhaps these considerationsexplain why there is a relative silence in the literature on methodsand issues in the aggregation and use of GAS data to monitorservices.

CONCLUSIONS

The recent policy focus on outcome measurement and userinvolvement is to be welcomed for emphasizing the importanceof demonstrating the effectiveness of services. However, the prac-ticalities of establishing such systems within community healthand social care services may not be as straightforward as at firstseems.

It is possible that an alternative approach based on individu-alized goals, but with a different measurement strategy (e.g., withabsolute anchors at either end of a scale), might be more appro-priate. Such an approach needs to be both transparent andstraightforward in use and capable of yielding robust data foraggregated analyses. Alternatively, a more practical strategy mightbe to abandon the goal of aggregating individualized data, andinstead separate the measurement of individual change (for indi-vidual or small group effectiveness assessments) from overall ser-vice monitoring. This would be carried out by means of suitableperformance indicators that reflect desirable outcomes for a localservice (e.g., ratio of in-district to out-of-district placements orrate of emergency admissions to hospital).

As our research indicated, practitioner attitudes impact sig-nificantly on the success of implementation. Thus, future studiesshould also include methods to gather the views and experiencesof staff. Similarly, it would be important to gain information fromclients and carers about the use of GAS and appropriateness ofchange in score.

The question of whether it is possible to devise adequateaggregate measures of outcome, based on individualized goals, isstill an open one. If further research were to be conducted on theissues raised by this project, the authors recommend that itshould focus on the development of measures of individualizedoutcome that are more straightforward to use by staff, are easierto use with users and carers, yet have adequate measurementproperties.

ACKNOWLEDGMENTS

This study was undertaken by Manchester Learning DisabilityPartnership with funding received from the UK Departmentof Health as part of the Outcomes in Social Care for AdultsProgramme (OSCA). The views expressed in the publication arethose of the authors and not necessarily those of the Departmentof Health. We are grateful to Jim Lemon for his help with theinitial stages of implementation, Keith Allen for his help withinterviews, and to the staff and managers of Manchester LearningDisability Partnership for their participation.

REFERENCES

Bailey, D. B., & Simeonsson, R. J. (1988). Investigation of use ofgoal attainment scaling to evaluate individual progress ofclients with severe, and profound mental retardation. MentalRetardation, 26, 289–295.

Ball, T. (1996). Health support teams (learning disabilities) out-comes study. Sheffield, UK: Sheffield Consulting and ClinicalPsychologists, Learning Disabilities Services, Longley Mead-ows, Rivermead Unit, Northern General Hospital, Sheffield S57AU, UK.

Bannister, P., Burman, E., Parker, I., Taylor, M., & Tindall, C.(1994). Qualitative methods in psychology: A research guide.Buckingham, UK: Open University Press.

Burns, J., & Tobell, J. (1997). Goal attainment scaling for peoplewith learning disabilities. Oxford: Winslow Press.

Cardillo, J. E. (1994). Goal setting, follow-up, and goal monitor-ing. In T. J. Kiresuk, A. Smith, & J. E. Cardillo (Eds.), Goalattainment scaling: Applications, theory and measurement(pp. 39–59). Hillsdale, NJ: Lawrence Erlbaum Associates.

Cardillo, J. E., & Smith, A. (1994a). Perspectives on validity. InT. J. Kiresuk, A. Smith, & J. E. Cardillo (Eds.), Goal attainmentScaling: Applications, theory and measurement (pp. 243–272).Hillsdale, NJ: Lawrence Erlbaum Associates.

Cardillo, J. E., & Smith, A. (1994b). Psychometric issues. In T. J.Kiresuk, A. Smith, & J. E. Cardillo (Eds.), Goal attainmentScaling: Applications, theory and measurement (pp. 173–212).Hillsdale, NJ: Lawrence Erlbaum Associates.

Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data.Thousand Oaks, CA: Sage.

Department of Health. (1997). The new NHS: Modern anddependable. London: Department of Health.

Department of Health. (2001). Valuing people: A new strategy forlearning disability for the 21st century. London: Departmentof Health.

Glover, S., Burns, J., & Stanley, B. (1994). Goal attainment scalingas a method of monitoring progress of people with severelearning disabilities. British Journal of Learning Disabilities, 22,148–150.

Kiresuk, T. J., & Sherman, R. E. (1968). Goal attainment scaling:A general method for evaluating comprehensive community



128

mental health programs. Community Mental Health, 4, 443–453.

Kiresuk, T. J., Smith, A., & Cardillo, J. E. (Eds.). (1994). Goalattainment scaling: Applications, theory and measurement.Hillsdale, NJ: Lawrence Erlbaum Associates.

Lemon, J., & Burns, J. (1996). Applying GAS to evaluate AIT:Using goal attainment scaling (GAS) for service evaluation.London: Wandsworth Community Team for People withLearning Disabilities, Wandsworth Health Authority.

Miles, M. B., & Huberman, A. M. (1994). Qualitative dataanalysis: An expanded sourcebook. London: Sage.

NHS Executive. (1998). Signposts for success: Commissioning andproviding health services for people with learning disabilities.London: Department of Health.

SPSS. (2001). SPSS base II for Windows (statistical package for thesocial sciences). Chicago, IL: SPSS Inc.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research:Grounded theory techniques. Thousand Oaks, CA: Sage.

Turnbull, J. (1998). Evaluating health care using goal attainmentscaling. Nursing Standard, 12, 35–38.

United Kingdom Central Council for Nursing Midwifery andHealth Visiting. (2001). Professional self-regulation and clinicalgovernance. London: Author.

Young, A., & Chesson, R. (1997). Goal attainment scaling as amethod of measuring clinical outcome for children withlearning disabilities. British Journal of Occupational Therapy,60, 111–114.

Young, A., Harvais, V., Joy, D., & Chesson, R. (1997). Occupa-tional therapy: Made to measure. British Journal of Therapyand Rehabilitation, 4, 26–30.

Implementation of Goal Attainment Scaling in Community Intellectual Disability Services

Documents

Transcript of Implementation of Goal Attainment Scaling in Community Intellectual Disability Services