AGENCY-UNIVERSITY PARTNERSHIP FOR EVIDENCE-BASED · PDF fileAGENCY-UNIVERSITY PARTNERSHIP 57...

AGENCY-UNIVERSITY PARTNERSHIP FOR EVIDENCE-BASEDPRACTICE IN SOCIAL WORK

Jennifer L. BellamyUniversity of Chicago

Sarah E. BledsoeUniversity of Nortii Caroiina at Chapei Hill

Edward J. MullenColumbia University

Lin FangUniversity of Toronto

Jennifer I. ManuelNew York State Psyciiiatric Institute

Little is known ahout evidence-based practice (EBP) in social service agencies

beyond studies of researcher, practitioner, and educator opinions. The Bringing

Evidence for Social Work Trairüng (BEST) Project involved 16 participants from

3 social service agencies. The experiential training, delivered by 2 doctoral stu-

dents, focused on a team-identified practice issue and followed the EBP process

of motivation, question formulation, search, evaluation, and application plan-

ning. Posttrairiing focus group data were analyzed. Results suggest that univer-

sity researchers who based in schools of social work can successfully collabo-

rate with agencies to support the process of identifying, evaluating, and dis-

cussing the application of research evidence in practice. University-agency

training partnerships should be considered as 1 of many potential strategies for

advancing EBP in social work.

EVIDENCE-BASED PRACTICE (EBP) as a profes-

sional model is a frequently discussed and

debated topic in social work education.

Beyond studies based on descriptions of

researcher, practitioner, and educator opin-

ions of EBP, relatively little is known about the

implementation of EBP in social service agen-

cies. This article describes data from the

Bringing Evidence for Social Work Training

(BEST) Pilot Project, in which three New York

City social service agencies engaged in part-

nership with university-based researchers.

Specifically, the BEST Project used a team for-

mat to explore how agency-university collab-

orations might promote and support the use

of research evidence in practice. The BEST

Project design included 10 teaching modules

designed to increase the agency teams' knowl-

edge of EBP, improve research skills related to

EBP, foster positive attitudes toward EBP, and

Journal of Social Work Education, Vol. 44, No. 3 (Fall 2008).Copyright © 2008, Council on Social Work Education, Inc. Ail rights reserved. 55

56 JOURNAL OF SOCIAL WORK EDUCATION

identify factors that promote use of EBP insocial service agencies. This article describescommon themes across each of the three part-nerships related to knowledge and attitudesabout EBP, current and future use of EBP inpractice, barriers, solutions, and promotersidentified by team members through partici-pation in the BEST Project.

Background and Significance

The conceptual framework of EBP used by theBEST Project defined EBP as a professionalmodel of practice. This definition, originallyproposed for use in medicine, is frequentlydescribed as a series of steps encompassingthe adoption and implementation of a collec-tion of values and activities (Sackett,Rosenberg, Gray, Haynes, & Richardson, 1996;Sackett, Straus, Richardson, Rosenberg, &Haynes, 2000). Gibbs (2003) applied thismodel to human service settings by outliningseven steps involved in EBP: motivation,question development, search for relevantresearch, research appraisal, application, eval-uation, and dissemination. These steps aredesigned to engage practitioners in a continu-al process of evidence identification, evalua-tion, and application in partnership withclients to improve practice outcomes. Variousframeworks, ethics, models, toolkits, andguidelines illustrating how the medical EBPapproach might be amended and adjusted forapplication in social service contexts havebeen developed and described by a number ofresearchers (Anderson, Cosby, Swan, Moore,& Broekhoven, 1999; Gibbs & Gambrill, 2002;Rosen, Proctor, Morrow-Howell, Auslander, &Staudt, 1993). However, a recent review of the

literature suggests that none of these modelsor practice guidelines has found wide accept-ance or been adopted on a large scale in socialwork agencies (Bellamy, Bledsoe, & Traube,2006).

Despite a growing interest in EBP as aprofessional model of practice for social work,few studies have systematically examinedstrategies to facilitate the adoption of EBP(Addis, 2002; Gira, Kessler, & Poertner, 2004).As Kirk and Reid (2002) noted, althoughsocial work researchers have identified usefulpractice knowledge, the processes by whichnew knowledge can be applied throughoutthe profession remain largely unexplored inresearch. Indeed, much of the literatureregarding EBP in social work has focused onthe numerous challenges to widespread incor-poration of EBP in practice (Bellamy, Bledsoe,& Traube, 2006; Mullen, Shlonsky, Bledsoe, &Bellamy, 2005).

Efforts to increase EBP can be informedby related studies of implementation and in-novation research. Two groups of researchers(Fixsen, Naoom, Blasé, Friedman, & Wallace,2005; Greenhalgh, Robert, Macfarlane, Bate, &Kyriakidou, 2004) have drawn from a range of

• disciplines to provide thorough reviews of therecent findings of implementation researchand theoretical development. Although it is arelatively new discipline, implementation re-search has emerged across diverse fields andhas identified mechanisms, such as interorga-nizational networking, that may support theuse of EBP as a model of professional practiceQohnson & Austin, 2006; Mullen, Bledsoe, &Bellamy, in press). However, interorgarüza-donal networking has not been widely ap-

AGENCY-UNIVERSITY PARTNERSHIP 57

plied to the diffusion of innovations (Green-halgh et al., 2004). Social work agencies andschools of social work often form organiza-tional networks to provide field-learningopportunities for students and venues forresearch on practice. These partnerships pro-vide a natural opportunity and rarely studiedvehicle to facilitate EBP in practice settings.

The BEST Project was a pilot demonstra-tion designed to address barriers to EBP im-plementation identified in prior research, testthe feasibility and acceptability of a universi-ty-agency partnership with social work agen-cies focused on EBP, and collect exploratorydata regarding the experience of that partner-ship from the agency perspective. Based on areview of the literature and interviews withexpert researchers (Bellamy, Biedsoe, & Traube,2006), four significant categories of barriers tothe implementation of EBP were identified: (1)lack of knowledge about how to best access,critically evaluate, and translate evidence foruse with clients; (2) lack of fit of currentlyavailable evidence with practice needs andpopulations; (3) suspicion and distrust for evi-dence and EBP based on objections related topolitical, ethical, or control issues; and (4) lackof resources for the training, materials, andstaff time necessary to research the evidence.The BEST Project sought to address these bar-riers by providing training to agency-basedteams that was designed to increase accept-ance and knowledge of EBP. Concurrently, theBEST Project established partnerships withagencies to both troubleshoot anticipated bar-riers and identify potential promoters of EBP.This article presents findings from data col-lected during focus group discussions con-

ducted with agency staff following the com-pletion of the university-agency EBP training.Other data and findings from the BEST Projecthave been published and presented elsewhere(Bellamy, Biedsoe, & Traube, 2006; Mullen, Bel-lamy, & Biedsoe, 2005; Mullen, Bellamy, Bied-soe, & Jean Francois, 2006; Mullen, Biedsoe, &Bellamy, 2008).

MethodSample

Given the exploratory nature of this study andthe emphasis on partnership, the researcherswho were based at a school of social work(hereafter, the research team) collaboratedwith a convenience sample of three New YorkCity agencies that were willing to collaboratewith researchers on an EBP-focused project.After initial meetings with senior agencyadministrators to secure their agreement toparticipate in the project, these administratorsselected a team of 4 to 6 project participants(hereafter, the agency team). These threeteams included 16 agency participants. Table 1presents the breakdown of participants bytheir agency affiliation and role.

All but 1 agency team member, who left astaff position before the project's end, wereincluded in pretraining and posttraining focusgroup discussions. Participants were not com-pensated for their participation; however,study activities occurred during workinghours. The university institutional reviewboard approved the project for human partici-pant research. The research team approachedfour agencies as potential partners, and threeagencies fully engaged in the project and par-ticipated in the EBP training. The fourth agency


expressed interest in parficipating and metwith the university team; however, this agencychose not to proceed because external policychanges prompted unforeseen restructuring inthe agency. No data from this agency areincluded in this arficle.

Intervention

To allow for collaboration, the research teampresented a flexible plan to the agencies dur-ing initial project meetings. This process start-ed with the chief executive officer at eachagency followed by meetings with supervi-sors and staff who would ulfimately composethe agency teams. Including the agencies ascontributing partners at the earliest stages ofthe project was an explicit effort to addresssuspicions and distrust of research andresearchers identified in previous research(Bellamy, Bledsoe, & Traube, 2006). As aresult, agency teams guided and shaped nianyof the project details. Each agency team indi-vidually collaborated with the research team.A senior faculty member led the BEST Project,and doctoral students served as project coor-dinators, developed resource materials, andengaged directly with the agency teams. Theresearch and agency team members firstworked together to select a pracfice-relevanttarget problem and developed a plan to

address the problem through a hands-on, EBP

training experience.

Training Description

Each agency team parficipated in a series oftrainings delivered by the research teamregarding EBP. The research team providedtechnical assistance and participated in thegroup process. The training, which was thecore of the BEST Project, was based on themodel of EBP for social work described byGibbs (2003). The braining consisted of 10modules (available. for download from theColumbia University Wilkna & Albert MusherProgram Web site):

1. EBP introducfion and overview

2. Question selection3. Overview of research evidence4. Search tools5. Search demonstrafion6. Troubleshooting the search7. Evaluating the evidence8. General findings and observafions9. Synthesizing evidence found

10. Action plan

These targeted the first four steps of EBP:motivafion, quesfion development, search forrelevant research, and research appraisal.

TABLE 1. Participants by Agency Affiiiation and Roie (/V=16)

Participant Type

AdministratorClinical supervisorFront-line staffStudent

Agency A(n)

2

4

0

0

Agency B(n)

1

3

0

0

Agency C(n)

1

0

4

1


Thus, the teams transformed target problemsinto researchable questions, and the researchteam provided technical support for theagency teams' efforts to locate, evaluate, andappraise evidence to seek answers to theirrespective EBP questions.

Data and Measures

A pretest-posttest design was used, and qual-itative data were collected during two roundsof focus groups conducted separately witheach of the three agency teams, for a total ofsix focus group discussions. Descriptive im-plementation-process notes, recorded by theresearch team members, were another princi-pal data source. Participants provided demo-graphic data using a self-administered, anon-ymous questionnaire completed prior to thefirst training module.

Focus groups were conducted using asemistructured format guided by a protocol ofquestions and probes developed by the re-search team based on earlier research (Bel-lamy, Bledsoe, & Traube, 2006). The initialfocus group discussions were designed to col-lect baseline data. Participants were asked todescribe their knowledge of and past experi-ence with EBP, barriers to EBP use in theiragency, and EBP-enabling factors (hereafter,promoters) they had experienced or anticipat-ed prior to using EBP in their agency. The sec-ond round of focus group discussionsoccurred 8 to 12 weeks following the training.In these follow-up focus groups, participantswere asked about current knowledge andopinions regarding EBP, the use of research inpractice, perceptions of barriers and promot-ing factors they perceived in using EBP, theirexperience with the training program used in

the study, and their plans for future use ofresearch and EBP.

Data Analysis

Quantitative analysis. Descriptive data includ-ing demographic and professional informa-tion collected through questionnaires wereanalyzed using SPSS software (2006). Allavailable data were reported, and each vari-able reported had no more than one missingdata point. •

Qualitative analysis. The following strate-gies were used to improve the reliability andvalidity of qualitative data: triangulation ofdata and sources, multiple coders, and mem-ber checking. The data were triangulatedthrough multiple forms of data: audio record-ings, interviewer notes, and verbatim tran-scriptions. Data from the focus group discus-sions were transcribed and then analyzedusing NVivo 7.0 (QSR International, 2006)qualitative software. Major themes weredeveloped a priori based on findings of anearlier project phase (see Bellamy, Bledsoe, &Traube, 2006, for description and review offindings) and implemented as tree nodesusing qualitative methods described byKrueger and Casey (2000). Focus group tran-scripts were coded according to each of themajor themes. The research team discussedother major themes that emerged during thiscoding process and when consensus wasreached, they were designated as tree nodesand coded accordingly Coding was reviewedand refined into subtheme categories for eachmajor theme. Two research team memberscoded the transcripts independently to ensurereliability. Coding classitication of the tran-scripts was cross-reviewed by each coder. Any


ambiguities and coding discrepancies wereresolved by reviewing the focus group tran-scripts and reaching consensus. Memberchecking was accomplished by presenting thepreliminary findings to the agency teammembers and administrators. Table 2 presentsthe coding themes and subthemes identifiedin the qualitative analysis of the focus groups.

Agency-University PartnershipDescription

Participant agencies differed in terms of theirprogram structure, location, and populationserved (Table 3). Agency A provides a varietyof child and family-oriented programs in ahistorically Latino immigrant neighborhood.

TABLE 2. Coding Tiiemes and Subtiiemes

Theme Subtheme

EBP Knowledge

Attitudes about EBP

Current and Future EBP Use inPractice

Barriers

Solutions and Promoters

Definition of EBPNew knowledge and skills

PositiveMixed or neutralNegative (suspicion)

Current use: Changes in making practice decisions andCurrent use: EBP skills and resources (from training) usedIndividual/agency likelihood of using EBP in the future

Agency-level factors/cultureSuspicion (see negative attitudes)Lack of fit/misuseLack of knowledge, skills, training, supervision/monitoringLack of resources (time, access, funding)

Broad organization factors/cultureResources (time, access, funding)Knowledge, skills, training, supervision/monitoring

Positive attitudeMore research

New trend/vogueUniversity-level collaboration/partnershipExternal influences(information/network sharing)Support from other agencies


Agency B is a large, multisite organization

serving diverse populations and needs across

the New York City metropolitan area. Ad-

ministrators at this agency selected a single

program focus for their BEST Project partici-

pation: a multisite housing program primarily

serving persons with both severe and persist-

ent mental illness and chemical abuse, alcohol

abuse, or addiction. The third participating

agency. Agency C, delivers health care servic-

es to primarily Asian communities in the con-

text of a community health center. At Agency

C, the project team worked within the mental

health services unit of the health center.

Sampie Ciiaracteristics

The sample characteristics of the BEST project

participants are presented in Table 4. The

mean age of the participants was approximate-

ly 35 years and the majority was female. Most

participants identified themselves as either

Asian or Caucasian. Over half of participants

reported an annual salary between $35,000

and $40,000. Participants represented a range

of educational backgrounds with the largest

percentage holding a master's degree. Most

participants reported that the focus of their

degree was social work, and half of the partic-

ipants held a social work license. Participants

had worked an average of more than 5 years at

their agencies, and all but one participant, who

was a student, reported working full-time.

Results

EBP Knowiedge

Definition of EBP. Comparison of participant

responses at baseline to the responses at foUow

up showed a general pattern of positive

changes in participant perceptions of EBP. At

baseline, definitions of EBP were relatively

vague in nature, with both staff and administra-

tors describing EBP in terms of individual ele-

ments, including methodology, results and out-

comes, particular models or tools, and stan-

dards of practice that they heard about from

various sources such as conferences or students.

Typical participant definitions of EBP included,

"It is associated with data collection and

results" and "Something that's been already

investigated, that's proven, that works—a stan-

dard for us to go by in our own practice." In

contrast to these imprecise definitions of EBP,

the definitions provided in the follow-up

TABLE 3. Brief Agency Overview

Variable Agency A Agency B Agency C

Structure Family Services Multiprogram, Commimity health center

multisite

Chosen program focus Child and family Housing and mental Mental health

health

Primary population Primarily Latino MICA^ population Primarily Chinese American

immigrant immigrant community

*MICA=Mental illness and chemical addition.


focus groups were more specific and related

to process, incorporating clinical experience,

keeping an open mind, and disseminating

knowledge. Typical participant comments

from the follow-up focus groups included

these defirütions of EBP:

[EBP is about] investigating certain

techniques and finding evidence to

back it up in research, [and efforts to]

standardize it where other people

could replicate [those techniques].

(Agency B participant)

It's like a process, not only based on

research findings, but also part of your

clinical experience of what works and

doesn't work and it's ongoing.

(Agency C participant)

[EBP] is trying to keeping an open

mind to testing empirically. (Agency C

participant)

The language used by participants in the

follow-up focus groups generally reflected the

language and concepts discussed throughout

the EBP training.

Overall, participants demonstrated a

clear understanding that the current flow of

knowledge from research to practice is inef-

fective and inefficient, and this lack of com-

municafion was a problem. Overall, most par-

TABLE 4. Sample Demographic and Professional Characteristics (JV=16)

Variable Mean or nStandardDeviation Percentage

AgeFemaleRace

AsianBlack/African AmericanCaucasian/White ,Hispanic/Lafino

Salary$35K or less$35K-$50K$50K or more

EducationBachelor'sMaster'sPhD

Degree focusSocial workOther

Social work licenseYears workedHours worked per week

34.914

6361

294

682

13386.3

36.17

9.1-

--

--

--

--7.55.8

87.5

37.518.837.5

6.3

12.556.325.0

37.550.012.5

81.318.750.0


ficipants expressed an interest and willing-ness to make efforts to address that issue. Thewords of 2 participants summarized the opin-ions expressed by many:

There's a gap between what's beendone on the research level and what'sbeing done everyday on the front lines.It would be great to break that and it'shard to do. I know we came up withsome ideas about trying to implementsome of this research knowledge inmore day-to-day functions here. Ithink it would be interesting and help-ful to do that. (Agency A parficipant)

I think it's exciting to learn new tech-niques and new interventions, and Ithink it's important to know if thoseintervenfions that you do use are suc-cessful or not and just to keep up todate with the current research. (AgencyC parficipant)

Some parficipants lirJced their positive per-ceptions of EBP directly to their experiencewith the BEST Project training. One agencyteam member reflected on the results of herhands-on search in the trairüng by saying,"I'm really excited about the assessmentand...the [evidence] we found. I thirik [theidenfified assessment instrument] is a goodthing to try out" (Agency B parficipant).

New knowledge and skills. Participantsspoke posifively about the skills learned in thetraining, but many comments centered onhow the parficipants felt that they would beable to apply some, but not all, of the steps

they learned. Most parficipants commentedthat learning newer, more efficient skills forconducting searches was among the mosthelpful aspects of the trairüng. One partici-pant remarked on the future value of thesesearch skills:

The searching stuff, with the Web sitesyou offered. We could get on the com-puter and [search] and see what wecome up with. I think if people couldread stuff together, almost like a bookreview, [and] talk about it that wouldfeel almost in sync with how weapproach issues and challenges in ourdepartment. (Agency A parficipant)

Furthermore, parficipant comments indicatedthat the hands-on nature of the trairüng wasuseful to parficipants, parficularly the integra-fion of these skills and approaches into theexisting culture and structure of the agency.Another pracfifioner noted how the trainingmade idenfifying and using evidence seempracfical and effecfive and how it was "stream-lining" her approach to researching evidence.

Despite the knowledge, skills, and in-crease in posifive atfitudes described by parfic-ipants, as well as the support and guidance ofthe research team, some steps of the EBPprocess remained challenging. Nearly all ofthe parficipants commented on their struggleto understand the sophisficated aspects ofresearch, whether that meant interpretingresearch methodology or calculating compari-son stafisfics. One parficipant said, "To me, theevaluafions of the research, it's like really com-plicated, and the stafisfical stuff, to me, I start


to zone out" (Agency A participant). Anotherparticipant commented on the difficulty ofevaluating the technical aspects of theresearch:

Unless you've really looked at a lot ofresearch—like I still don't know whichstudy's data is really that clean or theway they went about it, or if the meth-ods are really that good; and I thinkhaving a [statisfic] helps you decidethat. But, I can't see myself goingthrough 15 articles and calculating [theeffect size]. (Agency C participant)

Even though participants said they found theEBP process intimidating, a few participantsexpressed a desire to leam more regarding theevidence that supports practice mandates andguidelines that are required by funding andregulatory organizations like the state office ofmental health. One participant described thisas, "being able to really look at it [practiceguidelines] through research and really feel,based on what we were shown, that [the prac-tice guideline] was valid and appropri-ate"(Agency A participant). Some participantsalso wanted to have easier access to reviews ofevidence that suggested clinical practiceguidelines for clinical practice.

Most participants wanted more trainingwith additional hands-on experience to leamthe tools and techniques necessary to use anEBP model of practice in their work. Forexample, 1 participant said, "I think I wouldneed more time with [training]. You know,like how to pick [information] out of the arti-cle. That's something I would really have tostudy" (Agency B participant). Participants

said judging the quality of evidence was a"very challenging" task. In general, thesepractitioners indicated a lack of confidence orsaid their EBP skills were insufficient to usesuccessfully all of the EBP techniques includ-ed in the training modules.

Attitudes About EBP

The agency teams' understanding of EBPchanged not only in terms of a definition butin terms of acceptance of EBP as a profession-al model of practice. In the second roxmd offocus groups, participants described feelingmore positive toward the concept of EBP andmore motivated to use EBP in their agency.One participant said, "In the beginning myview was like, [EBP] is a dirty word" (AgencyA participant). After participating in the BESTProject EBP training, this same participantdescribed EBP as "intuitive" and "thought-ful." Similarly, some agency team membersreported that their participation in the train-ing had given them a "fuller view" of EBP anda clearer understanding of this model of prac-tice. This expanded understanding of the EBPprocess seemed to increase acceptance of theEBP model. The participants' identification ofthe benefits associated with engaging in EBP,such as the potential to improve services andincrease accoiintability to clients, was relatedto their changes in attitude.

Positive changes in attitudes toward EBPalso appeared to stem from the clarification ofEBP as a model that includes client prefer-ences and clinical expertise. In particular, par-ticipants seemed to appreciate dispelling theirmisconceptions about EBP with the affirma-tion that clinical knowledge and expertise areessential components of EBP. One participant


said, "It feels much more tailored to wherewe're coming from and what our interests andvalues are, and not like there's going to put[constraints] onto our practice" (Agency Aparticipant). Another participant remarked onhow the EBP training had changed his/herperception of EBP from an imposed practice toa model that could be tailored to the specificsneeds of the agency:

It's helping us understand how to findthe areas of research that are missingand figure out places where other con-tributions need to be made; rather thanfeeling like this is the cookie cutter thatwe have to go with and that's the endof the story. (Agency A participant)

The importance of flexibility and contextualdifferences across agencies and programswere consistently mentioned in the focusgroups.

In addition, participants described howengaging in EBP was affirming and couldbuild confidence in their work. A few partici-pants noted that their agencies had currentpractices in use that they felt appropriate andeffective, but they had not identified evidenceto support these opinions. One participantnoted that EBP would be helpful to agenciesin this type of situation and said of readingabout research findings, "I think we wouldlearn new things and we'd be [affirming prac-tices] that we had a good hunch about"(Agency A participant). This participant alsoreported a willingness to modify current prac-tices if research evidence did not supportthose approaches. "We could jettison thembecause we [would] feel confident that there

isn't much reason to believe that such andsuch [intervention] would be helpful." EBPwas described as a way to improve practiceoutcomes, develop professional skills such ascritical thinking, be responsible to clients, andbe more efficient in staying up to date on cur-rent research knowledge.

However, after participating in the train-ing, a few participants expressed negativefeelings or concerns regarding the basic con-cept of EBP. Comments offered by 1 practi-tioner indicated that this participant still asso-ciated EBP with a "watered down" attempt toincorporate a "medical model" into socialwork in a "very unhelpful" way (Agency Aparticipant). Another participant at the sameagency said that theory did not play a largeenough role in EBP. Although other partici-pants voiced concerns about implementingEBP, these concerns were regarding practicalbarriers to the application of EBP, rather thanany fundamentally negative attitudes or per-ceptions of the concept as an approach topractice.

Current and Future Use of EBP inPractice

The steps of the EBP process included in theBEST Project training were applied by partici-pants to their practice, but because of the chal-lenges and complexities in practice, the EBPsteps were only partially applied and fidelityto the full EBP model of practice was notmaintained. Participants described doing "alittle bit of research" and "some of the steps"or applying certain tools that they found par-ticularly helpful "here and there." Althoughmany of the participants described plans touse their new EBP skills in the future, most


participants reported that they had not usedEBP tools and approaches outside of the train-ing sessions. Practitioners described theiranticipated use of EBP training tools asincluding simple approaches and adaptationsto fit the unique needs of their agency'sresources, culture, and context. They de-scribed their planned initiation of EBP as pro-ceeding on a smaller scale than the EBP modelpresented in the training. One participantsaid, "You might find something useful, butgiven the resources and the reality of whatyou have, you may not be able to totally usethe entire thing" (Agency B participant).Another participant who was both hopefuland cautious said, "If it's something that wecare about and talk about and try to model toa certain extent, I think we can definitely makeprogress" (Agency A participant). A piecemealapproach to EBP was apparent in commentsparticipants shared regarding both current useand future plans to use an EBP model of prac-tice. For example, a consistent strategy acrossagencies was to incorporate an EBP model intothe current agency context by focusing exist-ing staff training or supervision time on EBPrelated activities, rather than asking staff toadd to their current workload. This approachwas largely in response to a common barrier,lack of time, identified by all three agencies.Participants described incorporating EBPpractices into "daily practice," either by usingflexible resources, such as asking students infield placement rotations to search for evi-dence, or by using existing mechanisms, suchas journal clubs and staff meetings, supervi-sion, or in-house continuing education pro-grams, to secure staff time to focus on EBP.

In addition, participants mentioned need-ing help moving from the research aspects ofEBP to the implementation of interventions,techniques, or tools identified through theirsearch for evidence. One team member said.

If we did decide that there were someapproaches or interventions that wereuseful, there is still a whole piecemissing, and that's how we wouldpractice and how we would actuallyimplement—with whom, why, when—and that's not in the literature, that'swhere the creative part is (Agency Aparticipant).

The application of knowledge was some-thing that participants described as both nec-essary and challenging. Another participantsaid,

I think it's worthwhile. It just seems thatthere are always time constraints inactually implementing it. Also [it has tobe] a priority, or you can't just go alongwith what you know, and snatch stuff,and try to incorporate [EBP] with whatyou're doing (Agency B participant).

Another participant described needing help tofind ongoing support for training, supervising,and evaluating any new practices implement-ed in their agency (Agency A participant).

Participants also noted the need to alterthe approaches described in the literaturebased on the needs and culture of the popula-tions being served. One participant's com-ments summarized the thoughts of many:


In terms of the fit, so it almost wouldrequire another piece of training orknowledge around how do you judgeevidence that's been produced out of acompletely different population...wehad difficulty finding research dedicat-ed to the population that looks like[our clients] (Agency C parficipant).

Parficipants were parficularly troubled by thelack of research dedicated to ethnic or racialminority and immigrant groups who consti-tuted a large percentage of the client base ofthe three agencies. One parficipant reacted tothe narrow scope of research evidence that sheuncovered throughout the training: "This isthe closest to our [pracfice] quesfion we couldget? With all White samples, like, inMinnesota?" (Agency A participant). Shevoiced a concern shared by many of the par-ticipants that research samples were not simi-lar to the clients served by the parficipatingagencies. Although most participants wereskepfical about their ability to fully incorpo-rate EBP into their work and many reportedfeeling overwhelmed by barriers, most voiceda desire to keep fiying.

Barriers

Participants in the follow-up focus groupsidenfified numerous barriers to adopting EBP,including (a) lack of resources (e.g., fime,access to research, and funding), (b) lack ofknowledge (e.g., research skills, training, andsupervision for staff), (c) suspicion of the EBPconcept (i.e., it is a medical model that doesnot include theory), and (d) lack of fit (i.e.,available evidence is either irrelevant or insuf-

ficient). These barriers were consistent withthose identified in the baseline focus groups.In addition, follow-up focus group discus-sions reaffirmed the idenfificafion of practi-fioner experience as an EBP barrier. At base-line, participants described how EBP was auseful approach for students or new practi-tioners but not necessarily helpful for sea-soned practitioners. Similarly, commentsshared at follow up showed that parficipants'percepfions of EBP were unchanged in thisrespect, as reflected in the comments of 1 par-ticipant, who said.

It depends on where you're comingfrom. If somebody is coming from away of pracfice of doing something fora long period of fime, you're going tohave a harder fime changing them.While otherwise, you won't have theresistance. (Agency B participant).

Some parficipants also noted that accredi-tafion, government, or other monitoring bodieshanded down EBPs or related direcfives with-out considerafion for real-world pracfice issuesor the need for support when implementingpracfice innovafions. Parficipants said theywanted a pracfice model that offered flexibilityand a way to make adjustments if the recom-mended approach was not working in pracfice.It is worth noting that parficipants describedconflicting pressures that either advocated forEBP or discouraged the use of the EBP model.In part, this conflict appeared to stem from abreak in communicafion between researchersand front-line agency pracfifioners. Theorefi-cally, EBP includes a constant feedback loop


from research to practice and back; however,participants reported that they did not feelthat the infrastructure for this kind of commu-nicafion was in place to connect their pracficalexperience back to researchers, or to providethem with ongoing support by researchers.

Soiutions and Promoters

Agency teams idenfified a number of promot-ers for EBP as a model of professional practice.Many of the factors promoting EBP mirroredthe barriers to EBP implementation previous-ly mentioned and included funding, recruit-ment, and retention of qualified staff, time,access to Internet resources, and an increasedresearch base. Team members identified bothagency culture and structure as targets forchange in promoting EBP. Participants furtheridenfified the explicit commitment of timeand resources toward EBP-related activifies asone of the important promoters related to theculture of an agency. One participantremarked,

I think you realize [EBP is] pretty fimeconsuming, and I don't know if theagency is really pushing that way, buteven if they did, there is going to be aclash between the fime we're going tospend on this versus the time thatyou're supposed to be practicing clini-cally. (Agency B participant)

The university-agency partnership pro-vided this type of protected time dedicated toEBP-related activities in the context ofplanned group work and training sessions. Inaddifion, agency and staff acceptance of theEBP concept, described as "buy-in," were also

important promoters of EBP. For example,participants representing one agency wherefew front-line staff had training at the collegelevel expressed the opinion that better com-pensated and more skilled professional staffwould be essential to the success of EBP with-in their agency.

Parficipants clearly saw the group contextas key to implementing EBP. When asked howthey anticipated continuing EBP at theiragency, 1 participant said, "Probably as agroup process. We probably would divide itinto different tasks, smaller tasks, among peo-ple" (Agency C parficipant). A parficipantfrom a different agency said, "What we wouldhave to do is group process. Everybodywould share the things that are working [best]and everyone studies together" (Agency Bparticipant). Whereas some participantsviewed breaking down EBP responsibilifiesinto manageable steps as a key to moving for-ward, others spoke about larger organization-al level changes. For example, 1 team membersaid that EBP should be, "Not really a case-by-case thing, but how we want to approachthings" (Agency B participant). In addifion,participants described the need for an agency-wide assessment of needs and resources toensure that EBP programs are feasible.

Parficipants also described roles for pro-moting EBP that could be played by profes-sional entities such as licensure boards,accreditation committees, and funding bodies.The role for these entities was described mostfrequently as providing access to the researchliterature, facilitating service quality bystrengthening state licensure requirements,and providing continuing education pro-grams on EBP. One participant said, "There's


no requirement for ongoing skill developmentlike there is in the medical field. You don'thave to receive continuing social work educa-tion hours to maintain your license" (AgencyA participant; at the time the BEST Project wasconducted. New York state did not requirecontinuing education to maintain a socialwork license.)

Furthermore, participants addressed theimportance of combining clinical and researchknowledge to implement EBP. This conver-gence was described as both ongoing support,such as EBP skills training for supervisors,and technical assistance delivered from anoutside source. Several participants expressedan interest in continuing the agency-universi-ty partnership as a way of meeting the needfor support and assistance. One participantstated,

I think if the universities could seethemselves as having an ongoing rolenot just graduating people out brandnew but having a role maintainingworkers while they're in their work,and helping them find these kinds ofresources or hone their skills; to havean [university] available for thatwould be really nice. We don't general-ly have someone like you [the researchteam] or anyone to call up and say,"We really wanted to get an article onX or y."...or "What do you think aboutthe key words or where would youstart?" (Agency A participant)

Moreover, participants recognized their strug-gles with evaluating and implementingresearch evidence, and wanted help with

engaging in and monitoring these aspects ofthe EBP process.

Participants also recognized that limitedaccess to research resources was a barrier toimplementing EBP in their agencies. Par-ticipants felt that other professions do a betterjob of keeping graduates connected toresearch-evidence resources (e.g., the medicalprofession) and might serve as a model forsocial work. Participants described ideasabout university-agency bridges such aslibrary privileges for alumni, access to currentcourse syllabi and literature reviews pro-duced in graduate coursework. The commentof 1 participant summarized the opinionshared by many: "I think school has a role...toprovide alum with access [to evidence]. Youlook at medical school, and all MDs haveaccess to libraries at universities" (Agency Cparticipant). In addition, several participantsnoted a potential role for students as bridgesbetween their tiniversities and the agencies.One participant related her agency's plan tomove toward EBP:

Next year, we are hoping that our grad-uate students will buy into our newrequirement, which is that for each ofour topics that we address in our case-work meeting, they will provide litera-ture from a lit review source. . . . Withthat process in place, hopefully we willstart to inform some of our decisionsabout what we do in clinical settingsand with clients from the articles weread. (Agency A participant)

Many participants noted that students hadgreater access to research resources, such as


university subscriptions to journals, whichwere generally unavailable to agencies. Inaddition, participants observed that studentshad more experience and training in how tosearch for evidence as well as how to evaluateevidence. One participant stated, "Most of usthink that once we are out of school, we justput this [research] behind, so the link for uswould be our students" (Agency A partici-pant). However, participants also acknowl-edged the limitations of students. A differentparticipant at the same agency said, "The stu-dents can only take us so far. This is about ourpractice and students are visitors." Anotherparticipant said better coordination betweenthe classroom research experience and thefield experience would benefit both agenciesand students.

The final promoter of EBP identified byparticipants was novelty or innovation.Nearly all participants said that they thoughtof EBP as a cutting-edge approach to practice,and, therefore, desirable. In addition, the per-ception of EBP as an innovative approach topractice also appeared to be related to pres-sure from external forces on agencies to adoptEBP. Most participants in the follow-up focusgroups expressed the opinion that agenciesthat are under pressure to do something "newand improved" or demonstrate improvementin service delivery might find the "trend" or"vogue" associated with EBP alluring.

Discussion

Results from this study suggest that EBP train-ing and partnership with researchers atschools of social work can be effective formotivating social work practitioners to adoptthe EBP model but are not sufficient to sup-

port the implementation of EBP. An objectiveof the BEST Project was to explore if such anintervention could increase practitioner moti-vation to use research evidence and promoteincreased use of empirical evidence in prac-tice. Although the National Association ofSocial Workers (NASW) Code of Ethics (1996)states that "social workers should criticallyexamine and keep current with emergingknowledge" and "fully use evaluation andresearch evidence in their professional prac-tice," this professional commitment is not eas-ily translated into the complex envirorunent ofsocial service agency practice. Studies haveconsistently indicated that social workersinfrequently draw on research to inform prac-tice and often prefer sources other thanresearch knowledge to inform their practice(see Gibbs & Gambrill, 2002; Kirk & Rosen-blatt, 1981; Mullen & Bacon, 2004; Rosen,1994).

Future work should carefully considerthe important first step of enhancing motiva-tion. If social workers do not accept and"buy-in" to the fundamental principles, val-ues, and processes of EBP, it is difficult, if notimpossible, to advance EBP as a professionalmodel of practice. A critical first step in pro-moting acceptance of the EBP model is toaddress and clarify the misconceptions of EBPand to demonstrate its usefulness for commu-nity agencies. Consistent with Gibbs' (2003)description of EBP, we found that motivatingsocial workers to use EBP is only the begin-ning of the process. Engaging in EBP requiresskill, resources, and commitment of time fromsocial work organizations and social workpractitioners, most of whom are busy deliver-ing, managing, securing funding for, and

AGENCY-UNIVERSITY PARTNERSHIP 7 1

tracking services to clients. Even when teamparticipants expressly protected their time forthe purposes of engaging in EBP over thecourse of this project, scheduling was difficult,interruptions were common, and competingdemands left little time for EBP-related activi-ties outside of the project.

If the successful application of EBP is tobe achieved, the profession will have to com-mit scarce resources—including time andmoney—toward the advancement of andinfrastructure for EBP-related activities suchas (a) seeking out and evaluating the fit andquality of emerging evidence, (b) securingtraining and resources necessary to imple-ment identified practices, and (c) continualunbiased reassessment of the quality, fidelity,and currency of agency practice. Consumingand applying research are challenging.Participants in the current study expressed aneed for ongoing assistance with these activi-ties and felt that they did not have the skills ortraining to engage fully in the EBP processwithout support. Furthermore, influencesbeyond the control of social service agenciesand universities, such as funding and policymandates, could influence the adoption andimplementation of EBP over time. Participantsin the current study felt that they receivedmixed messages about EBP from externalsources and did not have the support andinfrastructure they needed to apply researchin practice.

Given the complexity of these tasks,many stakeholders in the profession, includ-ing schools of social work, must supportsocial service agencies in this work. A univer-sity-agency partnership can offer a venue toimprove practitioners' attitudes toward.

knowledge about, and willingness to use EBP;however, these improvements alone are insuf-ficient to produce substantial changes in serv-ice delivery. Other resources at schools ofsocial work, which were not used in the cur-rent project or were used only to a limiteddegree, including libraries) information tech-nology departments, continuing educationprograms, and faculty experts, could be usedto support the advancement of EBP. The socialwork profession must adopt a multilevel,multimodal approach to support the wide-spread acceptance and use of EBP as a modelof practice (Proctor, 2004). In the absence ofefforts to support EBP as a professional modelof practice in the field, students and newlygraduated social workers will likely continueexperiencing a disconnect between classroom-based learning and practice in the field.

The BEST Project focus groups identifiedsome potential solutions toward addressingsome of the barriers described here and else-where (Bellamy, Bledsoe, & Traube, 2006). Theurüversity-agency partnership appeared notonly to be feasible and acceptable but desir-able to participant agencies. Furthermore,there is some evidence suggesting that inno-vations, such as EBP, are more likely to beadopted if the perspectives of potentialadministrators-users are engaged at the earli-est developmental stages (Gufstason et al.,2003). Our experience in the BEST demonstra-tion pilot project supports this finding. Thecollaborative partnership of the BEST Projectwas an explicitly planned and executed com-ponent intended to address anticipated barri-ers and challenges, including distrust ofresearch, researchers, and research knowledgethrough relationship building. Collaboration


supports the continuous communication feed-back loop inherent to EBP, which benefits bothresearch and practice communities. Par-ficipants in the BEST Project stated that theyneeded technical support and experfise, espe-cially the research knowledge that schools ofsocial work can supply, to continue theirefforts to incorporate EBP. Likewise, theresearch team gained access to a "natural lab"for EBP-related research at each agency.

Another key finding was the importanceof teams. Teams have proven a successfulstrategy in contexts involving the implemen-tafion of a complex technology (e.g., Edmond-son, Böhmer, & Pisano, 2001). Learning theEBP process can be intimidating to socialworkers attracted to the profession for itsperson-centered, as opposed to research-centered, focus. Working as a team diffusedthis anxiety and allowed team members tocontribute to the effort according to their indi-vidual skills and strengths. The teamapproach leveraged participants' tendency toseek knowledge from colleagues (Mullen,Bellamy, Biedsoe, & Jean Francois, 2006), andteams could capitalize on individual membertalents. Some parficipants were more experi-enced in the use of technology and otherresources. For example, students and otherswith university affiliations had access toimportant fee-based research documents,technology, and support resources. All agencyteams discussed ways to use student fieldplacements as a bridge to increasing EBPprocess at their agency in addition to betterconnecting the classroom and agency experi-ence using EBP as a framework. However, thissingle bridge between imiversities and agen-cies is tenuous and temporary. Instead, effec-

tive and consistent collaborative efforts per-formed by professionals who are skilled man-agers of interorganizational contexts, orboundary spanners, are recommended toencourage greater acceptance of EBP as amodel of professional practice (Williams,2002).

Last, flexibility and fit were importanttools. Users are more likely to adopt innova-fions, such as EBP, if users can modify theapproach to suit individual needs (Meyer,Johnson, & Ethington, 1997). Although theBEST Project used a similar basic process ineach agency (i.e., moving from administrafivenegofiations to agency team construcfion totraining implementation), each of the steps inthe process played out with flexibility. Forexample, although each of the agenciesreceived the same training curriculum, theyreceived the modules over different timeintervals to suit agency scheduling needs. Inaddition, even though all agencies involvedteams, the research team did not prescribe thestaff and administrative membership of eachagency team. There is evidence that a keyattribute of irmovafion is the adaptability ofthis so-called soft periphery, composed ofthose malleable elements of any technology orintervention (Denis, Herbert, Langley,Lozeau, & Trotfier, 2002).

Limitafions related to the design of thisstudy constrain the generaüzabüity of the find-ings presented. As previously described, thestudy sample was small and of unknown repre-sentativeness. Furthermore, this was anexploratory study examining the feasibuity ofimplementing hands-on EBP training in agencypracfice and identifying barriers and facilita-tors. Future research is needed to explore more


fuUy other dimensions of EBP training and sup-

port in a broader set of agencies, perhaps using

standardized and objecfive measures of impor-

tant processes and outcomes.

Conciusion

Efforts to support the widespread implemen-

tafion of EBP in social service agencies repre-

sent one way to address the current

research-practice gap. The BEST Project

demonstrates an attempt to address the cur-

rent research-practice gap, particularly

around barriers related to practifioners' nega-

tive atfitudes toward research evidence and

lack of understanding or misperceptions

about EBP concepts and process. Our findings

also suggest that university researchers can

collaborate with agencies and form effective

partnerships to support the process of identi-

fying, evaluating, and applying research evi-

dence in practice. However, a long-term part-

nership commitment is necessary to achieve

sustainable EBP. Social work pracfifioners and

social service agencies may not be equipped

to face the challenges presented by adopting

EBP as professional model of pracfice on their

own. Some agencies and practitioners are

willing to invest in partnerships with univer-

sities and engage in EBP; however, the social

work profession must identify and apply mul-

fiple multimodal strategies to change prac-

tices effectively on a broad scale.

References

Addis, M. E. (2002). Methods for disseminat-

ing research products and increasing

evidence-based practice: Promises, obsta-

cles, and future directions. Clinical

Psychology: Science and Practice, 9,367-378.

Anderson, M., Cosby, J., Swan, B., Moore, H.,

& Broekhoven, M. (1999). The use of

research in local health service agencies.

Social Science & Medicine, 49,1007-1019.

Bellamy, J., Bledsoe, S. E., & Traube, D. (2006).

The current state of evidence based prac-

fice in social work: A review of the litera-

ture and qualitative analysis of expert

interviews. Journal of Evidence-Based Social

Work, 3, 23-Í8.

Columbia University Willma & Albert Musher

Program, (n.d.). The Evidence Based Policy

and Practice Online Resource Center.

Retrieved August, 26 2007, from ht tp: / /

www.columbia.edu/cu/musher/Website

/Website/index.htm

Denis, J. L., Herbert, Y, Langley, A., Lozeau,

D., & Trottier, L. H. (2002). Explaining dif-

fusion patterns for complex health care

innovations. Health Care Management

Review, 27, 60-73.

Edmondson, A. C , Böhmer, R. M., & Pisano,

G. P (2001). Disrupted routines: Team

learning and new technology implemen-

tafion in hospitals. Administrative Sciences

Quarterly, 46, 685-716.

Fixsen, D. L., Naoom, S. F, Blasé, K. A.,

Friedman, R. M., & Wallace, F. (2005).

Implementation research: A synthesis of the

literature. Tampa: University of South

Florida.

Gibbs, L. E. (2003). Evidence-based practice for

the helping professions: A practical guide to

integrated multimedia. Pacific Grove, CA:

Brooks/Cole.

Gibbs, L. E., & Gambrill, E. (2002). Evidence-

based practice: Counterarguments to

objections. Research on Social Work Practice,

12, 452-476.


Gira, E. C , Kessler, M. L., & Poertner, J. (2004).

Influencing social workers to use research

evidence in practice: Lessons from medi-

cine and the allied health professions.

Research on Social Work Practice, 14, 68-79.

Greenhalgh, T., Robert, G., Macfarlane, E.,

Bate, P, & Kyriakidou, O. (2004).

Diffusion of innovations in service organ-

izations: Systematic review and recom-

mendations. The Milbank Quarterly, 82,

581-629.

Gufstason, D. H., Sinfort, R, Eichler, M., Adams,

L., Bisognano, M., & Steeudel, H. (2003).

Developing and testing a model to predict

outcomes of organizational change. Health

Services Research, 38, 751-776.

Johnson, M., & Austin, M. J. (2006). Evidence-

based practice in the social services:

Implications for organizational change.

Administration in Social Work, 30, 75-104.

Kirk, S. A., & Reid, W. J. (2002). Science and

social work: A critical appraisal. New York:

Columbia University Press.

Kirk, S., & Rosenblatt, A. (1981). Research

knowledge and orientation among social

work students. In S. Briar, H. Weissman,

& A. Rubin (Eds.), Research utilization in

social work education (pp. 29-35). New

York: Council on Social Work Education.

Krueger, R. A., & Casey M. A. (2000). Focus

groups: A practical guide for applied research

(3rd ed.). Thousand Oaks, CA: Sage.

Meyer, M., Johnson, D., & Ethington, C.

(1997). Contrasting attributes of preven-

tive health innovations. Journal of Com-

munication, 47,112-131.

Mullen, E. J., & Bacon, W. (2004). A survey of

practitioner adoption and implementa-

tion of practice guidelines and evidence-

based treatments. In A. R. Roberts & K.

Yeager (Eds.), Evidence-based practice man-

ual: Research and outcome measures in health

and human services. New York: Oxford

University Press.

Mullen, E. J., Bellamy, J. L., & Bledsoe, S. E.

(2005). Implementing evidence-based

social work practice. In P. Sommerfeld

(Ed.), Evidence-based social work—Towards

a new professionalism? (pp. 149-172). Bern,

Switzerland: Peter Lang.

Mullen, E. J., Bellamy, J. L., Bledsoe, S. E., &

Jean Erancois, J. (2007). Teaching evidence

based practice. Research on Social Work

Practice 17, 574-582.

Mullen, E. J., Bledsoe, S. E., & Bellamy J. L.

(2008). Implementing evidence-based

social work practice. Research on Social

Work Practice, 18, 325-338.

Mullen, E. J., Shlonsky, A., Bledsoe, S. E., &

Bellamy, J. L. (2005). Erom concept to

implementation: Challenges facing evi-

dence based social work. Evidence and

Policy: A Journal of Debate, Research, and

Practice, 2(1).

National Association of Social Workers.

(1996). Code of ethics. Washington, DC:

Author. Retrieved February 1, 2007, from:

http://www.socialworkers.org/pubs/co

de/code.asp

Proctor, E. K. (2004). Leverage points for the

implementation of evidence-based prac-

tice. Brief Treatment and Crisis Intervention,

4, ITJ-IM.

QSR International. (2006). NVivo qualitative data

analysis program. Version 7 [Computer soft-

ware]. Melbourne, Australia: Author.


Rosen, A. (1994). Knowledge use in direct

practice. Social Service Review, 68,561-577.

Rosen, A., Proctor, E. K, Morrow-HoweU, N., Aus-

lander, W. E., & Staudt, M. (1993). Systematic

planned practice: A tool for planning, implementa-

tion, and evaluation. Unpublished manual.

Sackett, D. L, Rosenberg, W. M. C, Gray J. A. M.,

Haynes, R B., & Richardson, W. S. (1996).

Evidence-based medidne: What it is and what

it isn't. British Medical Journal, 312,71-72.

Sackett, D. L., Straus, S. E., Richardson, W. S.,

Rosenberg, W., & Haynes, R. B. (2000).

Evidence-based medicine: How to practice and

teach EBM (2nd ed.). New York: Churchill

Livingstone.

SPSS, Inc. (2006). SPSS Base 14.0 for Windows

[Computer software]. Chicago: Author.

Walter, I., Nutley, S., & Davies, H. (2005).

What works to promote evidence-based

practice? A cross-sector review. Evidence &

Policy: A Journal of Research, Debate and

Practice, 1,335-364.

Williams, P. (2002). The competent boundary

spanner. Public Administration, 80,103 -124.

Accepted: 08/07

Jennifer L. Bellamy is an assistant professor at the School of Social Service Administration,University of Chicago; Sarah E. Bledsoe is an assistant professor at the Schooi of Sociai WorKUniversity of North Carolina at Ghapei Hiii; and Edward J. Mullen is the Willma and Aibert Musher Chairprofessor at the Schooi of Social WorK Columbia University. Lin Fang is an assistant professor at theUniversity of Toronto, and Jennifer I. Manuel a postdoctoral research associate at the New YorkState Psychiatric institute.

This work was supported by the Wilma and Albert Musher Program at the Columbia University School ofSocial Work and a grant from the National Institute of Mental Health, 5T32MH014623-26.

Address correspondence to Jennifer L. Bellamy, School of Social Service Administration, The Universityof Chicago, Chicago, IL; e-mail: [email protected].

AGENCY-UNIVERSITY PARTNERSHIP FOR EVIDENCE-BASED · PDF fileAGENCY-UNIVERSITY PARTNERSHIP 57...

Documents

Transcript of AGENCY-UNIVERSITY PARTNERSHIP FOR EVIDENCE-BASED · PDF fileAGENCY-UNIVERSITY PARTNERSHIP 57...