high fidelity use of EBPs. Our shared goal: never ... · Coaching Training Selection Integrated &...
11
Michelle A. Duda, Ph.D., BCBA-D Associate Director/Scientist National Implementation Research Network FPG Child Development Institute University of North Carolina at Chapel Hill The ACCFCR 10 th Anniversary Research Showcase May 22, 2013 Applied Implementation Science The NIRN Team Acknowledgement Big Ideas Congrats on all of the great work in Alberta! Alberta Response Model (2000) Child, Youth and Family Enhancement Act (2004) Casework practice model (2008) Build upon, improve, and sustain initiatives making a difference for children, families and communities Systematically and Intentionally apply the principles of Applied Implementation Science to serve as the platform for any initiative Human Service Systems Increasing populations Shrinking resources Increasing knowledge Modest outcomes Science to Service Gap Challenges Our shared goal: Create the infrastructure (capacity, systems and leadership) needed to support and sustain the high fidelity use of EBPs. Individuals Cannot Benefit from Interventions they never experience Socially Significant Solutions Good Intentions Actual Supports Years 1-3 Outcomes Every Teacher Trained Fewer than 50% of the teachers received some training Fewer than 10% of the schools used the CSR as intended Every Teacher Continually Supported Fewer than 25% of those teachers received support Vast majority of students did not benefit Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006 Longitudinal Studies of a Variety of Comprehensive School Reforms Research Example
Transcript of high fidelity use of EBPs. Our shared goal: never ... · Coaching Training Selection Integrated &...
Mic
helle
A. D
ud
a, P
h.D
., BC
BA
-D
Asso
cia
te D
irecto
r/Scie
ntis
t
Natio
nal Im
ple
men
tatio
n R
esearc
h N
etw
ork
FP
G C
hild
Develo
pm
en
t Institu
te
Un
ivers
ity o
f No
rth C
aro
lina a
t Ch
ap
el H
ill
Th
e A
CC
FC
R
10
thA
nn
ive
rsa
ryR
esearc
h S
ho
wcase
May 2
2, 2
013
Ap
plie
d
Imp
lem
en
tatio
nS
cie
nce
Th
e N
IRN
Team
Acknow
ledg
em
ent
Big
Ideas
�C
on
gra
ts o
n a
ll of th
e g
rea
t wo
rk in
Alb
erta
!�
Alb
erta
Re
sp
on
se
Mo
de
l (20
00
)
�C
hild
, Yo
uth
an
d F
am
ily E
nh
an
ce
me
nt A
ct (2
00
4)
�C
ase
wo
rk p
ractic
e m
od
el (2
00
8)
�B
uild
up
on
, imp
rove
, an
d s
usta
in in
itiativ
es
ma
kin
g a
diffe
ren
ce
for c
hild
ren
, fam
ilies a
nd
co
mm
un
ities
�S
yste
ma
tica
lly a
nd
Inte
ntio
na
lly a
pp
ly th
e
prin
cip
les o
f Ap
plie
d Im
ple
me
nta
tion
Scie
nce
to
se
rve
as th
e p
latfo
rm fo
r an
y in
itiativ
e
Hu
ma
n S
erv
ice
Sy
ste
ms
Inc
rea
sin
g p
op
ula
tion
s
Sh
rink
ing
res
ou
rce
s
Inc
rea
sin
g k
no
wle
dg
e
Mo
de
st o
utc
om
es
Scie
nce to
Serv
ice G
ap
Ch
alle
ng
es
Ou
r sh
are
d g
oal:
Cre
ate
the in
frastru
ctu
re
(capacity
, syste
ms a
nd
leaders
hip
) needed to
support a
nd s
usta
in th
e
hig
h fid
elity
use o
f EB
Ps.
Ind
ivid
ua
ls C
an
no
t Ben
efit
from
Inte
rven
tion
s th
ey
never e
xp
erie
nc
e
So
cia
lly S
ign
ifica
nt
So
lutio
ns
Go
od
In
ten
tion
s
Ac
tua
l Su
pp
orts
Ye
ars
1-3
Ou
tco
me
s
Every
Teach
er
Tra
ined
Few
er th
an
50%
of
the te
ach
ers
re
ceiv
ed
so
me
train
ing
Few
er th
an
10%
of
the s
ch
oo
ls u
sed
the
CS
R a
s in
ten
ded
Every
Teach
er
Co
ntin
ually
S
up
po
rted
Few
er th
an
25%
of
tho
se te
ach
ers
re
ceiv
ed
su
pp
ort
Vast m
ajo
rity o
f stu
den
ts d
id
no
t ben
efit
Ala
dje
m &
Bo
rman
, 2006; V
ern
ez, K
ara
m, M
aria
no
, & D
eM
artin
i, 2006
Lo
ng
itud
inal S
tud
ies o
f a V
arie
ty o
f Co
mp
reh
en
siv
e S
ch
oo
l Refo
rms
Re
se
arc
h E
xa
mp
le
Na
tion
al A
ss
es
sm
en
t of E
du
ca
tion
Pro
gre
ss
(NA
EP
)
•$100 b
illion
for in
no
vativ
e p
rog
ram
s(U
SD
E)
•$63 b
illion
for m
ate
rnal h
ealth
p
rog
ram
s(U
SA
ID)
•$4 b
illion
for h
om
evis
iting
pro
gra
ms
(AC
F)
•N
o fu
nd
ing
set a
sid
e fo
r im
ple
men
tatio
n s
up
po
rts fo
r these
pro
gra
m in
itiativ
es
So
cia
lly S
ign
ifican
t So
lutio
ns
Fe
de
ral S
PE
ND
ING
on
K-1
2 E
du
ca
tion
un
de
r the
ES
EA
an
d N
AE
P R
EA
DIN
G S
co
res
(Ag
e 9
)
Exp
erim
en
tal D
ata
Sh
ow
Th
ese M
eth
od
s,
Wh
en
Used
Alo
ne, A
re In
su
fficie
nt:
•D
iffus
ion
/ Dis
se
min
atio
n o
f info
rma
tion
•T
rain
ing
•P
as
sin
g la
ws
/ ma
nd
ate
s/ re
gu
latio
ns
•P
rov
idin
g fu
nd
ing
/ inc
en
tive
s
•O
rga
niz
atio
n c
ha
ng
e/ re
org
an
iza
tion
Data
: 5%
to 1
5%
Realiz
e In
ten
ded
Ou
tco
mes
Wh
at D
oes N
ot W
ork
Reco
gn
izin
g G
ap
sT
he
Ch
alle
ng
e
•S
cien
ce to
Se
rvice
Ga
p–
Wh
at is k
no
wn
to b
e e
ffectiv
e is n
ot w
ha
t is se
lecte
dto
he
lp co
nsu
me
rs
•Im
ple
me
nta
tion
Ga
p–
Wh
at is se
lecte
d is n
ot u
sed
with
fide
litya
nd
go
od
o
utco
me
s
–W
ha
t is use
d w
ith fid
elity
is no
t susta
ine
dfo
r a
use
ful p
erio
d o
f time
–W
ha
t is use
d w
ith fid
elity
is no
t use
d o
n a
scale
sufficie
nt to
bro
ad
ly im
pa
ct ou
tcom
es
GO
OD
NE
WS
!
�A
gric
ultu
re�
Bu
sin
ess
�P
reve
ntio
n�
Ch
ild W
elfa
re�
Me
nta
l & B
eh
avio
ral H
ea
lth
�E
du
ca
tion
�Ju
ve
nile
Ju
stic
e
�P
ub
lic H
ea
lth�
Ea
rly c
hild
ho
od
�
Su
bsta
nce
Ab
use
Se
rvic
es
Imple
menta
tion is
imple
menta
tion a
cro
ss
dom
ain
s a
nd s
erv
ice s
ecto
rs…
It’s tra
nsdis
cip
linary
!
“Im
ple
me
nta
tion
sc
ien
ce
is
the
sys
tem
atic
stu
dy o
f v
aria
ble
s a
nd
co
nd
ition
s
tha
t lea
d to
full a
nd
e
ffec
tive
us
e o
f ev
ide
nc
e-
ba
se
d p
rog
ram
s a
nd
oth
er
effe
ctiv
e in
no
va
tion
s in
ty
pic
al h
um
an
se
rvic
e
se
tting
s.”
Bla
se
an
d F
ixse
n, 2
01
0
Na
tion
al Im
ple
me
nta
tion
Re
se
arc
h N
etw
ork
“Im
ple
men
tatio
n S
cie
nce”
Do
wn
loa
d a
t: h
ttp://w
ww
.fpg
.unc.e
du
/~n
irn/re
so
urc
es/p
ublic
atio
ns/M
ono
gra
ph
/
Inven
t a n
ew
futu
re
•S
top
wastin
g tim
e a
nd
mo
ney o
n th
ing
s
that d
on
’t wo
rk (a
nd
never h
ave!)
–S
ug
ai’s
La
w: F
or e
ve
ry n
ew
initia
tive
, sto
p
two
cu
rren
t on
es
.
–D
e-s
ca
le; A
vo
id la
ye
ring
an
d fra
gm
en
tatio
n
•S
et a
sid
e 1
5%
of fu
nd
s fo
r imp
lem
en
tatio
n
•R
eq
uire
qu
arte
rly re
po
rts o
f fidelity
data
Syste
m R
ein
ven
tion
© F
ixsen a
nd B
lase 2
013
Ap
plie
d Im
ple
me
nta
tion
S
cie
nc
eA
ctiv
e Im
ple
me
nta
tion
Imp
lem
en
tatio
n
Imp
lem
en
tatio
n
Team
s u
se
evid
en
ce b
ased
im
ple
men
tatio
n
meth
od
s
Cap
acity
bu
ildin
g a
nd
lead
ers
hip
d
evelo
pm
en
t of Im
ple
men
tatio
n
Team
Pra
ctitio
ners
use
EB
P/in
no
vatio
n
with
fidelity
Po
sitiv
e
ou
tco
mes fo
r ch
ildre
n a
nd
fa
milie
s
Makin
g It H
ap
pen
Activ
e Im
ple
men
tatio
n
•L
ettin
g it h
ap
pen
(Diffu
sio
n)
–R
ec
ipie
nts
are
ac
co
un
tab
le
•H
elp
ing
it hap
pen
(Dis
sem
inatio
n)
–R
ec
ipie
nts
are
ac
co
un
tab
le
•M
akin
g it h
ap
pen
(Activ
e Im
ple
men
tatio
n)
–P
urp
os
efu
l an
d p
roa
ctiv
e u
se
of
imp
lem
en
tatio
n p
rac
tice
an
d s
cie
nc
e
–Im
ple
me
nta
tion
Te
am
s a
re a
cc
ou
nta
ble
Ba
se
d o
n H
all &
Ho
rd (1
98
7); G
ree
nh
alg
h, R
ob
ert, M
ac
Fa
rlan
e, B
ate
, & K
yria
kid
ou
(20
04
); F
ixs
en
, Bla
se
, Du
da
, Na
oo
m, &
Va
n D
yk
e (2
01
0)
Ac
tive
Imp
lem
en
tatio
n F
ram
ew
ork
s
•O
rgan
ize Im
ple
men
tatio
n K
no
wle
dg
e
•Id
en
tify G
ap
s
•G
uid
e D
evelo
pm
en
t of Im
ple
men
tatio
n
Cap
acity
•C
reate
Testa
ble
Hyp
oth
eses
So
cia
lly
Sig
nific
an
t O
utc
om
es
Effe
ctiv
e
Inte
rven
tion
s
Effe
ctiv
e
Imp
lem
en
tatio
n
Meth
od
s
En
ab
ling
C
on
texts
Fo
rmu
la fo
r Su
ccess
Th
e w
ell-b
ein
g o
f A
LL
of A
lberta
’s
ch
ildre
n, fa
milie
s
an
d c
om
mu
nitie
s
co
ntin
ues to
im
pro
ve
Ac
tive
Imp
lem
en
tatio
n F
ram
ew
ork
sA
ctiv
e Im
ple
me
nta
tion
Fra
me
wo
rks
WH
O
Imple
menta
tion
Team
s
WH
EN
Imple
menta
tion
Sta
ges
HO
W
Imple
menta
tion
Driv
ers
HO
W
Impro
vem
ent
Cycle
s
WH
AT
Usable
Inte
rventio
ns
�U
sab
le In
terv
en
tion
s
�Im
ple
me
nta
tion
Sta
ge
s
�Im
ple
me
nta
tion
Driv
ers
�Im
pro
ve
me
nt C
ycle
s
�Im
ple
me
nta
tion
Te
am
s
Activ
e Im
ple
men
tatio
n
Fra
mew
ork
s
DIL
EM
MA
Most o
f our e
vid
ence-b
ased in
terv
entio
ns d
o
not m
eet u
sability
crite
ria fo
r effe
ctiv
e
imple
menta
tion
•A
bo
ut 1
8%
of o
utc
om
e s
tud
ies
(N=
1,2
00+
) assessed
the in
dep
en
den
t varia
ble
(“it”
)
–A
bo
ut 7
% lin
ke
d e
ss
en
tial c
om
po
ne
nts
(fid
elity
) to o
utc
om
es
(we
kn
ow
wh
at
pro
du
ce
s d
es
ired
ou
tco
me
s)
Mo
nc
he
r & P
rinz, 1
99
1;G
res
ha
m, e
t al., 1
99
3; D
an
e &
Sc
hn
eid
er, 1
99
8; D
urla
k &
Du
Pre
, 20
08
Wh
at is
“it”
?
1.
Cle
ar d
es
crip
tion
of th
e
pro
gra
m�
Ph
ilos
op
hy, v
alu
es
, prin
cip
les
�In
clu
sio
n –
ex
clu
sio
n c
riteria
2.
Ide
ntifie
d c
ore
inte
rve
ntio
n c
om
po
ne
nts
(ak
a,
ac
tive
ing
red
ien
ts, e
ss
en
tial fu
nc
tion
s)
3.
Op
era
tion
al d
efin
ition
so
f co
re in
terv
en
tion
c
om
po
ne
nts
(wh
at p
rac
tition
ers
do
, sa
y)
4.
Pra
ctic
al p
erfo
rma
nc
e/fid
elity
as
se
ss
me
nt
�H
igh
ly c
orre
late
d (0
.70
+) w
ith d
es
ired
ou
tco
me
s
Usab
le In
terv
en
tion
Crite
riaW
hat s
ho
uld
we d
o a
bo
ut th
is?
Focus in
terv
entio
ns re
searc
h m
ore
on
identify
ing c
ore
inte
rventio
n c
om
ponents
Wh
y is
this
imp
orta
nt?
1.
Bu
ildin
g s
taff
ca
pa
city
�S
ele
ctio
n
�T
rain
ing
�C
oa
ch
ing
�P
erfo
rma
nc
e/F
ide
lity
As
se
ss
me
nt
2.
En
ab
ling
C
on
tex
tua
liza
tion
�A
vo
idin
g th
e “
Zo
ne
of
Dra
stic
Mu
tatio
n”
(Baum
an, S
tein
, & Ire
ys, 1
991)
�P
rac
tition
er e
xp
ertis
e
�C
lien
t pre
fere
nc
es
an
d
va
lue
s
3. E
na
blin
g re
plic
atio
n w
ith s
ca
le
Pra
ctitio
ners
and m
anagers
use in
terv
entio
ns,
they d
o n
ot u
se s
tandard
s fo
r rigor
Co
mp
ete
nc
y D
rive
rsa
nd
Pra
ctitio
ne
r Tu
rno
ve
r
N=
153
Aaro
ns e
t al., 2
009
So
cia
lly
Sig
nific
an
t O
utc
om
es
WH
AT:
Effe
ctiv
e
Inte
rven
tion
s
HO
W/W
HO
:E
ffectiv
e
Imp
lem
en
tatio
n
Meth
od
s
En
ab
ling
C
on
texts
Fo
rmu
la fo
r Su
ccess
© F
ixsen a
nd B
lase 2
013
Th
e w
ell-b
ein
g o
f A
LL
of A
lberta
’s
ch
ildre
n, fa
milie
s
an
d c
om
mu
nitie
s
co
ntin
ue to
im
pro
ve
AP
PL
IED
Imp
lem
en
tatio
n
Scie
nce
�U
sa
ble
Inte
rve
ntio
ns
�Im
ple
men
tatio
n
Sta
ges
�Im
ple
me
nta
tion
Driv
ers
�Im
pro
ve
me
nt C
ycle
s
�Im
ple
me
nta
tion
Te
am
s
•P
urp
ose
ful m
atch
ing
of critica
l
imp
lem
en
tatio
n a
ctivitie
s to th
e
ap
pro
pria
te sta
ge
of th
e
pro
cess
Sta
ge
s o
f Imp
lem
en
tatio
n
Sta
ges
Imp
lem
en
tatio
n S
tag
es
Imp
lem
en
tatio
n S
tag
es
2-4
Years
•A
ssess n
eed
s•
Exam
ine
inte
rven
tion
co
mp
on
en
ts•
Co
nsid
er
Imp
lem
en
tatio
n
Driv
ers
•A
ssess F
it
•A
cq
uire
R
eso
urc
es
•P
rep
are
O
rgan
izatio
n•
Pre
pare
Im
ple
men
tatio
n
Driv
ers
•P
rep
are
Sta
ff
•A
dju
st
Imp
lem
en
tatio
n
Driv
ers
•M
an
ag
e C
han
ge
•D
ep
loy D
ata
S
yste
ms
•In
itiate
Im
pro
vem
en
t C
ycle
s
•M
on
itor &
M
an
ag
e
Imp
lem
en
tatio
n
Driv
ers
•A
ch
ieve F
idelity
&
Ou
tco
me
Ben
ch
mark
s•
Fu
rther Im
pro
ve
Fid
elity
&
Ou
tco
mes
Ex
plo
ratio
nIn
sta
llatio
nIn
itial
Imp
lem
en
tatio
nF
ull
Imp
lem
en
tatio
n
AP
PL
IED
Imp
lem
en
tatio
n
Scie
nce
�U
sa
ble
Inte
rve
ntio
ns
�Im
ple
me
nta
tion
Sta
ge
s
�Im
ple
men
tatio
n
Driv
ers
�Im
pro
ve
me
nt C
ycle
s
�Im
ple
me
nta
tion
Te
am
s
•H
elp
to d
evelo
p, im
pro
ve, a
nd
su
sta
in p
ractitio
ners
co
mp
ete
nce a
nd
co
nfid
en
ce to
imp
lem
en
t effe
ctiv
e
pra
ctic
es a
nd
su
pp
orts
•H
elp
en
su
re s
usta
inab
ility a
nd
imp
rovem
en
t at th
e
org
an
izatio
n a
nd
syste
ms le
vel
•H
elp
gu
ide le
ad
ers
to u
se th
e rig
ht le
ad
ers
hip
stra
teg
ies
for th
e s
ituatio
n
Imp
lem
en
tatio
n D
rive
rs
Driv
ers
© F
ixsen a
nd B
lase 2
013
Tra
inin
g a
nd
Co
ach
ing
Imp
lem
en
tatio
n
Driv
ers
OU
TC
OM
ES
% o
f Pa
rticip
an
ts w
ho
De
mo
ns
trate
Kn
ow
led
ge
, De
mo
ns
trate
N
ew
Sk
ills in
a T
rain
ing
Se
tting
,
an
d U
se
ne
w S
kills
in th
e C
las
sro
om
TR
AIN
ING
CO
MP
ON
EN
TS
Kn
ow
led
ge
Sk
ill
De
mo
ns
tratio
n
Us
e in
the
Cla
ss
roo
m
Th
eo
ry a
nd
D
iscu
ssio
n10%
5%
0%
..+D
em
on
stra
tion
in
Tra
inin
g30%
20%
0%
…+
Pra
ctic
e &
F
eed
back in
Tra
inin
g60%
60%
5%
…+
Co
ac
hin
g in
C
las
sro
om
95%
95%
95%
—Joyce a
nd S
how
ers
, 2002
© F
ixse
n &
Bla
se, 2
00
8
So
cia
lly S
ign
ifican
t Ou
tco
mes
Co
ns
iste
nt U
se
of E
BP
s
Inte
gra
ted
&
Co
mp
en
sa
tory
Lead
ers
hip
Lead
ers
hip
Co
ach
ing
Tra
inin
g
Sele
ctio
n
Syste
ms
Inte
rven
tion
Facilita
tive
Ad
min
istra
tion
Decis
ion
Su
pp
ort
Data
Syste
m
Perfo
rman
ce A
sses
sm
en
t(F
idelity
)
Lo
cu
s o
f Re
sp
on
sib
ility
Perfo
rman
ce A
sses
sm
en
t (F
idelity
)
Co
ach
ing
Tra
inin
g
Sele
ctio
n
Inte
gra
ted
&
Co
mp
en
sato
ry
Syste
ms
Inte
rven
tion
Facilita
tive
Ad
min
istra
tion
Decis
ion
Su
pp
ort
Data
Syste
m
Ad
ap
tive
Tech
nic
al
Le
ad
ers
hip
Driv
ers
Inte
gra
ted
&
Co
mp
en
sato
ry
Sta
te G
ov’t. D
ept.
Local P
artn
ers
hip
Dire
ct S
erv
ice P
rovid
er
Contra
cte
d T
A G
roup
Natio
nal P
urv
eyor
OM
ITT
ED
Inn
ovatio
n
Re
su
lts fro
m C
hild
W
ellb
ein
g P
roje
ct
Ca
se
Ex
am
ple
: Me
tz e
t al.
Co
mp
on
en
tT
1T
2T
3
Se
lectio
n1
.44
2.0
0*
1.8
9*
Tra
inin
g1
.33
1.5
0*
1.1
0
Co
ach
ing
1.2
71
.73
*1
.83
*
Pe
rf. Asse
ssme
nt
0.7
81
.34
2.0
0*
DS
DS
0.1
81
.36
2.0
0*
Fac. A
dm
inistra
tion
1.3
82
.00
*2
.00
*
Sy
stem
s Inte
rve
ntio
n1
.29
1.8
6*
2.0
0*
Av
era
ge
Co
mp
osite
Sco
re1
.11
.68
*1
.83
*
Fid
elity
(% o
f
case
s)
18
%8
3%
83
%
Su
ccess C
oach
mo
del in
vo
lved
inte
nse p
rog
ram
develo
pm
en
t of c
ore
in
terv
en
tion
co
mp
on
en
ts a
nd
acco
mp
an
yin
g im
ple
men
tatio
n d
rivers
© D
ea
n F
ixse
n, K
are
n B
lase
, Ro
be
rt Ho
rne
r, Ge
org
e S
uga
i, 20
08
—R
. Sp
en
cer D
arlin
g
Bu
sine
ss Exp
ert
All o
rga
niza
tion
s [an
d sy
stem
s]
are
de
sign
ed
, inte
ntio
na
lly o
r
un
wittin
gly
, to a
chie
ve
pre
cisely
the
resu
lts the
y g
et.“
EX
IST
ING
SY
ST
EM
Effe
ctiv
e In
no
va
tion
s a
re
Ch
an
ge
d to
Fit th
e
Sys
tem
Or O
pe
rate
in th
e S
ha
do
ws
(Gh
os
t Sys
tem
)
Effe
ctiv
e S
yste
m C
han
ge
EX
IST
ING
SY
ST
EM
IS
CH
AN
GE
D T
O S
UP
PO
RT
TH
E E
FF
EC
TIV
EN
ES
S O
F
TH
E IN
NO
VA
TIO
N
(Ho
st S
ys
tem
)
EF
FE
CT
IVE
INN
OV
AT
ION
Ho
w S
yste
ms A
re C
han
ged
�U
se
of D
ec
isio
n-S
up
po
rt Da
ta
Sys
tem
s�
Tim
ely
, Va
lid, R
elia
ble
�A
ctio
na
ble
Un
its
�U
se
d to
Info
rm D
ec
isio
ns
�F
ac
ilitativ
e A
dm
inis
tratio
n�
Th
ing
s A
dm
inis
trato
rs C
an
Co
ntro
l
�R
ed
uc
e B
urd
en
�Im
pro
ve
Fid
elity
an
d O
utc
om
es
�S
ys
tem
s In
terv
en
tion
�G
ain
su
pp
ort o
f a L
eve
l “A
bo
ve
”
�In
form
the
sys
tem
of b
arrie
rs,
so
lutio
ns
, an
d fa
cilita
tors
�W
ork
tog
eth
er to
find
so
lutio
ns
an
d
tes
t the
m
Effe
ctiv
e S
yste
m C
han
ge
EX
IST
ING
SY
ST
EM
IS
CH
AN
GE
D T
O S
UP
PO
RT
TH
E E
FF
EC
TIV
EN
ES
S O
F
TH
E IN
NO
VA
TIO
N
(Ho
st S
ys
tem
)
EF
FE
CT
IVE
INN
OV
AT
ION
Organization Drivers Organization Drivers
AP
PL
IED
Imp
lem
en
tatio
n
Scie
nce
�U
sa
ble
Inte
rve
ntio
ns
�Im
ple
me
nta
tion
Sta
ge
s
�Im
ple
me
nta
tion
Driv
ers
�Im
pro
vem
en
t Cycle
s
�Im
ple
me
nta
tion
Te
am
s
•N
ew
Ap
pro
ac
he
s N
ee
d N
ew
Wa
ys
of W
ork
•T
ran
sp
are
nt, p
roto
co
l-driv
en
fee
db
ac
k lo
op
s
an
d p
roc
es
se
s
•A
lign
ed
po
licie
s, fu
nd
ing
, gu
ida
nc
e to
su
pp
ort
ne
w w
ays
of w
ork
•T
he
re a
re n
o a
dm
inis
trativ
e d
ec
isio
ns
, the
y a
re
all q
ua
lity d
ec
isio
ns
.
Imp
rovem
en
t Cycle
sC
ycle
s
© F
ixsen a
nd B
lase 2
013
Imp
rovem
en
t Cycle
s
Ra
pid
cy
cle
(PD
SA
) pro
ble
m s
olv
ing
Sh
ew
hart (1
931); D
em
ing
(1986)
Us
ab
ility te
stin
g
Ru
bin
(1994); N
iels
en
(2000)
Pra
ctic
e-p
olic
y c
om
mu
nic
atio
n lo
op
Fix
sen
, Bla
se, M
etz
, & V
an
Dyke (2
013)
Usa
bility
vs. P
ilot T
estin
g
Usa
bility
•C
lea
r de
scriptio
n o
f the
pro
gra
m
•Tria
l an
d le
arn
ing
ap
pro
ach
•S
ma
ll nu
mb
er o
f
pa
rticipa
nts (N
= 3
-5
)
•M
ultip
le ite
ratio
ns to
de
tect a
nd
corre
ct
pro
ble
ms a
s the
y a
rise
•Le
arn
HO
W to
do
the
wo
rk
effe
ctive
ly
Pilo
t
•C
lea
r de
scriptio
n o
f the
pro
gra
m
•Tria
l an
d a
ssessm
en
t
ap
pro
ach
•S
ufficie
nt n
um
be
r of
pa
rticipa
nts fo
r statistica
l
po
we
r (N=
20
–5
0)
•S
ufficie
nt tim
e to
rea
lize
po
ten
tial re
sults
Po
licy P
ractic
e F
eed
ba
ck L
oo
ps
Po
licy (P
lan
)
Pra
ctic
e (D
o)
Stru
ctu
re
Pro
ced
ure
Po
licy
Pra
ctic
e
Fo
rm N
ee
ds
to S
up
po
rts F
un
ctio
ns
at E
ve
ry L
eve
l
Feedback
Study -Act
Policy Enabled Practices(PEP)
Practice Informed Policy(PIP)
“External” System Change Support
AP
PL
IED
Imp
lem
en
tatio
n
Scie
nce
�U
sa
ble
Inte
rve
ntio
ns
�Im
ple
me
nta
tion
Sta
ge
s
�Im
ple
me
nta
tion
Driv
ers
�Im
pro
ve
me
nt C
ycle
s
�Im
ple
men
tatio
n
Team
s
Imp
l. Team
NO
Imp
l. Team
Effe
ctiv
e
Makin
g it H
ap
pen
IMP
LE
ME
NTA
TIO
N
INTERVENTION
80%
, 3 Y
rs14%
, 17 Y
rs
Ba
las &
Bo
ren
, 20
00
Gre
en
& S
eife
rt, 20
05
Fix
se
n, B
lase
, Tim
be
rs, &
Wo
lf, 20
01
Lettin
g it H
ap
pen
H
elp
ing
it Hap
pen
Imp
lem
en
tatio
n T
eam
s
Imp
lem
en
tatio
n T
eam
s A
RE
NO
T
•C
om
mitte
es
•A
dvis
ory
Gro
up
s
Imp
lem
en
tatio
n T
eam
s D
O
•K
no
w th
e in
no
va
tion
–th
e W
HA
T
•K
no
w a
nd
us
e im
ple
me
nta
tion
sc
ien
ce
an
d b
es
t pra
ctic
es
–th
e
HO
W
•U
se
Pla
n, D
o, S
tud
y, A
ct C
yc
les
to “
ge
t sta
rted
” a
nd
“g
et b
ette
r”
•S
tay o
n “
mis
sio
n”
•E
ng
ag
e in
the
righ
t wo
rk a
t the
righ
t leve
l at th
e rig
ht tim
e.
Imp
lem
en
tatio
n T
eam
s
Team
s
© F
ixsen a
nd B
lase 2
013
Pro
vid
e Im
ple
me
nta
tion
Su
pp
ort
•F
ocu
s is
on
–
Qu
ality
, inte
gra
tion
, su
sta
ina
bility
of D
rive
rs–
Da
ta-b
as
ed
de
cis
ion
-ma
kin
g
–P
rob
lem
-so
lvin
g a
nd
an
aly
tica
l en
ga
ge
me
nt
–P
urp
os
efu
l ad
ap
tatio
n a
nd
pla
nn
ed
su
pp
orts
–A
lign
me
nt a
nd
Su
sta
ina
bility
•P
rovid
e a
cco
un
tab
le s
tructu
re to
mo
ve
inte
rven
tion
thro
ug
h s
tag
es o
f im
ple
men
tatio
n fo
r an
y n
ew
inn
ovatio
n
Bu
ildin
g a
nd
Div
ers
ifyin
g L
ead
ers
hip
�H
ow
can w
e b
uild
the c
apacitie
sof
Imple
menta
tion T
eam
s?
�H
ow
can w
eexpand
the le
aders
hip
base?
�H
ow
can w
e b
uild
adaptiv
e p
roble
m
solv
ing u
nits
?
�H
ow
can w
e b
uild
team
s th
at c
an
addre
ss a
ny in
novatio
n?
© F
ixsen a
nd B
lase 2
013
So
cia
lly
Sig
nific
an
t O
utc
om
es
WH
AT:
Effe
ctiv
e
Inte
rven
tion
s
HO
W/W
HO
:E
ffectiv
e
Imp
lem
en
tatio
n
Meth
od
s
WH
ER
E:
En
ab
ling
C
on
texts
Fo
rmu
la fo
r Su
ccess
Th
e w
ell-b
ein
g o
f A
LL
of A
lberta
’s
ch
ildre
n, fa
milie
s
an
d c
om
mu
nitie
s
co
ntin
ues to
im
pro
ve
Imp
lem
en
tatio
n
Team
Execu
tive
Man
ag
em
en
t Team
Pra
ctitio
ners
Inn
ovatio
ns
Ch
ildre
n, In
div
idu
als
System Change
Ad
ap
tive
Ch
alle
ng
es
•D
up
lica
tion
•F
rag
me
nta
tion
•H
iring
crite
ria
•S
ala
ries
•C
red
en
tialin
g
•L
ice
ns
ing
•T
ime
/ sc
he
du
ling
•U
nio
n c
on
trac
ts
•R
FP
me
tho
ds
•L
aw
s/ m
an
da
tes
“External” System Change Support
Practice-Policy Communication Loop
Policy Enabled Practice
Rein
ven
t Syste
ms: M
an
ag
e C
han
ge
“T
he fa
ult
can
no
t lie in
th
e p
art
resp
on
sib
le
for th
e
rep
air.”
Ash
by
(1956)
Initia
te a
nd
Man
ag
e C
han
ge
•To
beg
in, d
o o
ne th
ing
well (E
BP
)
–S
ing
ula
r foc
us
for c
ha
ng
e
–B
uild
ca
pa
city
as
yo
u le
arn
to d
o it w
ell
•D
ecid
e w
hat to
keep
, wh
at to
revis
e, a
nd
w
hat is
no
t need
ed
–8
0%
sim
ilar / th
e s
am
e (U
lrich
, 20
02
)
–2
0%
ch
an
ge
d / d
rop
pe
d
–H
ow
to d
ec
ide
wh
ich
is w
hic
h?
?
–G
et s
tarte
d, g
et b
ette
r (on
pu
rpo
se
!)
•A
ble
to le
arn
from
the
ir ow
n e
xp
erie
nc
e
–A
ctio
n: M
od
ify th
eir s
tructu
re a
nd
desig
n to
refle
ct
that th
ey h
ave le
arn
ed
•S
taff a
nd
org
an
iza
tion
al e
lem
en
ts a
re m
ulti-
sk
illed
–A
ctio
n: S
yste
matic
ally
dete
ct a
nd
co
rrect e
rrors
aris
ing
in o
ther p
arts
of th
e s
yste
m
•S
ys
tem
ha
s th
e c
ap
ac
ity to
se
arc
h fo
r erro
rs
an
d fa
ulty
op
era
ting
as
su
mp
tion
s
–A
ctio
n: M
ake c
han
ges n
eed
ed
to d
efra
gm
en
t an
d
alig
n fu
nctio
ns w
ithin
the s
yste
m
Learn
ing
Syste
ms
Next “
Rig
ht”
Ste
ps
Earm
ark
funds w
ithin
inte
rventio
ns re
searc
h
gra
nts
and s
erv
ices c
ontra
cts
Re
se
arc
h G
ran
ts
•M
ea
su
ring
the
pre
se
nce
an
d s
tren
gth
of c
ore
inte
rve
ntio
nco
mp
on
en
ts
•C
on
ne
ctin
g fid
elity
da
ta to
pro
gra
m o
utc
om
es
•M
ea
su
ring
co
re
imp
lem
en
tatio
nco
mp
on
en
ts
Se
rvic
es
Co
ntra
cts
•S
ecu
re F
TE
for
Imp
lem
en
tatio
n T
ea
m
me
mb
ers
•F
un
d d
ecis
ion
-su
pp
ort d
ata
syste
ms
•F
un
d u
sa
bility
testin
g
•Q
ua
rterly
fide
lity re
po
rts
Pu
tting
it all T
og
eth
er
Ex
tern
al
Influ
en
ce
Fa
ctors
Org
an
izatio
n
an
d Le
ad
ersh
ip
Co
mp
on
en
ts
Activ
e
Imp
lem
en
tatio
n
Co
mp
on
en
ts
Pre
dicte
d
Fid
elity
Ou
tcom
es
Pre
dicte
d
Su
stain
ab
ility
Ou
tcom
es
Ge
ne
rally
En
ab
ling
Stro
ng
Stro
ng
Hig
hLo
ng
Term
We
ak
Low
/Me
diu
mM
ed
ium
Term
We
ak
Stro
ng
Hig
h
Me
diu
m Te
rm
We
ak
Low
Sh
ort Te
rm
Ge
ne
rally
Hin
de
ring
Stro
ng
Stro
ng
Hig
hM
ed
ium
Term
We
ak
Low
Me
diu
m Te
rm
We
ak
Stro
ng
Me
diu
m/H
igh
Sh
ort Te
rm
We
ak
Low
Sh
ort Te
rm
Bu
reau
cra
cy
Pra
ctitio
ners
Po
licie
s
Pro
vid
er
Org
an
izatio
ns
Fu
nctio
nal S
yste
ms
WH
Y:
So
cia
lly
Sig
nific
an
t O
utc
om
es
WH
AT:
Effe
ctiv
e
Inte
rven
tion
s
HO
W/W
HO
:E
ffectiv
e
Imp
lem
en
tatio
n
Meth
od
s
WH
ER
E:
En
ab
ling
C
on
texts
Fo
rmu
la fo
r Su
ccess
Th
e w
ell-b
ein
g o
f A
LL
of A
lberta
’s
ch
ildre
n, fa
milie
s
an
d c
om
mu
nitie
s
co
ntin
ues to
im
pro
ve
Su
mm
ary
�P
ositiv
e A
ssum
ptio
ns a
nd B
uild
Upon S
trength
s
�C
om
pete
nce n
eeds to
be D
evelo
ped a
nd S
usta
ined
�S
ele
ctio
n, T
rain
ing
, Co
ach
ing
, Fid
elity
Me
asu
res h
elp
ch
an
ge
an
d s
up
po
rt ne
w
pra
ctitio
ne
r be
ha
vio
r an
d s
kills
�O
rganiz
atio
ns a
nd S
yste
ms n
eed to
change
�D
ata
syste
ms n
ee
d to
be
use
d to
ma
ke
de
cis
ion
s
�F
acilita
tive
ad
min
istra
tive
pra
ctic
es &
syste
ms in
terv
en
tion
s c
rea
te h
osp
itab
le
en
viro
nm
en
ts
�P
olic
y e
na
ble
s n
ew
pra
ctic
e b
ut p
ractic
e n
ee
ds to
info
rm p
olic
y
�C
onte
nt E
xperts
and Im
ple
menta
tion T
eam
s h
elp
with
syste
m a
nd s
erv
ice c
hange
�Y
ou a
re n
ever d
one –
The e
nviro
nm
ent is
in m
otio
n�
Imp
rove
me
nt C
ycle
s a
re C
ritica
l
�T
he
‘righ
t’ lea
de
rsh
ip s
trate
gie
s a
re n
ee
de
d fo
r the
issu
es a
t ha
nd
Fra
nk P
orte
r Gra
ham
Ch
ild D
evelo
pm
en
t Institu
te
Un
ivers
ity o
f No
rth C
aro
lina a
t Ch
ap
el H
ill
http
://nirn
.fpg
.un
c.e
du
/w
ww
.scalin
gu
p.o
rg
Sta
y C
on
necte
d!
Learn
Mo
re:
http
://imp
lem
en
tatio
n.fp
g.u
nc.e
du
/@Im
pS
cie
nce
NIR
N
Mic
he
lle A
. Du
da
, Ph
.D., B
CB
A-D
du
da
@u
nc
.ed
u
Au
gu
st 1
9-2
1, 2
013
ww
w.g
lobalim
ple
me
nta
tion.o
rg/g
ic
Wash
ing
ton
Hilto
n H
ote
lW
as
hin
gto
n D
.C.
HT
TP
://NIR
N.F
PG
.UN
C.E
DU
Fix
se
n, D
. L., N
ao
om
, S. F
., Bla
se
, K. A
., Frie
dm
an
, R. M
. & W
alla
ce
, F. (2
00
5).
Imp
lem
en
tatio
n R
ese
arc
h: A
Syn
the
sis
of th
e L
itera
ture
. Ta
mp
a, F
L: U
niv
ers
ity o
f S
ou
th F
lorid
a, L
ou
is d
e la
Pa
rte F
lorid
a M
en
tal H
ea
lth In
stitu
te, T
he
Na
tion
al
Imp
lem
en
tatio
n R
ese
arc
h N
etw
ork
(FM
HI P
ub
lica
tion
#2
31
).
Imp
lem
en
tatio
n
Re
se
arc
h:
A S
yn
the
sis
of
the
Lite
ratu
re
Imp
lem
en
tatio
n S
cie
nce
•A
nn
ie E
. Ca
se
y F
ou
nd
atio
n
(EB
Ps a
nd
Cu
ltura
l Co
mp
ete
nce
)
•W
illiam
T. G
ran
t Fo
un
da
tion
(Im
ple
me
nta
tion
Lite
ratu
re R
evie
w)
•S
ub
sta
nc
e A
bu
se
an
d M
en
tal
He
alth
Se
rvic
es
A
dm
inis
tratio
n (S
AM
HS
A)
(Imp
lem
en
tatio
n S
trate
gie
s G
ran
ts;
Na
tion
al Im
ple
me
nta
tion
Aw
ard
s)
•C
en
ters
for D
isease C
on
trol &
P
reven
tion
(CD
C)
(Imp
lem
en
tatio
n R
ese
arc
h)
•D
uke E
nd
ow
men
t (C
hild
Welfa
re R
efo
rm)
•N
atio
na
l Ins
titute
of M
en
tal
He
alth
(NIM
H)
(Re
se
arc
h A
nd
Tra
inin
g G
ran
ts)
•J
uve
nile
Ju
stic
e a
nd
D
elin
qu
en
cy P
reve
ntio
n
(Pro
gra
m D
eve
lop
me
nt A
nd
Eva
lua
tion
G
ran
ts)
•U
S D
ep
t. of E
du
ca
tion
: Offic
e o
f S
pe
cia
l Ed
uc
atio
n P
rog
ram
s
(Sca
ling
up
an
d C
ap
acity
De
ve
lop
me
nt
Ce
nte
r)
•A
dm
inis
tratio
n fo
r Ch
ildre
n a
nd
F
am
ilies
(AC
F)
(Ch
ild W
elfa
re L
ea
de
rsh
ip; C
ap
acity
D
eve
lop
me
nt C
en
ter)
Th
an
k Y
ou
for Y
ou
r Su
pp
ort
©C
op
yrig
ht D
ean
Fix
sen
an
d K
are
n B
lase
Th
is c
on
ten
t is lic
en
se
d u
nd
er C
rea
tive
Co
mm
on
s lic
en
se
CC
BY
-N
C-N
D, A
ttribu
tion
-No
nC
om
me
rcia
l-No
Deriv
s. Y
ou
are
free
to
sh
are
, co
py, d
istrib
ute
an
d tra
ns
mit th
e w
ork
un
de
r the
follo
win
g
co
nd
ition
s: A
ttribu
tion
—Y
ou
mu
st a
ttribu
te th
e w
ork
in th
e
ma
nn
er s
pe
cifie
d b
y th
e a
uth
or o
r lice
ns
or (b
ut n
ot in
an
y w
ay th
at
su
gg
es
ts th
at th
ey e
nd
ors
e y
ou
or y
ou
r us
e o
f the
wo
rk);
No
nc
om
me
rcia
l —Y
ou
ma
y n
ot u
se
this
wo
rk fo
r co
mm
erc
ial
pu
rpo
se
s; N
o D
eriv
ativ
e W
ork
s —
Yo
u m
ay n
ot a
lter o
r tran
sfo
rm
this
wo
rk.
An
y o
f the
ab
ove
co
nd
ition
s c
an
be
wa
ive
d if y
ou
ge
t p
erm
iss
ion
from
the
co
pyrig
ht h
old
er.
http
://cre
ativ
ec
om
mo
ns
.org
/lice
nse
s/b
y-n
c-n
d/3
.0
